OPSYN Vol.3 How to Set Up Leap Motion and Utilize it in Unreal Engine
OPSYN Progress
This week, I was able to control the game object with my hands using leap motion and control the music event in Max/MSP.
I can send the 3 dimensions (X,Y,Z) at the same time, and it can be assigned to any parameter in Max or DAW. With this, I can do something like MIDI Polyphonic Expression (MPE). For example, you can control the modulation in addition to controlling the pitch and velocity with one finger movement. I’d like to make a kind of virtual keyboard feature based on this.
Setting Up Leap Motion Controller in Unreal Engine
Here is the step for using leap motion with your project in UE4. As a prerequisite, you need to have a Leap Motion controller and attach it to your headset with Leap Motion VR Developer Mount. If you don’t have a Leap Motion, there is a Leap Motion Universal VR Developer Bundle that comes with Leap Motion and VR Mount Kit.
Necessary SDK/Drivers:
- Leap Motion Driver (ORION BETA) – driver for using Leap Motion with your HMD
- LeapUnreal Plugin – enable leap motion in your UE4 project
Setting up leap motion in Unreal Engine is not too hard. You can set up in 1 minutes once you downloaded all softwares. All informations are found here. I’ll explain the details that I found useful.
Step
1. Download necessary softwares from the list above. You need to install the Leap Motion Driver (ORION BETA) first.
2. Create a new project (if you’ve not created yet). Either blueprint project or C++ project is fine.
3. Save and close your UE4 project
4. Copy the LeapUnreal Plugin into your project’s root folder.
5. Reopen your UE4 project
6. Enable Show Plugin Content
7. Drag and Drop the LeapHandsPawn(LeapMotion Content>Examples) into Viewport.
8. Set Auto Possess Player to Player 0
9. Hit the VR Preview
Now, you are able to see your virtual hands in HMD.
How to Utilize Leap Motion Controller in Unreal Engine
Next, I’ll show how to use LeapUnrealModules in your project. It is not difficult and no coding skills are required. You can just copy and paste from the Leap Unreal Module.
Required Content
- LeapUnrealModules – Leap motion example contents that can be incorporated in your project.
Step
1. Open the LeapModules project that you downloaded from here.
2. Go to the Content folder and right click on Modules and click Migrate
3. Hit Ok
4. Select the Destination (your project’s Content folder)
5. Go to your project and you will see the Module folder under your Content
6. You can pick whatever contents you want to use in your project. I will use PickupAndDrop module here.
7. Drag and Drop the LeapPhysicsLowPolyHand into ViewPort
8. Drag the LeapPhysicsLowPolyHand under LeapHandsPawn
9. Set KinematicPickupItemActor or PhysicsHandlePickupItemActor in Viewport and hit play. You should be able to interact with these objects.
Set and Get an Actor Reference and Using OSC
In order to send the location information of the actor via OSC, I set the PhysicsHandlePickupItemActor as reference and set the blue print like below.
For how to get the actor as reference, here is the instruction.
