Setting Up the Virtual Hand Meshes

Setting up Virtual Hand Meshes involves two key steps:

1. Importing the virtual hand meshes into your project.
2. Configuring the virtual hand settings.

In this section we focus on the first part. For detailed information on step two, please visit the Virtual Hand configuration section.

Compatible Virtual Hand Meshes

The SenseGlove Unreal Engine Plugin is compatible with any virtual hand mesh that adheres to the Epic rig and bone structure. Additionally, the virtual hand meshes must be exported with specific settings to meet all requirements. If you're planning to model and rig your own virtual hand meshes, the Epic FBX Skeletal Mesh Pipeline is a useful starting point.

However, if you're looking to get up and running with the SenseGlove Unreal Engine Plugin quickly, the process is much simpler. Unreal Engine has included two sets of compatible virtual hand models with the Unreal Engine VR Template since version 5.1. This guide will walk you through how to export these virtual hand models from the VR Template and import them into your VR simulation.

Exporting the Virtual Hand Meshes from the VRTemplate

1. Start by creating a new Unreal Engine project using the VR Template. In the Unreal Project Browser, select GAMES > Virtual Reality.

2. Once the Unreal Editor opens with your new project, navigate to the Content Browser. Go to All > Content > Characters > MannequinsXR > Meshes. Here, you'll find two sets of virtual hand meshes: SDKM_MannyXR_left and SDKM_MannyXR_right (male hands), and SDKM_QuinnXR_left and SDKM_QuinnXR_right (female hands).

3. Choose the pair of hand meshes you want to export. Right-click on them, then select Asset Actions followed by Bulk Export... from the context menu.

4. In the file dialog that appears, choose a folder to save the exported hands, and click the Select Folder button to export the meshes in FBX format.

5. The Unreal Editor will then display the FBX Export Options dialog. Leave the default settings unchanged and click Export All to proceed.

6. After exporting, you can find the FBX files for both hands in the directory you selected: /path/you/chose/for/bulk/export/Game/Characters/MannequinsXR/Meshes/.

Importing the Virtual Hand Meshes into Your Own Project

1. Start by creating a new folder inside your project's Content Browser. Navigate to that folder, then press the Import button next to the + Add button at the top of the Content Browser.

2. In the Import dialog that appears, navigate to the folder containing the virtual hand meshes. Select both FBX files and click the Open button.

3. The Unreal Editor will display the FBX Import Options dialog. Leave the default settings unchanged and click Import All to proceed.

4. After the import process is done, a dialog will display the import logs. Any errors or warnings encountered during the import process will be shown here.

5. The imported virtual hand meshes should now appear in the folder you selected in the Content Browser. Unreal Engine will create a Skeletal Mesh, a Skeleton, and a Physics Asset for each imported mesh, along with a default Material asset shared between both virtual hand meshes.

6. You can choose to keep or modify the default material. However, since the SenseGlove Unreal Engine Plugin provides a default material, we choose to delete the default material created by Unreal Engine during the import process. We'll assign the SenseGlove default material to the imported virtual hand meshes in the next steps. Right-click on the default material and select Delete.

7. In the Delete Assets dialog, click Force Delete to confirm the deletion of the default material.

8. Open the Skeletal Mesh asset for the left hand and assign the M_SenseGlove_VirtualHand material from the Asset Details panel.

9. Repeat the process for the Skeletal Mesh asset of the right hand, and assign the M_SenseGlove_VirtualHand material in the Asset Details panel.

10. Return to the Content Browser by closing all asset windows and click the Save All button to save all imported virtual hand mesh assets to disk.

11. In the Save Content dialog, choose Save Selected to confirm the saving all action.

Setting up the Rigid Bodies

1. Open the Physics Asset for the left virtual hand mesh by double-clicking it in the Content Browser. This will open the PhAT (Physics Asset Tool) editor, where the virtual hand mesh for the left hand will appear with a default physics body, usually shaped as a capsule.

2. In the Tools panel, under the Body Creation section, locate the Primitive Type dropdown and select Box instead of the default Capsule shape. Then, click the Generate All Bodies button at the bottom of the Tools panel to create a new physics body.

3. After generating the new body, some adjustments are required for optimal interactions inside your VR simulations. Press the r key on your keyboard to enter scaling mode and use the arrows to resize the physics body. To reposition the body, press the w key to switch to translation mode. For adjusting the rotation, press the e key. Toggle between these modes as needed to fine-tune the physics body to your requirements.

4. You can always revisit and adjust the rigid body later after testing its impact in your VR simulations. For now, save the asset and close the PhAT editor.

5. Repeat the same procedure for the right virtual hand mesh.

Setting up the SenseGlove Grab and Touch Sockets

To ensure the Grab/Release and Touch systems function correctly, multiple sockets must be set up on each virtual hand mesh with precise locations and rotations. Before version v2.1.0 of the SenseGlove Unreal Engine Plugin, this was a manual and time-consuming process. However, with the v2.1.0 release, the plugin now includes the SenseGlove Sockets Editor, a built-in tool specifically designed for this task.

Accessing the SenseGlove Sockets Editor

The SenseGlove Sockets Editor can be utilized in three ways:

1. By right-clicking on any Skeleton or Skeletal Mesh asset inside the Unreal Content Browser.

2. From the Asset menu in the Skeleton Editor or Skeletal Mesh Editor for any open Skeleton or Skeletal Mesh asset.

3. From the Skeleton Editor or Skeletal Mesh Editor toolbar for any open Skeleton or Skeletal Mesh asset.

The SenseGlove Sockets Editor currently offers two actions:

1. Add SenseGlove Sockets: which adds and sets up the SenseGlove grab and touch sockets to any virtual hand mesh that adheres to the Epic rig and bone structure.
2. Clear Existing Sockets: which destructively clears all existing sockets; SenseGlove or otherwise, from any mesh.

Adding the SenseGlove Sockets

When you invoke the Add SenseGlove Sockets action, the Sockets Editor will prompt you for confirmation:

If it succeeds at adding the standard SenseGlove sockets, you will receive a confirmation message:

After closing the dialog, the editors for the affected Skeleton and Skeletal Mesh assets will open, displaying the newly added sockets:

To ensure the changes persist, save the assets to disk.

In any case, the SenseGlove Sockets Editor reports all failures in the Unreal Editor logs. To view and investigate the logs, simply head to the Window menu and click on Output Log:

For example, in the following screenshots the following errors are stated: Socket 'GrabAttachPoint' already exists on '/Game/SGHandbook/SKM_MannyXR_left.SKM_MannyXR_left'; refuse to add a duplicate!.

LogGeneric: Error: [ERROR C:\Users\mamadou\Desktop\dev\SGHandbook\Plugins\SenseGlove\Source\SenseGloveEditor\Private\SGEditor\SGAssetUtils.cpp FSGAssetUtils::FImpl::AddSocket 394] Socket 'GrabAttachPoint' already exists on '/Game/SGHandbook/SKM_MannyXR_left.SKM_MannyXR_left'; refuse to add a duplicate!
LogGeneric: Error: [ERROR C:\Users\mamadou\Desktop\dev\SGHandbook\Plugins\SenseGlove\Source\SenseGloveEditor\Private\SGEditor\SGAssetUtils.cpp FSGAssetUtils::FImpl::AddGrabAttachPointSocket 442] Failed to add the socket 'GrabAttachPoint' to '/Game/SGHandbook/SKM_MannyXR_left.SKM_MannyXR_left'!
LogGeneric: Error: [ERROR C:\Users\mamadou\Desktop\dev\SGHandbook\Plugins\SenseGlove\Source\SenseGloveEditor\Private\SGEditor\SGAssetUtils.cpp FSGAssetUtils::FImpl::AddSenseGloveSockets 587] Failed to add the grab attach point socket to asset '/Game/SGHandbook/SKM_MannyXR_left.SKM_MannyXR_left'!
LogGeneric: Error: [ERROR C:\Users\mamadou\Desktop\dev\SGHandbook\Plugins\SenseGlove\Source\SenseGloveEditor\Private\SGEditor\SGAssetUtils.cpp FSGAssetUtils::FImpl::AddSenseGloveSockets 741] Failed to add the SenseGlove sockets to the asset '/Game/SGHandbook/SKM_MannyXR_left.SKM_MannyXR_left'!

Clearing All Existing Sockets

When you invoke the Clear Existing Sockets action, the Sockets Editor will ask for your confirmation:

If successful, you will receive a message indicating all the existing sockets have been cleared:

After closing the dialog, the editors for the affected Skeleton and Skeletal Mesh assets will open, displaying the affected assets with all sockets cleared:

Configuring the SGPawn and Plugin Virtual Hand Mesh Settings

The final step in setting up the virtual hand meshes is to configure the SGPawn and Plugin Virtual Hand Mesh Settings to ensure they utilize the newly created virtual hand meshes.

Please visit Setting Up SGPawn, The Virtual Hand Mesh Settings, and how to setup the Wrist Tracking Hardware sections for more information.

SGPawn Configuration

In the SGPawn Blueprint class, make sure to assign the appropriate Skeletal Mesh Asset to the following components:

• HandLeft
• HandRight
• RealHandLeft
• RealHandRight

This ensures that the correct hand meshes are used for both virtual and real hands.

Plugin Virtual Hand Mesh Settings

Next, navigate to Project Settings > Plugins > SenseGlove > Virtual Hand Settings > Mesh Settings and specify the correct left and right-hand meshes for:

• Left Hand Reference Mesh
• Right Hand Reference Mesh

This configuration guarantees that the tracking system correctly interprets the bone transforms of the virtual hand meshes when generating FXRMotionControllerData. Additionally, it allows the animation system to accurately use these bone transforms when processing FXRMotionControllerData and animating the virtual hand meshes.