This guide will provide
an overview of how to use
HandEngine to connect to a
hand and create an
StretchSense Hand Engine: Already installed (see StretchSense Hand Engine Installation Guide (DOC-5009)) with IP address binding configured. To download login to your account page on www.stretchsense.com/my-account or email firstname.lastname@example.org for details.
Hand Engine License Key (viewable on your account page on https://stretchsense.com/my-account/)
Operating System: Windows 10 Pro
1. Setting up the Glove Wired USB/Wireless
- Connect your glove to the PC running Hand Engine using either a USB-MicroUSB wired connection or the StretchSense USB-Bluetooth Dongle.a. NOTE 1: If connecting via the USB-MicroUSB cable, you may need to do a one time configuration of
the COM port settings in Windows Device Manager. See Setting up COM Port for Wired USB Connection (DOC-5002) dongle for further instructions.
b. NOTE 2: Windows will automatically assign a COM port number to the glove and/or USB Bluetooth dongle when they are connected to a USB port. You can find which COM Port has been assigned to a particular glove/dongle by looking at the Ports (COM & LPT) category in Windows Device Manager. When you connect the glove/dongle, the associated COM port will appear in the Ports (COM & LPT) category, and unplugging the glove/dongle will make the associated COM port disappear from the Ports (COM & LPT) category.
- Power on the glove by pushing the button on top of the glove electronics once to enter Bluetooth mode. Once the glove is on (blinking blue) press the button twice to enter USB/wired mode. If you want to return back to Bluetooth mode press the button twice again. The glove LED will blink Blue when in Bluetooth mode, and green when in USB mode.
a. When starting in Bluetooth mode, the glove LED is set to a long blue blink to signify it is searching for an available StretchSense USB-Bluetooth Dongle. If it has previously been paired with a USB-Bluetooth Dongle and it is in range of that dongle it will automatically connect and the glove LED will switch to a short blue blink and the LED on the connected dongle will go solid blue. If this is the first time using the glove with an unpaired dongle (a dongle is unpaired if when plugged into a USB port the dongle LED is solid red), move the glove within <1 ft (<30cm) of the dongle until the red light on the dongle turns first green to indicate pairing has occurred, and then blue to indicate data is streaming from the glove.
2. Getting Started with Hand Engine
- Launch HandEngine using the Windows Start Menu or Desktop shortcuts.
- Once launched, Hand Engine will open in a web browser tab for you to use.
a. NOTE: StretchSense recommends running Hand Engine in a Google Chrome tab.
b. NOTE: To avoid performance degradation, StretchSense recommends you do not have any other tabs open in the web browser window that is running Hand Engine.
c. NOTE: If at any point Hand Engine exhibits persistent unexpected behavior, you can save the state of the Hand Engine UI by clicking File→ Save Session to download a configuration *.session file, then you can reset Hand Engine by clicking the Refresh button on your browser tab which will set the Hand Engine UI back to its default state. You can then click File → Load Session and navigate to your configuration *.session file to reload the previous state of the UI. This does not affect any calibration that you have done previously. If you have not restored the state of the Hand Engine UI.
- To connect the glove to Hand Engine, expand the Glove 1 dropdown by clicking on it.
a. NOTE: you can use the following process for the other Gloves to add more gloves as well.
- Once expanded it will display the glove controls panel for Glove 1.
- Press Scan to populate the COM Port list and select the COM port of your USB dongle/Glove, then toggle the switch next to the COM PORT box from to On to open a connection.
- Inspect the raw data plots in the graphical view at the bottom of StretchSense Device 1 to make sure all 15 (for MoCap Pro Original gloves)/16 (for MoCap Pro SuperSplay gloves) sensors are present and accounted for. Curling your gloved fingers will make the histograms grow taller, and vice versa.
NOTE: MoCap Pro Original gloves do not have a wrist sensor so you should expect this histogram to be empty.
3. Calibrating the
To calibrate a glove first you will need to create an Actor and Profile which is used to save and recall calibration profiles. Press the Create button next to the Actor field and fill in the details when prompted.
NOTE: Calibration profiles relate to a specific hand. StretchSense recommends including the hand (i.e. “L. Hand”/”R. Hand”) in the Profile Description.
NOTE: If you have previously created an Actor and/or Profile on the same PC you are currently using, select them from the Actor and/or Profile drop down boxes.
NOTE: If you have previously created an Actor and/or Profile on another PC, you can transfer this data to you current PC by:
- Export Actor and/or Profiles from the other PC by selecting File → Export Actor or File → Export Calibration to download a JSON file containing the Actor and/or Profile data.- Exporting an Actor exports all calibration profiles associated with that actor
- Exporting a calibration Exports that Actor but with only the current calibration profile
Copy the JSON file to a location your current PC can access.
Import the Actor and/or Profile by selecting File → Import Actor or File → Import Calibration
Actor and Profile
have been created
you can start
poses to the
profile using the Pose
You can select
which poses you
want to use
depending on your
libraries to help
Quick: A few simple poses for good general use.
Performance: A more detailed pose set for more nuanced movement.
Expert: An even more detailed pose set for expert hand data output.
ASL: The complete American Sign Language set of poses.
Game_Starter: A compilation of common hand poses in games.
example we will
use the Quick
Pose set. First, ensure
that Preview in
the Glove controls
panel is set to Off.
Next, expand the Quick menu
in the Pose
Controls and select the
first item on the
ensure Auto Advance Pose is On as
progress to the
next pose on the
list once the
previous pose is
NOTE: If you wish to calibrate multiple gloves to the same pose at the same time then toggle Group Calibration to On. When Group Calibration is On, every glove that is connected to Hand Engine and Hand Engine is receiving data will be trained to the selected pose at the same time. Whether you are doing a pair of gloves on a single actor, or calibrating multiple actors simultaneously, data is captured from every glove at the same time so all of the hands/gloves/actors must be in the correct pose at the same time during capturing.
NOTE: Even if Auto Advance Pose is On, you can still manually select the next pose to be trained from the Pose Library Controls using your mouse, Auto Advance Pose will not interfere and simply jump to the next pose afterwards.
NOTE: If you’ve already calibrated before, make sure the Preview button is turned Off before you try to add more poses from the library to the profile.
get the actor to
match the pose in
IMPORTANT! For best results, have the actor initially copy the reference pose with their hand with the wrist straight. It is very important to make sure all of the joints on the actor’s hand match the reference hand pose from the pose library, if the actor cannot achieve the pose, it is better that they do not train to that pose than to train it incorrectly. Furthermore it is highly beneficial to ensure that if a particular joint is in the same position in more than one pose, that for each of these poses the corresponding joint on the actor’s hand is also in the same position. This provides higher quality data to the hand solver that results in optimal calibration performance. Once the actor is set in the hand pose, ask them to hold the hand pose steady and hold their forearm steady while gently rotating at their wrist (e.g. like the wrist would move when using a skipping rope), and to keep rotating their wrist in this circular motion until the capture is complete. Start the capture process by clicking the Capture button. The capture process takes <2 seconds for each captured pose.
progress to the
next pose (for
example in the
case of the Quick
library the second
pose is the Thumbs
Up pose). Repeat Step
4 above using this
new pose. Once all
poses in the
library have been
captured the view
port will stop
updating, you will
see the poses
saved in the Captured
Poses tree in the Glove
NOTE: You can delete individual poses from the Captured Poses list by right clicking on the pose and clicking Delete.
NOTE: You can delete all of the poses from the Captured Poses list by clicking the RESET button underneath. This empties the calibration profile but does not delete the calibration profile.
can select how to
A continuous movement that interpolates between poses
Good for realistic hand movement
Pose Detection Mode
Snaps between the captured poses selecting the one most suitable
Good for keyframing as the poses captured are the only ones that are output
Pose library best
suits the Blend
press the Train
Mode button to
To view the
pre-viz of your
toggle the Preview button
The hand in the
Viewport will now
show the live
output of the
MoCap Pro glove
hand capture in
NOTE: if you like to add more poses from the pose library to your profile, you need to turn the Preview button Off.
If you intend to stream hand data to your animation software, set Streaming to On. The guides on how to configure your animation software to receive this data stream using StretchSense plugins are provided here: https://kb.stretchsense.com/streaming-to-other-software.
NOTE: Each glove requires a unique TCP port to stream data on. By default TCP port 9000 - 9011 are used for Glove 1 - Glove 12 respectively for Hand Engine Studio, or TCP port 9000-9003 for Glove 1 - Glove 4 respectively for Hand Engine Animator..
Mode uses the same
as Blend Mode. To
Mode simply press the Train
Detection button. In this
mode, you are able
to specify a Debounce value
measured in frames
Debounce value is
the number of
that a pose output
must be stable
before the pose
will be applied to
asset. The Animate
enable you to
specify the type
of animation (from
left to right: Smooth
in and out,
and the number of
steps above for
each glove you are
4. Configuring Timecode in Hand
From the Master Timecode section, select the timecode source you wish to use for your glove data from the System TC dropdown box. The options available are:
Local System Time (default): use the system clock of the PC running Hand Engine as the timecode source. Hand Engine converts the time from the system clock of your PC into a 30fps timecode value.
Glove Clock: use the Glove time from each individual glove as the Hand Engine timecode source when recording data. The glove sends time measured in milliseconds since the glove connected to the PC running Hand Engine. Hand Engine converts the time to a 30fps timecode value.
NOTE: if you have a properly configured UltraSync One module (see DOC-5008 - StretchSense UltraSync One Module Configuration) physically connected to your glove, the glove will always default to output Glove Timecode (see below) instead of time in milliseconds.
Glove Timecode: Use this option if you have an UltraSync One connected to each glove or have used an UltraSync One module to jam sync each glove.
StretchSense Circuit: Connect an UltraSync One module to a spare glove/circuit and use that single spare glove as the timecode source for all glove data. Requires selecting the COM port associated with the spare glove/circuit.
5. Setting the Local Recording
Directory and Timer Based
Changing the directory Hand Engine will save local raw data files to: To change the root directory where files are saved:
First create that directory in Windows File Explorer, then copy the full path name of that directory.
In Hand Engine, click Settings → Preferences, then paste the path name to the Recording Location text box.
NOTE: It is important to ensure a trailing backslash (“\”) character is at the end of the path name.
through the poses
in a pose library
you to click
Capture for each
pose by using the
Timer. Set the Auto
Capture option to On,
and specify Time
capture in seconds to hold
each pose before
NOTE: StretchSense recommends starting with an 8 second delay to begin with and reducing that time as appropriate.
When Capture is clicked, the Capture button changes to Stop (Auto On) and:
A timer pops up on screen counting down from the time specified in Time between capture and the actor must adopt the displayed pose.
When the timer reaches 3 seconds to go, the actor will be asked to start rotating their wrist(s).
At 0 seconds the capture process begins. The capture process takes ~1 second to complete.
Once the capture is complete, if Auto Mode is On, the viewport will be updated with the next pose in the pose library and the timer will restart.
Steps i-iv repeats until the last pose in the pose library is captured
NOTE: The Auto Capture process can be stopped by clicking Stop (Auto On)
Recordings to open a Windows
window at the
Hand Engine will
save local raw
6. Saving and Restoring the
State of Hand Engine
Once you have
each glove and setup Hand
Engine in the
configuration you desire,
you can save the state of
Hand Engine by clicking File
→ Save Session to
download a *.session file
of the state of Hand
Engine. To restore a
previous state of Hand
Engine, click File
Session and navigate to the
previously saved *.session
7. Recording MoCap Pro Glove
Data using Hand Engine
assumes you have already
connected each glove you
are using to Hand Engine,
created a calibration
profile with the desired
training poses for each
glove, have selected Blend
or Pose Detection mode,
and have configured your
Hand Engine timecode
In order to record raw capacitance data, joint angle data and timecode data first define a Label for the recording, then toggle the Record Switch to Rec. To Stop the recording toggle the Record Switch to ‘Stop’.
NOTE: A subfolder is created for each take in C:\Users\<user_id>\Documents\StretchSense Hand Engine with the naming convention <take name>_<date>_<time>
within this subfolder,
two files are created
for each glove
connected to Hand
1x Comma Separated Variable (CSV) text file with the naming convention Raw_GLOVE-<x>_<actor name>_<profile name>_Hand_<left/right>_<take name>_<date>_<time>.csv containing (from left to right):
Raw capacitance data (16 columns)
Raw IMU data (if applicable) (10 columns)
Glove time (1 column)
Hand Engine time (1 column)
Solved joint angles (60 columns)
1x JSON file with the naming convention Cal_GLOVE-<x>_<actor name>_<profile name>_Hand_<left/right>_<take name>_<date>_<time>.json containing a snapshot of the calibration profile used at the time the data was recorded.
8. Playing Back a Raw Data File
Recorded in Hand Engine
Select File mode, then click Browse and navigate to a raw data CSV file previously recorded in Hand Engine.
Select the Actor and Profile you wish to use to interpret the raw data file, and then click Train Blend or Train Pose Detection depending on your needs.
Click Play to start the playback. Click Pause to pause the playback at the current frame and click Play again to resume from that point. Click the Stop button to reset the animation back to the start of the animation file.
NOTE: You can reprocess the raw data file using any Actor/Profile and any mode (Blend or Pose Detection). You do not need to use the same that was used for the original recording.
NOTE: If you wish to play several recorded raw data files simultaneously, repeat steps 1 and 2 in each of Glove 1-12 in Hand Engine as required, then click the Play button in Global File Playback. This option is useful for triggering the streaming of multiple raw data files to your animation software at the same time.