¶ General Information
Currently the package VR IK Avatar Controller, created by students at the Delft University of Technology, is avalaible to create a full-body tracked avatar in Unity with inverse kinematics for the arms and legs. This makes use of the Unity engine with the Steam VR plugin. The package, as for now, works with 3 body trackers (HTC Vive tracker 2.0 or 3.0) next to the HTC Vive Headset and two HTC Vive Controllers.
This package can either be integrated into an existing project or be used to create a new project.
¶ Getting Started
Using this package will be divided into four parts:
- Getting started with the package
- Setting up the avatar
- Calibration
¶ Getting started with the package
The package can be downloaded from the GitLab.
¶ Steam VR plugin
First of all, the Steam VR plugin should be downloaded and installed as described below:
- Download Unity if needed (minimum version 5.6.7).
- Create a new project or open an existing project.
- Download the Steam VR plugin from the Unity Asset store if needed.
- In Unity open the Package Manager from the Window tab. Select the Steam VR plugin from “My Assets” and download and import the package.
¶ VR IK Avatar Controller
- Open the Package Manager from the Window tab. Click the plus sign in the top left corner and “Add package from disk…”. Browse for your package and select the “package.json” file in the VR IK Avatar Controller folder.
- Now in the Project window the package should be visible in the Packages folder. There is a Sample Scene to see what a working setup looks like. In the sample scene click “Import TMP Essentials” if asked.
¶ Setting up the avatar
For this project any avatar can be used, which has a humanoid skeleton. These can be downloaded from the Unity Asset Store or imported in any other desired way, like creating your own avatar.
-
Open your new or existing scene. When working with an empty/new project, it is advisable to first create a floor surface by adding a plane 3D GameObject. Make sure to delete the “Main Camera”.
-
From the SteamVR > Prefabs folder, which is in the Assets folder in the Project window, place the [CameraRig] GameObject in the scene on a floor surface.
-
Create two empty GameObject (“Create Empty”) and child them to both Controllers (found in the hierarchy under [CameraRig]) named “Left Hand Target” and “Right Hand Target”. Optionally add a Mesh Filter and Mesh Renderer by clicking “Add Component” in the Inspector window to make them visible. For the Mesh Filter choose “Sphere” as Mesh. If you do add Meshes, scale them down to 0.1 in each axis to make them smaller.
-
Add an Empty GameObject to the Camera in the same way and name it “Head Target”.
-
Create an empty GameObject named “Targets” as child to the [CameraRig] and add three empty GameObjects named “Left Foot”, “Right Foot” and “Hip” as child to “Target”. To each of these three Add Component with the name “Steam VR_Tracked Object”.
- Child each of the three GameObjects from last step with a GameObject named “Left Foot Target”, “Right Foot Target” and “Hip Target”, respectively. Optionally add Mesh Filters and Renderers with a Sphere mesh to make them visible. If you do, scale them down to 0.1 in each axis to make them smaller.
Make sure all “Targets” have the Transform position 0 in all axis.
-
Import your desired avatar into the scene.
-
Attach the script “IKController” to the avatar. Attach each target made in previous steps to the corresponding body part.
-
On the tracked body parts Left Foot, Right Foot and Hip in the Steam_VR Tracked Object script, it is important to select the correct device under ‘Index’. When using 2 lighthouses these will most likely be 5, 6 and 7 and with 4 lighthouses 7, 8 and 9, etc.
-
Click the avatar in the Hierarchy. In the Inspector there should be a Component named “Animator”. Add the “VRIK Animation Controller” from the package as controller to the Animator. Alternately, a new Animation Controller could be created. Right-click in the Project window and Create > Animation Controller. Open de Animation Controller and open the settings of the “Base Layer”. Here make sure the “IK Pass” is ticked. The avatar is now usable in VR.
-
Make an empty GameObject named “Calibration” and attach the scripts “Calibration” and “Scaling” to it. In the Calibration script attach the tracked objects and targets to the corresponding Transform. For the Scaling script do the same.
Note that the “Right Foot” and “Left Foot” in the “Scaling” script should contain the Transforms of the ‘Toe’ of the avatar’s skeleton.
- Attach the “CalibrationTrigger” script to each of the Controllers ([CameraRig] > Controller). Add the Calibration GameObject once to the CalibrationTrigger script within one controller and twice to the other. On one controller select the CalibrateHip function and on the othercontroller add both the CalibrateLength (which is in the Scaling script) and Calibrate functions.
- The avatar should now be fully functional. It can be calibrated using the triggers.
¶ Calibration
-
Calibration of the Hip: The calibration of the hip always works, regardless of your position and orientation. It uses the initial rotation of the hip when starting the scene. This means calibrating the hip can be done whenever. Stand up straight and use the trigger.
-
Calibration of the feet: Calibration of the feet uses world coordinates to find the axis over which to rotate and uses the camera direction to aim the feet forwards. For now this only works when facing in the positive Z-direction. Otherwise the feet will rotate over wrong axes.
It is important to stand up straight with both arms hanging next to the body when doing the scaling. Otherwise, the arms will scale in the wrong direction.
-
Calibrating the length/scaling: Scaling uses the height of the avatar, which must be defined in the inspector, and the height of the user, which it finds by using the Y-coordinate of the camera. Note that due to room setup the length of the user is never found to be exactly correct. This can be corrected for by changing the offset in the inspector. To find this, first try to scale the user with the trigger, read the measured height in the debug log and then change the offset in the inspector.
-
The script will also find the height of the hip tracker. Since the entire body follows the hip tracker, the avatar will float when it is worn too high and the knees will bend when it is worn too low. The hip tracker should manualy be adjusted to achieve the correct height. (This requires further work and should eventually be done automatically with code)
-
The feet are also scaled by an doing estimation. (This requires further work)
-
¶ How the package works
¶ Inverse kinematics
This package makes use of the internal animation system of Unity. This system has the ability to use inverse kinematics. A file named “IKController” can be found inside the package which does all the controlling of the avatar. It does the following things:
- A weight is set to certain body parts, which determines how much that body part wants to move after its target or wants to stay in its original position.
- Determine the direction in which the head faces. For this it uses the rotation of the headset and creates a “target” which the head will always try to look towards.
- Set the IK position and rotation of the elbow joint for the hands and the knee joint for the feet. These are all seperate, so they can all be adjusted as required.
- Set the position of the avatar based on the hip tracker position and rotation.
The weights used for the avatar are:
- Overall weight
- Body weight
- Head weight
- Eyes weight
- Clamp weight
- Position and rotation weight for right foot
- Position and rotation weight for left foot
- Position and rotation weight for right hand
- Position and rotation weight for left hand
¶ Calibration
By making use of the “Targets”, adjustments can be made to the position and rotation of body parts. Especially during calibration this is useful and makes it possible to do calibration automatically. By finding the rotation of, for example, the foot and hip trackers, and knowing what their rotation should be to get the avatar in the right position, the rotation can be calculated which to apply to the targets to get the avatar to the same position as the user.
The same goes for the position of the foot trackers. These can now be used to get the feet in the correct position. This is currently done by making an estimate of how far the tracker is in front of the ankle joint.
¶ What the package does not do
Currently the package has a few shortcomings, which should be improved in the future.
- Since the head only faces towards a target, it can only rotate over two axes. Tilting the head sideways cannot be replicated by the current Controller. This could be fixed with an Avatar Mask.
- The hints, which is the direction in which joints want to move, are currently not tweaked to get more realistic movement of the avatar. A sample code to do this can be found in the “IKController” script.
- The height of the user is automatically scaled. The height of the hip tracker, however, does not automatically go to the correct height. This can easily be added in the “Scaling” script by looking at the current height of the hip tracker and the ideal height of the hip tracker. The feet are also scaled, but this is currently done by an estimation. This then only scales the length of the feet and not the width, which could also be added.