Ximmerse SDK for Unity3D (v1.0)

UNCONTROLLED COPY: The master of this document is stored on an electronic database and is “write protected”; it may be altered only by authorized persons. While copies may be printed, it is not recommended. Viewing of the master electronically ensures access to the current issue. Any hardcopies taken must be regarded as uncontrolled copies.

Update History

• 01/23/2016: Support Windows 10.
• 12/30/2015: Native C++ SDK is released.
• 12/16/2015: Add “Developing with PC (Windows)” chapter.
• 12/01/2015: First release.

• 1 Release Statement
  • 1.1 Foreword
  • 1.2 Test Environment
  • 1.3 Notes
  • 1.4 Technical Support
• 2 SDK Setup
  • 2.1 Import Resource Package
  • 2.2 Setup Ximmerse SDK under Unity VR
  • 2.3 Input Setup for Ximmerse Hardwares
    • 2.3.1 Stereo Camera Component
    • 2.3.2 Ximmerse’s Joystick (X-COBRA)
    • About Config Files
  • 2.4 Programming
    • 2.4.1 Access Axes and Buttons
    • 2.4.2 Access Position and Rotation
• 3 Ximmerse.Core Namespace
  • 3.1 Overview
  • 3.2 Brief Introduction
• 4 Ximmerse.CrossInput Namespace
  • 4.1 Overview
  • 4.2 Selected Classes Introduction
    • 4.2.1 CrossInputManager
    • 4.2.2 UnityInput
    • 4.2.3 HandJoystick
  • 4.3 Extensions
• 5 Other Namespaces
• 6 Samples
  • 6.1 CrossInputManagerGUI
    • 6.1.1 InputTest (X-Hawk)
    • 6.1.2 InputTest (X-Cobra)
    • 6.1.3 InputTest (X-Swift)
  • 6.2 HandController
  • 6.3 HydraToXimmerse
• 7 Developing with Gear VR
  • 7.1 Hardware Requirement
  • 7.2 Steps
• 8 Developing with PC (Windows)
  • 8.1 Hardware Requirement
  • 8.2 Steps
• 9 Misc.

1 Release Statement

1.1 Foreword

This SDK is designed mainly for Unity3D Engine. In future, we will provide support for other developing environments.

1.2 Test Environment

• Operating System (OS): Windows 7/8/10 PC
• Unity3D Engine Version: Unity 5.1.3f1 (32-bit)

1.3 Notes

1. This SDK does not support Linux or Mac OS X, please develop using Windows (PC).
2. You can use other versions of Unity3D. However, versions 4.X are not verified or technically supported.
3. It is recommended to use Versions 5.X due to its support for VR.
4. In terms of compatibility or script update issues for different versions of Unity3D, it is recommended to use preprocessor macros.

1.4 Technical Support

• Email: support@ximmerse.com

2 SDK Setup

2.1 Import Resource Package

1. In an empty Unity project, click Unity menu Assets→ Import Package→ Custom Package...
   
2. In the file selection dialog, select Ximmerse SDK 1.0.unitypackage.
   
3. In Importing package, click Import.
   
4. Done! There will be two folders named Plugins and Ximmerse SDK under folder $(ProjectDir)/Assets.
   

2.2 Setup Ximmerse SDK under Unity VR

Note: Please make sure that your Unity version supports VR mode, i.e. v5.1.0+.

1. Click Unity menu File→ Build Settings...
   
2. In Build Settings, click Player Settings....
   
3. In Inspector view → Settings for PC, Mac & Linux Standalone tab → check the Virtual Reality Supported checkbox.
   
4. Click Unity menu File→ New Scene, to create a scene.
   
5. Select Main Camera game object in the Hierachy view → click Unity menu GameObject→ Create Empty Child, to add a child object.
   
6. Right click the added object in the Hierachy view → Rename, to rename it to be Pivot.
   
7. In Project Browser→ Assets→ Ximmerse SDK→ Prefabs→ Input→ drag the prefabs named_CrossInput to the Hierarchy view to instantiate it.
   
8. In Hierarchy→ _CrossInput→ _XimInput→ XimBlobTracking.
   
9. In Inspector, change ‘Pivot’ key to be the transform of game object Pivot, which is the child game object of Main Camera.
   

2.3 Input Setup for Ximmerse Hardwares

2.3.1 Stereo Camera Component

Item Pose Name under Points refers to the key of the corresponding VirtualPose.

2.3.2 Ximmerse’s Joystick (X-COBRA)

In file $(ProjectDir)/Assets/Ximmerse SDK/Prefabs/Input/X-Cobra Alpha 0/Sensors, there are 4 subclasses ofSensorBase, which refers to IMU, button, trigger and axis respectively.

1. IMU: Item Pose Name refers to the key of the corresponding VirtualPose.
   
2. Button: Enter edit mode by checking the Edit Mode checkbox in Inspector view. The left columns are display names (for easy editing) and right columns refer to the keys of the correspondingVirtualButton.
   
3. Trigger: Axis Name refers to the key of the corresponding VirtualAxis and Button Name refers to the key of the corresponding VirtualButton.
   
4. Axis: Enter edit mode by checking the Edit Mode checkbox in Inspector view. The left columns are display names (for easy editing) and right columns refer to the keys of the correspondingVirtualAxis.
   

About Config Files

There is one config file CrossInput.txt under folder $(ProjectDir)/Assets/Ximmerse SDK/Resources/Configs, through which you can setup detailed parameters of Ximmerse hardwares after the release of Unity build applications.

In this SDK, there are two MonoBehaviour scripts, i.e. XimBlobTracking_Usb.cs and DeviceBase.cs, which use the config file. In both scripts, there are two shared keywords, i.e. configPath and configSec. configPathrefers to the path of the config file and configSec refers to the referred section in the config file. Please refer here to know more about the file formats of config files.

In the following two screenshots of the Inspector view, the red boxes highlight the setups of the config files and blue boxes highlight the fields that can be edited in the config file.

 

You can see the corresponding sections and fields in file CrossInput.txt.

If you don’t want to use config files, just set configPath to be empty and write the parameters of config files to the corresponding fields of MonoBehaviour script.

2.4 Programming

2.4.1 Access Axes and Buttons

1. Import Ximmerse.CrossInput namespace.using Ximmerse.CrossInput;
2. Access the Horizontal axis of left joystick and the Vertical axis of right joystick.float left_h =CrossInputManager.GetAxis("Left_Horizontal"); float right_v =CrossInputManager.GetAxis("Right_Vertical"); These information can also be obtained by HandJoystick as follows, which are equivalent to the above functions.float left_h =HandJoystick.joysticks[0].GetAxis("Horizontal"); float right_v =HandJoystick.joysticks[1].GetAxis("Vertical");

3. Access the Fire1 button of left joystick and the Jump button of right joystick.

   if(CrossInputManager.GetButtonDown("Left_Fire1")) { // Let's fire... } if(CrossInputManager.GetButtonDown("Right_Jump")){ // Let's jump... }

   Similarly, these information can also be obtained by HandJoystick as follows, which are equivalent to the above functions.

   if(HandJoystick.joysticks[0].GetButtonDown("Fire1")){ // Let's fire... } if(HandJoystick.joysticks[1].GetButtonDown("Jump")){ // Let's jump... }

2.4.2 Access Position and Rotation

Taking the sample script HandController.cs under folder $(ProjectDir)/Assets/Ximmerse SDK/Scenes/Example Scripts as an example:

1. Import Ximmerse.CrossInput namespace.using Ximmerse.CrossInput;

2. Obtain the reference of VirtualPose by CrossInputManager.VirtualPoseReference(poseName).

   m_Pose=CrossInputManager.VirtualPoseReference(poseName);

   The properties and fields of VirtualPose are as follows:

   public class VirtualPose:SmartPointer{ public string name{ get; private set; } public bool matchWithInputManager { get; private set; } public bool active=true; public int id; public Vector3 position; public Quaternion rotation; ... }
3. Set position and rotation of Transform to be position and rotation of VirtualPose in functions like MonoBehaviour.FixedUpdate(). ///
   /// ///
   protected virtual void FixedUpdate(){ // m_HandTrans.position=m_Pose.position; m_HandTrans.rotation=m_Pose.rotation; }

3 Ximmerse.Core Namespace

3.1 Overview

This namespace is used for accessing the APIs of Ximmerse’s hardwares. As our beta versions are still in a fast iteration stage, there will be different versions for the supported hardware. Please pay attention to what your DK version is. Though there will be big changes for this namespace, it should be OK to use Ximmerse hardwares with general software as long as the name and address of the equipment are set properly. Note that in most cases, you will not need to access this namespace. In terms of input interfaces, you only need to access Ximmerse.CrossInput. You can also apply Razer’s Hydra in developing asXimmerse.CrossInput is a compatible input solution.

3.2 Brief Introduction

In Unity, Ximmerse’s joystick (X-COBRA) is abstracted as DeviceBase. Information related to the Ximmerse’s joystick (e.g. axes, buttons, triggers and IMU) is abstracted as SensorBase.

Prefabs should set up automatically in the SDK. Developers only need to set the corresponding config file under folder project_folder/Ximmerse SDK/Resources/Configs/ in order to setup Ximmerse’s X-COBRA after the formal release.

DeviceName is the device number. Please refer to the txt file under the crossponding samples.

DeviceAddress is the serial port address under Windows, which is the Bluetooth address under Android.

FmtJoy refers to the character string of the input formatted key. For example, key "Fire1" will be formatted to "Left_ Fire1" in CrossInputManager (under Ximmerse.CrossInput namespace) when joystick’sFmtJoy is "Left_{0}".

USB is used for communication of the stereo camera module, which is recommended to setup in the txtfile under folder project_folder/Assets/StreamingAssets/Configs/.

In general cases, VID and PID should be set to 0 (will be set to the default values while running, i.e.VID=0x1f3b; PID=0x10ff;) and assign Pivot properly, which stands for one Transform of the stereo camera module in Unity (as the next child transform under the head-mounted display coordinate system in VR SDK). Sensitivity is the coordinate scaling value. PoseName in the Points list is the projected pose name.

Remarks:

• Config files are located under folders below:
  • Unity editor: project_folder/Ximmerse SDK/Resources/Configs/
  • Windows release version: release_folder/release_folder_Data/StreamingAssets/Configs/
  • Android release version: Android_SD_card_root_folder/Ximmerse Runtime/Configs/

4 Ximmerse.CrossInput Namespace

4.1 Overview

Compatible solution for multiple input. CrossInputManager is the access entry point, which is modified from the official sample UnityStandardAssets.CrossPlatformInput. In order to run Ximmerse.CrossInputsuccessfully, GameObject with CrossInputManager component (whose properties source list should also be set up properly) should be added to the game scene.

Tip: You can turn off CrossInputManager‘s useCrossInput before running so thatCrossInputManager will use the embedded input APIs of Unity.

On the other hand, you can customize other input solutions, such as Razer’s Hydra, Xbox 360 controller or Ximmerse’s joystick (X-COBRA). Instead of writing different access interfaces for different input solutions, you only need to access the static functions of CrossInputManager.

For example, assuming you want to acquire the status of the joystick and button (pressed or not), you will need to call Input.GetAxis(axisName) and Input.GetButton(buttonName) in native Unity programming. WithXimmerse.CrossInput input system, they can be easily replaced by CrossInputManager.GetAxis(axisName) andCrossInputManager.GetButton(buttonName).

Current input solutions can be divided into two categories: traditional controllers (keyboard & mouse and joystick) and motion controllers (with position and orientation in space). With Ximmerse.CrossInput, motion sensing input is abstracted as VirtualPose class (with position and orientation in space). You can obtain the reference of the VirtualPose by simply calling CrossInputManager.VirtualPoseReference(name).

Tip: In the UnityStandardAssets.CrossPlatformInput sample, joystick and button are abstracted asVirtualAxis and VirtualButton. Please refer to the Extensions section on how to extendXimmerse.CrossInput.

In this SDK, the full preset hierarchy is as follows. To disable one particular input component, simply setGameObject‘s active to false on the respective script.

 

4.2 Selected Classes Introduction

4.2.1 CrossInputManager

Input manager, similar to UnityStandardAssets.CrossPlatformInput.
The related input APIs of CrossPlatformInputManager are as follows:

Remarks:

1. A mouse’s axis information can be obtained by CrossInputManager.mousePosition. Due to poor user experience, it is not recommended to use traditional mouse as VR input.
2. Alike UnityEngine.Input, CrossInputManager‘s interface can only be accessed inMonoBehaviour.Update().
3. The priority of CrossInputManager script should be set higher than the normal ones in order to make sure input can be written to CrossInputManager‘s cache while being called. This can be set in Edit→Project Settings→Script Execution Order:
   

4.2.2 UnityInput

You can write the embedded input of Unity to CrossInputManager‘s input cache. The embedded input of Unity can be further obtained through CrossInputManager.

4.2.3 HandJoystick

Input interface to distinguish left and right hands. It’s not easy to distinguish left and right hands as strings are used as keys in CrossInputManager. With the handy accessor HandJoystick, developers can use formatted strings and the string arrays of input keys to instantiate an instance. Input can be accessed byHandJoystick.GetAxis(axisName) and HandJoystick.GetButton(buttonName). Please refer to the following example:

4.3 Extensions

As mentioned above, different input solutions are abstracted as IInputSource interfaces in SDK. Thus, developers can customize multiple input solutions of their own.
The interface function of IInputSource is as follows:

Remarks:

1. Input cache refers to the list of VirtualAxis, VirtualButton and VirtualPose of CrossInputManager.
2. Input frame refers to the time stamp when the program reads the input (MonoBehaviour.Update()).

In order to run in Ximmerse.CrossInput, it only needs to implement the MonoBehaviour of IInputSource‘s interface and add it to the property source list of CrossInputManager.

Similarly, Razer’s Hydra can also be used due to its compatibility with Ximmerse.CrossInput by adding its SDK SixenseUnityPlugin to the project and compatible scipt Sixense Input.cs to CrossInputManager.

5 Other Namespaces

Some scripts belong to the Utility class. Demos are available for them where you can edit to fit your needs. Some namespaces belong to low level encapsulation of the SDK which in general cases, will not be used in project development.
There are some other namespaces in this SDK, including:

• Ximmerse.Animation: for easy implementation of animation solutions, like HandAnimator.
• Ximmerse.IO: for IO Stream under Windows and Android and encapsulation of IO tools class.
• Ximmerse.UI: for extension of UGUI and the compatible solution of traditional UI and VR UI.

6 Samples

Note: all samples can be found in folder $(ProjectDir)/Assets/Ximmerse SDK/Scenes/.

6.1 CrossInputManagerGUI

This sample can be found in folder $(ProjectDir)/Assets/Ximmerse SDK/Scenes/InputTest, which further contains three samples, i.e. InputTest (X-Hawk), InputTest (X-Cobra) and InputTest (X-Swift).

6.1.1 InputTest (X-Hawk)

This sample demonstrates how to obtain the detailed X-Cobra’s information (including position, rotation, axis, trigger and buttons, etc.) through X-Hawk.

6.1.2 InputTest (X-Cobra)

This sample demonstrates how to obtain the detailed X-Cobra’s information (including rotation, axis, trigger and buttons, etc.) through Bluetooth USB dongle plugged in PC.

6.1.3 InputTest (X-Swift)

This sample demonstrates how to obtain the detailed X-Swift’s information (i.e. rotation) through Bluetooth USB dongle plugged in PC.

6.2 HandController

This sample can be found in folder $(ProjectDir)/Assets/Ximmerse SDK/Scenes/InputTest/Examples, which demonstrates how to obtain the VirtualPose and how to control hand stretches using the trigger value.

6.3 HydraToXimmerse

This sample can be found in folder $(ProjectDir)/Assets/Ximmerse SDK/Scenes/InputTest/Other Examples, which demonstrates how to quickly transfer a Razer’s Hydra project to Ximmerse.CrossInput. Some modifications exist for the interfaces to be accessed.

If SixenseHands of SixenseInput SDK is used in this sample, the hierarchy will be

If HydraToXimmerse of this SDK is used in this sample, the hierarchy will be

1. First remove the original SixenseInput and add the preset object of CrossInputManager then, check the As Hand Joystick checkbox of the compatible CrossInput‘s SixenseInput, HandJoystick thus will create two joystick accessors automatically.
   
2. Remove SixenseHandsController script and add the parent transform with PoseMover for Hand-Rightand Hand-Left to achieve no rotation shift of the Transform controlled by PoseMover, i.e. no rotation for the Transform controlled by PoseMover when the corresponding VirtualPose has no rotation. Meanwhile, set localPosition of Hand-Right and Hand-Left to Vector3.zero.
   
   
    
   
3. Move the left and right pivots of SixenseInput that is compatible to CrossInput to where PoseMoveris.
   
4. Modify SixenseHand.cs to import Ximmerse.CrossInput namespace.//
5. Replace SixenseInput.Controller in the script by HandJoystick.//
6. Obtain m_controller from HandJoystick.joysticks, reference 0 for left hand and 1 for right hand.//
7. When accessing input interface, replace the original SixenseInput function with the HandJoystickfunction of Unity style.//
8. Map trigger value to axes and buttons, i.e. HandJoystick.GetAxis(triggerName) andHandJoystick.GetButton(triggerName).//
9. Ensure that triggerName is not a constant. It should be consistent with the crossponding value ofXimmerse.CrossInput.CrossInputManager. It’s more convenient to obtain the information of given joystick from multiple ones by replacing the full key of CrossInputManager with short keys.

7 Developing with Gear VR

7.1 Hardware Requirement

1. Ximmerse X-Hawk
2. Ximmerse X-Hawk USB Connector
3. Gear VR2
4. Samsung mobile phone: Galaxy S6
   
   
   

7.2 Steps

1. Enable developer mode in Galaxy S6. This can be done in Settings→ Device→ Applications→Application manager→ Gear AR Service→ Storage→ MANAGE STORAGE→ Developer→ Developer mode.

   Tip: If you do not see the Developer mode option in Gear AR Service, tap VR Server Versionat least 6 times. By re-entering the dialog, you will be able to see it.

   
2. Connect the X-Hawk’s USB Connector port C to a power bank (make sure power bank is on).
3. Connect the X-Hawk’s USB Connector port A to the X-Hawk.

4. Connect the X-Hawk’s USB Connector port B to the Galaxy S6. Wait until Choose an app for the USB device dialog pops up. Note that all the available apps that use Ximmerse’s SDK will be displayed here.

   

   Tip: If the Choose an app for the USB device dialog does not pop up after a while, you will need to clear the cache of the previous apps that use Ximmerse’s SDK. This can be done inSettings→ Device→ Applications→ Default applications→ Clear defaults→ CLEAR.

   

5. Choose any of the available apps to continue. Take Ximmerse SDK 1.00 for example, select it and click the ALWAYS option to continue. You will be able to see the stereo camera demo running on the Galaxy S6.

   
   
   
   
6. Remove port B from the Galaxy S6 and connect it to the Gear VR.
7. Insert Galaxy S6 into Gear VR. You will be able to see the stereo camera demo running on the Galaxy S6.
8. You are now ready to enjoy VR on the Gear VR.

Tip: If you want to switch to other apps, for example from Ximmerse SDK 1.00 to UsbListener, you will also need to clear the cache of previous apps that used Ximmerse’s SDK as mentioned previously.

8 Developing with PC (Windows)

8.1 Hardware Requirement

1. Ximmerse X-Hawk
   
   
   
2. Ximmerse X-Cobra
   
   
   
3. Power Cable
   
   
   

8.2 Steps

1. Connect Micro port of Power Cable to X-Hawk.
2. Connect USB port of Power Cable to PC.
3. Replace X-Hawk’s mount by Oculus DK2’s (Windows version) and install it to Oculus DK2.

   Tip:

   • By now, X-Hawk should be connected with PC (Windows) successfully. During the process, no extra driver is needed, which will be automatically installed by Windows after plugging it to PC.
   • This can be verified via right click Computer→ Manage→ Device Manager→ Human Interface Devices, extra HID-compliant device and USB input device will be added.
     
     
     It can be also verified via Unity3D: Ximmerse SDK→ right click Xim Blob Tracking_Usbcomponent → Print UsbDevices→ X-Hawk device name will appear in the Console log.
     
      
4. Connect X-Cobra to X-Hawk. This can be done by:
   1. Turn on X-Cobra by press its power button.
   2. Put X-Cobra close to X-Hawk. It will be successfully connected once it vibrates. It’s recommended to first connect the left hand X-Cobra, then the right one.

      Tip: button combinations of X-Cobra:

      • Enter calibration mode: hold button [1], [2] and [3] simultaneously for 2 seconds.
      • Switch left/right joystick:
        1. Hold power button for 3 seconds to turn off.
        2. Hold button [4] and press power button to turn on.

5. Unreal Engine 4 Development
   To develop with X-Hawk in Unreal Engine 4, you only need to download Hydra Plugin for Unreal Engine 4 from here and replace the sixense.dll by our modified sixense.dll.

   All the Sixense SDK interfaces are the same as Hydra joystick except the following aspects:

   1. The 3D position of the controller is respected to the coordinate system of HMD (Helmet Mounted Display), not the world system.
   2. The pose and rotation of the controller is respected to the world coordinate system, not the base station.

   3. The number of buttons of X-Cobra is less than Hydra joystick. Default mappings are:

      X-Cobra	Hydra Joystick
      Button [1]	Button [ONE]
      Button [2]	Button [START]
      Button [3]	Button [TWO]
      Button [4]	Button [THREE]

      Tip: You can re-map buttons by via sixenseSetButtonRemap() function of our modifiedsixense.dll, whose description is as follows:

      /* buttons is a 7-int array. It is the button references of X-Cobra that correspond to Hydra's joystick buttons. If the value of given button is -1, it means not using X-Cobra button mapping for this button */ int sixenseSetButtonRemap(int* buttons);

      The purpose of using our modified sixense.dll is to access X-Hawk via Sixense SDK interfaces, not to make it fully compatible with Hydra joystick. Please follow the above differences to modify Hydra’s SDK.

9 Misc.

All required files can be downloaded from our SDK github page, including

1. SDK Unity package: Unity → Ximmerse SDK 1.0.unitypackage.
2. Native C++ SDK: Native C++.
3. Tools:
   • Tracking HID demo: Tools → Tracking_hid.zip.
   • CrossInputManager demo: Tools → CrossInputManager.zip.
   • X-Console tool: Tools → X-Console.zip.
   • IMU calibration tool: Tools → IMU_cal_tool_V1.zip.
4. Driver of Bluetooth USB Dongle: Driver → CSR8510 Bluetooth USB Dongle.zip.
5. Binary files: Binary → x86 / x64 → sixense.dll.