highfidelity/hifi — pull request #3145 by samcake (2014-07-09 08:12:25+00:00)
Title: 19734 - Leapmotion support

We finally got it to work!

Here is what i did for this job:
- A generic "Device Tracker" framework which will allow to add any kind of devices to the interface application.
- A generic way to expose a "Motion Tracker" which can map directly its "Joint Tracker" to the corresponding joints of the skeleton ( this feature need to be coded but the foundations are there ).
- Each "Joint Tracker" of a device can be accessed from the javascript engine though the "factory" approach, such an "InputController" can be either used event driven or actively pulled. We did a first implementation exposing the "SPatial" input driver, aka the proxy to a Joint Tracker of a Motion Device.
- And we could get the Leapmotion to work in that new framework.
- The script "leapOfFaith,js" demonstrates that the leapmotion works by displaying particules representing all the active joints detected by the leap at every frame.
And also all the code to inject the joint rotation into the Avatar as soon as we have that missing feature coded (see below).

What remains to be done in follow up jobs:
- provide a way to assign the absolute joint rotation from the bind space to an avatar bone (in javascript and c++ ): Thanks to Andrew's help, we could understand this problem. We are missing a method on the MyAvatar interface ( both in javascript and c++) to assign an absolute joint rotation ( from the root of the model). so far all the js examples are assigning local rotations to the joints through the MyAvatar.setJointData( ) which is different from what we need to do with what we get from the leap motion.
We could demonstrate that when we modified the current code path of "MyAvatar:setJointData()" by ultimately using JointState::setRotationFromBIndFrame() instead of JointState::setRotationInParentFrame() in the method Model::setJointState(... )
We started thinking about how to expose that but it's not trivial and needs to be done as a separate job.

- add in the Avatar::simulate() function a place to directly map the joints of a motion tracker to the skeleton in order to replace the currently hard coded path for the PrioVR and Hydra. Javascript would be to slow to do this efficiently

- create a clean spec for the naming convention for accessing the devices and their category

- provide a way to create plugins libraries that will expose new devices to the interface app without having to modify it's code.

- re-clean everything and extend to the currently supported devices