yanatan16/RoboNUI — Issue #13: Hand Resolution
Author: yanatan16
Created: 2011-11-22 00:50:41+00:00
State: open

Integrate Hand tracking and fingertip tracking into RoboNUI to control the operation of the manipulator.

This can be done in one of two ways:

Integrate OpenNI (another pending issue) and use built in hand and finger tracker

Or

Use an OpenCV derivative to find it myself

Hand Tracking (see ref section for places to get more info on this) :
#1: Get hand position (also convert raw depth to z coordinate depth)
#2: cut out hand blob using threshold on depth data. (70cm in depth?)
#3: use opencv to get defect points and lines
#5: determine that distance minus filter radius to open/close palm
#6: find thumb ???
#7: use thumb/palm position in wrist rotation angle

We can use OpenCV for comp vision (.net version?): engucv or opencvsharp or sharpcv

Reference:
Tracking of Fingertips and Centers of Palm Using KINECT (I have a copy)
Frati, V., Prattichizzo, D., Using Kinect for hand tracking and rendering in wearable haptics, IEEE World Haptics Conference (WHC 2011), 21-24 June 2011, pp. 317-321.