This prototype used stationary cameras to observe the user and to react to his gestures. Even with high resolution cameras the interaction area is always restricted by the camera's view cone. For this reason pan-tilt units are necessary to track the user without restrictions. A difference image is calculated from two succesive frames by pixelwise subtraction. Motion Detection is necessary to find newly appearing objects, for example, if the user puts his hand into the pocket, the hand disappears and the system must be able to find it, if it appears again.
Tracking means that the vision system observes and follows the users motion.
Tracking reduces errors through motion noise, improves segmentation and speeds up calculation. Tracking in a polar coordinate system is used for the rotating cameras. The initialization of the Kalman Filter must be adapted for human motion and polar coordinates. The Kalman Filter tries to follow the moving objects. If an object moves, additional motion appears in the difference image from the motion detection.