At that age feedback control is not yet used for grasping, but only a feedforward like the one currently implemented on our robot, is applied. Fixation before grasping is also a standard behavior, although adults can sometimes do without. The Human Robot Interaction is conveyed with gestures. While the cameras are looking down over the table with objects, the hand of the human user is constantly tracked. The tracking is based on the detection of motion and skin color.
Tracking ends when the hand stops moving.
Then the user is supposed to point at one of the objects. The final position of the hand is fixated, and gesture recognition is done only on the central parts of the images in order to save computing time. With different postures the user tells the robot how to grasp the particular object, e.g., from above or from the side. The hand posture classification is done by elastic graph matching and is robust with respect to different users and complex backgrounds. The camera head performs a corresponding motion, depending on the size of the seen hand, which is interpreted as information about the approximate distance from hand and object, respectively.