A Segmentation Algorithm For the Comparison of Human Limb Trajectories

In this article, we present an analytical approach for movement evaluation and comparison. We develop a joint-space-based segmentation and comparison algorithm that calculates pairwise similarity of arm movement trajectories. To validate the algorithm, we collected sixty sample arm movement sequences from a subject imitating video demonstrations. We present the performance of our algorithm on that data set, and discuss the contribution of each of its components. SEGMENTATION ALGORITHM 2 A Segmentation Algorithm for the Comparison of Human Limb Trajectories Comparing and evaluating movement is a known diÆcult problem in psychophysics, as well as in robotics. Our particular area of interest is in motor skill learning by imitation, both in humans and humanoid robots (Byrne & Russon, 1998; Matari c, 2001), where the need for analytical movement comparison is particularly acute. Our earlier work (Matari c & Pomplun, 1997, 1998) in the area of psychophysics of imitation involved studying the attentional behavior of people watching movements for subsequent imitation. In more recent experiments, we aimed to answer more precise questions about the quality of the subjects' imitation under various conditions, by recording the demonstrator's and the subjects' movements for subsequent analysis and comparison. This presented an important evaluation challenge. What is the most e ective means of evaluating arm movements across subjects and tasks? In the present paper, we try to answer this question by developing suitable evaluation metrics that allow us to make quantitative statements about human movement, resulting from imitation or otherwise. We are not aware of any other work aimed directly at analytically evaluating imitated behavior, and the work presented here is at its very beginning. Research into copying written 2D characters, drawings, and trajectories has been done (see Cooke, Kitts, Matari c & Sekuler, 1997; Kitts, Cooke, Matari c & Sekuler, 1997). In our previous work on evaluation of motor behavior, but not imitation behavior (Matari c, Zordan & Mason, 1998), we explored end-e ector jerk as a quantitative measure of motor behavior, but found it to be overly sensitive to speci c parameters of each implementation. Other measures, such as minimum torque change (Uno, Kawato & Suzuki, 1989) and minimum energy (Nelson, 1983), have also been explored. None of these alone provides a suÆcient description of the quality of a movement, and in particular of a motor imitation, where the underlying intentions of the demonstrator, however complex, may be relevant. Depending on the nature of the task, imitation can be as simple as achieving the observed task (so-called \task level" imitation, see Tomasello, Kruger & Rather, 1993), something that is quite easily evaluated, to being as complex as conveying the underlying message of a dance or a gesture. The latter has so far only been studied qualitatively. Studies evaluating task-irrelevant movements (see Goncalves, Di Bernardo, Ursella & Perona, 1995; Goncalves, Di Bernardo & Perona, 1998, 1999) compared human movements to the output of computer vision systems tracking and modeling these movements by using a simple mean square error (MSE) metric. They demonstrated that in many cases even pairs of movements with large MSE were still perceived as identical by human observers. This motivates the use of more sophisticated metrics to quantitatively assess the similarity of limb trajectories. Evaluation of imitation is an open problem, and the evaluation metrics developed in the present paper are meant as a general tool for both human and robot imitation as well as motor control evaluation. SEGMENTATION ALGORITHM 3 Method Our experiment was conducted at the National Institutes of Health Resource for the Study of Neural Models of Behavior, at the University of Rochester. Subjects. The experimental trials reported in this article were performed by one of the authors (M. M.), who is right-handed. Stimuli. The stimuli consisted of 20 video clips, each showing an individual sequence of human arm movements. These movements had been performed by one of the authors (M. M.) and video recorded. The video clips presented her arm moving in front of a black background; each clip was about 3 to 5 seconds long. All sequences started with a straightened arm and a clenched st and typically involved movements of the shoulder, elbow, wrist, and ngers. Apparatus. Stimuli were presented on a 22-inch monitor at a viewing distance of 100 cm. Arm movements were tracked with the FastTrak motion tracking system. For this purpose, four position sensors were fastened to the subject's right arm, one in each of the following four positions (see Figure 1): 1. at the center of the upper arm, 2. at the center of the lower arm, 3. immediately above the wrist, and 4. on the upper phalanx of the middle nger. |||insert Figure 1 here |||Each sensor was about 2 1 1 cm in size. We used elastic bands to hold the sensors and their cables in place. The cables were taken to the subject's right shoulder and then connected to the FastTrak processing computer. The spatial position of each sensor was measured every 34 ms with an accuracy of about 2 mm in each dimension. Procedure. Prior to each recording trial, one of the 20 stimuli was shown to the subject. To initiate the imitation process after each stimulus presentation, the experimenter gave the verbal signal \ready". The subject then straightened out her arm horizontally towards the monitor screen and clenched the st. Next, the subject received the instruction \imitate" and started to imitate the previously viewed sequence of movements as precisely as possible. When nished imitating, the subject said \done" and put the arm back into the resting position. Between the \imitate" and \done" signals, the subject's arm movements were recorded by the FastTrak system. In the trials reported here, M.M. imitated each of the 20 stimuli three times, resulting in a total of 60 reference trajectories being recorded. SEGMENTATION ALGORITHM 4