Detection and Cl assification of M otion Bo undaries

We segment the trajectory of a moving object into piecewise smooth motion intervals separated by motion boundaries. Motion boundaries are classified into various types, including starts, stops, pauses, and discontinuous changes of motion due to force impulses. We localize and classify motion boundaries by fitting a mixture of two polynomials near the boundary. Given a classification of motion boundaries, we use naive physical rules to infer a set of changing contact relationships which explain the observed motion. We show segmentation and classification results for several image sequences of a basketball undergoing gravitational and nongravitational motion. Introduction Given the trajectory of a moving object, what physically meaningful aspects of the motion can be recovered? Here we segment the trajectory of a moving object into piecewise smooth motion intervals separated by motion boundaries. We consider the motion of a single passive object, such as a basketball, undergoing gravitational and nongravitational motion. (See Fig. 1.) The ball may roll on a surface, fall, or bounce off the floor or walls. In addition an active object, such as a hand, may act on the object by carrying, lifting, hitting, etc. A sample sequence is shown in the first row of Fig. 2. Here a hand pushes a ball along a table top. The ball rolls on the table, falls, and bounces several times on the ground. Our eventual goal is to characterize events based on qualitative scene dynamics . For example, given the sequence in Fig. 3 we should infer that an “active” hand is moving a “passive” ball by applying a force. Once released, the ball is undergoing passive motion as it rolls and falls off the table. In (Mann, Jepson, & Siskind 1997) a system was presented that infers scene dynamics based on the Newtonian mechanics of a simplified scene model. However, that system was limited to the instantaneous analysis of continuous motion. Sequences were processed on a frame by frame basis, and discontinuous motions (due to contact changes, collisions, or starts and stops of motion) were explicitly removed. To apply dynamics analysis to extended sequences, we require a way to identify the motion boundaries, and to deCopyright c © 2002, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: A composite of the tracking results for a sequence where a subject throws a basketball. termine the allowable motion at such boundaries. Here we present a simplified system that considers the trajectory of a single object (the ball) and extracts motion boundaries corresponding to contact changes, application or removal of forces, starts and stops of motion, etc. In particular, we show that from the trajectory of the ball, and possibly the proximity of the hand, we can infer motion boundaries, as well as changes in hand and surface contact. This information should provide suitable input for event description and recognition (Siskind 2000). This paper makes three contributions. First, we present a characterization of motion boundaries, based on the velocity and acceleration changes around a discontinuity. Second, given an initial segmentation into piecewise quadratic segments, we present an algorithm that classifies the motion boundaries into various categories. Our classification is based on a novel fitting process that fits a mixture of two polynomials within the neighborhood of the discontinuity. Finally, we show how physical knowledge of plausible motion boundaries can be used to infer surface contact, even though this is not directly observed in the input. We show segmentation and fitting results for several image sequences of a basketball undergoing gravitational and nongravitational motion. 764 AAAI-02 From: AAAI-02 Proceedings. Copyright © 2002, AAAI (www.aaai.org). All rights reserved. offtable: 2242 2251 2262 227