Abstract-The problem considered here involves the use of a sequence of noisy monocular images of a three-dimensional (3-D) moving object to estimate both its structure and kinematics. The object is assumed to be rigid, and its motion is assumed to be "smooth." A set of object match points is assumed to be available, consisting of fixed features on the object, the image plane coordinates of which have been extracted from successive images in the sequence. Structure is defined as the 3-D positions of these object feature points, relative to each other. Rotational motion occurs about the origin of an object-centered coordinate system, while translational motion is that of the origin of this coordinate system. In previous work [SI- [8] we have developed a model based approach for motion/structure estimation using a long sequence of monocular images. This approach provides a great deal of flexibility, by allowing the use of arbitrarily many image frames and feature points, and each model can easily be modified or extended for different problems. Our earlier work involved assumptions about object structure and/or motion, were primarily tested on simulated imagery, and did not address the issue of uniqueness of the model parameters. In this paper, which is a continuation of the research started in [q, results of an experiment with real imagery are presented, involving estimation of 28 unknown translational, rotational, and structural parameters, based on 12 images with 7 feature points. Uniqueness results are summarized for the case of purely translational motion. A test based on a singular value decomposition is described that determines whether or not noise-free data from an image sequence uniquely determines the elements of any given parameter vector, and empirical support of this test is given. Index Terms-Image sequence analysis, motion analysis, 3-D motion estimation, uniqueness. T a time sequence of 2-D projections (images) has been the focus of a significant amount of research during the past decade. This problem occurs in a wide variety of situations, which range from estimating rigid object structure and motion (e.g., [lo], [22] , [32] ), to self-motion estimation and navigation (e.g., [9] , [27]), to tracking certain 3-D parameters of "point" objects (e.g., [28]), and to many others. In all cases, there is either an implicit or explicit use of various models for structure, motion, and imaging, the parameters of which are to be estimated, whether they are translational or rotational increments, rates, feature positions, etc. Most of the advantages of model-based methods discussed in [9] (for recursive estimation of self-motion) apply to model-based batch and hybrid batch/recursive estimation procedures. There are several aspects of this modeling that are of particular relevance in distinguishing among the various research endeavors: the number and type of the unknown parameters; the amount and type of data to be used; and the suitability of the model for its intended purpose. The number of unknown parameters (the number of "degrees of freedom") is directly related to estimation performance and data requirements. If many parameters are unknown, as is the case with the research presented in this paper, the estimation process can be delicate and difficult, and batch methods may be required to extract as much information as possible from the data. If fewer parameters are involved, such as in [13] and (271, the estimation process is more robust, as evidenced by the successful use of real-or nearreal time recursive techniques in these three cases. Similarly, the additional information contained in stereo imagery allows the use of simpler models and solutions as in [17] ; a robust linear solution for monocular vision has proved more elusive. For example, an approximate linear method for monocular vision discussed in [34] is used to generate initial guesses for a nonlinear least squares objective function similar to ours, with the intent being to reduce the number of unknown parameters in the iterative optimization. However, this method is tested only on very large objects for simulated imagery and fixed scenes (self-motion) for real imagery (both having very low noise levels as quantified by the measure discussed below), and appears to be limited to motion with small rotation. Since the problem is nonlinear (central projection imaging and rotational motion), the particular values of the unknown parameters are also important. Pure translational motion and object (or scene) structure are more easily estimated than rotational motion, and the presence of significant rotational motion can lead to estimation problems, especially with regard to convergence, due to qualitative changes in the objective function (creation of multiple local minima). Also, self-motion estimation is more robust than object motion estimation, to the extent that self-motion estimation involves a very large "object" (a fixed scene). The use of long sequences (more than four or five images) gives a substantial improvement in achievable estimation accuracy, and accuracy improves as the object image gets larger. Since these theoretical lower bounds on variances of estimate [8] are independent of any particular 0162-8828/91/060CL0497$01.00 0 1991 IEEE
One of the first important results on recursive structure and motion estimation was the work of REF .
Estimating the kinematics and structure of a rigid object from a sequence of monocular images
IEEE Trans. Pattern Anal. Mach. Intell.
IEEE Trans. Pattern Anal. Mach. Intell.
Computer Science