IWSSIP 2010-17 th International Conference on Systems , Signals and Image Processing Estimation of Camera Parameters in Video Sequences with a Large Amount of Scene Motion

Most of existing techniques to estimate camera motion is based on analysis of the optical flow. However, such methods can be inaccurate and/or inefficiently when applied in video sequences which have a large amount of motion or a large number of scene changes. In this paper, we present an approach to estimate camera motion based on analysis of local invariant features. Such features are robust across a substantial range of affine distortion. Experiments on synthesized video clips with a fully controlled environment show that our technique is more effective than the optical flow-based approaches for estimating camera motion with a large amount of scene motion. Keywordsvideo indexing; camera motion; local invariant features; affine model. I. I NTRODUCTION The estimation of camera motion is one of the most important aspects for video processing, analysis, indexing, and retrieval. Most of existing techniques to estimate camera motion is based on analysis of the optical flow [1]–[6]. However, such methods can be inaccurate and/or inefficiently when applied in video sequences which have a large amount of motion or a large number of scene changes [7], [8]. To address this problem, we present an approach for the estimation of camera motion with a large amount of scene motion. Our technique relies on analysis of local invariant features obtained from extrema in the scale space rather than on analysis of the optical flow. Such features are robust across a substantial range of affine distortion. In order to validate our approach, we use synthetic videos sequences based on POV-Ray scenes including all kinds of camera motion and many of their possible combinations. The main advantage of such a synthetic test set is that the camera motion parameters can be fully controlled. Further, we have conducted several experiments to show that our technique is more effective than the optical flow-based ones for estimating camera motion with a large amount of scene motion. The remainder of the paper is organized as follows. Section II presents our approach for the estimation of camera motion. The experimental settings and results are discussed in Section III. Finally, Section IV presents conclusions and directions for future work. Thanks to Microsoft EScience Project, CAPES/COFECUB Proje ct, and Brazilian agencies FAPESP, CNPq, and CAPES for funding. II. OUR APPROACH In presence of a substantial range of affine distortion, the methods for estimating camera motion by analysis of the optical flow can fail [3]. To address this problem, we present an approach for the estimation of camera motion based on the analysis of the local invariant features. It consists of three main steps: (1) feature matching; (2) motion model fitting; and (3) robust estimation of the camera parameters. A. Feature Matching The first step for estimating camera motion in video sequences is to extract and match features between consecutive frames. Here, we use a framework to detect and describe local invariant features in images, called Scale Invariant Features Transform(SIFT) [9]. This approach is composed by four major stages: (1) scale-space peak selection; (2) keypoint localization; (3) orientation assignment; and (4) keypoint description. The scale-invariant features are efficiently identified by using a staged filtering approach. The first stage identifies key locations in scale space by looking for locations that are maxima or minima of a difference-of-Gaussian function. Next, for each candidate keypoint, interpolation of nearby data is used to accurately determine its position. Moreover, this information allows to reject candidate keypoints that have low contrast or are poorly localized along an edge. Thereafter, it identifies the dominant orientations for each keypoint using local image gradient directions. Finally, the method builds a local descriptor for each keypoint based on the image gradients in its local neighborhood. To match keypoints from two images, we use the Euclidean distance between the local descriptors. To ensure correct match, the ratio of the distance for the best match and the second best match must be less than 0.6 [9]. B. Motion Model Fitting A camera projects a 3D world point into a 2D image point. The motion of the camera may be limited to a single motion such as rotation, translation, or zoom, or some combination of these three motions. Such camera motion can be well categorized by few parameters. In our case, we use a two-dimensional affine model to estimate a parametric form for describing the displacement of the video frame content from the correspondence between local invariant features. The affine model was employed in the following considerations. First, the affine model is more resilient IWSSIP 2010 17th International Conference on Systems, Signals and Image Processing to noisy data. In addition, it can represent all of the basic camera motions often used in video indexing. If we denote the position in the first image by (x, y) and the corresponding position in the second image by (x̂, ŷ), we can formulate the two-dimensional affine motion model as