Three-dimensional (3D) Model-Based Vision enables observed image features to be used to determine the pose (ie. position and attitude) of a known 3D object with respect to the camera (or alternatively, the viewpoint of the camera with respect to the model) [1] . The knowledge concerning the object that is used to perform Model-Based Vision is the 3D locations of salient and easily observed object features, such as corners and high-contrast edges (eg. surface marking sand crease edges). Tracking of a moving object can be achieved by performing such a vision task on successive frames of an image sequence. By making use of the continuity of object motion found in the closely-spaced frames of video imagery, the process of tracking becomes simplified, permitting real-time object tracking to be accomplished. By noting the difference in image position between the observed image features and the projection of the 3D model features with the model in its currently estimated pose, the required small corrections to the object pose can be calculated. The particular model features that we use to perform tracking are points located on high contrast edges. The projection of these edges into the image is easy to perform, and the corresponding image edges simple to locate by searching the image pixels perpendicularly to the edge direction. Only the perpendicular component of motion is sought because of the aperture problem, and this is why we need the edges to be locally straight. The set of measured displacements of these edges are used to update the estimate of model pose by linearising the resulting equations. For the edges to be successfully detected, the edges need to be locally straight and uncluttered. Rudimentary edge detection is performed by considering the row of pixels perpendicular to the model edge that pass through the projected control point, and finding the pixel position of the largest brightness gradient along that row. In general, the projected model control points will not lie precisely on the observed image edges because of errors in the estimate of the object pose due to object motion. However, frames captured at video rate give errors in the object pose which are small so allowing edges to be correctly matched. To first order, small changes in the model pose will cause image displacements of the control points which are linear in the change in the pose, and so the perpendicular distances between the projected control points and the observed image edges will vary linearly with the change in the pose. This linearity enables the small variations of the object pose that serve to minimise these perpendicular distances to be determined by solving a set of simultaneous linear equations, hence producing a fast pose correction. Applying the resulting correction to the pose estimate enables the pose estimate to rapidly converge on the correct object pose as subsequent frames are processed. If the object is moving across the image, the above method of updating the pose vector will produce a result that lags behind the actual pose. It is advantageous to counter this deficiency by predicting ahead, as inter-frame motion in excess of the half-length of the row of pixels can then be tolerated. This predicting ahead is most simply achieved by using a position and velocity tracker, the so-called alpha-beta tracker [2] . The basic tracker algorithm as described above has limited uses. The necessary enhancements on the basic process described above are: • to enable and disable control points appropriately as the object attitude varies, and control points are revealed and obscured respectively; • to ignore control points on weak, error prone edges; • to define an edge polarity (ie. light/dark or dark/light) so that strong edges with wrong polarity caught in the search region are ignored; • to make use of profile edges whose 3D control point positions change with object pose (eg. on cylinders, cones, spheres, etc.); • to predict ahead and to reduce the variability of the pose estimates by using a Kalman Filtering technique [3] . Define the (Cartesian) camera coordinate system, which has its origin at the camera pin-hole, Z-axis aligned along the optical axis of the camera, and X and Y axes aligned along the horizontal (rightward) and vertical (downward) image axes respectively. Imaging of points in 3D will be handled by the introduction of conceptual image-plane situated at unit distance in front of the camera pin-hole. A point at
RAPID REF was one of the first methods where the object pose was estimated based on the correspondences between points on a 3D model contour and points on the edges of the image.
RAPID-A Video Rate Object Tracker
BMVC
Computer Science