39 espite the increasing sophistication of augmented reality tracking technology, a recent survey of AR notes that "tracking in unprepared environments remains an enormous challenge." 1 Although camera-based tracking for AR environments using fiducial (reference) markers has been highly successful, 2 there are many scenarios in which we might want to add virtual content without preparing the environment by inserting artificial markers. A prototypical example is the labeling of geographical features for navigation in an outdoors AR setting. Here, we must track the camera using natural scene features alone. In the past, this problem has been approached in one of two distinctly different approaches. The geometric approach reconstructs the full 3D camera motion based on mathematical constraints between feature positions in multiple images. For example, the fundamental matrix is an algebraic entity that embodies the information about camera movement between two frames of the same static scene. (See the literature on the fundamental matrix calculation-and the subsequent extraction of motion parameters-especially Hartley and Zisserman's theory review. 3 ) Commercial movie studios typically use these and similar methods to introduce computer graphics into film sequences. However, such algorithms are unsuitable for real-time applications. These methods, which must generally batch-process the entire data sequence at once (an operation known as global bundle adjustment), are too time-consuming for interactive applications. The second approach to natural-feature tracking is image based. Instead of estimating the full 3D camera motion, the camera tracks and labels features across a sequence of images by measuring the 2D image motion. Such "optical flow" methods are faster than the geometric approach but, because they can't estimate 3D camera motion, they can't be used to introduce 3D augmentations into scenes. Moreover, optical-flow tracking is unstable and requires heavy regularization to produce a reasonable solution. We offer an intermediate approach we developed to the tracking problem that encompasses the advantages of both geometric and image-based tracking. We apply this technique to cases where feature positions in adjacent frames of an image sequence are related by a homography, or projective transformation. We describe this transformation's computation and demonstrate several applications. First, we use an augmented notice board to explain how a homography, between two images of a planar scene, completely determines the relative camera positions. Second, we show that the homography can also recover pure camera rotations, and we use this to develop an outdoor AR tracking system. Third, we use the system to measure head rotation and form a simple lowcost virtual reality (VR) tracking solution. A homography is a one-to-one mapping between two images, which is defined by only eight parameters. This model exactly describes the image motion between two frames of a video sequence when I the camera motion is pure rotation, or I the camera is viewing a planar scene. Usually, feature displacement between two images depends on both the camera movement and the camera's distance from the feature. A simple parameterized mapping is therefore not possible. However, in many circumstances, the homography represents a good approximation of the true image flow, particularly when the image structure is near planar, or the camera movement is small and the scene structure is mostly distant. A number of previous systems have applied this information. For example, Uenohara and Kanade's 4 system achieves medical-image registration based on five coplanar points. However, their system's disadvantage
REF studied planar tracking for augmented reality applications.
Augmented Reality Camera Tracking with Homographies
IEEE Computer Graphics and Applications
IEEE Computer Graphics and Applications
Computer Science