Binocular Vision-Based Augmented Reality System with an Increased
Registration Depth Using Dynamic Correction of Feature Positions
Steve Vallerand, Masayuki Kanbara and Naokazu Yokoya
Graduate School of Information ScienceNara Institute of Science and Technology
8916-5 Takayama, Ikoma, Nara
Japan, 630-0101
steve-v@is.aist-nara.ac.jp

Abstract
In vision-based augmented reality systems, the relationship between the real and virtual worlds needs to be estimated to perform the registration of the virtual objects.
This paper suggests a registration method which increases
the registration depth of video see-through augmented reality systems using binocular cameras. The method uses
both monocular and stereoscopic vision-based techniques
in order to perform the registration. Also, the registration
method can be combined with a facultative correction of the
2D positions of the feature points. The correction increases
the stability and the accuracy of the proposed registration
method.

tance if the quality of the registration is not influenced when
the distance between the camera and the feature points increases. However, if the distance influences the quality of
the registration, the registration method is said to be dependant on the distance, and the registration method is consequently limited in registration depth.
Presently, most AR systems use MR since MR is independent of the distance. Although SR is dependent on thedistance, stereoscopic computation provides additional information that may help to perform the registration. This
paper proposes a new registration method which uses elements of MR and SR to perform the registration. Also, afacultative correction method is presented to increase theregistration stability and accuracy of the proposed registration method.

1. Introduction
2. Proposed methods
Augmented reality (AR) systems enhance user’s perception and interaction with the real world. Nowadays,
the geometric registration is one of the most important aspects of AR systems. In other words, the real and virtualworlds must be properly aligned to give the illusion thatboth worlds coexist. Different techniques exist to align thereal and the virtual world. In vision-based AR systems,
two types of registration techniques exist: monocular andstereoscopic vision-based registration. Monocular registration (MR) uses only one camera. Basically, a MR performs
the registration by solving the perspective pose problem
from the positions of three or more features in the camera images[1]. On the other hand, stereoscopic registration(SR) uses two cameras and performs the registration froman estimation of the 3D positions of feature points obtainedby triangulation[2].
An important characteristic of a registration method is
the influence of the distance on the registration quality. A
registration method is said to be independent of the dis-2.1. Registration method
Our registration method is based on MR to take advantage of the distance independency of this method. We
choose to use three points to perform the MR for two principal reasons. First, a registration method using three points
is an important aspect of any vision-based system, even forsystem using MR with four points, since those systems mayneed to succeed the registration in degenerated conditions,in particular, when one of the four feature points is missing.
Secondly, we also want to use stereoscopic computation toperform the registration. Therefore, our registration method
is based on MR using three feature points. Then, stereoscopic computation is used to compensate the lack of one
feature point.
At first, the 3D positions of three observed points must
be retrieved. Determining the 3D positions of three points
from the 2D positions of those points in an image is calledthe three point space resection problem. Many direct solutions of the three point resection problem have been proposed. Haralick et al., reviewed most of those direct solutions in their paper[3]. The 3D positions of the featurepoints are retrieved using the Finsterwalder’s approach inour system. The multiple solutions obtained are discriminated using stereoscopic projection.
Instead of evaluating all the solutions individually in order to find the best solution, pairs of solutions are evaluated. A pair of solutions (PS) is created with a solution
computed from the left image L and a solution computedfrom the right image R. Projection errors EL and ER arecomputed for each feature point k like illustrated in Figure
1. The global error (GE) of a PS is given by the sum of
all the projection errors given by each of the three featurepoints(k = 1; 2; 3). The PS which gives the smallest error
is chosen as the best PS. Then, the left image registrationis performed with the left image component of the best PS
and the right image registration is performed with the rightimage component.

eras (Canon) is used to acquire stereoscopic images. The
three corners of a blue triangle marker give the three featurepoints needed to perform the registration. Colored regions
inserted in the triangle allow the system to discriminate between the markers.
The proposed registration method improves the qualityof the registration compared to standard SR and standard
MR using three points. The correction method influences
positively the registration. Once the 2D positions of thefeature points have been corrected, those corrected points
can be used to perform a SR with an increased registration depth. All in all, stereoscopic vision applied concurrently with MR improves the registration quality in binocular vision-based AR system using three feature points based
registration. Figure 2 shows some registration images obtained with the proposed registration and correction methods.

Figure 2. Examples of registration images.
Figure 1. Projection errors for a feature point2.2. Correction method
Instead of performing the registration independently foreach image, a facultative correction method to improve
the quality of the registration is proposed. The correction
method is based on the fact that the 3D positions of corresponding points in binocular images may differ from eachimage. The difference between the two 3D positions mostly
arises from the lack of accuracy when computing the 2D positions of the feature points. The idea is to correct the 2D
positions of the points in the images in order to obtain thesame 3D positions for each image.
In other words, we want the GE to equal zero. Corrected
2D positions are computed by applying a rule which tends
to diminish the GE. But, the difference between the two
3D positions computed with the corrected points may still
be significant. Therefore, the correction process is iterated
until the GE is smaller than a predefined threshold value.

3. Experimental results
The proposed approach is tested on a desktop PC (SGIvisual workstation 330) and a HMD with internal cam-4. ConclusionThis paper has proposed a new method for performing
the registration in a binocular vision-based AR system. The
proposed registration method is independent of distanceeven if only three feature points are used to perform theregistration. Also, the correction method increases the registration depth of SR systems. In future work, we want toevaluate the effect of the correction method in a system using MR with four feature points.

References[1] T. Okuma, K. Kiyokawa, H. Takemura and N. Yokoya, “AnAugmented Reality System Using a Real-time Vision BasedRegistration”, ICPR 98: Proceedings 14th International Conference on Pattern Recognition, Vol. 2, pp. 1226-1229, 1998.
[2] M. Kanbara, T. Okuma, H. Takemura and N. Yokoya, “A
Stereoscopic Video See-through Augmented Reality System
Based on Real-time Vision-based Registration”, Proc. IEEE
Virtual Reality 2000, pp. 255–262, 2000.
[3] R.M. Haralick, C.-N. Lee and K. Ottenberg, “Analysis andSolutions of The Three Point Perspective Pose Estimation
Problem”, Proc. CVPR ’91, pp. 592-598, June 1991.