Wearable Augmented Reality System
Using Invisible Visual Markers and an IR Camera
Yusuke Nakazato, Masayuki Kanbara, and Naokazu Yokoya
Nara Institute of Science and Technology
8916–5 Takayama, Ikoma, Nara, 630–0192, Japan,{yuusu-n, kanbara, yokoya}@is.naist.jp
Abstract

Translucent
Retro-reflective
MarkerReflection

This paper describes a wearable augmented reality (AR)system using invisible visual markers. Some AR systems use
visual markers in order to measure the position and orientation of user’s viewpoint. However, conventional visualmarkers usually impair the scenery. This paper proposes asystem using invisible visual markers consisting of translucent retro-reflectors. The system can be realized without
undesirable visual effects and power supply for infrastructures.

1

Optical Seethrough Display(ND2000[MICROVISION])

IR camera
with IR-LEDs

Wearable Computer(MP-XP7310 [JVC])

Introduction

The AR technique which merges the real and virtualworlds has received a great deal of attention as a new
method for displaying location-based information in thereal world. Therefore, AR systems using wearable computers like navigation systems have been proposed. To realizean AR system using a wearable computer, the exact positionand orientation of a user are required. Especially in indoor
environments, since a GPS cannot be used, many localization methods have been proposed[1, 2]. One of the methods
estimates the user’s position by recognizing visual markers
pasted up on the ceilings or walls. For example, Baratoff, etal. [3] have used ARToolKit [4] square markers. Naimark,et al. [5] have developed a system that stably estimates theposition and orientation of the user by combining an accelerometer with a camera which captures circular markers.
However, these methods have some problems concerning
undesirable visual effects.
In order not to impair the scenery, we employ a localization method which is based on using invisible visualmarkers that consist of translucent retro-reflectors[6]. In
the method, user’s position and orientation are estimated byrecognizing invisible visual markers which are illuminatedfrom IR LEDs. This paper describes a wearable AR system
which is based on using the invisible marker based localization method.

InfraredLightFigure 1. Wearable augmented reality system
overview.

2Wearable Augmented Reality System

Figure 1 illustrates the overview of the proposed wearable AR system. Markers consisting of translucent retroreflectors are set up on the ceilings or walls in indoor environments as infrastructures. Since the markers are translucent, it is difficult for a human to observe the markers. The
user equips an optical see-through display and an IR camera upward for capturing images. The camera captures thereflection of the IR LEDs that are attached to it. The retroreflector reflects a light toward a light source and thus its
reflection can be captured clearly by the IR camera locatednear the IR LEDs as shown in Figure 2. From a captured image, the regions of markers are extracted, and IDs associatedwith the markers are recognized. In addition to identifying
the received marker, it is possible to estimate the relative position and orientation of camera with respect to the marker
coordinate system from four vertices of a square marker ofknown size using a standard computer vision technique[4].
Finally, the system draws the virtual object images such as

Figure 2. Image captured by an IR camera withIR LEDs.

Figure 4. Annotation overlay image.
reflectors and an IR camera. In the system, user’s positionand orientation are estimated by recognizing the markers illuminated from IR LEDs. In the experiment of a prototype
system, we have confirmed that the system can be realized
without undesirable visual effects and power supply for infrastructures. In future work, we should carry out experiments in wide indoor environments.
Acknowledgements This research is partially supported byFigure 3. Experimental environment: Invisible
visual markers are setup on the ceiling.
annotation overlay images based on estimated user’s position and orientation.

3Experiments

In this experiment, a prototype wearable AR system
shown in Figure 1 is used. View angle of the IR camera
is 92.6◦ and the image size is 320 × 240 pixels. The experimental environment is shown in Figure 3. In the environment, invisible visual markers are setup on the ceilings.
There are 78 markers on 18m × 3m. When the distance
between IR camera and ceilings was 1.2m, the error of estimated user’s position and and its standard deviation wereabout 10cmand 8cm, respectively. Moreover error of estimated user’s orientation was about 5◦ and its standard deviation was 5◦ .
Figure 4 shows an example of annotation overlay images
generated by the prototype system. In this experiment, we
confirmed that wearable AR system can be realized without
undesirable visual effects and power supply for infrastructures.

4Conclusion

This paper has proposed a wearable AR system using
invisible visual markers which consist of translucent retro-Core Research for Evolutional Science and Technology (CREST)
Program “Advanced Media Technology for Everyday Living” ofJapan Science and Technology Agency (JST).

References[1] D. Hallaway, T. Höllerer and S. Feiner: “Coarse, inexpensive,infrared tracking for wearable computing,” Proc. 7th IEEE Int.
Symp. on Wearable Computers (ISWC’03), pp. 69–78, 2003.
[2] M. Maeda, T. Ogawa, K. Kiyokawa and H. Takemura: “Tracking of user position and orientation by stereo measurement ofinfrared markers and orientation sensing,” Proc. 8th IEEE Int.
Symp. on Wearable Computers (ISWC’04), pp. 77–84, 2004.
[3] G. Baratoff, A. Neubeck and H. Regenbrecht: “Interactive
multi-marker calibration for augmented reality applications,”
Proc. 1st IEEE/ACM Int. Symp. on Mixed and AugmentedReality (ISMAR2002), pp. 107–116, 2002.
[4] H. Kato and H. Billinghurst: “Marker tracking and hmd calibration for a video-based augmented reality conferencing system,” Proc. 2nd IEEE/ACM Int. Workshop on Augmented Reality (IWAR’99), pp. 85–94, 1999.
[5] L. Naimark and E. Foxlin: “Circular data matrix fiducial system and robust image processing for a wearable vision-inertial
self-tracker,” Proc. 1st IEEE/ACM Int. Symp. on Mixed andAugmented Reality (ISMAR2002), pp. 27–36, 2002.
[6] Y. Nakazato, M. Kanbara and N. Yokoya: “Localization ofwearable users using invisible retro-reflective markers andan IR camera,” Proc. SPIE Electronic Imaging, Vol. 5664,
pp. 1234–1242, 2005.