A Localization SystemUsing Invisible Retro-reflective Markers
Yusuke Nakazato, Masayuki Kanbara and Naokazu Yokoya
Nara Institute of Science and Technology
8916–5 Takayama, Ikoma, Nara, 630–0192, Japan{yuusu-n, kanbara, yokoya}@is.naist.jp
Abstract
This paper describes a method for localizing wearable computer users. To realize applications of wearable computers
like a human navigation system, the position of a user is required for location-based services. Many localization methods in indoor environments have been proposed. One of themethods estimates user’s position using IR beacons or visualmarkers. However, these methods have some problems concerning power supply and/or undesirable visual effects. In
order to avoid the problems, we propose a new localization
method which is based on using an IR camera and invisible
markers consisting of translucent retro-reflectors. In the proposed method, to extract the regions of the markers from thecaptured images stably, the camera captures the reflection ofIR LEDs which are flashed on and off synchronously.

1

Introduction

Since computers have made remarkable progress in resentyears, a wearable computer can be realized. At the same time,
the augmented reality (AR) technique which merges the realand virtual worlds has received a great deal of attention as a
new method for displaying location-based information in thereal world[1, 2]. Therefore, AR systems using wearable computers like navigation systems[3, 4] are proposed. To realizean AR system using wearable computer, the exact positionand orientation of a user are required. Especially in indoor
environments, since a GPS cannot be used, many localization
methods have been proposed[5, 6]. One of the methods estimates the user’s position using infrared markers[7, 8]. The
methods specify the user’s position using the position IDs received from IrDA markers which compose positioning infrastructures. Tenmoku, et al[7] have estimated user’s position
using infrared markers and pedometer, and the orientation of
the user by gyro sensor. Maeda, et al[8] have developed ahybrid tracking system that estimates the position and orientation of the user by combining a gyro sensor with a stereo
camera which captures infrared markers. Another type ofmethods estimates the user’s position by recognizing visualmarkers pasted up on the ceilings or walls[9, 10]. For example, Baratoff, et al[9] used ARToolKit[11] square markers,and Naimark, et al[10] have developed a system that stably
estimates the position and orientation of the user by combining an accelerometer with a camera which captures circularmarkers. However, these methods have some problems con-cerning power supply of positioning infrastructure and/or undesirable visual effects.
In order not to impair the scenery, we propose a new localization method which is based on using invisible markers
that consist of translucent retro-reflectors. In the proposed
method, translucent retro-reflective markers, which are invisible and do not need power supply, are captured by an IRcamera. In order to avoid the influence of infrared other than
the reflection from retro-reflective markers, infrared LEDs
are flashed on and off continuously. Thus, the images containing the markers are captured synchronized with the flash.
The regions of the markers are robustly extracted from thedifference between images as LED on and off. Herewith,the position of a user can be estimated without power supplyof infrastructures and undesirable visual effects in the realscene.
This paper is structured as follows. Section 2 describes alocalization system using invisible markers and IR camera.
In Section 3, experimental results with a prototype system
are described. Finally, Section 4 gives conclusion and futurework.

2 User Localization Using Invisible
Markers
In this section, our proposed localization system is describedin detail. In Section 2.1, invisible markers are described. Section 2.2 discusses marker patterns, and finally Section 2.3
explains a method for estimating user’s position and orientation.

2.1 Invisible Markers
The markers are set up on the ceilings or walls in indoor environments as infrastructures. When visual markers are setup on the ceilings, the markers impair the scenery as shownin Figure 1(a). Figure 1(b) shows the scene where invisible
markers consisting of translucent retro-reflectors are set up.
Since the markers are translucent, it is difficult for a user toobserve the markers. However, when the image is captured
with a flash, the markers can be clearly observed as shownin Figure 1(c). Because the retro-reflector reflects a light toward a light source, its reflection can be captured clearly by
the camera which is located near the flashing light.

(a) Standard visual markers.

(b) Invisible markers.

(c) Invisible markers with a flash.

Figure 1: Markers as positioning infrastructures.

IR-LED switch off
Capturing image
Figure 2: Instances of markers (N = 4) (the black part illustrates a retro-reflector).

Capturing image

Translucent
retro-reflective
markerReflection

IR-LED switch on

Image subtraction
InfraredLight

Wearable computer(MP-XP7310 [Victor])

Marker detection &Position estimation

RS-232C

Figure 4: Flow diagram of capturing markers.
Video Capture Unit
(NV-UT200 [NOVAC]) IR LEDs andIR camera

Figure 3: Overview of localization system.

2.2

Pattern of Markers

Figure 2 illustrates instances of marker patterns. In thisfigure, the black part represents the retro-reflector, and themarker has a square frame. To generate a marker pattern,
the inside of the frame contains evenly spaced N × N grid
points, and dots are allocated on the grids. To determine thedirection of the marker uniquely, one dot is always allocatedto one of four corners of N ×N grids, and other three corners
are blank. Therefore, the total number of IDs associated the2patterns is 2N −4 .

2.3

Estimation of Position and Orientation

Figure 3 illustrates the overview of the proposed localization system. Invisible markers consisting of translucent retroreflectors are set up on the ceilings or walls. The user equips
the head with an infrared camera upward for capturing images. The camera captures the reflection of the infrared LEDs
that are attached to it. The reflection of retro-reflector can becaptured clearly by the infrared camera.
However, the camera also captures lights other than the in-frared LEDs; for example, fluorescent and sun light. To avoid
such a problem, infrared LEDs are flashed on and off synchronously under control of a wearable computer as shownin Figure 4. In the image without infrared light from the infrared LEDs, invisible markers are not captured. On the otherhand, in the image with infrared light from the infrared LEDs,
these are clearly captured. By calculating the difference between the images with and without infrared light from theLEDs, the influence of infrared other than the reflection fromretro-reflective markers can be eliminated. From the subtraction images, the regions of markers are extracted, and IDs
associated with the markers are recognized. To extract theregions of markers, ARToolKit[11] can be used. In addition
to identifying the received marker, it is possible to estimate
the relative position and orientation of camera with respect tothe marker coordinate system from four vertices of a square
marker of known size using a standard computer vision technique.

3 Experiments
We have carried out experiments with the proposed localization system. View angle of the infrared camera shown inFigure 3 is 92.6◦ , and six infrared LEDs are attached aroundthe camera. We made a circuit which controls the LEDs using RS-232C serial communication with a PC. The used PC
is a mobile computer “InterLink MP-XP7310 (Pentium M

(a) Captured images when infrared
LEDs are switch-off.

(b) Captured images when infrared
LEDs are switch-on.

(c) Results of marker recognition.

Figure 5: Images obtained by infrared camera and results of marker recognition.
0 6030 120210360

7201080LightLight1440

1800
y [cm]

060

LightLightLightLight

120x[cm]

Figure 6: Layout of markers.

1GHz),” and the input image size is 320 × 240 pixels. The
distance between the camera and the ceiling is about 120cm.
In Section 3.1, recognition results of invisible markers aredescribed. Section 3.2 gives experimental results about theaccuracy of localization.

3.1

MarkerResults of Marker Recognition

In this experiment, we confirm that the proposed system canrecognizes position IDs from markers without power supCamera
ply of infrastructures and undesirable visual effects in thereal scene. When the infrared LEDs are switch-off, retroreflective markers are not captured as shown in Figure 5(a).
Figure 7: Estimated camera position and orientation.
On the other hand, when infrared LEDs are switch-on, themarkers are clearly captured as shown in Figure 5(b). Figure5(c) shows results of marker recognition. These images are Figure 6. We employ two different size markers to recoggenerated by recognizing subtraction images between Fig- nize the markers even if the distance between a camera andures 5(a) and 5(b). Note that the position IDs associated markers is changed. The edge length of the small marker iswith the markers, and cones representing the direction of user 16cm and the size of its square element allocated on the grids
are superimposed on the subtraction image. We confirm that is 1cm × 1cm. On the other hand, the large marker’s edge
marker IDs can be correctly recognized. The system can cal- length is 28cm, and the size of its square element is 2cm ×culate the relative position and direction of camera with re- 2cm. The both markers’ frame width is 1cm. The number ofspect to the marker coordinate system. The processing rate grids N in marker pattern is 4 as shown in Figure 2.
is about 15 frames per second, because two images when theIn this experiment, x-y plane represents the ceiling plane,
infrared LEDs are switch-on and switch-off are required for and the z axis corresponds to the vertical direction. The pitchrecognition.
,roll and yaw represent the rotation angles around x, y, and zaxes in the marker coordinate system, respectively. The dis3.2 Accuracy Evaluation of Localization Re- tance between the camera and the ceiling is about 120cm,
and the camera which looks toward the ceiling moves tosults(50,0,120) from (50,1800,120) in parallel to the ceiling. FigIn this experiment, we evaluate the accuracy of the proposed ure 7 illustrates estimated camera positions and orientations
system. The markers are set up on the ceiling, as shown in㪜㫉㫉㫆㫉㫊㩷㫆㪽㩷㪧㫆㫊㫀㫋㫀㫆㫅㪲㪺㫄㪴

㪌㪇㪋㪇
㪊㪇㪉㪇
㪈㪇
㪄㪈㪇㪄㪉㪇㪇 㪈㪄㪊㪇㪄㪋㪇㪄㪌㪇㫏 㫐 㫑㪈㪇㪈㪉㪇㪈㪊㪇㪈

㪋㪇㪈㪌㪇㪈㪽㫉㪸㫄㪼㫊㪍㪇㪈㪎㪇㪈㪏㪇㪈㪐㪇㪈㪈㪇㪇㪎㪇㪈㪏㪇㪈㪐㪇㪈㪈㪇㪇

Figure 8: Errors of estimated camera position.

㪜㫉㫉㫆㫉㫊㩷㫆㪽㩷㪦㫉㫀㪼㫅㫋㪸㫋㫀㫆㫅㪲㫦㪴㪉㪌㪈㪌㪌㪄㪌 㪈㪄㪈㪌㪄㪉㪌㫇㫀㫋㪺㪿 㫉㫆㫃 㫐㪸㫎㪈㪇㪈㪉㪇㪈㪊㪇㪈㪋㪇㪈㪌㪇㪈㪽㫉㪸㫄㪼㫊

㪍㪇㪈

Figure 9: Errors of estimated camera orientation.

when the camera moves in parallel with the y axis. Figures
8 and 9 show errors of estimated position and orientation,respectively. From the experiment, we can confirm that errors of estimated camera position and orientation are at mostabout 30cm and 15◦ , respectively. The averages of estimatedposition and orientation errors are about 15cm and 8◦ . We
conclude that the accuracy of estimated camera position iscomparable with the case of using conventional visual markers. However, the results of estimation are vibrated.

4Conclusion

This paper has proposed a new localization method using invisible markers and infrared camera. In our method,user’s position is identified by recognizing translucent retroreflective markers which are illuminated from infrared LEDs.
In order to eliminate the influence of infrared other than thereflection from retro-reflective markers, the infrared LEDs
are flashed on and off synchronously with image capturing.
In experiments, we have confirmed that the accuracy of estimated camera position and orientation is comparable withthe case of using conventional visual markers. In future work,
we should carry out experiments at extensive indoor environments. We will also stabilize the result of position estimationin order to construct an AR system.

Acknowledgments This research was supported in partby Core Research for Evolutional Science and Technology(CREST) Program “Advanced Media Technology for Everyday Living” of Japan Science and Technology Agency (JST).

References[1] M. Kanbara, T. Okuma, H. Takemura and N. Yokoya: “Astereoscopic video see-through augmented reality system
based on real-time vision-based registration,” Proc. IEEE Int.
Conf. on Virtual Reality 2000, pp. 255–262, 2000.

[2] S. Julier, M. Lanzagorta, Y. Baillot, L. Rosenblum, S. Feiner,T. Höller and S. Sestito: “Information filtering for mobile augmented reality,” Proc. 1st IEEE/ACM Int. Symp. on AugmentedReality (ISAR 2000), pp. 3–11, 2000.
[3] T. Höller, S. Feiner, T. Terauchi, G. Rashid and D. Hallaway:“Exploring mars: Developing indoor and outdoor user interfaces to a mobile augmented reality system,” Computers andGraphics, Vol. 23, No. 6, pp. 779–785, 1999.
[4] P. Daehne and J. Karigiannis: “ARCHEOGUIDE: System architecture of a mobile outdoor augmented reality system,”
Proc. 1st IEEE/ACM Int. Symp. on Mixed and Augmented Reality (ISMAR2002), pp. 263–264, 2002.
[5] M. Kourogi and T. Kurata: “Personal positioning based onwalking locomotion analysis with self-contained sensors andwearable camera,” Proc. 2nd IEEE/ACM Int. Symp. on Mixedand Augmented Reality (ISMAR 03), pp. 103–112, 2003.
[6] L. Vacchetti, V. Lepetit and P. Fua: “Combining edge and texture information for real-time accurate 3D camera tracking,”
Proc. 3rd IEEE/ACM Int. Symp. on Mixed and Augmented Reality (ISMAR 04), pp. 48–57, 2004.
[7] R. Tenmoku, M. Kanbara and N. Yokoya: “A wearable augmented reality system using positioning infrastructures and apedometer,” Proc. 7th IEEE Int. Symp. on Wearable Computers (ISWC’03), pp. 110–117, 2003.
[8] M. Maeda, T. Ogawa, K. Kiyokawa and H. Takemura: “Tracking of user position and orientation by stereo measurement ofinfrared markers and orientation sensing,” Proc. 8th IEEE Int.
Symp. on Wearable Computers (ISWC’04), pp. 77–84, 2004.
[9] G. Baratoff, A. Neubeck and H. Regenbrecht: “Interactivemulti-marker calibration for augmented reality applications,”
Proc. 1st IEEE/ACM Int. Symp. on Mixed and Augmented Reality (ISMAR2002), pp. 107–116, 2002.
[10] L. Naimark and E. Foxlin: “Circular data matrix fiducial system and robust image processing for a wearable vision-inertial
self-tracker,” Proc. 1st IEEE/ACM Int. Symp. on Mixed andAugmented Reality (ISMAR2002), pp. 27–36, 2002.
[11] H. Kato and H. Billinghurst: “Marker tracking and HMD calibration for a video-based augmented reality conferencing system,” Proc. 2nd IEEE/ACM Int. Workshop on Augmented Reality (IWAR’99), pp. 85–94, 1999.