Localization System for Large Indoor Environments Using Invisible Markers
Yusuke Nakazato∗

Masayuki Kanbara†
Nara Institute of Science and TechnologyAbstract
We propose a user localization system that uses invisible markers
for wearable augmented reality (AR) in large indoor environments.
Wearable AR systems have received a great deal of attention as a
new method for displaying location-based information in the realworld. For using wearable AR systems, it is necessary to measure
the position and orientation of a user using a positioning infrastructure without the undesirable visual effects that arise from merging real and virtual worlds. In addition, the infrastructure of thelocalization environment must be constructed easily and cheaply.
The proposed system can estimate the position and orientation ofa user precisely by affixing wallpapers containing printed invisible
markers on ceilings or walls. The user’s position and orientation
are estimated by recognizing the markers using an infrared camera
with infrared LEDs. To construct environments for the localization
system, we developed an initialization tool that calibrates the alignment of the markers from photographs taken by flash illumination
using a digital still camera.

Naokazu Yokoya‡

Initialization
Administrator

Pasting up Marker
Wallpapers

Invisible Marker Setup
Capturing Markers Images
Invisible Marker Calibration

Creating Marker Alignment Infousing a Calibration Tool

Marker Alignment Information(markers’ position & orientation)

User Localization

InfraredLight

User
User Localization System

Infrared Camera

Invisible Markers

User’s Position and Orientation (Output: UDP or Shared Memory)

Figure 1: Overview of a user localization system using invisible
markers.

CR Categories: H.5.1 [Information Interfaces and Presentation]:
Multimedia Information Systems—Artificial, augmented, and virtual realities; I.5.4 [Pattern Recognition]: Applications—Computervision
Keywords: localization, invisible marker, augmented reality

1Introduction

One of the localization methods uses images captured by the user’sview camera and a feature database of a real environment. Thesemethods estimate the user’s position and orientation by matching
features on an image from the user’s viewpoint and the known feature points or a 3D model of the environment [Klein and Murray2007]. However, it is difficult to estimate the user’s position in environments that have many similar patterns.

(a) Capturing without a flash.

(b) Capturing with a flash.

Figure 2: Environment where Invisible markers wallpaper is installed on the ceiling.

2 Localization System Using Invisible Markers
2.1 First phase: Initialization

Another approach estimates the user’s position and orientation
by recognizing visual markers affixed on the ceilings or walls[Naimark and Foxlin 2002]. In this method, it is relatively cheap
to set up visual markers to form a positioning infrastructure. In addition, it is possible to estimate the user’s position and orientation
robustly, because recognition of the markers from an image captured using a wearable camera is relatively easy. However, a visualmarker is often so prominent that it creates undesirable visual effects. Therefore, invisible markers[Park and Park 2004; Nakazato
et al. 2006] or markers designed to blend with the scene[Saito et al.
2007; Tenmoku et al. 2007] need to be used. These methods canachieve localization without disturbing the scenery.
In order to realize a localization system in indoor environments inpractice, we propose a localization system and marker initialization tool using invisible markers. The proposed method has twophases as shown in Figure 1: The first phase is the initialization ofinvisible-marker environments, and the second phase is installing auser localization system in these environments.
∗ e-mail:nakazato.yusuke@canon.co.jp (Presently with Canon Inc.)† e-mail:kanbara@is.naist.jp‡ e-mail:yokoya@is.naist.jp

2.1.1 Invisible Marker Setup

In this system, wallpapers with pre-printed invisible markers areused because it is easy to set them up. Figure 2 shows an example
of an environment where an invisible-marker wallpaper is installedon a ceiling. In general, the user cannot see the markers, as shown in
Figure 2(a). However, the markers on the wallpaper can be captured
clearly using a camera with a flash because the markers consist of
retro-reflectors.
The markers are square, and their inside contains evenly spaced
N × N grid points, and dots are allocated to the grids. To determine
the direction of the marker uniquely, one dot is allocated to one of
the four corners of the grids, and the other three corners are blank.
These dots correspond to bits, which are encoded using a cyclic
redundancy check (CRC) error-detecting code. If the number ofCRC check bits is C, the total number of IDs associated with the2IDs is 2N −4−C .
It is difficult to print markers that have unique ID patterns becauseof the marker printing process. Therefore, all markers on the wallpaper have a pattern in which all bits are initially 1. The administrator creates a unique ID pattern using the proposed pattern-creatingtool. The tool outputs marker design information, which includes atable of random ID patterns. The administrator scratches dots thatTable 2: Errors in estimated position and orientation of the camera
under the invisible marker wallpapers.
positionorientation(a) Infrared camera L.

(b) Infrared camera S.

Figure 3: Infrared camera with infrared LEDs.
Table 1: Specifications of the infrared camera.
size[mm]weight[g]resolution[pixel]
view angle[◦ ]number of LEDs
interfaceframe rate[fps]IR camera L
75 × 55 × 47
1151024 × 768
110
12USB2.0 (×2)
29.12

IR camera S
25 × 20 × 10
20640 × 480
70
6USB2.030

correspond to 0 bits on the marker, and then the wallpaper is affixed
onto the ceilings or walls.
2.1.2 Invisible Marker Calibration

In the marker-based approach, the alignment information of markers, i.e., the position and orientation of markers located in the realenvironment, is needed in advance. In order to obtain the alignment
information easily, the proposed system uses a marker calibration
tool. The administrator captures images of invisible markers via anormal digital still camera, and the images are inputted into the calibration tool in order to estimate the position and orientation of themarkers. The administrator also inputs intrinsic camera parameters,
the position and orientation of reference markers, and the markers’
design information, which consists of the size, interval of the markers, grids, number of CRC checkbits, and junction information ofmarkers’ ID patterns. Finally, the tool outputs marker alignment
information, i.e., the markers’ position and orientation in the realworld. This information is used in the localization phase to estimate the user’s position and orientation.

2.2 Second phase: User Localization
In the proposed system, the user’s position and orientation can be
estimated from the intrinsic parameters of an infrared (IR) camera
and the marker alignment information from the initialization phase.
The user is equipped with a wearable computer, and an IR camera
with IR LEDs is fitted on his head. The IR camera captures invisible markers clearly by detecting the reflectance of IR light. The
estimated position and orientation of the user can then be written toshared memory on the wearable computer, or can be sent to anothercomputer using UDP connection.
We developed two types of infrared cameras, as shown in Figure
3. The specifications of these cameras are shown in Table 1. Onecamera (IR camera L) has a high resolution and wide angle in orderto estimate the position and orientation of the user precisely. Thecamera also has user’s view camera for AR in front of the camera.
The other camera (IR camera S) is small and lightweight.

3

Experiment of User Localization

We evaluated the accuracy of the estimated position and orientationof markers using the calibration tool. In this experiment, invisiblemarker wallpaper with an area of about 65m2 was affixed on theceiling, as shown in Figure 2. The markers with an area of 16cm
and unique IDs were aligned on the wallpaper. The density of markers was about 24 markers/m2 . The number of marker grids was
5, and the number of CRC check bits was 3. One marker’s position and orientation was input as the reference marker, and 100

average error
standard deviation
average error
standard deviation

IR camera L
9 mm
4 mm
0.41 ◦
0.13 ◦

IR camera S
13 mm
6 mm
0.52 ◦
0.31 ◦images (3072 × 2304pixel) of markers captured by a digital stillcamera (Olympus, µ 770SW) were input. The intrinsic camera parameters were estimated via Tsai’s method in advance. The grandtruths of the markers’ position and orientation were manually measured using a total station (Topcon, GPT-9005A, accuracy: ±5mm).
In this experiment, the error in the estimated marker positions ofmarker alignment information was about 13mm, and standard deviation was about 12mm.
In the next experiment, we evaluated the accuracy of the estimatedposition and orientation of the two types cameras in an environmentthat was constructed in the initialization phase. These cameras weredirected toward the ceiling (the distance between the cameras andthe ceiling was about 1.2 m), and its position and orientation wereestimated using the alignment information, which was outputtedthrough the initialization experiment. The true positions and orientations were measured manually using the total station. Table 2shows the error and standard deviation of the estimated position andorientation of the cameras. In this experiment, the frame rate of localization was about 29 fps (CPU utilization: 15%) when we use a
ThinkPad X60 (Lenovo; CPU: Intel T7200 2 GHz, memory: 2 GB)
as a wearable computer.

4 Conclusion
We proposed a user localization system and initialization tool using invisible markers for wearable AR. Since it is possible to affix
numerous markers closely using an invisible-marker wallpaper, thissystem can estimate the position and orientation of a user precisely.
Furthermore, this system assists the administrator in calibrating themarkers using the initialization tool. In experiments, we have confirmed that the proposed system is able to estimate the position andorientation of a user with high accuracy.

References
K LEIN , G., AND M URRAY, D. 2007. Parallel tracking and mapping for small AR workspaces. Proc. 6th IEEE/ACM Int. Symp.
on Mixed and Augmented Reality, 225–234.
NAIMARK , L., AND F OXLIN , E. 2002. Circular data matrix fiducial system and robust image processing for a wearable visioninertial self-tracker. Proc. 1st IEEE/ACM Int. Symp. on Mixedand Augmented Reality, 27–36.
NAKAZATO , Y., K ANBARA , M., AND YOKOYA , N. 2006. An
initialization tool for installing visual markers in wearable augmented reality. Proc. 16th Int. Conf. on Artificial Reality andTelexistence, 228–238.
PARK , H., AND PARK , J. 2004. Invisible marker tracking forAR. Proc. 3rd IEEE/ACM Int. Symp. on Mixed and AugmentedReality, 272–273.
S AITO , S., H IYAMA , A., TANIKAWA , T., AND H IROSE , M. 2007.
Indoor marker-based localization using coded seamless pattern
for interior decoration. Proc. IEEE Virtual Reality 2007, 67–74.
T ENMOKU , R., YOSHIDA , Y., S HIBATA , F., K IMURA , A., AND
TAMURA , H. 2007. Visually elegant and robust semi-fiducials
for geometric registration in mixed reality. Proc. 6th IEEE/ACM
Int. Symp. on Mixed and Augmented Reality, 261–262.