A Wearable Augmented Reality System
Using an IrDA Device and a Passometer
Ryuhei Tenmokua Masayuki Kanbaraa and Naokazu Yokoyaa
a Graduate School of Information Science, Nara Institute of Science and Technology,

8916-5 Takayama, Ikoma, Nara, 630-0101 Japan
ABSTRACTThis paper describes a wearable augmented reality system with an IrDA device and a passometer. To realize
augmented reality systems, the position and orientation of user’s viewpoint should be obtained in real time foraligning the real and virtual coordinate systems. In the proposed system, the orientation of user’s viewpoint ismeasured by an inertial sensor attached to the user’s glasses, and the position is measured by using an IrDA
device and a passometer. First, the user’s position is speciﬁed exactly when the user comes into the infrared ray
range of the IrDA markers which are set up to the appointed points. When the user goes out of the infrared ray
range, the user’s position is estimated by using a passometer. The passometer is constructed of an electronic
compass and acceleration sensors. The former can detect the user’s walking direction. The latter can count how
many steps the user walks. These data and the user’s pace make it possible to estimate the user’s position in
the neighborhood of the IrDA markers. We have developed a navigation system based on using the techniquesabove and have proven the feasibility of the system with experiments.
Keywords: Wearable Computer, Augmented Reality, IrDA, Passometer

1. INTRODUCTION
Since computers have made remarkable progress in resent years, a wearable computer can be realized.1 At
the same time, Augmented Reality (AR) which merges the real and virtual worlds has received a great deal ofattention as a new method for displaying information.2–4 If we construct an annotation overlay system for realscene using an AR technique with a wearable computer, it can be applied for many applications.5–9 Figure
1 shows an example of the annotation overlay system which imposes annotation on the real scene using AR
techniques with wearable computers. To realize AR systems, the position and orientation of user’s viewpoint areneeded for acquiring a relationship between the real and virtual coordinate systems. Usually, Global Positioning
System (GPS) is used to measure the user’s position outdoors.10 In AR systems which use GPS, a diﬀerential

Annotation

Figure 1. An example of a wearable AR application.
Further author information: (Send correspondence to R.T.)
R.T.: E-mail: ryuhei-t@is.aist-nara.ac.jp, Telephone: +81 743 72 5296
M.K.: E-mail: kanbara@is.aist-nara.ac.jp, Telephone: +81 743 72 5292
N.Y.: E-mail: yokoya@is.aist-nara.ac.jp, Telephone: +81 743 72 5290

GPS11 or a real time kinematic GPS is used to measure the user’s position accurately. However, since both of themneed radio communication to correct GPS data, the hardware conﬁguration of the system will be complicated.
Moreover, GPS cannot be used indoors. Therefore, if we construct a wearable AR system which can be usedboth indoors and outdoors, we must combine other positioning sensors which can be used indoors such as anelectro-magnetic sensor.12 Accordingly, the hardware conﬁguration of the system will be more complicated andthe system needs to switch the sensors between indoors and outdoors.
The purpose of the present work is to construct a wearable annotation overlay system which can be usedboth indoors and outdoors seamlessly. We have already proposed the wearable annotation overlay system using
RFID tags which can be used at appointed points both of indoors and outdoors.13 In this system, RFID tags are
set up to appointed points, and the system can identify the user’s position reading the RFID tag with a REID
tag reader equipped by the user. However, the system needs the user’s action bringing an RFID tag reader to
an REID tag to acquire the user’s position. Using an IrDA device instead of RFID tags, any special action but
walking close to an IrDA marker is not required for the user.
In this paper, we propose a wearable annotation overlay system using an IrDA device and a passometer.
First, the proposed system speciﬁes the user’s position exactly at appointed points where IrDA markers are set
up to. The IrDA marker always send a beacon including its position ID. The IrDA receiver which the user
equips receives the beacon and the system speciﬁes the user’s position when the user is in the infrared ray rangeof the IrDA marker. When the user recedes from the IrDA marker, the user’s position is estimated by using a
passometer which includes an electronic compass and acceleration sensors. The former can detect which direction
the user walks and the latter can count how many steps the user walks. Moreover, an inertial sensor which is
attached to the headset detects the orientation of the user’s viewpoint. The system generates annotation overlay
images from information such as the position, the orientation, and the database of annotations which is held ina wearable computer, and shows annotation overlay images to the user through the display device attached tothe headset.
This paper is structured as follows. Section 2 describes the proposed wearable AR system using the IrDA
device and the passometer. In Section 3, experimental results with a prototype system are described. Finally,
Section 4 describes summary and future work.

2. WEARABLE AUGMENTED REALITY SYSTEM
AR systems need three kind of information to overlay annotations on the real world in the correct position: aposition and an orientation of the user’s viewpoint, and an annotation database including annotation information.
The proposed system acquires the user’s position using an IrDA device and a passometer. The system gets theorientation of the user’s viewpoint using the inertial sensor attached to the user’s headset. The annotation
database is stored in a wearable computer. In Section 2.1, the hardware conﬁguration of the proposed system is
described. Section 2.2 describes how the system measures the user’s position in detail. Section 2.3 describes themethod of generating annotation overlay images.

2.1. Hardware Configuration of Wearable Augmented Reality System
Figure 2 shows the hardware conﬁguration and data ﬂow of the proposed wearable augmented reality system. The
user equips four sensors, a notebook PC and a display device. Four sensors obtain the position and orientation of
the user’s viewpoint and the real scene image. These data are sent to a notebook PC. The notebook PC generates
annotation overlay images from these data and a database of annotations which is stored in the notebook PC.
The notebook PC sends annotation overlay images to a display device which is attached to the user’s headset.
The user can see it through the display device. Components of the proposed system are described in more detailin the following.
Sensors The user equips following four sensors.
Inertial sensor (Intersense: InterTrax2) It is attached to the user’s glasses and measures the orientationof the user’s viewpoint. It can transmit data at 256Hz to a computer through USB connection.
Electric power is supplied from the notebook PC through USB connection.

Inertial Sensor

INTERTRAX2(INTERSENSE)

Display Device

User’s Orientation

CameraClip On Display(Micro Optical)

Dragonfly (Point
Grey Research)
Real Scene Image

IrDA Device
Annotation
Overlay Image
Computer

Passometer

3D Motion Sensor(NEC TOKIN)
User’s Position

Database of
Annotations
INSPIRON 8100
(DELL)

Figure 2. Hardware configuration and data flow of the proposed system.

Camera (Point Grey Research: Dragonﬂy) The camera is attached to the user’s glasses and captures thereal scene image from the user’s viewpoint. It can capture RGB 24bit color image of 640 × 480 pixels.
It can transmit the image 30 frames per second to a computer through IEEE1394 connection. Electric
power is supplied from the notebook PC through IEEE1394 connection.
IrDA device (Original) An IrDA receiver is attached to the user’s bag, and IrDA markers are set up
at appointed points as shown in Figure 3 (c). The IrDA marker can send a beacon as far as about
5 meters. The IrDA markers always send the unique position ID. The IrDA receiver transmits thedata through serial connection. Electric power of the receiver and the marker is supplied from a 9V
battery.
Passometer (NEC TOKIN: 3D motion sensor) A 3D motion sensor can measure pitch, roll, and yaw. It
can also measure accelerations in the horizontal directions. It detects what direction and how many
steps the user walks from the point where an IrDA marker is set up to. It can transmit data at 125Hz
to a computer through USB connection. Power is also supplied from the notebook PC through USB
connection.
Computer (DELL: Inspiron8100, PentiumIII 1.2GHz, 512Mbytes memory) The notebook PC has a databaseof annotation information for generating annotation overlay images.
Display device (MicroOptical: Clip On Display) A small display device is attached to glasses as shown in
Figure 2. The display device present a 640 × 480 color image to the user. Power is supplied from a control
box which is carried in the user’s shoulder bag. The user can see annotation overlay images through it;that is, the proposed system is a video see-through augmented reality system.

(a) IrDA receiver.

(c) IrDA marker set up at theceiling of the door sector.

(b) IrDA marker.
Figure 3. IrDA receiver and marker.
IrDA marker
IrDA marker

Identifying position
with the IrDA
Estimating position
with the passometer

Figure 4. Method for measuring the user’s position.

2.2. Measurement of User’s Position
The proposed system measures the user’s position indoors and outdoors using an IrDA device and a passometeras shown in Figure 4. The system identiﬁes the user’s accurate position using an IrDA device when the user isin the infrared ray range of IrDA markers. If the user goes out of the infrared ray range, the user’s position isestimated by using a passometer. An accumulative error of the passometer is corrected when the user comes
into the infrared ray range again. In the following, Section 2.2.1 describes the method for identifying the user’s
position using an IrDA device. Section 2.2.2 describes how the system estimates the user’s position using a
passometer.
2.2.1. Identification of User’s Position Using an IrDA device
To specify the user’s position at appointed points exactly, the system uses an IrDA device. Figure 3(a) shows an
IrDA receiver, and (b) shows an IrDA marker. The IrDA markers are set up to appointed points as a positioning
infrastructure. Figure 3(c) shows the IrDA marker set up at the ceiling of the door sector. IrDA markers always
send their unique position IDs. On the other hand, the IrDA receiver is set up to the user’s shoulder bag asshown in Figure 2. When the user comes into the infrared ray range of an IrDA marker, the user’s position is
identiﬁed by the position ID which is received from the IrDA marker. If it is diﬃcult to set up IrDA markers
constantly, IrDA markers should be set up only at the points where the user needs guiding information, such asat branch points or in front of guide boards.
2.2.2. Estimation of User’s Position Using a Passometer
When the user goes out of infrared ray range, the user’s position is estimated by the passometer. The passometerconsists of a compass and acceleration sensors. The former can detect which direction the user walks toward.
The latter can count how many steps the user walks.14, 15 These data and the user’s pace make it possible toestimate the user’s position in the neighborhood of IrDA markers.
The proposed system regards the direction of the user’s waist as the walking direction. When the user walks,
the periodical pattern of the user’s waist acceleration in the back-and-forth direction appears. The acceleration
sensor of a passometer detects the pattern, and counts the number of user’s steps. Figure 5 shows the typicalAcceleration [G]

1.2

0.8

]
[G 0.4
n
o
it
ar
e
l
e
c
c
0
a

User’s Waist
Acceleration in the
Back-and-forthDirection
1

Number of
User’s Steps

-0.4

-0.8

0

2000

4000

time[msec]

6000

Time [msec]

8000

Figure 5. Relationship between the number of user’s steps and the user’s waist acceleration in the back-and-forth
direction.

Pointing Arrow
Indicators

Guiding Annotation
Detail Information

Figure 6. An example of annotation overlay image.

relationship between the number of user’s steps and the user’s waist acceleration in the back-and-forth direction.
In Figure 5, a solid line shows the user’s waist acceleration in the back-and-forth direction. A pair of ups anddowns of dotted line means a user walk of one step. Note that the user’s steps are manually measured. The
graph indicates that the user’s waist acceleration has a pair of a peak and a valley when the user walks one step.
From Figure 5, the system can count the user’s steps by counting the number of pairs of a peak and a valley inthe user’s waist acceleration pattern.

2.3. Generation of Annotation Overlay Image
As mentioned above, the system needs the database of annotations to generates annotation overlay images thatare presented to the user. The database of annotations is consisted of the following data:• Image ﬁles of guiding annotations and detail annotations,• Directions of pointing arrows,• Positions of annotations and IrDA markers.
The system selects appropriate annotations from this database according to the user’s estimated position in thereal world.
The proposed system generates an annotation overlay image from the user’s position and the orientation of
the user’s viewpoint, and the database of annotations. Figure 6 shows an example of annotation overlay image.
In our system, Annotations include four kinds of information: indicators, pointing arrows, guiding annotations,and detail annotations. Indicators show users a state of the system. Pointing arrows are used to link guiding
annotations to the objects in the real scene. The guiding annotations are used for showing the names of the real

objects to the user. The pointing arrow and the guiding annotation are placed in the appointed points in thereal scene. The closer the user walks to a guiding annotation, the larger it is shown to the user. Therefore, theuser can intuitively link guiding annotations to real objects. Detail annotations show the user detail information
about the real object. A detail annotation appears on the lower right side of an image while the user notices thecorresponding guiding annotations. The system recognizes that the user notices a guiding annotation when it ison the center of an annotation overlay image.

3. EXPERIMENTS AND RESULTS
3.1. Experiments
We have carried out some experiments using the proposed wearable augmented reality system. We have developeda navigation system which works in our campus where a number of IrDA markers are placed at appointed points.
Sections 3.1.1 and 3.1.2 describe indoor and outdoor navigation experiments, respectively.
3.1.1. Navigation in Indoor Environment
Figure 7 illustrates an indoor experimental environment. In Figure 7, the points numbered P1, P2, and P3 mean
the positions where the IrDA marker is set up. The user walked along the dotted arrows in Figure 7 from P1
to P3. Figure 8(a),...,(f) are annotation overlay images presented to the user when the user was at the point(a),...,(f) in Figure 7, and the user’s orientation was along the solid arrows at the point (a),...,(f) in Figure 7,
respectively. Assuming the position P1 as an origin, a horizontal plane as the x-y plane, and the z axis as avertical direction as shown in Figure 7, the coordinates of the points P2, P3, and A, ..., G are mentioned in
Figure 7 (the unit of them is millimeter). Figure 8(a), (c), and (e) are the annotation overlay images when thesystem identiﬁed the user’s position using IrDA devices and (b), (d), and (f) are ones when the system estimatedthe user’s position using the passometer. The point (b) in Figure 7 is the position six steps away from the point(a), the point (d) is the position six steps away from the points (c), and the point (f) is the position four steps
away from the point (e), respectively. Through this experiment, even if the user moves some steps in the indoor
environment, the user can obtain annotations correctly overlaid on real scene images in real time.
3.1.2. Navigation in Outdoor Environment
Figure 9 illustrates an outdoor experimental environment. In Figure 9, the points numbered P1, P2, and P3
mean the positions where the IrDA marker is set up. The user walked along the dotted arrows in Figure 9
from P1 to P3. Figure 10(a),...,(f) show annotation overlay images when the user was at the point (a),...,(f) in
Figure 9, and the user’s orientation was along the solid arrows at the point (a),...,(f) in Figure 9, respectively.
Assuming the position P1 as an origin, a horizontal plane as the x-y plane, and the z axis as a vertical direction
as shown in Figure 9, the coordinates of the points P2, P3, and A, ..., G are mentioned in Figure 9 (the unitof them is millimeter). Figure 10(a), (c), and (e) are the annotation overlay images when the system identiﬁed
the user’s position using IrDA devices and (b), (d), and (f) in Figure 9 are ones when the system estimated theuser’s position using the passometer. The points (b), (d), and (f) are the positions eight steps away from thepoints (a), (c), and (e), respectively. Through this experiment, even if the user moves some steps in the outdoor
environment, the user can obtain the images overlaid annotations correctly.

3.2. Discussion
The annotation overlay image can be presented to the user at 25fps. The weight of the user’s headset is about
200g and the user’s shoulder bag which includes the notebook PC and batteries is about 5kg. The system canrun with the battery more than two hours. Experimental results have shown the feasibility of the proposedsystem both indoors and outdoors.
As shown in (b) and (d) of Figure 10, when there exist occlusions among objects in the scene, the system mayoverlay wrong annotations. Therefore, the user cannot intuitively link annotations to the correct real objects.
Using a simple 3D model of the real scene, we could represent annotations with correct occlusion. This should
further be investigated.

E(5450, 26500, 800)

P1(0, 0, 0)
YZ

P3(2450, 16500, 0)

(a)

(e)

X

(f)

G(2450, 31500, 2000)

IrDA marker(b)F(1450, 28500, 800)
P2(5750, 5750, 0)(d)Guiding annotationand pointing arrow

D(4450, 17750, 1800)(c)
A(8000, 8000, 2000)

C(7050, 19750, 1800)
B(8000, 5000, 2000)

Figure 7. An indoor experimental environment.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 8. Examples of annotation overlay images.

4. SUMMARY
This paper has described a wearable augmented reality system which can be used in indoor and outdoor environments seamlessly using an IrDA device and a passometer. We have shown the feasibility of the proposedsystem demonstrating with experiments in both indoor and outdoor environments. In the future, we should use
wireless LAN system to update annotation database automatically and solve the occlusion problem in presenting
annotations.

ACKNOWLEDGMENTS
This research is partially supported by Core Research for Evolutional Science and Technology (CREST) Program“Advanced Media Technology for Everyday Living” of Japan Science and Technology Corporation (JST) andalso by the Grant-in-Aid for Scientiﬁc Research from the Ministry of Education, Culture, Sports, Science andTechnology.

C(-27000, 15000, 3000)
B(-22000, 20000, 3000)

A(-22000, -2000, 2000)

IrDA marker(d)P2(-10000, 0, 0)(c)(e) (f)Guiding annotationand pointing arrow

P3(-10000, 10000, 0)
D(-12000, 40000, 5000)

(b)
(a)
Y
P1(0, 0, 0)Z

X

Figure 9. An outdoor experimental environment.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 10. Examples the annotation overlay images.

REFERENCES
1. S. Mann: “Wearable Computing: A First Step Toward Personal Imaging, ” IEEE Computer, Vol. 30, No.
2, pp. 25-32, 2002.
2. R. Azuma: “A Survey of Augmented Reality, ” Presence, Vol. 6, No. 4, pp. 355-385, 1997.
3. M. Kanbara, T. Okuma, H. Takemura and N. Yokoya: “A Stereoscopic Video See-through Augmented
Reality System Based on Real-time Vision-based Registration, ” Proc. IEEE Int. Conf. on Virtual Reality
2000, pp. 255-262, 2000.
4. S. Julier, M. Lanzagorta, Y. Baillot, L. Rosenblum, S. Feiner, T. Holler, and S. Sestito: “Information
Filtering for Mobile Augmented Reality, ” Proc. IEEE/ACM 1st Int. Symp. on Augmented Reality, pp.
3-11, 2000.

5. S. Julier, M. Lanzagorta, Y. Baillot, L. Rosenblum, S. Feiner, T. Höller, and S. Sestito: “Personal Guidance
System for the Visually Impaired, ” Proc. Int. Conf. on Assistive Technologies, pp. 85-90, 1994.
6. M. Kourogi, T. Kurata, and K. Sakaue: “A Panorama-based Method of Personal Positioning and Orientation
and Its Real-time Applications for Wearable Computers, ” Proc. 5th Int. Symp. on Wearable Computers,
pp. 107-114, 2001.
7. K. Satoh, K. Hara, M. Anabuki, H.Yamamoto, and H.Tamura: “TOWNWEAR: An Outdoor Wearable MR
System with High-precision Registration, ” Proc. 2nd Int. Symp. on Mixed Reality, pp. 210-211, 2001.
8. M. Billinghurst, S. Weghorst, and T. Furness : “Wearable Computers for Three Dimensional CSCW, ”
Proc. 1st Int. Symp. on Wearable Computers, pp. 39-46, 1997.
9. P. Daehne, J. Karigiannis: “ARCHEOGUIDE: System Architectureof a Mobile Outdoor Augmented Reality
System, ” Proc. 1st Int. Symp. on Mixed and Augmented Reality, pp. 263-264, 2002.
10. H. Petrie, V. Johnson, T. Strothotte, A. Raab, S. Fritz, and R. Michel: “MoBIC: Designing a Travel Aid
for Blind and Elderly People, ” Jour. of Navigation, Vol. 49, No. 1, pp. 44-52, 1996.
11. S. Feiner, B. MacIntyre, T. Höller, and A. Webster: “A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment, ” Proc. 1st Int. Symp. on Wearable Computers, pp. 74-81, 1997.
12. A. State, G. Horita, D. Chen, W. Garrett, and M. Livingston: “Superior Augmented Reality Registration
by Integrating Landmark Tracking and Magnetic Tracking, ” Proc. SIGGRAPH’96, pp. 429-438, 1996.
13. R. Tenmoku, M. Kanbara, N. Yokoya, and H. Takemura: “Annotation Overlay with a Wearable Computer
Using Augmented Reality, ” Proc. 1st CREST Workshop on Advanced Computing and Communicating
Techniques for Wearable Information Playing, pp. 27-32, 2002.
14. S. W. Lee and K. Mase: “A Personal Indoor Navigation Syatem Using Wearable Sensors, ” Proc. Int. Symp.
on Mixed Reality, pp. 147-148, 2001.
15. S. W. Lee and K. Mase: “Incremental Motion-based Location Recognition, ” Proc. 5th Int. Symp. on
Wearable Computers, pp. 123-130, 2001.