Shared Annotation Database for
Networked Wearable Augmented Reality System
Koji Makita, Masayuki Kanbara, and Naokazu Yokoya
Graduate School of Information Science,Nara Institute of Science and Technology,
8916-5 Takayama, Ikoma, Nara, 630-0192 Japanfkoji-ma, kanbara, yokoyag@is.naist.jp

Abstract. This paper describes a database of annotation informationfor augmented reality (AR) on wearable computers. With the advanceof computers, AR systems using wearable computers have received agreat deal of attention. To overlay annotations on the real scene image,a user's computer needs to hold annotation information. The purposeof this paper is to construct a networked database system of annotation
information for wearable AR systems. The proposed system provides
users with annotation information from a server via a wireless network
so that the wearable computers do not need to hold it in advance andinformation providers can easily update and add the database with a
web browser. In experiments, the user's position-based annotations havebeen proven to be shown to the user e ectively.
1 Introduction

Since computers have made a remarkable progress in resent years, a wearable
computer can be realized[1]. At the same time, the augmented reality (AR)technique which merges the real and virtual worlds has received a great deal ofattention as a new method for displaying location-based information in the realworld[2-4]. Therefore, AR systems using wearable computers will open up a newvista to the next generation wearable computing[5,6]. Figure 1 shows an exampleof annotation-overlay using a wearable AR system. Since the wearable AR system
can intuitively display information to user on the real scene as shown in Figure
1, it can be applied to a number of di erent elds[5,7-12]. To realize a wearable
AR system, the position and orientation of user's viewpoint and annotation
information are needed. The position and orientation of user's viewpoint areneeded for acquiring the relationship between the real and virtual coordinate
systems. Many researchers have proposed a number of di erent methods formeasurement of the position and orientation of user's viewpoint with some kindsof sensors[6,7,13-15]. To overlay annotations on the real scene image, a user's
computer needs to hold user's location-based information. Up to this time, sincea database of annotation information is usually held in the wearable computerin advance, it is dicult for the database of annotation information to be easily
updated or added by information providers (including normal PC users andwearable PC users).
The purpose of the present work is to construct a shared database systemof annotation information for wearable AR systems. To realize the system, we2

Koji Makita et al.
Annotation
Detailed
information
Wearable
PC user

User’
User’s viewFig. 1. An example of annotating a real scene.
install a database server which can be accessed with a wireless network. Thedatabase is shared by multiple users of wearable AR systems and information
providers. Thereby, the information providers can provide users with the newest
annotation information by updating the annotation database. On the other hand,users of AR systems can obtain the newest annotations without holding theannotation information in advance. The information providers can eciently
update and add the database with a web browser. Moreover, a wearable AR
user can also edit the database of annotation information easily because theuser's position acquired by a positioning sensor is used to determine the user'sposition on the map.
This paper is structured as follows. Section 2 describes the shared databasesystem of annotation information using a wireless network. In Section 3, experimental results with a prototype system are described. Finally, Section 4 givessummary and future work.
2 Shared Database of Annotation Information

Figure 2 shows an outline of shared database system of annotation information.
In this study, the database is shared via a wireless network. The database ofannotation information is stored in the server and is shared by multiple users ofwearable AR systems and information providers. Consequently, users of wearable
AR systems can obtain annotation information at anytime via a wireless networkand can see the newest annotation overlay images without holding the databaseof annotation information in advance. On the other hand, information providers
can provide eciently the newest annotation information for users of wearable
AR systems by updating and adding the database with a web browser. In Section
2.1, the composition of the database of annotation information is described.
Section 2.2 describes how to update the database with a web browser. Section
2.3 describes how the user obtains annotation information.
2.1 Composition of annotation database

The database contains some kinds of location-based contents. Each annotation iscomposed of a pair of contents(name and detail) and their positions. Componentsof the annotation information are described in detail in the following.

Shared Annotation Database
Pos

Info: Wired network

WearablePC

Annotation
database

DesktopPC

Position ofannotation＋Wearable
PC
MobilePC
Information
providersInfo•Image•Movie•SoundInfo3

etc
Server・・・・・・: Wireless network
Info: Annotation information

Pos: User’s position

Wearable
PC
Users ofwearable
AR systemsFig. 2. Shared database of annotation information.
Position: Three-dimensional position of an annotation in the real world. Threeparameters (latitude, longitude, height) are stored in the database.
Name: A name of the object which is overlaid in the real scene as annotation
information.
Detail: Detailed information about the object. When user's eyes are xed on
the object, the detail about the object is shown in the lower part of the user'sview.
2.2 Updating the shared database

The annotation information can be corrected, added and deleted by information
providers with a web browser. An interface for information providers to update
the database is a web browser as shown in Figure 3. Information providers caneasily update the database by accessing a prepared web page and by transmitting the data of annotation information. The annotation updating procedure is
described below.
1. Speci cation of position
Information providers can zoom in and out to maps (Figure 3: C, D) using
buttons (Figure 3: A). Besides, the providers can move by clicking any pointon the map. In this way, the providers can specify the position of a newannotation to be added. It should be noted that position parameters suchas latitude, longitude and height are automatically determined based on thespeci ed position on the map.
2. Input of name
Information providers input the object name to web page. The name is sent
to the server and the picture of annotation is automatically generated in theserver.
3. Input of details
Using the same method as in the input of the name, information providers
send details of objects. The providers also can send a picture, a sound leand a movie le as details.

4Koji Makita et al.

Fig. 3. Input form of annotation information.
The providers can eciently send the newest annotation information using
a web browser. For that reason, a user of wearable AR systems can also update
the database. In this case, the server shows the user a map of his neighborhood
according to the user's position acquired by positioning sensors. Since the user
is able to update the database, the user can immediately correct the position
error of annotations by con rming the overlaid image.
2.3 Getting annotation information

In this work, a database server is prepared assuming that the user's wearable
computer can access the database via a wireless network. Annotations to bepresented to the user are determined based on the user's current position. First,
the user's position is measured by some sensors (positioning infrastructures,GPS, and so on) which are equipped by the user. The user's wearable computerthen obtains proper annotation information based on the measured user's position. The server automatically decides which annotation should be provided.
Consequently, the user's wearable system can obtain the newest annotation information at anytime. The user's wearable system obtains the newest annotation
information periodically when the user moves for a xed distance or a xed time
is passed.
3 Experiments

We have carried out some experiments using the proposed database of annotation
information in a server in our campus where users of wearable AR systems canuse a wireless local area network. Figure 4 illustrates a hardware con gurationof a wearable augmented reality system which is used in these experiments.
The user equips some positioning sensors, a notebook PC and a display device.
Three sensors described later can obtain the position and orientation of the user's
viewpoint and the real scene image[12]. These data are sent to the notebook PC.
The notebook PC obtains annotation information from the database server via
a wireless local area network. The notebook PC sends annotation overlay images

Shared Annotation Database
CCDcamera

5

Display
DeviceQcam(Logicool)•

Inertial Sensor

INTERTRAX2(INTERSENSE)•

Clip On Display(Micro Optical)•

Wearable Computer
Positioning Sensor

DRM(Point ResearchCorporation)
ＧＰＳ•

InterLink
MP-XP7310(Victor)

3D Motion Sensor

Fig. 4. Hardware con guration of wearable augmented reality system.
to a display device attached to the user's headset. The user can see it throughthe display device. Components of the system are described in more detail below.
Sensors The user equips the following three sensors. Electric power is supplied
from the notebook PC or a 9V battery. The data is transmitted to thecomputer through USB or serial connection.
Inertial sensor (Intersense: InterTrax2 ) The inertial sensor is attached tothe user's headset and measures the orientation of the user's viewpoint.
The inertial sensor can obtain data at 256Hz.
Camera (Logicool: Qcam) The camera is attached to the user's headset andcaptures the real scene image from the user's viewpoint. It can capture
a color image of 640 2 480 pixels at 30fps.
Positioning sensor (Point Research Corporation: Dead Reckoning Module) The positioning sensor can measure the latitude and longitude. It
can also measure accelerations in the horizontal direction.
Computer (DELL: Inspiron8100, PentiumIII 1.2GHz, memory 512Mbytes) The
computer is carried in the user's shoulder bag. It can use a wireless local area
network with a network card.
Display device (MicroOptical: Clip On Display) The display device is a video
see-through device. It is attached to user's headset. It can present a 640 2
480 color image to the user.
In this experiment, we have developed a database of annotation informationin the server (CPU Pentium4 2.0GHz, memory 512Mbytes) in our laboratory.
Figure 5 illustrates the experimental environment. In Figure 5, the points 1,...,5indicate the positions where the annotations exist. The user obtained the annotation information and looked around at the points A,...,D. The criterion concerning which annotations should be obtained is based on the distance between eachannotation and the user's position. In this experiment, we set empirically thecriterion at 70 meters. Besides, in order to check that the database is correctly
updated, the user moving in our campus updated the annotation information
about the "Cafeteria" with a web browser.

6Koji Makita et al.
Database Serverin Campus

1D（h）

3

2

（i）
（g）

C

（f）

（e） （d）

B（c）

A （a）

4（b）

WirelessLAN

5User

1 ,…, 5：Position of Annotations

A ,…, D：Position of UserFig. 5. Environment of the outdoor experiment.
Figures 6 and 7 show the annotation overlay images. Figure 6 shows theannotation overlay images when the user was at the points A,...,D in Figure 5 andthe user's orientation was along the arrows (a),...,(i) in Figure 5, respectively. As
shown in Figure 6 (a), the annotation of "Information Science" is overlaid on thefront of the building, so that the user can recognize the annotation information
intuitively. The same conclusion is obtained from Figure 6 (b),...,(i). Thereby, wehave con rmed that the user can obtained and perceive the shared annotation
information intuitively. Figure 7 shows the annotation overlay images before
and after updating the annotation information. The annotation information inFigure 7 (a) was changed to the new one in Figure 7 (b) automatically when axed time is passed. We have con rmed that the annotation information can beupdated by editing the shared database of annotation information.
Through the experiments, the user has successfully obtained the locationbased annotation information according to the user's position. Simultaneously,
the shared database can be easily and eciently updated and can provide theuser with the newest annotation information in real-time.
4 SummaryThis paper has described a database of annotation information for a wearable
augmented reality system which is shared by multiple users via network and iseciently updated with a web browser. In other words, proposed is a networked
wearable augmented reality system. We have shown the feasibility of the proposed database through the demonstration with experiments in our campus. In
the future, we should conduct experiments in a wider area and use other kindsof detailed location-based contents (movie, sound, and so on).
Acknowledgments This work was supported in part by Core Research for
Evolutional Science and Technology (CREST) Program "Advanced Media Technology for Everyday Living" of Japan Science and Technology Agency (JST).

Shared Annotation Database(a)(b)(c)(d)(e)

(f)

7(g)
(h)
(i)
Fig. 6. Overlay images at the points A, B, C, and D((a),(b) and (c) at A; (d)and (e) at B; (f) and (g) at C; (h) and (i) at D).

(a) before updating(b) after updating
Fig. 7. Example of updating annotation information.

8Koji Makita et al.

References

1. S. Mann: \Wearable Computing: A First Step Toward Personal Imaging," IEEEComputer, Vol. 30, No. 2, 2002.
2. R. Azuma: \A Survey of Augmented Reality," Presence, Vol. 6, No. 4, pp. 355-385,1997.
3. M. Kanbara, T. Okuma, H. Takemura and N.Yokoya: \A Stereoscopic Video Seethrough Augmented Reality System Based on Real-time Vision-based Registration," Proc. IEEE Int. Conf. on Virtual Reality 2000, pp. 255-262, 2000.
4. S. Julier, M. Lanzagorta, Y. Baillot, L. Rosenblum, S. Feiner, T. Holler, and S. Sestito: \Information Filtering for Mobile Augmented Reality," Proc. 1st IEEE/ACM
Int. Symp. on Augmented Reality, pp. 3-11, 2000.
5. K. Satoh, K. Hara, M. Anabuki, H.Yamamoto, and H.Tamura: \TOWNWEAR:
An Outdoor Wearable MR System with High-precision Registration," Proc. 2ndInt. Symp. on Mixed Reality, pp. 210-211, 2001.
6. R. Tenmoku, M. Kanbara, and N. Yokoya: \A Wearable Augmented Reality System
Using an IrDA Device and a Passometer," Proc. SPIE, Vol. 5006, pp. 478-486, 2003.
7. J. Loomis, R. Golledge, R. Klatzky, J.Speigle, and J. Tietz: \Personal Guidance
System for the VisuallyImpaired," Proc. Int. Conf. on Assistive Technologies, pp.
85-90, 1994.
8. M. Kourogi, T. Kurata, and K. Sakaue: \A Panorama-based Method of Personal
Positioning and Orientation and Its Real-time Applications for Wearable Computers," Proc. 5th IEEE Int. Symp. on Wearable Computers, pp. 107-114, 2001.
9. M. Billinghurst, S. Weghorst, and T. Furness III: \Wearable Computers for Three
Dimensional CSCW," Proc. 1st IEEE Int. Symp. on Wearable Computers, pp.
39-46, 1997.
10. T. Okuma, T. Kurata, and K. Sakaue: \Fiducial-less 3-D Object Tracking in AR
Systems Based on the Integration of Top-down and Bottom-up Approaches andAutomatic Database Addition," Proc. 2nd IEEE/ACM Int. Symp. on Mixed andAugmented Reality, pp.342-343, 2003.
11. D. Stricker, J. Karigiannis, I. T. Christou, T. Gleue, and N. Ioannidis: \Augmented
Reality for Visitors of Cultual Heritage Sites," Proc. Int. Conf. on Cultual andScienti c Aspects of Experimental Media Spaces, pp. 89-93, 2001.
12. R. Tenmoku, M. Kanbara, and N. Yokoya: \A wearable Augmented Reality System
Using Positioning Infrastructures and a Pedometer", Proc. 7th IEEE Int. Symp.
on Wearable Computers, pp. 110-117, 2003.
13. A. State, G. Horita, D. Chen, W. Garrett, and M. Livingston: \Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic
Tracking," Proc. SIGGRAPH'96, pp. 429-438, 1996.
14. H. Petrie, V. Johnson, T. Strothotte, A. Raab, S. Fritz, and R. Michel: \MoBIC:
Designing a Travel Aid for Blind and Elderly People," Jour. of Navigation, Vol.
49, No. 1, pp. 44-52, 1996.
15. S. Feiner, B. MacIntyre, T. Holler, and A. Webster: \A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment,"
Proc. 1st IEEE Int. Symp. on Wearable Computers, pp. 74-81, 1997.