Shared Database of Annotation Information
for Wearable Augmented Reality System
Koji Makitaa , Masayuki Kanbaraa and Naokazu Yokoyaa
a Graduate School of Information Science, Nara Institute of Science and Technology,
8916-5 Takayama, Ikoma, Nara, 630-0192 Japan

ABSTRACTThis paper describes a database of annotation information for augmented reality (AR) on wearable computers.
With the advance of computers, AR systems using wearable computers have received a great deal of attention
as a new method for displaying location-based information in real-time. To overlay annotations on the realscene image, a user's computer needs to hold annotation information. Up to this time, since a database ofannotation information is usually held in advance in the wearable computer, it is dicult for the database ofannotation information to be e ectively updated or added by providers of information. The purpose of this paper
is to construct a networked database system of annotation information for wearable AR systems. The proposedsystem provides users with annotation information from a server via a wireless network so that the wearable
computers do not need to hold it in advance and information providers can easily update and add the databasewith a web browser. In this study, we have developed a shared database of annotation information and have
proven the feasibility of the prototype system with preliminary experiments. In experiments, the orientation andthe position of user's viewpoint are measured by integrating several kinds of sensors. The user's position-based
annotations have been proven to be shown to the user automatically. Moreover, we have con rmed that thedatabase can be successfully updated and added by information providers with a web browser.
Keywords: Wearable Computer, Augmented Reality, Annotation Overlay, Networked Annotation Database,
Wireless Network

1. INTRODUCTION

Since computers have made a remarkable progress in resent years, a wearable computer can be realized1 . At thesame time, the augmented reality (AR) technique which merges the real and virtual worlds has received a great
deal of attention as a new method for displaying location-based information in the world204 . Therefore, ARsystems using wearable computers will open up a new vista to the next generation wearable computing5 6 . Figure
1 shows an example of annotation-overlay system using a wearable AR system. Since the wearable AR system
can intuitively display information to user on the real scene as shown in Figure 1, it can be applied to a numberof di erent elds5 7012 . To realize a wearable AR system, the position and orientation of user's viewpoint andannotation information are needed. The position and orientation of user's viewpoint are needed for acquiring
the relationship between the real and virtual coordinate systems. Many researchers have proposed methods formeasurement of the position and orientation of user's viewpoint with some kinds of sensors6 7 13015 . To overlay
annotations on the real scene image, a user's computer needs to hold user's location-based information. Up tothis time, since a database of annotation information is usually held in the wearable computer in advance, it isdicult for the database of annotation information to be updated or added by information providers (including
normal PC users and wearable PC users).
The purpose of the present work is to construct a shared database system of annotation information for
wearable AR systems. To realize the system, we install a database server which can be accessed with a wirelessnetwork. The database is shared by multiple users of wearable AR systems and information providers. Thereby,;;; ;Further author information: (Send correspondence to K.M.)
K.M.: E-mail: koji-ma@is.aist-nara.ac.jp, Telephone: +81 743 72 5296
M.K.: E-mail: kanbara@is.aist-nara.ac.jp, Telephone: +81 743 72 5292
N.Y.: E-mail: yokoya@is.aist-nara.ac.jp, Telephone: +81 743 72 5290

Figure 1. An example of annotation overlay system

the information providers can provide users with the newest annotation information by updating the annotation
database. On the other hand, users of AR systems can see the newest annotations without holding the annotation
information in advance. The information providers can eciently update and add the database with a webbrowser. Moreover, a wearable AR user can also edit the database of annotation information easily because theuser's position acquired by a positioning sensor is used to determine the user's position on the map.
This paper is structured as follows. Section 2 describes the shared database system for annotation information
using a wireless network. In Section 3, experimental results with a prototype system are described. Finally,
Section 4 describes summary and future work.

2. SHARED DATABASE OF ANNOTATION INFORMATION

Figure 2 shows an outline of shared database system of annotation information. In this study, the database isshared via a wireless network. The database of annotation information is stored in the server and is shared bymultiple users of wearable AR systems and information providers. Consequently, users of wearable AR systems
can obtain annotation information at anytime via a wireless network and can see the newest annotation overlay
images without holding the database of annotation information in advance. On the other hand, informationproviders can provide eciently the newest annotation information for users of wearable AR systems by updating
and adding the database with a web browser. In Section 2.1, the composition of a database of annotation
information is described. Section 2.2 describes the method updating the database with a web browser. Section
2.3 describes how the user obtains annotation information.

2.1. Composition of a database of annotation information

The database contains some kinds of location-based contents. Each annotation is constructed of a pair of contents
and their location. Components of the annotation information are described in detail in the following.

Position: Three-dimensional position of an annotation in the real world. Three parameters (latitude, longitude,height) are stored in the database.
Name: A name of the object which is overlaid in the real scene as annotation information.
Detail: Detailed information about the object. When user's eyes are xed on the object, the detail about theobject is shown at the lower of the user's screen.

Figure 2. Shared database of annotation information

2.2. Updating the shared database

The annotation information can be corrected, added and deleted by information providers with a web browser. An
interface for information providers to update the database is a web browser as shown in Figure 3. Information
providers can easily update the database by accessing a prepared web page and by transmitting the data ofannotation information. The annotation updating procedure is described in the following.
1. Speci cation of position
Information providers can zoom in and out to maps (Figure 3: C, D) using buttons (Figure 3: A) . Besides,
the providers can move by clicking any point on the map. In this way, the providers can specify the positionof a new annotation to be added. It should be noted that position parameters such as latitude, longitudeand height are automatically determined based on the speci ed position on the map.
2. Input of name
Information providers input the name of an object. The name is sent to the server and a picture of anannotation is automatically generated in the server.
3. Input of details
By the same method as in the input of the name, information providers send details of an object. The
providers also can send a picture, a sound le and a movie le as details.
The providers can eciently send the newest annotation information using a web browser. For that reason,
a user of wearable AR systems can also update the database. In this case, the server can show the user amap of his neighborhood according to the user's position acquired by positioning sensors. Since the user is able
to update the database, the user can immediately correct the position error of annotations by con rming theoverlaid image.

Figure 3. Input form of annotation information

2.3. Getting annotation information

In this work, a database server is prepared assuming that the user's wearable computer can access the database
via a wireless network. Annotations to be presented to the user are determined based on the user's current
position. First, the user's position is measured by some sensors (Positioning Infrastructures, GPS, and so on)which are equipped by the user. The user's wearable computer then obtains proper annotation information
based on the measured user's position. The server automatically decides which annotation should be provided.
Consequently, the user's wearable system can obtain the newest annotation information at anytime. The user'swearable system obtains the newest annotation information periodically when the user moves for a xed distanceor a xed time is passed.

3. EXPERIMENTS

We have carried out some experiments using the proposed database of annotation information. We have developed
a shared database of annotation in a server in our campus where users of wearable AR systems can use a wireless
local area network. Figure 4 illustrates a hardware con guration of a wearable augmented reality system which
is used in these experiments. The user equips some positioning sensors, a notebook PC and a display device.
Three sensors can obtain the position and orientation of the user's viewpoint and the real scene image. These
data are sent to a notebook PC. The notebook PC obtains annotation information from the database server via
a wireless local area network. The notebook PC sends annotation overlay images to a display device attachedto the user's headset. The user can see it through the display device. Components of the system are describedin more detail in the following.

Sensors The user equips the following three sensors. Electric power is supplied from the notebook PC or a 9Vbattery. The data is transmitted to the computer through USB or serial connection.
Inertial sensor (Intersense: InterTrax2) An inertial sensor is attached to the user's headset and measures
the orientation of the user's viewpoint. The inertial sensor can obtain data at 256Hz.
Camera (Logicool: Qcam) A camera is attached to the user's headset and captures the real scene image
from the user's viewpoint. It can capture a color image of 640 2 480 pixels at 30fps.
Positioning sensor (Point Research Corporation: Dead Reckoning Module) A positioning sensor canmeasure the latitude and longitude. It can also measure accelerations in the horizontal direction.

Figure 4. Hardware con guration of wearable augmented reality system
Table 1. List of sample annotation information

Computer (DELL: Inspiron8100, PentiumIII 1.2GHz, memory 512Mbytes) A computer is carried in the user's

shoulder bag. It can use a wireless local area network with a network card.
Display device (MicroOptical: Clip On Display) A display device is a video see-through device. It is attached
to user's headset. It can present a 640 2 480 color image to the user.

Figure 5. Environment of the outdoor experiment

In this experiment, we have developed a database of annotation information in the server (CPU Pentium42.0GHz, memory 512Mbytes) in our laboratory. Table 1 shows the list of sample annotation information. Figure 5
illustrates the experimental environment. In Figure 5, the points ➀,...,➄ mean the positions where the annotations
exist. The user obtained the annotation information and looks round at the points A,...,D. A criterion concerningwhich annotations should be obtained is based on the distance between each annotation and the user's position.
In this experiment, we set empirically the criterion at 70 meters. Besides, in order to check that the database iscorrectly updated, the user moving in our campus updated the annotation information about "cafeteria" with a
web browser.
Figure 6 and Figure 7 show the annotation overlay images. Figure 6 shows the annotation overlay images
when the user was at the points A,...,D in Figure 5 and the user's orientation was along the arrows (a),...,(i)in Figure 5, respectively. As shown in Figure 6 (a), the annotation of \information science" is overlaid on afront of building, so that the user can recognize the annotation information intuitively. The same conclusion isobtained from Figure 6 (b),...,(i). Thereby, we have con rmed that the user can obtained and perceive the sharedannotation information intuitively. Figure 7 shows the annotation overlay images before and after updating theannotation information. The annotation information in Figure 7 (a) was changed to the new one in Figure 7 (b)automatically when a xed time is passed. We have con rmed that the annotation information can be updatedby editing the shared database of annotation information.
Through experiments, the user has successfully obtained the location-based annotation information according
to the user's position. Simultaneously, the shared database can be easily and eciently updated and can provide
the user with the newest annotation information in real-time.

4. SUMMARY

This paper has described a database of annotation information for a wearable augmented reality system which isshared by multiple users via network and is eciently updated with a web browser. In other words, proposed is anetworked wearable augmented reality system. We have shown the feasibility of the proposed database throughthe demonstration with experiments in our campus. In the future, we should conduct experiments in a wider
area and use other kinds of detailed location-based contents (movie, sound, and so on).

ACKNOWLEDGMENTS

This work was supported in part by Core Research for Evolutional Science and Technology (CREST) Program\Advanced Media Technology for Everyday Living" of Japan Science and Technology Corporation (JST) andalso by the Grant-in-Aid for Scienti c Research from the Ministry of Education, Culture, Sports, Science andTechnology (MEXT).

REFERENCES

1. S. Mann: \Wearable Computing: A First Step Toward Personal Imaging," IEEE Computer, Vol.30, No. 2,2002.
2. R. Azuma: \A Survey of Augmented Reality," Presence, Vol. 6, No. 4, pp. 355-385, 1997.
3. M. Kanbara, T. Okuma, H. Takemura and N.Yokoya: \A Stereoscopic Video See-through Augmented
Reality System Based on Real-time Vision-based Registration," Proc. IEEE Int. Conf. on Virtual Reality
2000, pp. 255-262, 2000.
4. S. Julier, M. Lanzagorta, Y. Baillot, L. Rosenblum, S. Feiner, T. Holler, and S. Sestito: \Information
Filtering for Mobile Augmented Reality," Proc. IEEE/ACM 1st Int. Symp. on Augmented Reality, pp. 3-11,2000.
5. K. Satoh, K. Hara, M. Anabuki, H.Yamamoto, and H.Tamura: \TOWNWEAR: An Outdoor Wearable MR
System with High-precision Registration," Proc. 2nd Int. Symp. on Mixed Reality, pp. 210-211, 2001.
6. R. Tenmoku, M. Kanbara, and N. Yokoya: \A wearable augmented reality system using an IrDA device
and a passometer," Proc. SPIE Electronic Imaging, Vol. 5006, pp. 478-486, 2003.
7. J. Loomis, R. Golledge, R. Klatzky, J.Speigle, and J. Tietz: \Personal Guidance System for the VisuallyImpaired," Proc. Int. Conf. on Assistive Technologies, pp. 85-90, 1994.
8. M. Kourogi, T. Kurata, and K. Sakaue: \A Panorama-based Method of Personal Positioning and Orientationand Its Real-time Applications for Wearable Computers," Proc. 5th Int. Symp. on Wearable Computers,
pp. 107-114, 2001.
9. M. Billinghurst, S. Weghorst, and T. Furness III: \Wearable Computers for Three Dimensional CSCW,"
Proc. 1st Int. Symp. on Wearable Computers, pp. 39-46, 1997.
10. Takashi Okuma, Takeshi Kurata, and Katsuhiko Sakaue: \Fiducial-less 3-D Object Tracking in AR Systems
Based on the Integration of Top-down and Bottom-up Approaches and Automatic Database Addition," InProc. 2nd Int. Symp. on Mixed and Augmented Reality (ISMAR03) in Tokyo, Japan, pp.342-343, 2003.
11. D. Stricker, J. Karigiannis, I. T. Christou, T. Gleue, and N. Ioannidis: \Augmented Reality for Visitors ofCultual Heritage Sites," Proc. Int. Conf. on Cultual and Scienti c Aspects of Experimental Media Spaces(CAST 01), pp. 89-93, 2001.
12. R. Tenmoku, M. Kanbara, N. Yokoya, and H. Takemura: \Annotation overlay with a wearable computer
using augmented reality," Proc. 1st CREST Workshop on Advanced Computing and Communicating Techniques for Wearable Information Playing, pp. 27-32, March 2002.
13. A. State, G. Horita, D. Chen, W. Garrett, and M. Livingston: \Superior Augmented Reality Registration
by Integrating Landmark Tracking and Magnetic Tracking," Proc. SIGGRAPH'96, pp. 429-438, 1996.
14. H. Petrie, V. Johnson, T. Strothotte, A. Raab, S. Fritz, and R. Michel: \MoBIC: Designing a Travel Aidfor Blind and Elderly People," Jour. of Navigation, Vol. 49, No. 1, pp. 44-52, 1996.
15. S. Feiner, B. MacIntyre, T. Holler, and A. Webster: \A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment," Proc. 1st Int. Symp. on Wearable Computers, pp. 74-81, 1997.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Figure 6. Overlay images at the points A, B, C, and D(a) before updating(b) after updating

Figure 7. Example of updating annotation information