Nara Palace Site Navigator:
Device-Independent Human Navigation
Using a Networked Shared Database
Ryuhei TENMOKU, Yusuke NAKAZATO, Atsushi ANABUKI,
Masayuki KANBARA, and Naokazu YOKOYANara Institute of Science and Technology,
8916-5 Takayama, Ikoma, Nara, 630-0192 Japan
Abstract. Many human navigation systems which present location-based information to mobile device users aredeveloped in recent years. We have already developed a wearable tour guide system which presents locationbased tour guide information using augmented reality techniques. In this research, we try to improve theprevious system into a new device-independent human navigation system "Nara Palace Site Navigator" which
can present tour guide information according to the specifications of the user’s devices: wearable computer,
personal digital assistant (PDA), and cellular phone. Additionally, this paper describes the experiments using the
proposed system in the Nara palace site to show the feasibility of the proposed system.

1. Introduction
The explosive spread of a small computer and a PDA has made it possible to realize
human navigation systems which can present location-based information to mobile device
users according to the user’s location. These systems present navigation information or tour
guide information based on the user’s location data to cellular phone, PDA, or wearable
computer[1, 2] users.
A number of navigation systems using a PDA are proposed until now. For example,
CYBERGUIDE system presents context-aware tour guide to a PDA user[3]. CYBERGUIDE
system can presents the user’s circumference map with an arrow which points out thedestination. GUIDE system is a tour guide system for Lancaster City[4]. It can present
location aware tour guide information according to the user’s preferences using multimedia
contents. The mobile museum guide system in HIPS project presents to the user locationbased guide information in the museum[5]. REAL pedestrian navigation system cannavigate the user using the user’s circumference map and virtual reality contents in bothindoor and outdoor environments[6]. REAL system can be used with not only a PDA but
also a wearable computer using polygon reduction techniques for CG contents. Moreover,using powerful notebook computers, it can be realized that the mobile navigation systems
present more attractive contents. DEEP MAP system is a wearable personal guide systemusing a notebook computer[7]. DEEP MAP system realizes useful tour guide for foreign
tourists combining a language translation tool. MARS system is a wearable navigation
system which can be used in the Columbia University campus[8, 9]. MARS system overlays
annotations on buildings using augmented reality techniques[10, 11, 12]. ARCHEOGUIDE
system is a mobile guide system for guiding tourists[13, 14]. This system is intended topresent archaeological information to the user at cultural-heritage sites. The archaeologicalinformation in this system includes virtually reconstructed buildings which are overlaid on        

 
Figure 1. The aerial photographof the Nara palace site 
   

 
    

Figure 2. Nara Palace Site Navigatorruins of buildings using augmented reality techniques. This system demonstrates thataugmented reality provides an effective method for guiding tourists. We have also developed
a wearable annotation overlay system which can be used indoors and outdoors[15]. Our
previous system presents to the user location-based information using augmented reality.
However, these mobile navigation systems provide services which restrict the user’s
mobile device suitable for contents to be presented. In other words, the specification of theuser’s mobile device is determined by contents to be assumed. It is not a really useful service
model in these days when many kinds of mobile devices are put on the market. Therefore, inthis research, we try to improve our previous wearable augmented reality system[15] into a
new device-independent human navigation system “Nara Palace Site Navigator”. “NaraPalace Site Navigator” can present tour guide information according to the specifications of
the user's devices using the networked shared database[16].
This paper is structured as follows. Section 2 describes the outline of “Nara Palace Site
Navigator”. Section 3 and Section 4 describe the detailed information of the client systemusing a wearable computer and a PDA/cellular phone, respectively. In Section 5, experimentswith a prototype system in the Nara palace site are described. Finally, Section 6 summarizes
the present work and also describes future work.

2. Device-Independent Human Navigation: Nara Palace Site Navigator
2.1 Assumed environments
Nara palace site is the Japanese ancient capital palace which existed approximately 1300
years ago. Nara palace site which has an area of 2 square kilometers is shown in Figure 1.
The proposed system is supposed to be used in outdoor sightseeing sites which include somescattered sightseeing spots in wide areas just like the Nara palace site. Many of sightseeing
sites (ruins or temples) in the world are classified into such sites. Namely, the proposedsystem has much possibility to be applied for many other sightseeing sites. Additionally, theproposed system runs on the premise that a wireless network can be used in suchenvironments. To guide the user in such environments, the system is required to navigate theuser from spot to spot along the tour route and guide the user at scattered sightseeing spots.
2.2 Outline of Nara Palace Site Navigator
Figure 2 shows the outline of the Nara palace site navigator. Nara Palace Site Navigator isa server and client system. The server stores a networked shared database of location-based
contents, and clients are mobile devices which are equipped by tourists in the Nara palace site.
The proposed system realizes to guide the user by navigating him/her between sightseeing
spots and presenting guide information at every sightseeing spot. The system

User’s current
position

Indicator forsightseeing spot

Link to contents(a) Example of web map(b) Example of html content

Figure 3. Example of web map and html content
Table 1. Kinds of contents presented to each device
CONTENTSDEVICE TYPE
Wearable Computer (WC) type
PDA type
Mobile Phone type

AUGMENTEDREALITY

MOVIE SOUND TEXT AND IMAGE AUTOGENERATEDWeb MAP (resolution)（html）

O
O
O
O
X
O
O
O
X
X
X
O
(O: possible to present, X: impossible to present)

O (640*320)
O (320*320)
O (230*230)

navigates the user between sightseeing spots using autogenerated web maps. Figure 3 (a)shows an example of autogenerated web maps which are headup-displayed user’s
circumference maps with some information. Our system guides the user at every sightseeing
spot using multimedia contents: movies, sounds, texts and images (html files), and augmented
reality contents. Figure 3 (b) shows an example of text and image contents.
Our system assumes that all client devices can measure the user’s position and absolute
direction and send them to the server. Moreover, client devices must be able to brows web
pages. In the present system, such mobile devices are classified according to the performance
into three device types as follows: wearable computer type devices, PDA type devices, andmobile phone type devices. Table 1 summarizes the kinds of contents which are presented bythree device types.
We describe our planning service model of this system. We plan to construct the tour
guide system guiding every device users in every sightseeing spots. Namely, only
downloading the client application, the system guides the user in every sightseeing spots.
2.3 Networked shared contents database
The server stores a networked shared database of location-based contents. The databaseincludes the following three tables.

 Contents table Device information table User’s state tableTable 2. Example of the contents table
FILE NAMECONTNTS TYPELATITUDE

LONGITUDE

DEVICE TYPE

1.mpg

Movie

Latitude 1

Longitude 1

All

2.mp3

Sound

Latitude 2

Longitude 2All

3.htmlWeb

Latitude 3

Longitude 3

PDA type, Cellular phone type4.vrmlVirtual object

Latitude 4

Longitude 4

Wearable computer type

4.txt

Position of virtual object

Latitude 4

Longitude 4

Wearable computer type

5.jpg

Annotation

Latitude 5Longitude 5

Wearable computer type

5.txt

Position of annotation

Latitude 5Longitude 5

Wearable computer type

Table 3. Example of thedevice information table
USER ID

DEVICE TYPE

Table 4. Example of the user’s state table
USER IDLATITUDE

LONGITUDE

DIRECTION

ID 1

Latitude 1.1

Longitude 1.1

Direction 1

ID 1

Wearable computer type

ID 2

PDA type

ID 2

Latitude 2.1

Longitude 2.1

Direction2

Cellular phone type

ID 3

Latitude 3.1

Longitude 3.1

Direction 3

ID 3

Table 2, 3, and 4 show examples of these tables. The contents table stores contents file names
which are associated with contents-embedded positions and the kind of device which canpresent each file. The device information table manages specifications of all users’ devices.
The user’s state table manage every user’s present position and direction. The contents tableis prepared beforehand and the other two tables are created or updated when the data aretransmitted from clients. These tables are used for deciding the contents which should be sent
to the client.

2.4 Flowchart of Nara Palace Site Navigator
In this section, the flowchart of the system is described. The Nara Palace Site Navigatorrealizes device-independent human navigation with the processes described below.
1. When a client device starts the client program, the client device sends the device
information including specifications of the device to the server. In the present system,
the user input the device information manually.
2. When the server receives the device information, the server issues the user ID for theaccessing client. At the same time, the server adds the new client data into the device
information table.
3. The client sends the user ID, the user’s current position data (in latitude and longitude),and absolute direction to the server constantly. The data transmission rate is determinedby the band width of the wireless network.
4. The server updates the user’s state table based on the transmitted information. The
server generates the headup-displayed web map around the user’s current position basedon data of the user’s state table.
5. The server sends contents file which is determined by three tables. Concretely, a certain
user’s device type is found from the device information table and the user’s current
position is found from the user’s state table. Using these data, the server can find filenames whose contents can be presented by the user’s device and are embedded near theuser’s current position. Finally, the server sends these files to the client.
6. The client displays web maps and contents to guide the user.

Camera

Real
scene
image

Display device
Clip On Generated
Display
image(MicroOptical)

QV-700N(Logitech)

Earphone &microphone Sound

Inertial
sensorOrientation

Inertia Cube
(Intersense)

MS-HS59SC(ELECOM)

Dead reckoning
moduleVoicecommand

DRM-III (Point
Research
Corporation)

PositionContents
data

IrDA receiverComputer

Original(MI
Systems)

Figure 4. User’s appearanceof the wearable system
Annotation

Signs for tour routecontentsInterLink
MP-XP7310(Victor)Figure 5. Hardware configuration of the wearable system
Virtual object(virtual building drawn by CG)

Indicator of the system

Detailed information(a) Example of annotation overlay image(b) Example of virtual building overlay image

Figure 6. Examples of generated images of the wearable system

3. Client System Using Wearable Computer Type Device
3.1 Overview of wearable computer type
The client system using a wearable computer type device can present all prepared contents(movies, sounds, and augmented reality contents[15]) to the user. The user’s appearance andthe hardware configuration of the client system using a wearable computer is shown in
Figures 4 and 5, respectively. Since the system uses video see-through augmented reality, theuser equips a view camera. The client device composes augmented reality contents (virtual
objects and annotations) on real scene images from the user’s view camera based on the user’s
position and orientation measured by sensors. Then, positions of virtual objects used foraugmented reality contents have been obtained from the server. Generated images arepresented to the user through a display device which is attached to the user’s glasses.
Examples of generated images are shown in Figure 6. Figure 6 (a) is an example of thegenerated image when the system composes an annotation and signs for tour route on a realscene image. In Figure 6 (b), note that a virtual building is overlaid on the real scene of theruins of the building.
3.2 Measurement of user’s position and orientation
To overlay virtual objects in the correct position in the real scene image of user’s view, theuser’s position and orientation should be measured in real time. The user’s orientation, thePDA
Cellular phonewith built-in GPS
and digital compass

SL-C700(SHARP)

A5502K(KYOCERA)

GPS with
digital compass
eTrex
Summit(Garmin)(a) PDA system(b) Cellular phone system

Figure 7. Hardware configuration of the PDA and cellular phone systems

direction of the user’s viewpoint is measured with the inertial sensor attached to the user’s
glasses. The inertial sensor used in the proposed system can correct the inertial data measured
by a gyroscope with an electronic compass and a gravity sensor. Accordingly, the system canmeasure the user’s orientation without drift errors and accumulative errors that are principal
problems when a gyroscope is used solely. There still remains a problem in the inertial sensor:
that is, the data acquired from the inertial sensor are not synchronized with input images fromthe camera. In this system, the asynchronousness of two data is reduced by delaying the dataof the inertial sensor.
The system measures the user’s position by combining the user’s absolute position
measured by the GPS or IrDA devices and the user’s relative movement measured by thepedometer. When the GPS is used solely for measuring the position, the error fluctuates
between 5 meters and 50 meters which is caused by various reasons: the number of captured
GPS satellites, the weather, and so on. In the supposed environments which have no high-risebuildings, the error of the standard GPS is within 20 meters. This accuracy is sufficientenough for overlaying annotations on far and large real objects, but not sufficient enough foroverlaying virtual buildings on ruins of buildings close to the user. Therefore, IrDA markers
are set up at certain appointed points near the ruins on which virtual buildings are prepared to
be overlaid. The user equips the IrDA receiver to identify the infrared ray including position
IDs from the IrDA markers. Using IrDA sensors, the system can specify the user’s absolute
position with the error of less than 1 meter[15]. When the user goes away from one of IrDA
markers, the system estimates the user’s position by the pedometer. In outdoor environments
without high buildings, the positioning error of the pedometer used in the proposed system iswithin 5% of the distance traveled.
3.3 Interaction with system using speech recognition engine
In our wearable system, the user needs to input some information just like the selection of
the tour route and the selection of sightseeing contents to be presented. To realize theinteraction with the system, the use of a hand-held mouse or a wearable keyboard isconsidered. In this system, the user can make a hands-free interaction using voice recognition.
Our system uses the speech recognition engine “Julius”[17] for inputting information using
the user’s voice commands.

4. Client System Using PDA / Cellular Phone Type DeviceTable 5. States of data transmission and renewal times of web map
Device type
WC
PDA
Mobilephone

Data transmission
Maximum data
formattransmission rate
Wireless LAN
11MbpsIEEE802.11b
Wireless LAN
11MbpsIEEE802.11b
Data transmission format
144Kbpsof mobile phones

Renewal timeof web map
1sec.
3sec.
7sec.

Figure 8. Examples of user-presented images in Nara palace site

The user’s appearance and hardware configuration of the PDA system and the cellular
phone system are shown in Figures 7 (a) and (b), respectively. In both systems, it is necessary
to measure the user’s position and absolute orientation. In the PDA system, the GPS receiverwith digital compass is attached to the PDA to measure the user’s position and absolute
direction. In the cellular phone system, we use the cellular phone with built-in GPS receiver
and a digital compass.

5. Experiment in Nara Palace Site
We have carried out an experiment in Nara palace site using the prototype system. In thisexperiment, we used all of three client devices (a wearable computer type device, a PDA type

device, and a cellular phone type device) described in Sections 3 and 4. The datatransmission format and maximum data transmission rate of these devices are shown in Table
5. Experimental results including generated images are described below.
Figure 8 shows some examples of user-presented images. In Figure 8, (a), (b) and (c), andfrom (d) to (h) are presented to the user of the wearable computer, PDA, and mobile phone,respectively. We found that all contents files were presented to the user correctly as
established in Table 1 using the proposed system. Table 5 also shows the renewal time of
autogenerated web maps shown in Figure 8 (a), (b), (d), and (e). The renewal time means thetime after the client transmits the demand of updating to the server until the web map isupdated. The network delay in addition to these renewal times has an influence on the
comfortableness of tour guide. Since walking users cannot move so fast, the present network
delay and the poor update rate of the user’s position is a trivial problem. However, for theuser’s orientation, it is a significant problem. This is the problem which has occurred also inconventional human navigation systems using a wireless network. In the present system, theserver generates a web page of the headup-displayed user’s circumference map using theuser’s position and orientation data that are sent to the server. Figure 8 (f), (g), and (h)include augmented reality contents. Especially, in Figure 8 (g) and (h), a virtual building thatexisted in 1,300 years ago is overlaid on the real scene of the ruin of the building. In theexperiments, generated images including augmented reality contents are presented to thewearable computer user at more than 8 fps. We found that these virtual objects are overlaid
on correct positions of real scene images to present tour guide information to the user.

6. Summary and Future Work
This paper has described a device-independent human navigation system “Nara Palace Site
Navigator”. The proposed system presents location-based tour guide information according
to the user’s mobile device (wearable computer, PDA, and cellular phone) by sending device
information to the networked shared database server and receiving device-suitable contentsfrom the server. This paper has also described experiments with the prototype system in the
Nara palace site actually to show the feasibility of the system.
For future work, we plan to improve the present system into a more useful deviceindependent system. Even though the present system classifies the user’s mobile devices into
only three types (wearable computer type, PDA type, and cellular phone type), the futuresystem should classify the user’s devices more strictly based on the various capacity (network
power, rendering power, resolution of the display, and so on). Thereby, for example, thesystem can provide detailed virtual objects for a powerful wearable computer and roughvirtual objects for a less powerful wearable computer.

Acknowledgments
This work is supported in part by Core Research for Evolutional Science and Technology(CREST) Program “Advanced Media Technology for Everyday Living” of Japan Science andTechnology Agency (JST).

References[1] S. Mann: “Wearable Computing: A First Step toward Personal Imaging”, IEEE Computer,Vol. 30, No. 2, 1999.

[2] R. D. Vaul, M. Sung, J. Gips, and A. S. Pentland: “MIThril 2003: Applications andArchitecture”, Proc. of Int. Symp. on Wearable Computers 2003, pp. 4-11, 2003.
[3] S. Long, K. Kooper, G. D. Abowd, and C. G. Atkeson: “Rapid Prototyping of Mobile
Context-Aware Applications: The Cyberguide Case Study”, Proc. of the 2nd Int. Conf.
On Mobile Computing and Networking, pp. 97-107, 1996.
[4] K. Cheverst, N. Davies, K. Mitchell, A. Friday, and C. Efstatiou: “Developing a Contextaware Electronic Tourist Guide: Some Issues and Experiences”, Proc of CHI2000, pp.
17-24, 2000.
[5] R. Oppermann, and M. Specht: “Adaptive mobile museum guide for information andlearning on demand”, Proc. of HCI International '99, pp. 642-646, 1999.
[6] J. Baus, A. Krüger, and W. Wahlster: “A Resorce-Adaptive Mobile Navigation System”,
Proc. of the 7th Int. Conf. on Intelligent User Interfaces 2002, pp. 15-22, 2002.
[7] R. Malaka, and A. Zipf: “Deep Map – Challenging IT Research in the Framework of a
Tourist Information System”, Proc. 7th Int’l Congress on Tourism and Comm.
(ENTER2000), pp. 15-27, 2000.
[8] T. Höllerer, S. Feiner, and J. Pavlik: “Situated Documentaries: Embedding Multimedia
Presentations in the Real World”, Proc. of Int. Symp. on Wearable Computers '99, pp.
79-86, 1999.
[9] T. Höllerer, S. Feiner, T. Terauchi, G. Rashid, and D. Hallaway: “Exploring MARS:
Developing Indoor and Outdoor User Interfaces to a Mobile Augmented Reality System”,
Computers and Graphics, Vol. 23, No. 6, pp. 779-785, 1999.
[10] S. Feiner, B. MacIntyre, and D. Seligmann: “Knowledge-based Augmented Reality”,
Communications of the ACM, Vol. 36, No. 7, pp. 52–62, 1993.
[11] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Juiler, and B. MacIntyre: “Recent
Advances in Augmented Reality”, IEEE Computer Graphics and Applications, Vol. 21,
No. 6, pp. 34–47, 2001.
[12] M. Kanbara, N. Yokoya, and H. Takemura: “Registration for stereo vision-based
augmented reality based on extendible tracking of markers and natural features”, Proc. of
the 16th IAPR Int. Conf. on Pattern Recognition, pp. 1045–1048, 2002.
[13] D. Stricker, J. Karigiannis, I. T. Christou, T. Gleue, and N. Ioannidis: “Augmented
Reality for Visitors of Cultural Heritage Sites”, Proc. of Int. Conf. on Cultural andScientific Aspects of Experimental Media Spaces, pp. 89–93, 2001.
[14] P. Dähne, and J. N. Karigiannis: “Archeoguide: System Architecture of a Mobile
Outdoor Augmented Reality System”, Proc. Int. Symp. on Mixed and Augmented
Reality, pp. 263–264, 2002.
[15] R. Tenmoku, M. Kanbara, and N. Yokoya: “A Wearable Augmented Reality System
Using Positioning Infrastructures and a Pedometer”, Proc. of Int. Symp. on Wearable
Computers 2003, pp. 110-117, 2003.
[16] K. Makita, M. Kanbara, and N. Yokoya: “Shared Database of Annotation Information
for Wearable Augmented Reality System”, Proc. of SPIE Electronic Imaging, Vol. 5291,
pp. 464-471, 2004.
[17] A. Lee, T. Kawahara, and K. Shikano: “Julius — An Open Source Real-Time Large
Vocabulary Recognition Engine”, Proc. of European Conf. on Speech Communication
and Technology, pp. 1691–1694, 2001.