Intuitive Annotation of User-Viewed Objects for Wearable AR Systems
Ryuhei TENMOKU Masayuki KANBARA Naokazu YOKOYANara Institute of Science and Technology
8916-5 Takayama, Ikoma, Nara, 630-0192 Japan{ryuhei-t, kanbara, yokoya}@is.naist.jp
Abstract
By realizing augmented reality on wearable computers,
it is possible to overlay annotations on the real world basedon the user’s current position and orientation. However, it
is difficult for the user to understand links between annotations and real objects intuitively when the scene is complicated or many annotations are overlaid at the same time.
This paper describes a view management method which emphasizes user-viewed real objects and their annotations using 3D models of the real scene. The proposed method highlights the objects viewed by the user. In addition, when theviewed object is occluded by other real objects, the object
is complemented by using a synthetic image, which is madefrom 3D models, on the overlaid image.

1. Introduction
In recent years, augmented reality (AR)[1], whichmerges the real and virtual worlds, is realized on wearable computers[2] to make it possible to intuitively present
location-based information to users. We have already developed a wearable AR system which overlays annotations
on real scenes[3]. However, in most current wearable annotation overlay systems, it is hard for users to link annotations to their target objects intuitively when a lot of annotations are presented densely. In this paper, we propose a new
annotation presentation technique emphasizing user-viewed
annotations and their target objects.
Related work in this field is briefly reviewed below.
Azuma et al.[4] have proposed a view management methodfor AR scenes, which rearranges multiple annotations without mutual overlap. Bell et al.[5] have proposed a view
management method for VR and AR scenes, which can rearrange annotations using image regions where no object isprojected, based on 3D models of scenes. However, mostprevious work focuses on how to arrange annotations spatially on augmented scenes when there exist multiple annotations. The proposed method focuses on how to intuitively
present links between annotations and their target objects.

                             

    

Figure 1. Outline of conventional and proposed methods.

2. Intuitive Annotation of User-Viewed Objects
Figure 1 illustrates the outline of the conventional annotation overlay method and the proposed method. The proposed method consists of two intuitive annotating methodsof user-viewed objects. These two methods generate twokinds of images (a) object highlight image and (b) occlusion complement image as shown in Figure 1 to support
the user’s intuitive understanding of links between annotations and their target objects. To highlight the user-viewedobject, the system needs a 3D model of the real scene whichis painted a different color on every object distinctively. To
complement occluded portions of the user-viewed object,
the same 3D model with detail textures is needed additionally. The 3D model with two kinds of textures are called a“structure model” and a “texture model”, respectively. The
structure model can be made from CAD data of buildings
easily.
The proposed method shows a pair of a user-viewed an-Millennium HallNAIST Office Building            

Cafeteria       

Figure 2. Generated user’s view images.
notation and its target object intuitively. The present system assumes that the user views the object at the center of auser’s view image. To generate object highlight images, image regions which the user-viewed object is projected on arecalculated by projecting the structure model. Moreover, theoccluded portions of the user-viewed object is determinedby projecting the texture model. Occluded portions of themodel are overlaid on object highlight images to generate
occlusion complement images.

3. Experiments Using Prototype System
We have constructed a prototype system using the proposed method to carry out an experiment of intuitively presenting annotations to the user in our campus. The prototype system measures the user’s position and orientation using IrDA sensors whose positioning error is within 1 meterand an inertial sensor[3].
In this experiment, there are three buildings (“Cafeteria”, “NAIST Office Building”, and “Millennium Hall”) andthree annotations are overlaid on each building. The central
image of the lower line in Figure 2 shows an annotation
overlay image when the system detects no user-viewed object. This user’s view is generated by the conventional annotation overlay method[3]. The other images of Figure 2
show generated user’s view images of the proposed method
when the user views each object. Through this experiment,
we have confirmed that the proposed method can present
the correspondence between real and virtual objects to theuser clearly.

4 Summary and Future Work
This paper has proposed an intuitive annotation technique using 3D models of real scenes as a new view man-agement method for wearable AR systems. We have also
carried out preliminary experiments using a prototype system. In the future, we have to apply our proposed method tomuch more registration errors such as positioning errors ofa GPS. In our plan, we try to decrease the influence of errors
on intuitive annotation using image processing techniques.
Acknowledgments
This research is supported in part by Core Research for Evolutional Science and Technology (CREST) Program “Advanced Media Technology for Everyday Living” of Japan
Science and Technology Agency (JST).

References[1] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S.
Julier, and B. MacIntyre: “Recent Advances in Augmented Reality, ” IEEE Computer Graphics and Applications, Vol. 21, No. 6, pp. 34-47, 2001.
[2] S. Mann: “Wearable Computing: A First Step Toward
Personal Imaging, ” IEEE Computer, Vol. 30, No. 2,
pp. 25-32, 2002.
[3] R. Tenmoku, M. Kanbara, and N. Yokoya: “A Wearable Augmented Reality System Using Positioning Infrastructures and a Pedometer, ” Proc. 7th Int. Symp.
on Wearable Computers, pp. 110-117, 2003.
[4] R. Azuma, and C. Furmanski: “Evaluating Label
Placement for Augmented Reality View Management,” Proc. 2nd Int. Symp. on Mixed and Augmented Reality, pp. 66-75, 2003
[5] B. Bell, S. Feiner, and T. Höllerer: “Information at aGlance, ” IEEE Computer Graphics and Applications,vol. 22, no. 4, pp. 6-9, 2002.