EGVE Symposium (2008)
B. Mohler and R. van Liere (Editors)

Visible Portion Estimation of Moving Target Objects
for Networked Wearable Augmented Reality
K. Makita1 , M. Kanbara1 and N. Yokoya1
1 Nara Institute of Science and Technology, JapanAbstract
This paper describes a new visible portion estimation method of moving target object for networked wearable
augmented reality (AR) system. In annotation overlay applications using AR systems, it is important to improvereadability and intelligibility of annotations in a user’s view. View management makes it possible to appropriately
generate annotation overlay images for users so as to intuitively understand annotations. For instance, overlappings of annotations and other objects can be prevented by using a view management technique. View management
requires visible portions of 3D target objects in a 2D view plane. This paper proposes a visible portion estimation method for moving target objects based on positions and shapes of the moving target objects. The wearable
augmented reality system obtains positions of target objects via wireless network for estimating visible portions oftarget objects in the user’s view. Annotations are able to be overlaid by using view management techniques with
our proposed visible portion estimation.
Categories and Subject Descriptors (according to ACM CCS): H.5.1 [Information Interfaces and Presentation]:
Artificial, augmented, and virtual realities

1. Introduction
Augmented reality (AR) can enhance the real world by superimposing virtual objects on a user’s view. Augmented reality applications can be divided broadly into several categories. Above all, appearance simulation applications andannotation overlay applications have been extensively studied. In appearance simulation, virtual objects have to bedrawn in an appropriate position and orientation as they are
in real world [ES06, FBH 97]. On the other hand, the appearance of an annotation does not need to look like a realobject. Figure 1 shows an example of an overlay image ofannotations for users as moving objects. Annotation overlay applications give us additional information about real
objects. It is absolutely essential for annotations to be understood as information of a target object in the real scene.
In annotation overlay applications, therefore, it is important to improve readability and intelligibility of annotations
in a user’s view [TMN05, BFH01, AF03, ZS05, UMK 05,
BFH02]. View management makes it possible to appropriately generate annotation overlay images for users so as tointuitively understand annotations. One critical issue withannotation view management is how to overcome overlap-­c The Eurographics Association 2008.

Figure 1: Example of annotation overlay image.
pings of annotations with one another. The Hiding of unexpected objects by annotations must also be prevented. To
overcome overlapping and hiding, annotation overlay systems have to estimate visible portions of 3D target objects ina 2D user’s view.
This paper proposes a visible portion estimation method
for view management techniques. The wearable augmented
reality system obtains positions of target objects via wire-K. Makita, M. Kanbara & N. Yokoya / Visible Portion Estimation of Moving Target Objectsfor Networked Wearable Augmented Reality

less network for estimating visible portions of target objects
in the user’s view. Annotations are overlaid at appropriate
positions without overlapping as much as possible by viewmanagement techniques.

2. Visible portion estimation for annotation overlay
Our proposed method to estimate visible portions of moving
target objects in a user’s view plane is to use positions andshapes of target objects. The proposed method includes thefollowing two types of estimation methods.

¯ Method A) Quadrangle estimation
The visible portion of the moving target object is estimated as a quadrangle.
¯ Method B) Existing probability estimation
The visible portion of the moving target object is estimated as a probability map of location probability density.
Two types of estimation methods are shown in Figure 2 and3, respectively. The following sections describe each methodin detail.

2.1. Quadrangle estimation
Figure 2 shows an outline of quadrangle estimation method.
The outline shown in Figure 2 takes for instance a wearable
computer user as a moving target object. Each step of theestimation is described in detail in the following.
In first step, the wearable AR system obtains positionsof target objects. In this study, we assume an environment
where the wearable AR system can obtain positions of target objects in the real world by using a framework that wehave already proposed [MKY06]. In the framework, highfrequency of updating and low communication delay areachieved by using hybrid P2P network. Positions of targetobjects in a user’s view are estimated based on positions oftarget objects, the user’s position, and the user’s orientation
as shown in Figure 2(a).
In the next step, visible portions of target objects are estimated by using given shapes. In this case, a region of thetarget is approximated by using a rectangular solid. This approximation is based on an assumption that the position of
the target’s head is estimated as the target’s position and thetarget stands vertically on the floor. Figure 2(b) shows an
estimated region of the target object.
Finally, a circumscribed quadrangle of the rectangularsolid is calculated as a region that is used for view management. This step is for simplification of calculation in checking overlapping and hiding. Figure 2(c) illustrates a rectangular solid and a circumscribed quadrangle. Figure 2(d)shows only a circumscribed quadrangle that is an eventual
visible portion of a target object. Regions of target objects ina user’s view are estimated as described.

2.2. Existing probability estimation
Figure 3 shows an outline of existing probability method.
The outline shown in Figure 3 also takes for instance a wearable computer user as a moving target object. This method
is designed by extending quadrangle estimation method byusing probability density. Each step of the estimation is described in detail in the following.
In first step, as shown in Figure 2, the wearable AR system obtains positions of target objects (Figure 3(a)). In thenext step, visible portions of target objects are estimated byparts of primitive models. Each primitive model is generated by subdivision of the shape in quadrangle estimation
method, and existing probability is added. Figure 3(b) showsa group of primitive models. In this case, the original rectangular solid is divided into some rectangular solids. Next,
each primitive model is drawn by off-screen rendering forthe purpose of existing probability estimation. Figure 3(c)shows a group of primitive models colored by using existing
probability. Each primitive model is drawn with permeation
rate in inverse proportion to probability density.
Finally, the probability map is generated by using the result of off-screen rendering. A color of a point on the image
generated by off-screen rendering can be treated as a existing probability of the moving target object at the point. So
the probability map can be generated in response to a property of each view management method.
3. Experiment3.1. Experimental Environment
We have carried out several experiments using the proposedmethod in indoor environment where wearable AR systems
can use a wireless LAN. We have developed a networkshared database of annotation information in a user information management server (CPU: 3.0GHz, RAM: 3.25GB).
We have also pasted invisible visual markers on the ceilingfor wearable computer users to estimate their position andorientation [NKY05].
In this experiment, four wearable system users exist in
the environment as moving target objects. Users’ positions
are estimated by their wearable systems and are transmitted
to wearable AR systems. The hardware configuration of thewearable AR system is as follows. (PC CPU: 2.0GHz, Memory: 2.0GB, Camera: Original).
To generate annotation overlay images, position and orientation of a camera which is attached to user’s head areneeded. In experiments, the wearable system estimates theposition and orientation of the user’s viewpoint by using theconventional marker-based registration method [NKY05].
The position and orientation errors of this method are about
5 centimeters and 3 degrees. Each wearable computer user isequipped with an infrared camera on the user’s head for thepurpose of estimating user’s position and orientation.

­c The Eurographics Association 2008.

K. Makita, M. Kanbara & N. Yokoya / Visible Portion Estimation of Moving Target Objectsfor Networked Wearable Augmented Reality
Infrared
cameraPosition of theuser （point）

Approximate region（rectangular solid）

Circumscribed
quadrangle

Region of target object（quadrangle）

(a)

(b)

(c)

(d)

Figure 2: Estimation procedure of a visible portion of a target object using a quadrangle.

(a)

(b)

(c)

Figure 3: Estimation procedure of a probability density of the target object.
3.2. Estimation of visible portions of target objects in auser’s view

¯ Method A) Quadrangle estimation
The visible portion of the user is approximated by using a
rectangular solid stands vertically on the floor. The center
point of the upper surface of the rectangular solid corresponds to user’s head. Both the upper and lower surfacesof the rectangular solid are square in shape and 60cm ona side. Figure 4(a) and (b) show experimental results byusing quadrangle estimation. A camera of the wearable
AR system dynamically changes its position and orientation. This is because the wearable AR system has a certain
margin of estimation error.
¯ Method B) Probability density estimation
In this experiment, we use four groups of rectangularsolids. Because human body is not a rigid, we divide thebody of the target into 4 parts; head, neck, upper body, andlower body. Furthermore we divide each zone by using a
rectangular solid. Both the upper and lower surfaces ofrectangular solids are square in shape and 5cm on a side.
Next, we set up a probability density for each rectangularsolid. We can conclude that the head has high probabilitydensity because the camera is set to the head. The lower
body has low probability density to the contrary. We use
weight to each zone, weights (head: 0.7, neck: 0.5, upper
body: 0.3, lower body: 0.1) are used as an alpha value by
off-screen rendering to acquire probability density map inreal time. Figure 5(a) and (b) show experimental results
using probability density estimation. Rectangular solids­c The Eurographics Association 2008.

are shown in figures to confirm the result. In a normalsituation, rectangular solids are used only off-screen rendering.

3.3. Annotation overlay experiment
Annotations are placed using visible portions of target objects in a user’s view estimated in the previous step. In thispaper, the shape of annotations is a rectangle and fixed insize. We apply a greedy algorithm based on the result of aprevious survey of evaluating label placement [AF03]. As
advance preparation, 21 slate points for placing an annotation are set around and inside the region of a target object.
Each point has search priority.
Figure 4(c) shows an example of annotation overlay image generated by using quadrangle estimation. In this experiment, four users are walking in the area and the AR user
receives other users’ positions at about 20 times per second.
Annotations in Figure 4(c) are placed without overlappingsor hiding. We have generated annotation overlay images at a
rate of about 15 frames per second in this experiment.
Figure 5(b) and (c) show examples of annotation overlay
image generated by using probability density estimation. In
this experiment, we apply a new method based on greedy algorithm with our proposed probability map. We have defined
an annotation energy function as a sum of existing probability. The energy is a sum of color density of points both included in an area of annotation and off-screen rendering. InK. Makita, M. Kanbara & N. Yokoya / Visible Portion Estimation of Moving Target Objectsfor Networked Wearable Augmented Reality

(a)

(b)

(c)

Figure 4: Experimental result using quadrangle estimation.

(a)

(b)

(c)

Figure 5: Experimental result using probability density estimation.
this experiment, energy minimization calculation is always
done around the annotation.
4. Summary
This paper has proposed a new visible portion estimation
method for annotations on moving objects in networked
wearable augmented reality. By using our proposed method,users of an annotation overlay system can intuitively understand relationships between annotations and moving target
objects. We have demonstrated the feasibility of the methodwith experiments in indoor environment. In the future, weshould upgrade the layout quality of annotations by using
better approximation of the region of a target object.
References[ES06] J. Ehara and H. Saito: “Texture Overlay for Virtual Clothing Based on PCA of Silhouettes,” 5th IEEEand ACM Int. Symp. on Mixed and Augmented Reality(ISMAR 2006), pp. 139–142, 2006.
[FBH 97] S. Feiner, B. MacIntyre, T. Höllere, andA. Webster: “A Touring Machine: Prototyping 3D Mobile
Augmented Reality Systems for Exploring the Urban Environment,” Proc. 1st Int. Symp. on Wearable Computers,
pp. 74–81, 1997.

[TMN05] R. Tenmoku, M. Kanbara, and N. Yokoya: “Intuitive annotation of user-viewed objects for wearable ARsystems,” Proc. IEEE Int. Symp. on Wearable Computers,
pp. 200–201, 2005.
[BFH01] B. Bell, S. Feiner and T. Höllere: “View Management for Virtual and Augmented Reality,” ACM Symp.

on User Interface Software and Technology (UIST 2001),
pp. 101–110, 2001.
[AF03] R. Azuma and C. Furmanski: “Evaluating Label
Placement for Augmented Reality View Management,”
Proc. 2nd Int. Symp. on Mixed and Augmented Reality(ISMAR 2003), pp. 66–75, 2003.
[ZS05] F. Zhang and H. Sun: “Dynamic Labeling Management in Virtual and Augmented Environments,” 9th Int.
Conf. on Computer Aided Design and Computer Graphics(CAD/CG 2005), pp. 397–402, 2005.
[UMK 05] K. Uratani, T. Machida, K. Kiyokawa andH. Takemura: “A Study of Depth Visualization Techniques for Virtual Annotations in Augmented Reality,”
Proc. IEEE Virtual Reality (IEEE VR), pp. 295–296,
2005.
[BFH02] B. Bell, S. Feiner and T. Höllere, “Information
at a glance,” IEEE Computer Graphics and Applications,Vol. 22, No. 4, pp. 6–9, 2002.
[MKY06] K. Makita, M. Kanbara, and N. Yokoya: “Personal information annotation on wearable computer users
with hybrid peer-to-peer communication,” Proc. 16th Int.
Conf. on Artificial Reality and Telexistence (ICAT 2006),
pp. 217–227, 2006.
[NKY05] Y. Nakazato, M. Kanbara, and N. Yokoya: “Alocalization system using invisible retro-reflective markers,” Proc. IAPR Conf. on Machine Vision Applications(MVA2005), pp. 140–143, 2005.

­c The Eurographics Association 2008.