ISMAR 2003

Telepresence System Using High-resolution
Omnidirectional Movies and a Reactive Display
Sei IKEDA, Tomokazu SATO, Masayuki KANBARA and Naokazu YOKOYA
Vision and Media Computing Laboratory
Nara Institute of Science and Technology
8916-5 Takayama, Ikoma, Nara 630-0192, Japan{sei-i, tomoka-s, kanbara, yokoya}@is.aist-nara.ac.jp
Abstract(b)This paper describes a novel telepresence system thatuses high-resolution movies and a reactive display systemwith a treadmill. In this system, users can walk through avirtualized environment by actually walking on atreadmill. According to walking motion which is detectedby using 3-D position sensors put on both legs, thevirtualized environment captured by an omnidirectional
multi-camera system is projected on a multi-screen
display.

(4)
12 PCscontrol PC B(c)

4 × 3 projectors(5)

1. Introduction

screenTechnology that enables users to experience a remote site
virtually is called telepresence. A telepresence systemusing real environment images is expected to be used in
the field of entertainment and medicine. To realize a richpresence, the following functions are needed for thesystem: high-resolution displayed image, interlock withbody motion, free moving position and free viewing
direction. However, most existing systems [1, 2] lacksome of such functions.
In this paper, a novel telepresence system is proposed.
The system is equipped with a high-resolution immersive
display and a treadmill for input and output devices.
High-resolution omnidirectional movies acquired by anomnidirectional multi-camera system [3] are projected toan immersive screen by using detected user’s walking
motion. Our system realizes almost all of the abovementioned functions except that user’s motion is limited
to one degree of freedom. The following sections describe
the detail of the proposed system.

sensor(3)(1)

mirror(2)
treadmill(a)control PC AFigure 1. Configuration of the proposed system.

twelve images synchronized with the speed of walking
because each screen image is generated by four projectors.
As output data, these movies are displayed according tothe user’s motion. These components are described insome more detail below.

(a) Walking Motion Interface

2. Proposed Telepresence SystemThis interface is composed of a treadmill (WalkMaster), a
couple of 3-D position sensors (Polhemus Fastrak) andPC A (Intel Pentium 4 2.4 GHz) for control as illustrated
in Figure 1. User’s walking motion is detected by two 3D position sensors fixed on user’s legs (Figure 1(1)). The
treadmill is controlled by PC A based on position
information from the sensors (Figure 1(2)). The belt of
the treadmill is automatically rotated so that the center ofAs shown in Figure 1, the proposed system is composedof (a) a walking motion interface, (b) a graphics PC
cluster and (c) an immersive three-screen display. The
walking motion interface detects user’s walking motion
as input data, and sends calculated displacement
information to the PC cluster. The PC cluster draws

1ISMAR 2003
gravity of two sensors coincides with the center of thebelt area. Although a user can walk any direction on thisdevice [4], only the forward direction is used for thepresent telepresence system. PC A calculates
displacement of user’s position and sends it to thegraphics PC cluster (Figure 1(3)).

(b) Graphics PC Cluster
The graphics PC cluster is composed of twelve PCs(CPU: Intel Pentium 4 1.8 GHz, Graphics Card: Geforce4Ti4600) and a control PC B (Intel Pentium 4, 1.8 GHz).
Each graphics PC is networked through 100Mbps LANand controlled by PC B. PC B sends frame indexes totwelve PCs using UDP protocol simultaneously (Figure1(4)). Each machine draws synchronized frame XGA
images according to the user’s motion (Figure 1(5)). Notethat the images are accumulated in local hard disk inadvance and only the frame index is carried via network.

Figure 2. Appearance of telepresence system.

(c) Immersive Display
The immersive display is composed of three slanted rearprojection screens (Solidray VisualValley) and twelve
projectors. To obtain a wide field of view, the screens arelocated in user's front, left and right sides. To realize
high-resolution image projection, each screen image is
projected through a mirror by four projectors.
The resolution of each projector is 1024 × 768 (XGA)
pixels. Because there are some overlapping areas
projected by multiple projectors and some areas are notprojected on the screen, the resolution of each screen ispotentially about 2 million pixels.

Figure 3. A frame of input movie.

measurement; that is, the displayed image is notsynchronized with head motion but with leg motion.

4. Summary
In this paper, a novel telepresence system using an
immersive display and a treadmill is proposed. Thissystem can interactively present high-resolution realmovies according to walking motion. The experiment has
shown that the proposed telepresence system provides us
with the feeling of rich presence in remote sites.
The following investigations are left as important
future work. One is to control the belt rotation oftreadmill naturally. The other is to relax the limitation ofuser’s motion.

3. Experiment
The subjective evaluation experiment was conductedusing the proposed telepresence system shown in Figure 2.
Omnidirectional movies used in this experiment are
acquired by an omnidirectional multi-camera system
Ladybug mounted on a car moving at an approximately
constant speed. Figure 3 shows a frame of input movie fortwelve projectors.
We have confirmed that the proposed telepresence
system provides us with the feeling of rich presence inremote sites in this experiment. We have also confirmedthat geometric discontinuity between regions projected bydifferent projectors and synchronization error could not
be recognized. However, some image discontinuities wereobserved in scene areas very close to the camera system
due to its violation of single view point constraint andpoor presence was felt due to the limitation of user’smotion. We also felt unnatural in the control of thetreadmill when a user begins to walk, because the motionof upper part of the body is not considered in motion

References[1] Y. Onoe et al., "Telepresence by Real-time Viewdependent Image Generation from Omnidirectional VideoStreams", Computer Vision and Image Understanding,vol. 71, No. 2, pp. 154-165, 1998.
[2] H. Noma and T. Miyasato, "Design for Locomotion
Interface in a Large Scale Virtual Environment, ATLAS:
ATR Locomotion Interface for Active Self Motion", Proc.
ASME-DSC, vol.64, pp.111-118, 1998.
[3] S. Ikeda et al., “A Calibration Method for an
Omnidirectional Multi-camera System”, Proc. SPIE, vol.
5006, pp.499-507, 2003.
[4] H. Iwata, “Walking About Virtual Environments on
an Infinite Floor”, Proc. IEEE Virtual Reality ’99, pp.
286-293, 1999.

2