A Networked Telepresence System with an Internet Automobile
and its Transmission Issue
Kazutoshi Fujikawa1
Tomokazu Sato2

Kazumasa Yamazawa2
Tomoya Ishikawa2
Yuichi Butsuen2
1
1
Yutaka Nakaramura
Hideki Sunahara
Naokazu Yokoya2
1
Information Technology Center
Nara Institute of Science and Technology
Ikoma, Nara 630-0192, Japan{fujikawa,yutaka-n}@itc.naist.jp, suna@wide.ad.jp
2

Graduate School of Information Science
Nara Institute of Science and Technology
Ikoma, Nara 630-0192, Japan{yamazawa,tomoya-i,yuui-b,tomoka-s,yokoya}@is.naist.jp
Abstract
Recently, the telepresence which allows us to experiencea remote site through a virtualized real world has been investigated. We have implemented a networked telepresence
system using omni-directional images which enable multiple users to share a virtualized dynamic real scene in thedistance and to see any direction of the scene they want.
Adopting multicast transmission, our telepresence system
will provide a scene for a lot of users without increasing
network loads. In this paper, we introduce our networked
telepresence system and discusses its network communication issue when using multicast as a transmission protocol.

Keywords
Multicast, Wireless communication, Mobile communication, Video streaming

1. Introduction
Recently, there are many researches of telepresence thatacquires a dynamic real world into a virtual world and enables a user to be immersed in the remote environment [4].
The telepresence can be applied to various fields such asentertainment, medical service, and education. We have already developed a telepresence system which uses an omnidirectional camera and enables a user to look around thescene so that he/she increases the presence in telepresence[1, 5]. Our telepresence system acquires and transfers theremote omni-directional scene by an omni-directional camera, and shows a user the view-dependent images in realtime. The telepresence system also enables a user to lookaround the remote scene widely.
On the other hand, the progress of the wiewless network technology, an automobile can have the Internet connection at all times. If our telepresence system applies tothe automobile with the Internet connectivity using wireless communication channels, a variety of applications canbe realized. For instance, a lot of remote users can experience having a drive and share the scene among them, if
the omni-directional video stream is transmitted by multicast. Besides, each user can see any direction of the scene
he/she wants. Generally, for the telepresence system, thevideo stream should be transmitted constantly. Therefore,a robust transmission mechanism is required, since an automobile always moves and the condition of the wirelesscommunication channel might be unstable.
In this paper, we introduce our networked telepresence
system, which multiple users on the Internet to look arounda remote scene captured by omni-directional cameras, anddiscuss its network issue.
Section 2 describes the overview of the proposed system. Section 3 discusses the transmission issue and Section
4 describes the related work. Finally, this paper concludes
in Section 5.

2. Overview of our telepresence system
Figure 1 briefly shows the architecure of our telepresence
system. Our telepresence system consists of an automobilewhich mounts omni-directional camera “HyperOmni Vision [10]”, multicast relay server for video stream on the automobile, and the omni-directional video viewer. The omnidirectional video stream is transmitted to the campus LAN
from the automobile via a wireless communication channel of IEEE802.11a or IEEE802.11g. A user can receive theomni-directional video contents using a desktop PC, a laptop PC, or even an immersive client with a head mounted
display (HMD).

GPS navigator
Omni-directional camera

PC for GPS

PC for omni-directional camera

automobile with omni-directional camera

ZLUHOHVVFRPPXQLFDWLRQFKDQQHOLAN switch&DPSXV/$1
PC for wireless communication

Figure 2. Hardware comfiguration on the automobile of our telepresence system

video viewers

Figure 1. Overview of the proposed telepresence system

2.1. Hardware components on the automobile
Figure 2 shows the hardware configuration on the automobile of our telepresence system. In the automobile,
there are three PCs. One PC records GPS data including
the positioning data and the time. Another PC is for theomni-directional camera. The omni-directional camera acquires omni-directional progressive video surrounding theautomobile (see Figure 3). The acquired omni-directional
video is transferred to the PC through the FireWire channel.
The PC encodes the omni-directional video (640x480 pixels, 30 fps) to Windows Media Format (1Mbps) by Windows Media Encoder. The other PC is for the wirelesscommunication. This PC has both IEEE802.11a interface
and IEEE802.11g interface. The GPS data and the omnidirectional progressive video are transferred to the campus
LAN through this PC. Table 1 describes the detail specification of the hardware equipped on the automobile.

2.2. Omni-directional video viewer
The omni-directional video viewer converts the omnidirectional video (left image of Figure 4) to the common

Figure 3. Omni-directional camera acquiring
the scenery surrounding the automotile

perspective video (right image of Figure 4) to show theuser the common perspective video. Our telepresence system uses a texture mapping function of GPU (graphic processor unit) for converting video images in real-time by themethod of the literature [5].
A user on a desktop PC or a laptop PC can watch theomni-directional video contents through the viewer invokedby a web browser. We have been implementing an omnidirectional video viewer using Active-X as a plug-in software of a web browser (see Figure 5).
When the user access the web page which provides anomni-directional video content, the omni-directional video

Omni-directional camera
GPS navigatorwireless network interface
PC for omni-directional camera
PC for GPSPC for wireless communication
Multicast relay server

SONY DCR-TRV900+ Hyperboloidal mirror
GARMIN GPS V
IEEE802.11a, IEEE802.11g
CPU: Intel Pentium4 2.53GHz
OS: Windows XP
CPU: Intel Celeron 1.0GHz
OS: FreeBSD
CPU: Intel Pentium4 1.7GHz
OS: FreeBSD
CPU: Pentium4 2.5GHz
OS: Windows 2003 Server

Table 1. Hardware configuration of our telepresence system

Figure 5. Web based omni-directional videoviewer

3. Transmission issue
Figure 4. Omni-directional image and common perspective image

viewer implemented by Active-X is installed automatically.
The user can easily see an omni-directional video content
without care of an omni-directional camera type, parameters of camera, file path of the content, and so on, because a content provider embeds them in an HTML file.
The user can look around the omni-directional video by using a mouse-drag operation.

2.3. Omni-directional video contents
There are two kinds of omni-directional video contents
in the present implementation: stored video contents encoded in advance and live video contents encoded in realtime. Note that stored video contents can be provided as anon-demand-service.
The live video contents are used for simultaneously providing multiple users with the same contents similar to TV
broadcasting. The user can see the live video contents acquired by the omni-directional camera and transferred immediately. The live video contents can be transferred tomultiple users by multicast without increasing the networkload.

In our telepresence system, the communication channel between the automobile and the campus LAN uses
wireless communication facility of IEEE802.11a andIEEEE802.11g. Since the automobile runs in our campus, the condition of wireless communication channel
is not stable. Therefore, some robust transport mechanism is required.
We have implemented the prototype system of the proposed telepresence system to clarify the problem of the multicast transmission using wireless communication channels.
Figure 6 shows the experimental environment.
Now, we discuss the behavior of the network communication of the prototype system. In our experiment, weused “Foundry FirstIron WorkGroup” which has 24 ports of
10/100 Mbps as the LAN switch and “Avaya Access Point
III” as the wireless access point. Wireless access point (1)and (2) connected to port 2 and 1 of the LAN switch, respectively. A desktop PC for the omni-directional viewer connected to port 7 of the LAN switch, and we used three laptop
PCs which have IEEE802.11a interface and IEEE802.11g
interface.
In this experiment, each PC repeatedly did join/leave
operation to the multicast group for the omni-directional
video stream. Each laptop PC moved between wireless access point (1) and (2). Moreover, in the second experiment,

4. Related WorkMulticast relay server

Wireless access point (2)LAN switch

Wireless access point (1)ClientFigure 6. Experimental environment

the automobile moved around the wireless access point (1)and (2), that is, the multicast relay server move between
both wireless access points.
Actually four PCs received the distributed omnidirectional video. The four users could look around thescene in arbitrary directions. The video stream is displayed on the omni-directional viewer at 30 frames/sec,
if the wireless communication channel between the automobile and the campus LAN is stable and the PC uses thesame wireless communication protocol. In this time, theboth transmitting and receiving network loads of the relay server are 1Mbps. When the number of received
omni-directional video viewer increased, the network load
did not increase.
Since the multicast sender moves, the sender’s IP addressof the multicast group changes. In this case, the LAN switch
has to change the multicast tree. However, most multicastenabled LAN switch does not suppose a such situation.
Therefore, some addition multicast setting is required for
the LAN switch.
When the PC for the wireless communication changed
the wireless access point or the wireless communication
protocol, the time delay between the acquisition and thepresentation omni-directional video took about 10 seconds.
Therefore, we need some robust transport mechanism including multicast, which can handle multiple different wireless communication channels simultaneously.
Additionally, if no PCs joined to the multicast group, themulticast traffic sent to the all port of the LAN switch whichcaused the multicast storm. Therefore, the noble multicast
mechanism should be required, especially when multicasting high quality video stream.

In most telepresence systems [3, 4, 8, 11], active cameras
such as panning and tilting cameras are used to present auser view-dependent images of a scene in the distance. This
often suffers from the time delay from the change of viewing direction to the change of displayed image. The timedelay is mainly caused by both the communication between
an observation site and an observed world and the control ofcameras so as to follow the user’s viewing direction. Especially, the former factor depends on the actual distance between the observation and observed sites.
Moreover, in the existing telepresence systems, if a user
want to change the view direction of a scene, he/she haveto control a camera. Therefore, it is difficult that multiple
users share the same telepresence system.
In the IPv6 technology, some multicast protocols which
consider the mobility of the node have been proposed [2, 7].
These multicast protocols can handle the mobility of only
the multicast receivers. Source Mobility Support Multicast(SMM) [6] proposed a multicast protocol considering themobility of the sender. Since this multicast protocol supposes to be used by the Intelligent Transport System (ITS),
it uses the Cellular IP [9] as the underlying network protocol
between the automobile and the access point. This multicast
protocol can not be applied to the other underlying network
protocol except the Cellular IP. Therefore, the more generalmulticast protocol considering the mobility of the sender isrequired.

5. Conclusions
We have developed a networked telepresence system
which easily enables multiple users to look around a remote scene with the omni-directional camera. The system uses a web-browser, and enables users to see omnidirectional video such as common video. In the experiment,
the omni-directional video was distributed through wireless and wired network by multicast protocol, and multiple users could look around the scene in arbitrary directions in real time. The delay between the acquisition andthe presentation of omni-directional video is 10 seconds.
In future work, we should reduce the delay in transmitting
an omni-directional video stream. Also, we have to provide
some robust transport mechanism which can handle multiple different wireless communication channels simultaneously. Moreover, we have designed and developed a new
multicast protocol which has redundant multicast path considering the mobility of the sender.

References[1] S. Ikeda, T. Sato, M. Kanbara, and N. Yokoya. Immersive
telepresence system using high-resolution omnidirectional
movies and a locomotion interface. In Proceedings of SPIEElectronic Imaging, volume 5291, January 2004.
[2] D. Johnson, C. Perkins, and J. Arkko. Mobility Support inIPv6, May 2004. RFC 3775.
[3] A. Krishnan and N. Ahuja. Panoramic image acquisition. In
Proceedings of 1996 IEEE Computer Society Conference on
Computer Vision and Pattern Recognition, pages 379–384,June 1996.
[4] S. Moezzi(Ed.). Special issue on immersive telepresence.
IEEE MultiMedia, 4(1):17–56, January 1997.
[5] Y. Onoe, K. Yamazawa, H. Takemura, and N. Yokoya. Telepresence by real-time view-dependent image generation fromomunidirectional video streams. Computer Vision and Image Understanding, 71(2):154–165, August 1998.
[6] K. Sato, M. Katsumoto, and T. Miki. Source mobility support multicast (smm). IPSJ Journal, 45(2):412–425, February 2004.
[7] K. Suh, D.-H. Kwon, Y.-J. Suh, and Y. Park. Fast Multicast
Protocol for Mobile IPv6, January 2004. draft-suh-mipshopfmcast-mip6-00.txt.
[8] R. Szeliski. Video mosaics for virtual environments. IEEE
Computer Graphics and Applications, 16(2):22–30, March1996.
[9] A. G. Valkó. Cellular ip: a new approach to internet host mobility. ACM Computer Communication Review, 29(1):50–65,
January 1999.
[10] K. Yamazawa, Y. Yagi, and M. Yachida. Omnidirectional
imaging with hyperboloidal projection. In Proceedings ofInternational Conference on Intelligent Robots and Systems,
volume 2, pages 1029–1034, July 1993.
[11] J. Y. Zheng and S. Tsuji. Panoramic representation of scenes
for route understanding. In Proceedings of 10th International Conference on Pattern Recognition, pages 161–167,
June 1990.