Proceedings 2003 IEEE International Symposium on
Computational Intelligence in Robotics and Automation
July 16-20, 2003, Kobe, Japan

Networked Video Surveillance Using Multiple Omnidirectional Cameras
Shinji Morita
Graduate School of Information Science
Nara Institute of Science and Technology

Kazumasa Yamazawa
Graduate School of Information Science
Nara Institute of Science and Technology
yokoya.aist-nara.ac.jp/~yamazawa/and estimates the directions of the objects. The client
receives the directions and the omnidirectional videofrom the servers through network, estimates thepositions of the objects, and shows the user thepositions and the images of the objects.

Abstract
Remote surveillance is widely utilized at bank, shop,office, home and so on. In most conventional remotesurveillance systems, fixed or active cameras with anarrow field of view are generally used in order toacquire an image of the remote site. This paper proposes
new networked surveillance systems. The proposed
surveillancesystem,whichuses
multiple
omnidirectional cameras and network, is based on aserver/client model: the server computers, each ofwhich is connected to an omnidirectional video camera,
are placed in the surveillance area and the clientcomputer is placed in a user side. The servers detectmoving objects and estimate their directions fromsensors. The client estimates object positions from theirdirections received from the servers and presents
object-centered perspective images to the user. In
experiments, the implemented proposed system can dothose at real-time.

2 Omnidirectional Camera: HyperOmni Vision
We use HyperOmni Vision[5] as an omnidirectional
camera in the present work. HyperOmni Vision iscomposed of a hyperboloidal mirror and a video camera as
illustrated in Fig.1. The camera acquires anomnidirectional scene reflected by the hyperboloidal
mirror. The hyperboloidal mirror is constructed of a
hyperboloid of two sheets of revolution, which has two
focal points ( OM and OC ). The camera lens center is fixedat the focal point OC . Given a world coordinate ( X , Y , Z )and an image coordinate (x, y ) as shown in the Fig.1(b),the shape of hyperboloidal mirror and the two focal points
are represented as follows.

1 Introduction
Remote surveillance is widely utilized at bank, shop,office, home and so on. Although most of the methods
use a PTZ (panning, tilting and zooming) camera[1] ormultiple cameras[2]. An omnidirectional camera[3,4]has a number of advantages. For example, if we use
background subtraction for detecting moving objects,
with a PTZ camera, we must synchronize the PTZ of thecamera and the PTZ of the background image. When weuse multiple cameras, the system becomes complex. On
the other hand, an omnidirectional camera, which canacquire an omnidirectional view at a time, is suitable fordetecting moving objects in the surrounding
environment. However, as for most of conventional
methods
only
one
computermanagesallomnidirectional cameras. Moreover it suffers fromoscillations of intensity, such as fluorescent lights.
This paper describes networked video surveillanceusing multiple omnidirectional cameras and robustly
detecting moving objects with a background subtraction
technique based on adaptively updating backgroundimages. Our system is server/client model. Each server
has an omnidirectional camera, detects moving objects,

0-7803-7866-0/03/$17.00 ©2003 IEEE

Naokazu Yokoya
Graduate School of Information Science
Nara Institute of Science and Technology
yokoya.aist-nara.ac.jp/~yokoya/

X 2 +Y 2 Z 2− 2 = −1 ,a2b
Inner focal point OM : (0,0,+ c ) ,Mirror Shape :Outer focal point OC(1)(Camera lens center): (0,0,−c ) ,where c = a 2 + b 2 .
A ray going from the point P( X , Y , Z ) in 3D
toward the inner focal point OM is reflected by themirror and passes through the outer focal point OCintersecting the image plane at the point p(x, y ) .
Therefore, the projection of HyperOmni Vision is notcommon planar perspective, but satisfies the singleviewpoint constraint. The relationship between
P( X , Y , Z ) and p(x, y ) is given by:1245

x=()

f b2 − c2 X,(b + c )(Z − c ) − 2bc X + Y + (Z − c )f (b − c )Y.
y=(b + c )(Z − c ) − 2bc X + Y + (Z − c )
22222

22(2)

22

2

If we have multiple servers and multiple clients,
real-time surveillance is realized by distributing
processing as the above processing.

223.2 Moving Object Detection
The proposed moving object detection method is based on
background subtraction. The background subtraction iscarried out at each pixel.
We model the intensity of background pixel asfollows.

By using the above equation, an omnidirectional
input image can be converted to a common perspective
image at the viewpoint OM [6] (see Fig.2). In this study,
we generate common perspective images of detectedmoving objects.

I = Iave + σ sin(2π*ω*t) + k*noise,(3)whereI : intensity of background pixel,
Iave : average of I in time,
σ: amplitude of oscillation of intensity,ω : frequency of oscillation of intensity,t : time,k : -1, 0, or +1,noise : noise which is dependent only on the video camera.

Focal Point
Hyperboloidal Mirror

ZOMcbYOXxc
fy

P(X,Y,Z)p(x,y)

Image Plane

OCCamera CenterThe term σ sin(2π*ω*t) in Eq.(3) means a flicker of afluorescent light, CRT, etc. The term k*noise means anoise which is dependent only on the video camera due
to gain-up etc. Therefore, the intensity has the range of:(a) Overview(b) Construction
Fig. 1 HyperOmni Vision

Iave-σ-noise ≦ I ≦ Iave+σ+noise

If the intensity I of the pixel is outside the range, wedecide that the pixel is a part of a moving object. First,
we predetermine the parameter noise which isdependent only on the video camera setup. Next, weconsider the slow change of the background. When apixel is determined to represent the background, theparameters Iave and σ are updated by the following
equation.

Fig. 2 Input image (left) and generated image (right)
3(4)Networked Surveillance System

3.1 Outline of the method
Our surveillance system is server/client model. The eachserver is at surveillance area and has an omnidirectional
camera. The client is at the user side.

I’ave = Iave*(n-1)/n + I*1/n,σ’ =σ*(n-1)/n + 2*(I-Iave)*1/n,
wheren : parameter of the update speed .
(When n increases, updating speed decreases.)

The process of each server is the following steps:
1. It acquires omnidirectional image at video-rate.
2. It detects moving objects and estimates thedirections of the objects.
3. It sends the omnidirectional video and thedirections to the client through network.

(5)

Eq. (5) is computed for all the pixels determined as
the background on each frame. We use I’ave and σ’ as
Iave and σ in the next frame.
For a pixel estimated at a part of moving object, we
do not calculate Eq. (5). However, in order to handle
cases that an object is put or taken and the rapid changeof background, when the pixel is estimated as a part ofmoving objects, the parameter σ is updated using thefollowing equation.

The process of client is the following steps.
1. It receives the video and the directions from theservers.
2. It estimates the positions of the objects.
3. It shows the positions and the images of theobjects to the user.

1246

4.
σ’ = σ*(m-1)/m + 2*(I-Iave)*1/m,
(6)
where
m: parameter for the fading out speed of the detected object.
(When m increases, the speed of fading out decreases.)

5.

DVTS can change the bandwidth adaptive forthe end to end network condition.
Digital Video stream can be transported toanywhere through the network.

3.4 Object Position Estimation
The client receives the directions of the objects andestimates the positions of the objects as illustrated in
Fig.4. We estimate the polygonal area where thedirections overlap as the position of the objects. At last,a perspective image is computed from the received
omnidirectional video and shown to the user. The
computation of perspective images focusing on detectedobjects is important in some surveillance applications
and can be carried out in real time by using thetechnique in [6].

Normally, we determine m larger than n. Therefore, if
the detected moving object stays at a position, σincreases, and then the detected moving object fade out.
If we set m infinity, the detected new objects do not fadeout.
Next, we estimate the directions of the objects asfollows.
1. Compute a histogram with respect to thelongitude direction θ as illustrated in Fig.3.
2. Detect blobs whose values are more than 0 in thehistogram as the ranges of the longitude of themoving objects.
3. Compute the maximum and minimum latitude
for each blob as the direction of the object.
4. Send the directions of the objects to the client.

: Sensor

Range of direction: Range of direction

Θα=(θα1, θα2)
pixelsΘβ=(θβ1, θβ2)

: Existence area

θFig. 4 Existence areas of focused objectsr

Areas of objects

0Histogram

4Experiment
We have prototyped a remote surveillance system using
the proposed method. The system configuration and its
hardware specification are illustrated in Fig.5 andTable.1. In the experiment, the surveillance area is room
B308 [the second floor] of our institute, and the client isplaced in room B211 [the first floor]. The surveillancearea is 9.2m x 9.2m. Three omnidirectional cameras areplaced at (0.0m, 0.0m), (4.5m, 0.0m), and (3.0m, 4.5m).
The height of each omnidirectional camera from thefloor is 1.7m. We set the parameters as follows.
0 ≤ I ≤ 255 , noise=4, n=360, and m=1080.
The parameters are determined empirically. However,the selected parameters have been proven to be suitablefor different indoor environments such as lobby andmeeting rooms.
In this experiment, there were two walkers and aputted object under the condition that there werefluorescent lights and incoming lights from windows.
The displayed image to the user is illustrated as Fig.6.
Fig.7 illustrates the acquired omnidirectional images
and the results of the detection on the each server. Fig.8
illustrates the perspective images which are shown tothe users on the client. In the client side, the estimatedposition was refreshed at 0.2sec. The perspective image

θFig. 3 Generating a histogramusing polar-coordinates conversion

3.3 Omnidirectional Video Transport
Our system uses Digital Video Transport System(DVTS)[7] for transportation of omnidirectional video
through the network. The DVTS is an application fortransportation of IEEE1394 (FireWire) Digital Video
stream through the network and has the following
characteristics.
1. DVTS consumes about 33Mbps as network
bandwidth to transport the high quality digital
video stream (720x480 pixel, 30fps).
2. DVTS also transports auditory informationsynchronized with the visual scenes.
3. DVTS compress images to IEEE1394 DigitalVideo Stream and develops it with a DV
recorder being the product for the consumer
market.

1247

was refreshed at 0.05sec. The time from the acquisitionof the omnidirectional images to the display of theperspective images is 1sec.

HyperOmni Vision

5Conclusions
This paper describes networked video surveillanceusing multiple omnidirectional cameras and robustly
detecting moving objects with a background subtraction
technique based on adaptively updating backgroundimages. Our system is server/client model. Each server
has an omnidirectional camera, detects moving objects,
and estimates the directions of the objects. The client
receives the directions and the omnidirectional videofrom the servers through network, estimates thepositions of the objects, and shows the user thepositions and the images of the objects. In theexperiment, the system has been successfullydemonstrated in door environments.
The future work includes the improvement of
updating the areas of detected moving objects and thetracking of detected moving objects between adjacentframes.

ServerDV camera
IEEE1394
S cableClientDV recorder
S cableIEEE1394
Campus LAN(100Mbps)

observer

References

Fig. 5 Configuration of surveillance system[1] T.Wada and T.Matsuyama. Appearance sphere:
Background model for pan-tilt-zoom camera, In

Table 1 Hardware configurationof surveillance system
Computer
CPU: Pentium4 2GHz
(Sever, Client)Memory: 512MB
OS: Redhat 7.2
Network100Base-T
Image transport system
DVTS Ver.0.9α22

Proc. ICPR96, vol.A, pages 718-722, 1996.
[2] Mikic, S.Santini, and R.Jain. Tracking Objects in
3D using Multiple Camera Views, In Proc.
ACCV2000, pages 234-239, 2000.
[3] R.Miki, N.Yokoya, K.Yamazawa, and H.Takemura.
A Real-time Surveillance and Monitoring System
Using Multiple Omnidirectional Video Cameras, In
Proc. ACCV2000, pages 528-534, 2000.
[4] T.Sogo, H.Ishiguro and M.M.Trivedi. Real-timetarget localization and tracking by N-ocular stereo,
In Proc. IEEE Workshop on Omnidirectional Vision,pages 153-160, 2000.
[5] K.Yamazawa, Y.Yagi, and M.Yachida. Obstacle
Detection with Omnidirectional Image Sensor
HyperOmni Vision, In Proc. Int. Conf. on Robotics

Sensor3

and Automation, pages 1062-1067, 1995.
[6] Y.Onoe, K.Yamazawa, H.Takemura, and N.Yokoya.

Sensor2

Telepresence by Real-Time View-Dependent Image
Gen-eration from Omnidirectional Video Streams,Sensor1
Fig. 6 Displayed image to the user

Computer Vision and Image Understanding, 71(2):154-165, 1998.
[7] A.Ogawa, K.Kobayashi, O.Nakamura, and J.Murai.
Design and Implementation of DV Stream OverInternet, In IWS Internet Workshop, 1999.

1248

Camera1
Camera2
Camera3
Fig. 7 Input images and detected objects

1249

Upper left

Upper right
Lower rightFig. 8 Displayed image sequence

1250

Estimated position