DETECTING MOVING OBJECTS FROM OMNIDIRECTIONAL DYNAMIC IMAGESBASED ON ADAPTIVE BACKGROUND SUBTRACTION
Kazumasa Yamazawa and Naokazu Yokoya
Nara Institute of Science and Technology{yamazawa, yokoya}@is.aist-nara.ac.jp
ABSTRACTThis paper describes a new method for detecting moving
objects with an omnidirectional camera. The proposed
method uses a fixed omnidirectional camera which canacquire images surrounding the camera and a backgroundsubtraction technique based on adaptively updatingbackground images. In subtracting the background image
from the input image, we consider the oscillation ofintensity so that the proposed method can robustly detectmoving objects under the condition that there arefluorescent lights and incoming lights from windows. In
experiments, moving objects have been successfully
detected in indoor environments.
1. INTRODUCTION
When we detect moving objects with a common video
camera, the field of view is limited. There are somemethods for solving this problem. Although most of themethods use a PTZ (panning, tilting and zooming)
camera[1] or multiple cameras[2]. an omnidirectional
camera[3,4] has a number of advantages. For example, if
we use background subtraction for detecting moving
objects, with a PTZ camera, we must synchronize thePTZ of the camera and the PTZ of the background image.
Moreover, the camera has a limited field of view. When
we use multiple cameras, the system becomes complex.
On the other hand, an omnidirectional camera, which canacquire an omnidirectional view at a time, is suitable fordetecting moving objects in the surrounding environment.
However, most of conventional methods suffer fromoscillations of intensity, such as fluorescent lights. This
paper describes a new method for robustly detecting
moving objects with an omnidirectional camera and abackground subtraction technique based on adaptively
updating background images.
This paper is structured as follows. First, we describe
some detail of the proposed method for detecting moving
objects with a fixed omnidirectional video camera. Next,
we present experimental results. Finally, we summarize
the present work.

2. DETECTING MOVING OBJECTS
2.1. Outline of the method
The proposed method uses an omnidirectional camera anda background subtraction technique. The fixed
omnidirectional camera acquires omnidirectional images
surrounding the camera. First, we estimate thebackground image and amplitude image from the first
several frames of the input omnidirectional image
sequence. After the background image and amplitude
image are estimated, we detect moving objects from the
input images, the background image and the amplitude
image, updating the background image and the amplitude
image. Next, we generate common perspective images ofmoving objects, calculating fan-shaped bounding-boxes
which enclose the detected moving objects in theomnidirectional image.
This section is structured as follows. First, we briefly
describe the omnidirectional camera: HyperOmni Vision.
Next, we describe the method for detecting moving
objects from omnidirectional video stream. Finally, we
describe the method for calculating common perspective
views of detected moving objects.
2.2. Omnidirectional camera: HyperOmni Vision
We use HyperOmni Vision[5] as an omnidirectional
camera in the present work. HyperOmni Vision iscomposed of a hyperboloidal mirror and a video camera
as illustrated in Fig.1. The camera acquires anomnidirectional scene reflected by the hyperboloidal
mirror. The hyperboloidal mirror is constructed of a
hyperboloid of two sheets of revolution, which has two
focal points ( OM and OC ). The camera lens center is fixedat the focal point OC . Given a world coordinate ( X , Y , Z )and an image coordinate (x, y ) as shown in the Fig.1(b),the shape of hyperboloidal mirror and the two focal
points are represented as follows.

X 2 +Y 2 Z 2− 2 = −1 ,a2b(0,0,+c ) , (1)Mirror Shape :Inner focal point OM :Outer focal point OC (Camera lens center): (0,0,−c ) ,where c = a 2 + b 2 .
A ray going from the point P( X , Y , Z ) in 3D toward
the inner focal point OM is reflected by the mirror andpasses through the outer focal point OC intersecting theimage plane at the point p(x, y ) . Therefore, the projectionof HyperOmni Vision is not common planar perspective,
but satisfies the single viewpoint constraint. The
relationship between P( X , Y , Z ) and p(x, y ) is given by:x=()

f b2 − c2 X

2222222222

2

By using the above equation, an omnidirectional
input image can be converted to a common perspective
image at the viewpoint OM [6] (see Fig.2). In this study,
we generate common perspective images of detectedmoving objects.
Focal Point
Hyperboloidal Mirror

ZOMcbYOX

xcfOCyP(X,Y,Z)p(x,y)First several frames?

YN
Estimate background image Iave
and amplitude image σPart of moving object?

YNUpdate background image Iave
and amplitude image σUpdate amplitude image σ

Fig.3 Flow chart of updating Iave and σ at each frame.

,(b + c )(Z − c ) − 2bc X + Y + (Z − c ) (2)f (b − c )Y.
y=(b + c )(Z − c ) − 2bc X + Y + (Z − c )2For each pixel inomnidirectional input image

Image Plane
Camera Center(a) Overview(b) Construction
Fig.1 HyperOmni Vision.

2.3. Detecting moving object areas from omnidirectional input images
The proposed moving object detection method is basedon background subtraction. The background subtractionis carried out at each pixel. Figure 3 shows the flow chartof updating Iave and σ at each frame.
We model the intensity of background pixel asfollows.
I = Iave + σ sin(2π*ω*t) + k*noise,
(3)
where
I : intensity of background pixel,
Iave : average of I in time,
σ: amplitude of oscillation of intensity,ω : frequency of oscillation of intensity,t : time,k : -1, 0, or +1,
noise : noise which is dependent only on the video camera.
The term σ sin(2π*ω*t) in Eq.(3) means a flicker of afluorescent light, CRT, etc. The term k*noise means anoise which is dependent only on the video camera due togain-up etc. Therefore, the intensity has the range of:Iave-σ-noise ≦ I ≦ Iave+σ+noiseFig.2 Input image (left) and generated image (right).

(4)

If the intensity I of the pixel is outside the range, we
decide that the pixel is a part of a moving object. First, wepredetermine the parameter noise which is dependent
only on the video camera setup. Next, we consider theslow change of the background. When a pixel isdetermined to represent the background, the parameters
Iave and σ are updated by the following equation.

I’ave = Iave*(n-1)/n + I*1/n,σ’ =σ*(n-1)/n + 2*(I-Iave)*1/n,(5)wheren : parameter of the update speed .
(When n increases, updating speed decreases.)Eq. (5) is computed for all the pixels determined as
the background on each frame. We use I’ave and σ’ as
Iave and σ in the next frame.
For a pixel estimated at a part of moving object, we
do not calculate Eq. (5). However, in order to handle
cases that an object is put or taken and the rapid changeof background, when the pixel is estimated as a part ofmoving objects, the parameter σ is updated using thefollowing equation.

σ’ = σ*(m-1)/m + 2*(I-Iave)*1/m,

(6)

where
m: parameter for the fading out speed of the detected object.
(When m increases, the speed of fading out decreases.)Normally, we determine m larger than n. Therefore,
if the detected moving object stays at a position, σincreases, and then the detected moving object fade out.
If we set m infinity, the detected new objects do not fadeout.
2.4. Generating common perspective images ofdetected objects
The method described can detect pixels of moving objects
adaptively updating the background image continuously.
This section describes the generation of common
perspective images of detected moving objects. The
method is as follows.
1.

2.
3.
4.

Compute a histogram with respect to thelongitude direction θ of the detected movingobjects.
Detect blobs whose values are more than 0 in
the histogram as the ranges of the longitude ofthe moving objects.
Compute the maximum and minimum latitudeof the detected moving object for each blob.
Generate the common perspective images basedon the computed ranges of the longitude and thelatitude.

The generation of perspective images focusing on
detected objects is important in some surveillance
applications and can be carried out in real time by using
the technique in [6].

3. EXPERIMENT
We have prototyped a surveillance system using theproposed method on a Linux PC. The specification of thePC is as follows.
OS: Redhat 7.1
CPU: Pentium4 2GHz
Memory: 512MBFigure 4 shows the screen of the system. The screen
is divided into four images. The three images showcommon perspective images of the largest three movingobjects. The other shows an input omnidirectional image,histogram, etc.
We set the parameters as follows.
0 ≤ I ≤ 255
noise = 4n=360, m=1080

The parameters are determined empirically. However,the selected parameters have been proven to be suitablefor different indoor environments such as lobby andmeeting rooms.
In this experiment, there were two walkers and aputted doll under the condition that there were fluorescentlights and incoming lights from windows (see Fig.5).
Figure 6 shows the stream of screen images. The system
can detect moving objects robustly and can update images
at intervals of 0.2 sec.
4. CONCLUSIONS
We have proposed a new method for robustly detecting
moving objects with an omnidirectional video camera.
The proposed method uses a background subtraction
technique which is based on adaptively updatingbackground images. In subtracting the background image
from the input image, we consider the oscillation ofintensity so that the proposed method can robustly detectmoving objects under the condition that there arefluorescent lights and incoming lights from windows. We
have also developed a prototype system for surveillance
application. The system has been successfully
demonstrated in indoor environments.
The future work includes the improvement of updating
the areas of detected moving objects and the tracking ofdetected moving objects between adjacent frames.

5. REFERENCES[1] T.Wada and T.Matsuyama, “Appearance sphere:
Background model for pan-tilt-zoom camera,” Proc. ICPR96,vol.A, pp.718-722, 1996.
[2] I.Mikic, S.Santini, and R.Jain, “Tracking Objects in 3D
using Multiple Camera Views,” Proc. ACCV2000, pp.234-239,2000.
[3] R.Miki, N.Yokoya, K.Yamazawa, and H.Takemura, “A
Real-time Surveillance and Monitoring System Using Multiple
Omnidirectional Video Cameras”, Proc. ACCV2000, pp.528534, 2000.
[4] T.Sogo, H.Ishiguro and M.M.Trivedi, “Real-time target
localization and tracking by N-ocular stereo,” Proc. IEEE
Workshop on Omnidirectional Vision, pp.153-160, 2000

Fig.4 Screen image of the system.

2 walkers

Doll on a deskof reception[5] K.Yamazawa, Y.Yagi, and M.Yachida. “Obstacle De-tection
with Omnidirectional Image Sensor HyperOmni Vision,” Proc.
Int. Conf. on Robotics and Automation, pp.1062-1067, 1995.
[6] Y.Onoe, K.Yamazawa, H.Takemura, and N.Yokoya,“Telepresence by Real-Time View-Dependent Image Generation from Omnidirectional Video Streams,” Computer Vision
and Image Understanding, Vol. 71, No.2, pp.154-165, 1998.

Entrance

Fig.5 Experimental environment.

Figure 6 Stream of input image and detected object views.