International Journal of ITS Research, Vol. 2, No. 1, October 2004

Successive Observations of Trajectories of Vehicles with Plural Video Cameras
Hirokazu AKAHANE *1 Satoshi HATAKENAKA*2
Professor, Faculty of Engineering, Chiba Institute of Technology *1(2-17-1 Tsudanuma, Narashino-shi, Chiba 275-8588, JAPAN, TEL: +81-47-478-0444,
E-mail: akahane(at)ce.it-chiba.ac.jp )Narita International Airport Corporation*2
In this study, a system was developed for successive observations of trajectories of vehicles with images of plural video
cameras installed adjacently along roads through the use of techniques such as image processing and geometrytransformation. This system can more highly equilibrate the observation precision and the observation range than theuse of images of a single video camera. In addition, it can exclude the influence of occlusion where a vehicle isshadowed by another in video images. Furthermore, an observation equation was derived in combination of equationsof triangles that approximate road surfaces with equations of a 3D projective transform. The measurement accuracy ofthis method was evaluated in comparison with results of a precision survey of trajectories of a 1/22 scale vehicle model.
Keywords: vehicle tracking, vehicle behavior, image processing, Kalman smootherand not to disturb traffic flow as a whole.
These requirements for part-time automatic vehicle
control systems are totally different from requirements
for full–time automatic control systems because theformer have to override manual control in case of
emergency avoiding panicking either the drivers or thepassengers in the automatically controlled vehicles as

1. Introduction
In ITS, safe driving support systems that provide
drivers with safety information, warning or automatic
control that overrides manual driving maneuvers in order
to avoid danger are required so as not to confuse thedrivers as well as others who are driving in their vicinity

Table 1. Comparison of techniques for observing behavior of running vehicles
Observation
techniques

Traffic detector

Measurement car withsensors

Video camera withimage processing

Vehicles observed

Unspecified and many

Measurement car andits neighboring cars

Unspecified and many

Successive
observations oftrajectories of vehicles

Not possible

Possible

Possible but limited tocoverage of video
image

GPS, etc.

Transform of image
coordinates

Transform of rotation
speeds of wheels, useof spatial filter, etc.

Not possible( Calculation based on
differences in positions )Not possible( Calculation based on
differences in positions )Difficulty in dataacquisition for transformof coordinates and ininstallation of additional
camerasComparative items

Method for positioning
running vehicles

Method for measuringrunning speeds ofvehiclesDetection of passage ofvehicles at detector
locations
Calculations based ontime differences of
passage of vehicles
sensed by pairs ofdetectors and intervals of
locations of them

Method for measuringrunning accelerationsof vehicles

Not possible

Acceleration sensors of
distortion gauge type,gyroscope type, etc

Difficulty inmeasurement

Easy(Installation of additional
detectors is difficult)

Easy

1/7

Successive Observations of Trajectories of Vehicles with Plural Video Cameras

well as people in the neighboring vehicles that might
still be manually driven. It is indispensable to grasp
manual driving maneuvers, running behavior of thevehicles as their results, and interactions among vehicles
with high precision in order to satisfy such requirements.
In this study, a system for successive observations oftrajectories of vehicles with images of plural video
cameras installed adjacently along roads was developedin order to construct a database of serial numerical dataof positions, velocity, acceleration and so on that
represent microscopic behavior of individual vehicles.
Table 1 shows characteristics of the observation byvideo images in comparison with other observation
methods. The observation system developed in this study
can more highly equilibrate the observation precisionand the observation range than using images of a single
video camera. In addition, it can exclude the influence ofocclusion by framing a vehicle with plural video
cameras at different angles at the same moment.

2.3. Transform of image coordinates
Video image coordinates of vehicles acquired by themethod mentioned in section 2.2 were transformed into
world coordinates in images of each video camera
independently. In our past study, a 2D projective
transform was applied supposing that vehicles moved ona single plane [3]. The parameters of the transform wereidentified based on both image coordinates and world
coordinates of the required number of Ground Control
Points (GCPs) on a road surface.
The errors in transform of coordinates depend on thepositions of vehicles in video images because of theinfluence of characteristics, locations, and angles ofcameras. The evaluations of the variances of the errors
were applied to the smoothing algorithm mentioned
below.

2.4. Matching of vehicles between images ofadjacent cameras

2. Basic framework of observation system

Each vehicle has to match between images ofadjacent cameras in order to connect and smooth its
trajectories that consist of world coordinates transformed
through the use of the method mentioned in section 2.3
in images of each camera.
When a vehicle is framed by both of a pair ofadjacent cameras at the same moment, it can be
manually matched, based on the acquired video images.
When a vehicle is framed by a pair of adjacent cameras
successively, except for a very short spatial gap, it canbe manually matched, based on the video images
acquired at intervals that are roughly proportional to the
speed of the vehicle and the length the spatial gap.
This study applied the manual matching method
mentioned above because automatic and/or real-time
matching was not necessary for our present laboratory
use such as analysis of occurrence processes of traffic
accidents.

The basic composition of this system is thefollowing:

2.1. Synchronous acquisition of video images
Black burst signals and SMPTE time code signals aresupplied from a main camera to sub-cameras via daisy
chain connections in order to synchronize image
acquisition in each camera.
As for observations from open-air sites, GPS-based
synchronizers [1] that generate black burst signals and
SMPTE time code signals from GPS signals are used inorder to synchronize acquisition of images by video
cameras installed at separated sites with no cableconnection, for instance, from one rooftop to another.

2.2. Acquisition of video image coordinates

2.5. Estimation of trajectories of vehicles

The TrackPoint [2] was used for the acquisition of
the video image coordinates of vehicles in each frame.
This application automatically tracks objects designatedby an operator when they come in a video image. It has
an interface by which an operator who monitors video
images can manually modify automatic tracking when it
strays away from an actual trajectory.
So-called features such as corner parts of license
plates and bodies have to be chosen as tracking targets
because of requirements for image processing. Points
where tires contact road surfaces have the advantage oftransform of image coordinates because they are on theroad surfaces. However, those points are not necessarily
practical because they are frequently shadowed by othervehicles in congested traffic flow when video images are
acquired from the front and the rear of vehicles.

The Kalman smoothing algorithm was applied to theleast square estimations of unified trajectories based on
world coordinates transformed independently in imagesof each camera through the use of the method mentioned
in section 2.3 [3]. The estimation algorithm consists ofthe two processes as follows:
1) Management of errors in transform of coordinates
The errors evaluated in section 2.3 are processed.
When a vehicle is framed by both of a pair of adjacentcameras at the same moment, world coordinates can berationally unified using the error variance of coordinatetransform in each camera image.
2) Management of errors caused by heights of tracking
targets from road surfaces

2/7

International Journal of ITS Research, Vol. 2, No. 1, October 2004v

f (x (t )) =aa

XY-plane

Vehicle⎛ X ( t ) + v ( t ) ⋅ c o s θ ( t ) s in η ( t ) ∆ t⎞⎜⎟+ a l ( t ) ⋅ 1 c o s θ ( t ) s in η ( t ) ∆ t 2 ⎟
⎜
2
⎜
⎟
⎜
⎟
+ a r (t) ⋅ 1 cosη (t)∆ t2
2⎜⎟⎜ Y (t) + v (t) ⋅ cosθ (t) cosη (t)∆ t
⎟
⎜
⎟
2
⎜
+ al (t) ⋅ 1 cosθ (t) cosη (t)∆ t ⎟
2
⎜
⎟
⎜
⎟
+ a r ( t ) ⋅ 1 s in η ( t ) ∆ t 2
2
⎜
⎟
⎜
⎟
2
1
&
⎜η (t) + ω (t) ⋅ ∆ t + ω (t) ⋅ 2 ∆ t
⎟
⎜
⎟
2
1
⎜ v ( t ) + a l ( t ) ⋅ ∆ t + a& l ( t ) ⋅ 2 ∆ t⎟⎜⎟⎜ ω ( t ) + ω& ( t ) ⋅ ∆ t⎟⎜ a ( t ) + a& ( t ) ⋅ ∆ t
⎟
l
⎜ l
⎟
⎜ a r ( t ) + a& r ( t ) ⋅ ∆ t⎟⎜ ω& ( t )⎟⎜⎟⎜ a& l ( t )⎟⎜⎟&⎜ a r (t)⎟⎜ h (t)⎟⎝⎠Road surface

ω

Z
Y

X

x

η

θ

Figure 1. Vehicle motion model
The 2D projective transform in section 2.3 assumes
that tracking targets are on the road surfaces. Therefore,
the heights of tracking targets from road surfaces resultin errors in the 2D projective transform. The errors canbe compensated by values of heights.
When a vehicle is framed by both of a pair ofadjacent cameras at the same moment, the height of the
tracking target of the vehicle can be estimated by aprinciple similar to stereo photogrammetry. The Kalmansmoothing algorithm can compensate the errors causedby the heights that are simultaneously estimated by thealgorithm according to the least square criteria as well.

t: time∆t: scanning time
X , Y , Z：world coordinates
v: longitudinal velocity
θ : a gradient of a trajectory
η: a running direction of a vehicle
al : longitudinal acceleration

3. Models of vehicle motion and observation
In this study, the basic framework of the observation
system mentioned in section 2 was revised as follows:

3.1. The state equationar : lateral acceleration

Figure 1 illustrates the concept of the vehicle motion
model newly introduced in this study [4]. Equation (1)indicates the state equation derived from the vehicle
motion model where a height h(t) of a tracking target
from road surfaces was embedded in the Kalmansmoothing algorithm as a constant state variable.
x ( t + ∆ t ) = f ( x ( t )) + q ( t )ω : angular velocity
h: a height of a tracking target from road surfacesκ : random fluctuation in angular accelertion
λ: random fluctuation in longitudinal jerkµ: random fluctuation in lateral jerk&: notation for time derivative(1)

where,⎛ X (t ) ⎞
⎛ 0 ⎞
⎜
⎟
⎜ 0 ⎟
⎜ Y (t ) ⎟⎜⎟⎜η (t ) ⎟
⎜ 0 ⎟
⎜
⎟
⎜
⎟
⎜ν (t ) ⎟⎜ 0 ⎟⎜ ω (t ) ⎟
⎜ 0 ⎟
⎜
⎟
⎜
⎟
x(t ) = ⎜ al (t ) ⎟ , q(t ) = ⎜ 0 ⎟ ,⎜ a (t ) ⎟
⎜ 0 ⎟
⎜ r ⎟
⎜
⎟
⎜ ω& (t ) ⎟⎜ κ (t ) ⎟⎜&⎟⎜ λ (t ) ⎟⎜ al (t ) ⎟⎜⎟⎜ a& r (t ) ⎟⎜ µ (t ) ⎟
⎜⎜
⎟⎟
⎜ 0 ⎟
⎝
⎠
⎝ h (t ) ⎠3.2. 3D projective transform and identificationof its parameters
Equation (2) indicates the 3D projective transformthat was introduced instead of the 2D projective
transform in order to consider gradients of road surfaces:b j X + b2jY + b3j Z + b4j ⎫x j = 1j⎪b9 X + b10j Y + b11j Z + 1 ⎪⎬b5j X + b6jY + b7j Z + b8j ⎪
j
y = j
⎪
b9 X + b10j Y + b11j Z + 1 ⎭

3/7

(2)

Successive Observations of Trajectories of Vehicles with Plural Video Cameras

where,

x, y:

image coordinates

K kj = γ k b11j

bi :parameters

Lkj = α k b7j X 1k + β k b7jY1k + γ k b7j Z1k + γ k b8j

j:
a suffix denotes a camera
The parameters of the 3D projective transform wereidentified by the DLT (Direct Linear Transformation)[5] using world and image coordinates of GCPs that had
to be scattered not only on road surfaces, but in 3D
space.

u j : measurement error in x imge coordinate
v j : measurement error in y imge coordinate

The same pairs of Equation (4) as video cameras thatsimultaneously frame the identical vehicle from different
positions and angles are obtained and collectively
expressed by the following vectors:y (t ) = g ( x (t )) + e(t )(5)where,
⎛ M ⎞
⎛ M ⎞
⎜ j ⎟
⎜ j ⎟
⎜ x (t ) ⎟⎜ u (t ) ⎟, e( t ) = ⎜,y (t ) = ⎜⎟jj ⎟⎜ y (t ) ⎟⎜ v (t ) ⎟
⎜ M ⎟
⎜ M ⎟
⎝
⎠
⎝
⎠

3.3. Approximation of road surfaces by triangles
and formation of observation equation
Road surfaces that consisted of vertical andhorizontal alignments were approximated by triangles
whose vertices were GCPs on the surfaces as follows:

α k ( X − X 1k ) + β k (Y − Y1k ) + γ k ( Z − Z1k ) = 0(3)

where,

M
⎛
⎞
⎜ k
⎟
k
k
k
⎜ A j X (t ) + B j Y (t ) + C j h(t ) + D j ⎟
⎜ k
k
k
k ⎟
⎜ I j X (t ) + J j Y (t ) + K j h(t ) + L j ⎟g( x(t )) = ⎜⎟⎜ E kj X (t ) + F jk Y (t ) + G kj h(t ) + H kj ⎟
⎜ k
k
k
k ⎟
⎜ I j X (t ) + J j Y (t ) + K j h(t ) + L j ⎟
⎜⎜
⎟⎟
M
⎝
⎠

α k , β k , γ k： elements of a normal vectorof an approximation triangle
X 1k , Y1k , Z1k：

coordinates of a vertex of an

approximation trianglek:a suffix denotes an approximation
triangle
The observation equation (4) was approximately
derived when Equation (3) was substituted for Equation(2) eliminating coordinate Z and when h denotes a heightof a tracking target from road surfaces:

3.4. Application of extended Kalman smoothingalgorithm

⎫
j
u
(
t
)
+
⎪
I kj X ( t ) + J kj Y ( t ) + K kj h ( t ) + Lkj
⎪⎪
⎬ (4)
k
k
k
k
E j X (t ) + F j Y (t ) + G j h (t ) + H j⎪jjy (t ) = k+ v (t ) ⎪kkk
I j X (t ) + J j Y (t ) + K j h (t ) + L j⎪⎭
where,
Akj X ( t ) + B kj Y ( t ) + C kj h ( t ) + D kj

The extended Kalman smoothing algorithm wasapplied to the state equation (1) and the observation
equation (5) in order to obtain the least square estimatorof the state vector x(t ) that includes the height h as aconstant and the world coordinates X (t ) and Y (t ) of avehicle at every moment.
In the smoothing process, random fluctuation

Akj = γ k b1j − α k b3j

κ (t ), λ (t ), µ (t ) and measurement errors u j (t ), v j (t ) were

B kj = γ k b2j − β k b3j

assumed to be white noise. In addition, very large values
were set to the variances of measurement errorsx j (t ) =C kj = γ k b3j

u j (t ) and v j (t ) when the camera j did not frame avehicle tracked at time t in order to practically process
the information acquired just by other cameras thatframed it.
The information of image coordinates of a tracking
target of a vehicle at every moment by plural video
cameras could be effectively and rationally integrated
by the smoothing process considering the errors intransform of coordinates and errors caused by a height ofa tracking target from road surfaces.

D kj = α k b3j X 1k + β k b3jY1k + γ k b3j Z1k + γ k b4j
E kj = γ k b5j − α k b7j
F jk = γ k b6j − β k b7j
G kj = γ k b7j
H kj = α k b7j X 1k + β k b7jY1k + γ k b7j Z1k + γ k
I kj = γ k b9j − α k b11j
J kj = γ k b10j − β k b11j

4/7

International Journal of ITS Research, Vol. 2, No. 1, October 2004

Overlapped section A
0ｍ

1ｍ

2ｍ

Camera 3
Overlapped
section B
Camera 2

Camera 1

Camera 2

Camera 3

On-board camera

Camera 1
Figure 2. Horizontal alignment of the course
and angles of video cameras
Curve section
Course height [m]0.10

Figure 4. Sample images

0.08

Mirror0.06
0.04

On-board

0.02

Model vehicle

0.00
0.0

1.0

2.0 3.0 4.0 5.0 6.0
Length along course [m]Figure 3. Vertical alignment of course

4.

Precision verification
experimentbya0.05[m] Tape measure
Figure 5. On-board camera and measuremodel

sample images of the video cameras installed along thecourse. The model vehicle, only the speed of which wasradio-controlled, ran along the guidance groove installedin the center of the course.
Figure 5 illustrates the video camera and the mirror
carried by the model vehicle. This on-board camera thatwas syncronized with the other three cameras installed
along the course aquired images of the divisions of thetape scale pasted on the course. The positions of thevehicle model at every 6 video frames that was
equivalent to every two-tenth of a second were specified
by reading values of the divisions of the tape scale fromthe video images of the on-board camera. The specified
positions were precisely surveyed by a total station
afterwards.
The positions of the GCPs higher than the course
surfase as well as the GCPs located on the course wereprecisely surveyed in order to improve identification ofparameters of the 3D projective transform.

In this experiment, video images of a 1/22 scale
vehicle model that ran along a model course wereacquired using plural video cameras synchronized by themethod mentioned in section 2.1. A series of image
coordinates of tracking targets was obtained by themethod mentioned in section 2.2. Trajectories of themodel of a vehicle were estimated by the Kalmansmoothing algorithm where the 3D observation equationand the revised state equation mentioned in section 3
were embedded. The results of the estimation wereverified in comparison with the precise trajectories
measured by surveying instruments.

4.1. Settings in experiment
Figure 2 shows the horizontal alignment of theexperiment course as well as the locations and the anglesof the video cameras. Figure 3 illustrates the verticalalignment of the course where a flat curve section was
located between a descent gradient and an ascentgradient of about 4 percent. The total length of thecourse was about 100 meters in full-scale. Figure 4 is4.2. Results of precision verification
Figure 6 illustrates triangles that approximate thesurface of the course as well as the trajectory estimated

5/7

Successive Observations of Trajectories of Vehicles with Plural Video Cameras

97.0

Y [m]

96.5

96.0

Vertices of triangles
Surveyed trajectory

95.5

Estimated trajectory

97.0

97.5

98.0

98.5

99.0

99.5

95.0
100.0 100.5 101.0 101.5 102.0 102.5 103.0

X [m]Y coordinate difference[m]Figure 6. Triangles and trajectories
0.08

Height=0.16m

0.06

Height=0.01m

Speed [m/s]

Speed [km/h]
Estimated speed( 0.01m in height)
Estimated speed( 0.16m in height)
Precise difference
speed

1.2

0.04

1.0

0.02

0.8
0.6

X coordinate difference[m]Figure 7. Coordinate differences between
estimations and precise surveys

based on image coordinate values of the tracking target
that was 0.16 meters in height from the course surface,
together with the precise trajectory measured.
Figure 7 indicates differences in X and Y coordinates
between the estimation and the precise surveys
according to the heights of the tracking targets from thecourse surface. The differences tended to increase inproportion as the heights of the tracking targets
increased especially in the gradients at both ends of thecourse. However, the absolute values of the differenceswere less than about 0.1 meters which was equivalent toabout 2 meters in full-scale.
Figure 8 indicates speeds of the model vehicle
calculated from differences of estimated coordinate
values as well as the speeds calculated from differencesof precisely surveyed values. The right vertical axis of

75
60

0.00

-0.02
-0.10 -0.08 -0.06 -0.04 -0.02 0.00 0.02 0.04

90

45

0.4

30

0.2

15
0

0.0
0

2

4

6

8

10
Time [s]Figure 8. Speeds of model vehicle

Figure 8 shows the converted speeds of the model
vehicle in kilometers per hour into full-scale.
The considerable change in speeds during a shortperiod of time could be well estimated. No remarkablebias occurred, despite the defferences in positions.
However, the accuracy of the estimation of the speeds
tended to decrease in proportion as the heights of the
tracking targets increased, as did the accuracy of theestimated positions.
Figure 9 illusrates the differences in X and Y

6/7

Y coordinate difference[m]International Journal of ITS Research, Vol. 2, No. 1, October 2004

in diverging, in weaving, and so on.

2 overlapped sections (A&B in Figure 2)
1 overlapped section (Only A in Figure2)
No overlapped section

6. Acknowledgments

0.03This study was a part of the research project, “Basicand up-to-date research on ITS,” that the National
Institute for Land and Infrastructure Management, theMinistry of Land, Infrastructure and Transport, entrusted
to the Institute of Industrial Science, the University ofTokyo. The authors would like to express appreciation
to both of the institutes, especailly to the members of thereseach project.

0.02
0.01
0.00
-0.01
-0.02
-0.03

-0.01

0.01

0.03

0.05

0.07

X coordinate difference[m]7. References

Figure 9. Coordinate differences according to
the number of overlapped sections[1] http://www.totsu-eng.co.jp/[2] http://www.gen.co.jp/product/index.html[3] H. Akahane: “Vehicle Tracking with multiple Video
Cameras,” Proceedings of the 37th Symposium on
Infrastructure Planning and Management, JSCE, pp. 8996, 1998.
[4] R. Horiguchi, H. Akahane and K. Funabashi:“Estimation of 3D trajectories of probe cars and driving
behavior by the extended Kalman smoothing,”
Proceedings of the 57th Annual Meeting of JSCE, CDROM, 2003.
[5] Japan Society of Photogrammetory and Remote
Sensing: Analytical Photogrammetory (Revised Edition),
JSPRS, 1997.

coordinates between the precise surveys and theestimation where the tracking target was 0.01 meters inheight from the course surface according to the numberof overlapped sections shown in Figure 2. In theestimation with two overlapped sections, all of theobservations, including the overlapped observations by
the two cameras individually in sections A and B, wereused. In the estimation with one overlapped section, theoverlapped observations only in section A were used. In
the estimation with no overlapped section, theobservations by all cameras, except for one of two
cameras in the two overlapped sections, were used.
There was no remarkable increase in the differences ofthe estimation with one overlapped section in thepositions in comparison with the estimation with nooverlapped section. That meets the principle that theleast square estimator of the height of the tracking
targets from the course surface can be obtained based on
observations in even one overlapped section.

Prof. Hirokazu Akahane Received a Doctor ofEngineering degree from the Universityof Tokyo in 1986. Professor of Chiba
Institute of Technology from 1995.
Major research fields are road traffic
control and management. Member of
JSCE, JSTE and IATSS.

5. Conclusions
The system was developed for successive
observations of trajectories of vehicles with images ofplural video cameras installed adjacently along roads
through the use of techniques such as image processing
and projective transform.
The equations of triangles that approximated roadsurfaces were combined with the equations of the
projective transform in order to form the observation
equation. The Kalman smoothing algorithm where theobservation equation was embedded was applied to theobserved data of the 1/22 scale model of a vehicle thatran on the experiment course. As a result, this system
can more highly equilibrate the observation precisionand the observation range than the use of images of asingle video camera.
For further study, this system will be validated on realroads and will be applied as well to microscopic anddynamic analysis of vehicle behavior in traffic accidents,

Satoshi Hatakenaka Received a
Master of Engineering degree from
Chiba Institute of Technology in 2004.
Member of Narita International Airport
Corporation from 2004.

7/7

•
•
•

Received date
Received in revised form (if any)
Accepted date•

Editor