7th World Congress on Intelligent Transport Systems, Torino, 2000

A MANUAL OF VERIFICATION PROCESS FOR
ROAD NETWORK SIMULATION MODELS- AN EXAMINATION IN JAPANRyota Horiguchi
Doctor Researcher, Institute of Industrial Science, University of Tokyo
IIS-Building-C-Cw504 4-6-1 Komaba Meguro 153-8505 Tokyo Japan
Phone: +81-3-54526419, Fax: +81-3-54526420, E-mail: poepoe@nishi.iis.u-tokyo.ac.jp
Masao Kuwahara
Professor, Institute of Industrial Science University of Tokyo
IIS-Building-C-Cw504 4-6-1 Komaba Meguro 153-8505 Tokyo Japan
Phone: +81-3-54526419, Fax: +81-3-54526420, E-mail: kuwahara@nishi.iis.u-tokyo.ac.jp
Toshio Yoshii
Associate Professor, Kochi Institute of Technology
Tosayamada Kochi 782-8502 Japan
Phone: +81-887-57-2406, Fax: +81-887-57-2420, E-mail: yoshii@infra.kochi-tech.ac.jp

SUMMARY
This paper at first introduces an examination in Japan to standardize traffic simulation models.
The basic idea of the standardization here is to estimate abilities of existing models how to
reproduce traffic conditions through verification and validation. Verification implies
qualifying tests using virtual data sets in order to make a connection between the simulation
model and the traffic-engineering theory clear, while validation means an evaluation processwith real world data. Subsequently to the general introduction, the verification process will be
detailed with its philosophy and basic test configurations to verify models’ functions
concerning to 1) vehicle generation, 2) bottleneck capacity at simple road sections, 3)capacity of merging/diverging areas, 4) traffic jam growing/shrinking with propagation ofshock waves, 5) capacity of left/right turn at an intersection, and 6) drivers’ route choice
behavior. In the last part of this paper, we briefly state the on-going project to compare some
popular simulation models in Japan.

INTRODUCTION
Necessity of Standard to Evaluate Traffic Simulation Models
Nowadays, we may find dozens of simulation models in all over the world. In general,
network simulation models handle with wide urban area containing various traffic situations,so that their algorithms to reproduce traffic flow is necessary to be simplified in some degree.
Even in so-called “microscopic” models, vehicles are just moving by a-priori car following
principal that does not give a guarantee to be applicable in overall traffic context. Therefore it
is impossible for simulation models to perfectly reproduce traffic conditions, and we would
like to know how they do well.
Developers of simulation models would have provided the idea how to work the calculation1

7th World Congress on Intelligent Transport Systems, Torino, 2000

algorithms. However it is difficult to understand everything from literatures only. We still
have less information on what model parameters they have or how the traffic conditions they
will reproduce, as if their hearts are “black box”. Such situation might be a barrier for users to
select an appropriate simulation model for their subject.
One measure to remove this barrier is to apply simulation models to real traffic data and tovalidate their performance of reproduction. The importance of model validation using real
traffic data was suggested1) so far. However, this sort of filed application still gives us site
limited aspects. When the developer of each simulation model individually achieve validation,
we still face the difficulty to compare several models and to choose appropriate one among
them. We need some sort of standards to evaluate the simulation models.

Process of Simulation Model Development and Application
In order to cope with this problem, we have proposed the standard certification process for thedevelopment of simulation models. The certification process contains five stages described asfollows. Through the last two stages, a simulation model adduces evidence of its availability
and then certifies itself.
Specification – Determination of model requirements and specifications
If a model wants to be widely used, there must be a common recognition on its specifications,i.e. system input and output items and its behavior guaranteed by the system. For this purpose,
requirements must be organized to determine what kind of traffic phenomena must be handled
on the basis of consideration of application purpose, and then determine the model
specifications at this stage.
Modeling – Contrivance of the model operation principle
Contrivance of the model operation principle consists of the process of constructing thealgorithm complying with the model specifications, and of deciding how such algorithm is to
be incorporated into the model. Here the originality of each developer plays an important roleeven when the same specification is complied with.
Implementation – Programming and debugging
This process consists of programming to run the operation principle contrived in the previous
stage on a computer and debugging to check if the computer operates according to thealgorithm. Debugging must be distinguished as an operation different in nature fromverification described in the next.
Verification – Qualify tests with virtual data
This stage is to confirm that the implemented model can reproduce the traffic phenomena
considered in the model specification stage, thereby verifying that the operation principle isjustified. In this case, an object of comparison with the simulation result is the trafficengineering theory established already. Generally, verification involves extraction andverification of individual highlighted phenomena, one by one, while using virtual data with
ideal conditions so that they can be free from various actual restraints, such as the data
accuracy, reliability, availability, etc.
Validation – Evaluation of validity using real world data
This stage is to evaluate the practical applicability of the model using data available in an
actual world. Assume that the adequacy of the model operation principle has been verified in
the previous stage. The model is not considered to be practically applicable, however, if the2

7th World Congress on Intelligent Transport Systems, Torino, 2000

model specification itself is incomplete or the actual traffic situation can not be sufficiently
reproduced due to the realistic problem on input data acquisition or parameter calibration.
Furthermore, the model performance as a system should also be confirmed, such as whether
or not the execution of the model can be performed within a practically applicable time.

Japanese Activity for Standardization of Traffic Simulation (WG5/WS)
The standard certification process described above is currently discussed in the working group(WG5/WS) under the technical committee of Japan Society of Civil Engineers to get mutual
agreement among traffic engineers. WG5/WS is conducting “Clearing-House” of information
about traffic simulation models2) on Internet. The developers and the users of simulation
models can publish their experience with verification and validation results through theclearing-house. Following materials are also available on the clearing-house at present.
Draft Version of the Manual of Standard Verification Process for Traffic Simulation Models3)
This manual describes minimal verification items mainly for so-called network simulation
models. The subsequent chapter of this paper focuses on the main purports of the verification
manual and then give an outline of the substantive verification steps.
Standard Benchmark Data Set for Validation of Traffic Simulation Models
Validation requires data on simulation inputs including traffic demand and operation as wellas highly reliable data representing the traffic situation to be compared with the simulation
result. Such data collection has been a substantial burden for model developers, hindering
validation. With this background, we have proposed the desirable validation process using
common benchmark data sets, which is collected from real world with highly reliable
preciseness4). WG5/WS now prepare five data set for different categories of road networks
like urban streets, inner-city highways, etc.

MANUAL OF STANDARD VERIFICATION PROCESS
Fundamental Concept
Verification is a sort of virtual test using ideal network and demand configurations to qualify
basic phenomena on road traffic. Simulation has an advantage in applicability to theories
while theories can give us a general criterion to evaluate the traffic phenomena. Accordingly,
verification is not to expect exact agreement of the simulation result with the theoreticalvalues. The objective of verification is to define the model characteristics while confirming
correlation with or difference from the theory. Establishing this linkage will provide us very
helpful information to understand the model characteristics. At the same time, it is considered
important to define the relationship between certain model parameters and model behavior.
We may now find two categories of network simulation models. One is that maintain traffic
flows with vehicle lists or fluid approximation according to the macroscopic flow
characteristics, like flow-density or density-speed relationship, given to each link or section.
Another type of simulation models is employing vehicle driving behaviors including carfollowing and lane changing. Here, let us say former as Q-K type and latter as C-F (carfollowing) type.
The biggest difference of these two types is that Q-K type explicitly, or implicitly, gives thecapacity to a link or a section, while C-F type basically does not have a parameter directly
implies the capacity of link. As the concept of verification is to compare the simulation3

7th World Congress on Intelligent Transport Systems, Torino, 2000

models’ behavior with the established theory that is normally on macroscopic view, thedetailed description of verification step would be separately defined for each category, i.e.:a) For the simulation models of Q-K type, the verification implies that the simulation result
really agrees to the given flow characteristics. In this sense, the verification process of
this type is to be said as a kind of self-consistency check.
b) For the simulation models of C-F type, the model parameters are concerning toindividual driver’s characteristic. Therefore, the verification process of this type shouldmake clear the relationship between those model parameters and the flow characteristic
reproduced by the simulation. In this sense, we may say the verification is a sort ofsensitivity analysis.

Features to be Considered through Simulation
We have included six basic features in the verification manual, which should be at minimum
considered by the network simulation models.
Generation of Vehicles and Flow Conservation
For implementation of simulation, it is necessary to generate the traffic at the entry end
according to the arrival distribution of vehicles from outside the study area. Some type of
pattern such as random arrival or uniform arrival to be assumed according to the type or roadconcerned and the size of traffic volume. Verification process requires whether the generation
pattern assumed in the model really achieved. It should be also checked whether the numberof vehicles generated in a certain time period is equal to the given volume.
Once a vehicle generated, it must not disappear until it reaches its destination. Even in thecase that vehicle queue spills out of study area, newly generated vehicles are added to the endof the point queue outside the network and will flow into the network after sufficient time
period. Simulation models, so that, must keep this flow conservation law not only at every
links but also outside of network.
Bottleneck Capacity / Saturation Flow Rate at Link’s Downstream End
As the discharging flow rate from a bottleneck section like sags or tunnels contributes to thereproduction accuracy of the delay caused by the congestion at the bottleneck, it is essentialthat the capacity of the bottleneck should be reproduced in a stable manner during simulation.
Not only sags or tunnels but also merging and diverging sections can be the most remarkablebottleneck of highways. At the congesting merging section, the travel time on eachapproaching branch may vary with the merging ratio if the capacity of the merging section isconstant. Contrary, the capacity of the diverging section is constrained by the capacities ofdownstream links and may change depending on the proportion of the demand to each branch.
The verification step includes these merging and diverging configurations.
Even on surface streets in under-saturated conditions, a vehicle may have the delay comingfrom the stop and go at signalized intersections. The outflow from an intersection continues atthe saturation flow rate till vehicles retained during the red phase are gotten rid of. It is
important to clarify how the saturation flow rate is reproduced in the simulation model sameas the bottleneck capacity.
Growing and Shrinking Traffic Jam Consistent with Shock Wave Theory
When traffic jam beginning at a bottleneck grows to the upstream link, the traffic that need4

7th World Congress on Intelligent Transport Systems, Torino, 2000

Bottleneck capacity: C1
not pass through this bottleneck may be also affected. As Arrival: Aa difference in the jam’s growing/shrinking speed resultsin difference in the degree of influence on the total delay
Free flow
Congestion
Shock wave speed: v1
upon whole network, it is important to reproduce thisphenomenon by using physical-queue to reasonably Arrival: A
Bottleneck capacity: C2 > C1
maintain the traffic density of congestion. The
verification of this phenomena is made by comparing
Congestion
Free flow
the shock wave speed observed during the simulation
Shock wave speed: v2
Qwith the one derived from the shock wave kinematics, as
shown in Fig.1 .
A
v2

C2

For surface streets, on the other hand, even if a Flowv1
C1
signalized intersection is under-saturated, the vehicle
queue grows and shrinks in every cycle. The tail of theK
Densityqueue moves with some time lag from the begging ofgrowing speed of jam is determined bygreen phase because of drivers’ response delay at Fig.1: The
the arrival demand and the bottleneck capacity.
departure. When two signalized intersections are close
along a street, this time lag and the offset of signal phases are strongly affected to thethroughput along the street. Therefore, the simulation model that is considering signal control
effect must reasonably reproduce this phenomenon including shock wave propagation.
Gap Acceptance of Right(Left)-Turn at a Signalized Intersection
In ordinary streets, it is a daily observation that vehicles waiting for right(left)-turn in thesignalized intersection hinder travel of following vehicles, resulting in congestion. Suchvehicles are waiting to find out a gap in the opposing straight-through traffic in the green
phase. Model verification is made by changing the demand of opposing straight-throughtraffic and model parameters to define how decline the capacity of yield right(left)-turn.
Drivers’ Route Choice Behavior
Modeling for drivers’ route choice behavior considered in simulation is classified as follows:a) Dynamic route choice model not incorporatedb) Dynamic User Optimal (DUO) principle incorporatedc) Dynamic User Equilibrium (DUE) assignment incorporatedd) Probabilistic route choice incorporated
Of these models, the one using a) above is considered applicable to evaluation of theshort-term traffic management that need not consider the route of drivers, or to a network
without allowance for route choice. Verification of these models is not necessary because it isequivalent to the verification of merging/diverging section.
On the other hand, the simulation model using standards of b) c) and d) adopts a framework inwhich drivers select the route on the basis of presented route cost. This type of model isfrequently used to evaluate the operation policy of dispersing the traffic spatially by means of
informative service or road construction. Verification of these models can be made using asimplified network, e.g. with two routes for one O-D pair, to avoid the difficulty to figure out
the theoretical flow pattern to be compared with the simulation result. It is also interested
during the validation that the change on the settings of simulation model such as the update
interval of route costs and the location where the drivers can receive the information on thecosts will affect to the flow pattern.

57th World Congress on Intelligent Transport Systems, Torino, 2000

Description of Standard Verification Process
As mentioned before, the detailed verification step is separately described for Q-K type andC-F type with different fashion. In order to show how the verification of these two types aredifferent, the verification of bottleneck capacity is illustrated as an example in the next two
sections.
Demand = 1500 [veh./hr]

Verification of Bottleneck Capacity (Q-K type)
Given the sufficiently large demand to the bottleneckaccording to the procedure described below,
verification is made whether the flow ratio on thedownstream side is stable at the given bottleneck
capacity. You may read the meaning of“self-consistency check” from the last step iv) .

Bottleneck link capacity= 800, 1000, 1200 [veh./hr]

Link capacity = 2200 [veh./hr](a) Network and demand configuration
Cumulative
Counts [veh.]
1500

Case with Bottleneck
Capacity of 1000 [veh./hr]i) A network to be used consists of links whose
downstream ends become bottlenecks as shown 1000
Demandin Fig.2 (a). Set the model parameter so that the
Link throughput
bottleneck capacity becomes 800, 1000, and1200 [pcu/hr]. The capacity is set to be 2000
Hours
0
1
[pcu/hr] for other sections.
(b) Expected cumulative curve of traffic volumeii) The traffic demand of 1500 [pcu/hr] is provided
verification step of the bottleneck capacity
so that congestion occurs always in the Fig.2: The
for Q-K type simulation models.
bottleneck.
iii) Simulation is made for one hour using respective model parameters and the throughput
volume on the downstream side of bottleneck is recorded.
iv) As shown in Fig.2 (b), the throughput volume cumulative curve is plotted, andverification is made to see if the given bottleneck capacity is really achieved.
Verification of Bottleneck Capacity (C-F type)
Before to start verification, the major model parameters should be listed up with their default
values and plausible setting ranges, and roughly classified into those related to the driving
behavior and those concerned with a location and road section. In this section, a follow typemodel is assumed, which has the model parameters shown in Table 1. Simulation is made fora simple road network as in Fig.3.
Table 1: The model parameter of the C-F type simulation model assumed here.

Since this process is actually to Parameter Name
Default
Minimum
driving behavior of vehicles
determine what kind of traffic flow a) For
a1)
Responce delay
1.0 sec
0.7 sec
2.0 sec
1.7 seccharacteristics of a model demonstrates a2) Desired headway
a3)
Max. acceleration
2.0 m/sec2
1.8 m/sec2
when certain input data and parameter a4) Desired speed
60 km/hr
40 km/hrdemandsettings are used, the average Q-K curve b) For
b1)
Compositon ratio of drivers group of which the driving
characteristices are different
Set freely as requiredis derived according to the subsequentc) For link performancesteps by changing the parameter settings c1) Limit speed
60 km/hr
40 km/hrc2)
Gradient0%-6 %for all combination of their default, c3)
Driveway width
3.5 m/lane
2.75 m/lane
minimum and maximum values.
i)ii)

Maximum
1.5 sec
3.0 sec
2.5 m/sec2
100 km/hr

100 km/hr
6%
3.5 m/lanez

Set the traffic demand at some level and start simulation.
Wait till the link becomes a “steady state”, then observe the cumulative throughput
volume at upstream and downstream ends of the link during some time period.
iii) Take the average number of vehicles on the link from these two cumulative volume
6

7th World Congress on Intelligent Transport Systems, Torino, 2000

Upstream end

Downstream end

Varying the parameters forlink performance to form abottleneck here.

L
Cumulative
volumeQ

Area of this region: S

Plot each sampling statusvalues onto Q-K plane
Flow

dQ

Average density of this section:
K = S /LdT
Flow rate of this period:
Q = dQ /dT
dT

K

Time

DensityFig.3: The verification step of flow characteristic of C-F type simulation model.

curve, and calculate the average traffic density on the link. Then, plot the result on the
Q-K plane.
Lay all plots of Q-K plane into one, of which the parameter settings on driving behavior are
the same but the settings on link performance are different. The interpolate curve represents
the average Q-K curve for the parameter setting on driving behavior.
Other steps in the verification of C-F type are based on this “derived” flow characteristic
related to the model parameter settings. Therefore, the verification of this step must be madeat first.

CONCLUSION AND FUTURE TOPICS
In this paper, we have presented the basic idea of the certification process for the simulation
models through verification and validation. Also, the concept and the outline of the manual ofstandard verification process currently examined in Japan is introduced. The verification
manual is published3) through the “Clearing House” of simulation models2) on the Internet.
The URL is http://trans1.ce.it-chiba.ac.jp/ClearingHouse/main.html.
Several simulation models which are practically used in Japan will qualified with theverification process and will be evaluated its performance in WG5/WS. We have now sevenpilot models, such as AVENUE5), SOUND6), tiss-NET7), Paramics8), CORSIM9), REST10) andSIPA11), to be verified along the verification manual and validated with benchmark data set.
Only the result of AVENUE12) is available at present. The further discussion in WG5/WS is
expected that for to comprehend the results of the verification studies, and to estimate thecharacteristics of each model. Also, we will afford the movement of this standard certification
process for other simulation models world-wide.

ACKNOWLEDGEMENT
We would like to state special thanks to the members of WG5/WS, who contributed tomeaningful discussions on the verification manual.

77th World Congress on Intelligent Transport Systems, Torino, 2000

REFERENCE
1) Smartest Project: Review of Microscopic Simulation Models, http://www.its.leeds.ac.uk/smartest/index.html, since 1997
2) Clearing House of Traffic Simulation Models, http://trans1.ce.it-chiba.ac.jp/Clearing
House/main.html (in Japanese), since 1999
3) A Manual of Standard Verification Process for Traffic Flow Simulation Model (Draft in
English), http://trans1.ce.it-chiba.ac.jp/ClearingHouse/manual/VfyManE/VfyManE.html,
since 1999
4) R. Horiguchi, et. al: A Benchmark Data Set for Validity Evaluation of Road Network
Simulation Models, Proceedings of 5th World Congress on Intelligent Transport Systems,
Seoul, 1998
5) R. Horiguchi, et.al: A Network Simulation Model for Impact Studies of Traffic
Management 'AVENUE Ver. 2', Proceedings of the Third Annual World Congress onIntelligent Transport Systems, Orlando, 1996
6) T. Yoshii, et.al: An Evaluation of Effects of Dynamic Route Guidance on an Urban
Expressway Network, Proceedings of the 2nd World Congress on Intelligent Transport
Systems, Yokohama, 1995
7) K. Sakamoto, et.al: Traffic Assignment Method Considering Car-by-car Behavior for
Traffic Impact Studies -Development of the tiss-NET System-, Proceedings of 8th World
Conference on Transport Research, Antwerpen, 1998
8) Paramics: http://www.paramics.com/
9) CORSIM: http://www.fhwa-tsis.com/
10) T. Yoshida, et.al: A Basic Study for the Planning of ETC-dedicated Highway Interchangeusing a Traffic Simulator REST (in Japanese), Proceedings of Infrastructure Planning,
No.22 (2), pp.231-234, 1999
11) K. Yokochi, et.al: Development of Microscopic Traffic Simulator for AHS Evaluation,
Proceedings of 6th World Congress on Intelligent Transport Systems, Toronto, 1999
12) Verification of AVENUE: http://www.i-transportlab.jp/products/avenue/verification since
1999

8