Stability analysis of Activity-Based models
Case study of the Tel Aviv Transportation Model

Shlomo Bekhor – Technion, Israel Institute of Technology
Leonid Kheifits – Independent Consultant, Israel
Michael Sorani – Ayalon Highways, Israel

Introduction
Activity-Based Models (ABM) are disaggregate
models that simulate individual decisions as randomdraw from choice sets, thus inserting random
component to the results of model run.
The Tel Aviv ABM structure is similar to other activity
based models described in the literature.
The model run is supposed to converge to theequilibrium between generated tours andcorresponding level of service (LOS) data.

2

Goals of current study
Experience of working with Tel Aviv ABM has
revealed the need in conducting comprehensive
analysis of the randomness of the model results, andin developing practical methods for the model
stability monitoring and control.
Therefore, the goals of the current study are:– Analyze sources of randomness and their influence onmodel results– Produce practical recommendations for correct use ofthe ABM

3

Literature Review
Rasouli and Timmermans (2012) presented a reviewof uncertainty in travel demand forecasting models.
Veldhuisen et al. (2000): effect of Monte Carlo draws
on regional aggregate activity patterns
Castiglione et al. (2003): variability of the forecasts
due to random simulation error
Bowman et al. (2006): techniques to establish
convergence based on MSA
Vovsha et al. (2008): discussion of practical ways toreach equilibrium within an activity-based model
Cools et al. (2011): uncertainty related to thestatistical distributions of random components
4Tel Aviv Metropolitan Area

Tel Aviv Model
About 1,500 Km2 and3.3 million habitants in 2009
1,219 Traffic Analysis Zones
About 10,000 regular links
About 1,000 Transit lines
Main Modes: car driver, carpassenger, taxi, bus, rail, Mass
Transit (BRT/LRT)
Access modes: walk, transit,
Park&Ride, Kiss&Ride
6

Tel Aviv Model(structure and implementation)
Implementation features

POPULATIONGENERATOR

Disk space: 4 GB per project +3GB common space

List ofPersons

Memory required: 2GB
PG: stand alone application

TOUR
GENERATOR

TG: Special C# application
OD
Matrices
InitialDemandTRIP
ASSIGNMENTSResults

Level ofService

Assignments: Parallel workwith 3 EMME banks on single
PC
Run time: 40 min periteration, 10% population
sample, Intel i7 Quadprocessor

7Model Structure
The hierarchy of the models is fixed and the generalmodel structure is similar to the Bowman and BenAkiva (2001) approach.
The results from these models are translated into OD travel matrices that are assigned to the network.
For details on the model application, see Bekhor etal. (2011) and Shiftan et al. (2003).
The following slide shows the main components ofthe Tour Generator.

8TOURGENERATOR

Main Activity
WorkEducation

Shopping

OtherNo tour

Time Period (15 alternatives)

Destination of the Main Activity (1219 alternatives)
Zone 1

Zone 2

Zone 3…

Zone 1219

Mode of the Main Activity (14 alternatives)
Car Driver

Car Passenger

Taxi

Bus
P&R, K&R, WalkRail

LRT / BRT

P&R, K&R, Walk, BusIntermediate Stops
No stops

Before the main activity

After the main activity

Before and After

9Errors/randomness sources
There are three major model elements where therandomness or inaccuracy may occur:
POPULATIONGENERATOR

Randomly chosen persons

List ofPersons
TOUR
GENERATOR
OD
Matrices
TRIP
ASSIGNMENTS

Random choice of tour attributes
Level ofService

Assignment accuracy

Results10

Error sources to address
Assignment accuracy

TRIP
ASSIGNMENTS

Tour Generator random component

Population sampling

TOURGENERATORPOPULATIONGENERATOR

11

Assignment errors
Three variants of traffic assignment implementation
were compared:

TRIP
ASSIGNMENTS– Standard Frank-Wolfe algorithm (FW)– Frank-Wolfe algorithm with parallel computation(FWP)– Path-based traffic assignment (PBTA)

The implementation accuracy was evaluated using
distribution of errors in link segments.
This measure is convenient for evaluation andcomparison of errors of different nature.
12Assignment results: Distribution of link
volumes• Number of links: 10900• Average volume: 967 veh/hr• Amount of links with volume
higher than 80 veh/hr: 90%
TRIP
ASSIGNMENTS

13TRIPASSIGNMENTS

Volume difference from exact solution (veh/hr)

Assignment results: convergenceFrank-Wolfe

Path-Based100 300 500 1000 2000 5000

50 100 162-15-10-5051015
Iterations

14

Assignment results: run times
3.5

FW 1_processor
FW 2_processors
FW 4_processors
FW 12_processors
Path-BasedTRIP
ASSIGNMENTS

St.Dev of Auto Volume

3.02.5
2.0
1.5
1.0
0.5
0.0
0.0

10.0

20.0

30.0

40.0

50.0

60.0

Run time [min]15Assignment errors: conclusions

TRIPASSIGNMENTS

The analysis of the assignment implementations has
shown that usage of path-based algorithm maypractically eliminate the assignment error.
The time savings of the PB algorithm is further
expanded due to extensive use of path analysis
allowing to obtain various characteristics ofassignment results very fast.
The FW and FWP algorithms require conducting
additional time consuming assignments.

16

Tour Generator random component

TOURGENERATOR

17

Tour Generator - Simulation error

TOURGENERATOR

The starting point of the analysis is the demandmatrices that result from the TG component.
According to the flowchart presented in the previous
slide, the individual random choices are aggregatedto form the demand matrices.
There are over 30 demand matrices generated by themodel for different modes and periods of day.
To analyse the TG randomness effects, we consider
the car demand matrix for the AM period.

18

The demand matrices are random
The random draw of choices for each person in asample (Activity, Destination, Period of Day, Mode,etc.) results in random matrices, for example:
Car demand, Iteration nDestination
1506 1509 1601 1602 1605 1903 1921
1105 10 10 10 10 200201106 000010001107 0010000201108 010 10 10 20001109 002002000Origin

OriginTOURGENERATOR

Car demand, Iteration n+1
Destination
1506 1509 1601 1602 1605 1903 1921
1105 000010 10 101106 0010030001107 000010 10 101108 00010 10 20 101109 10 10001000Although the overall distribution of trips in the matrix
is quite stable, there are considerable changes at thecell level.
19

Distribution of trips in the car demandmatrix for different iterations
100000Iteration_14

TOURGENERATOR

Number of cells in demand matrix

Iteration_15
Iteration_16
10000

Iteration_17
Iteration_18
Iteration_19

1000

Iteration_20

1001005101520

253035

Number of trips in matrix cell20

Sparseness of demand matrices
Parameter

TOURGENERATOR

AverageStandard
Deviation

Total matrix cells (1219 * 1219)

1,485,961 (100%)

Total non-empty matrix cells

270,101 (18%)

403

Total matrix cells with one trip

150,362 (10%)

382

Total non-empty cells in iteration (n),corresponding to empty cells in
iteration (n+1)

117,068 (8%)

364

Total cells with one trip in iteration(n), corresponding to empty cells initeration (n+1)

93,152 (6%)

346

21

Distribution of OD time differences
500,000
Iteration 10
450,000

Iteration 11
Iteration 12400,000

Iteration 13
Iteration 14

350,000

Iteration 15
Iteration 17
Iteration 18

250,000

Iteration 19
Iteration 20

200,000
150,000
100,000
50,0000-0.95
-0.9
-0.85
-0.8
-0.75
-0.7
-0.65
-0.6-0.55
-0.5
-0.45
-0.4
-0.35
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
00.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95

TOURGENERATOR

Number of cells

Iteration 16
300,000

OD time difference (minutes)
22

Averaging methods

TOURGENERATOR

Different averaging methods can be used to stabilize
the model run results.
We consider 3 averaging methods, which arerespectively presented in the following slides:“MSA-R”:–

1“MSA-M”:–

1“Quasi-Aggregation”:
–1

∑,

23

MSA-R method

TOURGENERATOR

24

MSA-M method

TOURGENERATOR

25

“Quasi-aggregation” method

TOURGENERATOR

26

Convergence of averaging procedures

TOURGENERATOR

Averagingprocedure

Number ofiterations

MSA-R
MSA-M

20
20
16
(6 innereach)

QuasiAggregation

Standard Deviation ofVHT at last iteration(computed for 3 runs)
120
140

Average deviation of last
iteration VHT fromglobal average
-33
-935

43

• The convergence rate of all procedures follows the “ ” rule of thumb:• For MSA-R and MSA-M the VHT standard deviation decreases from theoriginal 420 to 120-140 after 20 iterations, that is comparable to theexpected 420/ 20 93.9;• The resulting VHT standard deviation of 35 for Quasi-aggregate
procedure after 16 iterations with 6 inner iterations each is even closer
to 420 / 16 ( 6 42.8 .

27

Convergence of averaging procedures
as a function of run time

TOURGENERATOR

Deviation of VHT from average (veh-hours)

1,000

Quasi-aggregate

800

MSA-M

MSA-R

600

400

200

0-200-400-600
0200

400

600

800

1000

1200

1400

1600

1800

2000

2200

2400

2600

2800

3000

Run time [min]28Tour Generator (conclusions)

TOURGENERATORAll three arrangements of model run with averaging
results converge similarly.
Note that the MSA-M procedure requires more time
for assignments than MSA-R, since for path-based
assignment used the run time depends on number of
non-zero cells in demand matrix, and in MSA-M
procedure the number of such cells increases with
the iterations, whereas in MSA-R this number isalmost constant.
Further, Quasi-aggregate procedure has different
proportion between number of TG runs and numberof assignment runs, depending on amount of inneriterations.
29

Population Generator random
component

POPULATIONGENERATORPG generates list of individuals with randomcharacteristics based on forecast of aggregate control
variables.
In addition, in order to accelerate the model’s run, asample from the full population is often used.
– In this case, a sample is taken randomly, and trips ofeach person in a sample take proper weight to assure
correct total number of trips in a system.

In this presentation only errors related to the
population sampling will be addressed.

30

POPULATIONGENERATOR
Volume differences between the average till the present iterationand the average till the previous (veh/h)-30

MSA on Assignment results
10% population-20-100102030
510Iterations
1520Volume differences between the average till the present iterationand the average till the previous (veh/h)

Effect of Sampling: Model Convergence-30

MSA on Assignment results
full population-20-100102030
510Iterations

152031Effect of Sampling: Model Convergence
10% of Population
10

Results_Average_10%

9SIGMA OF DIFFERENCES (auto volume)

POPULATIONGENERATORSIGMA OF DIFFERENCES (auto volume)

10

Full Population

FixLOS_10%

8MSA_10%7654321

Results_Average_full
9

FixLOS_full
8MSA_full
765432100024681012

ITERATIONS

141618202202468101214161820

22

ITERATIONS

32

Convergence of ABM for different
sample sizes

Samplesize sPOPULATIONGENERATOR

100%
50%
10%

AverageVHT
174,700
176,150
180,320

Standard deviation of VHT
variations
Observed
Theoretical
σ100%/ *
420
650
1,250

420
594
1,330

Standard deviation after
20 iterations
Observed Theoretical
120160
250

σ/ +,
92.9
145.3
279.5

The results indicate that the “ ” rule of thumb works both for thenumber of iterations and for the sampling rate.

33

Variations of VHT for different
population sample size
184,000

182,000

scen 2402 10% pop

POPULATIONGENERATOR

Total VHT (veh-hours)

scen 2406 10% pop
180,000

scen 2407 10% pop
scen 2418 50% pop

178,000

scen 2423 50% pop
scen 2424 50% pop
scen 2403 100% pop

176,000

scen 2422 100% pop
scen 2425 100% pop

174,000

172,000
0510152025

Iteration34

Population Generator - conclusions

POPULATIONGENERATOR

The sample size affects the stationary point of ABM
results.
In addition, population sampling does not bring any
significant savings in number of iterations, if the goal
is to assure certain accuracy of ABM results.
This is because the sampling would require moreiterations to converge to the same accuracy incomparison to the full sample.

35

Conclusions
Three sources of the ABM results instability wereanalyzed: random population sampling, random tour
generation, and assignment procedures.
In line with previous studies, the effect of assignment
procedures may be practically eliminated when using
path based assignment algorithms.
The effect of randomness of tour generation may bedecreased significantly by averaging the results ofmodel run.
The analysis of ABM stability allows developing
practical measures for performing estimation oftransportation projects with controlled accuracy
36

Further research
Population sampling increases the efficiency of the model
run, but the relationship between model steady stateswith different samples required further analysis.
Two major issues emerged from the presented work:
Analysis of the ABM steady states
More profound study of errors related to Population
generator: uncertainty of synthetic population createdfrom limited set of aggregate control variables
37