A calibration guideline for agent-based passenger mobility models
T. Sereginaa,∗ , M. Amelib , N. Coulombela and M. Zargayounab
a LVMT, Ecole des Ponts, University Gustave Eiffel, Champs-sur-Marne, France{tatiana.seregina,nicolas.coulombel}@enpc.fr
b COSYS, GRETTIA, University Gustave Eiffel, Champs-sur-Marne, France{mostafa.ameli,mahdi.zargayouna}@univ-eiffel.fr∗ Corresponding author
Extended abstract submitted for presentation at the 11th Triennial Symposium on
Transportation Analysis conference (TRISTAN XI)
June 19-25, 2022, Mauritius Island

January 15, 2022Keywords: Agent-based model, Passenger transportation, Traffic flow simulation, Model calibration, Parameter estimation

1INTRODUCTION

Agent-based simulation refers to the process of simulating the behavior and interactions of afinite number of self-interested “agents”. The key construct is the agent, a micro-level entity
behaving autonomously and capable of interacting with other agents through an environment.
The ultimate goal of the simulation is to represent complex systems in different socio-demographic
contexts and various interaction configurations between agents with respect to the environmentalconditions and constraints. The main advantage of the multi-agent paradigm is the ability torepresent non-linear phenomena with a high level of details that would be difficult to tackle
with analytical approaches (Bonabeau, 2002). Moreover, the resulting system behavior can beobserved and analyzed at an aggregated level and showcase macro-characteristics of the system.
Among different applications of agent-based models, simulation of mobility in transportation
systems is one of the most active areas of interest in transportation management, computerscience, and complex systems studies.
Multiple physical, behavioral and dynamic models are deployed in agent-based mobility modeling in order to capture and finely reproduce mobility behaviors of individuals and reactions
to changes in transport supply. These models consist of many parameters that must be determined according to the targeted real system. The optimal parameters lead the model to represent
agents behavior observed in the system. In other words, the development of a model that reliably
describes the mobility situation in a given territory requires the calibration of the model parameters to the conditions of this territory. “Calibration” is the process of defining the parameters
values of the model to fit with an expected outcome. Increasing the resolution of the simulation
increases its degrees of freedom that require more interactions to be modeled and, therefore, moreparameters to be identified and calibrated (Flötteröd et al., 2011). Thus, the complex nature ofagent-based mobility modeling and a high level of detail involves methodological challenges incalibrating such models. This study focuses on investigating the existing methodologies for thecalibration process of agent-based passenger mobility models.
A significant number of articles present agent-based modeling for specific cases studies in thecontext of passenger mobility (Zargayouna et al., 2020), using various calibration approaches,or are directly focused on the calibration techniques of such models (Djavadian & Chow, 2017).
TRISTAN XI Symposium

Original abstract submittal

T. Seregina, M. Ameli, N. Coulombel and M. Zargayouna

2However, in the absence of the most up-to-date and complete overview of the progress in thekey phase of building reliable transport models, this study aims to systematize the practiced anddeveloped calibration approaches.
We examine the history of the agent-based calibration problem and compile an overview
of the calibration approaches for agent-based mobility models described in the literature. The
main contribution of this work is twofold: (1) presenting key questions for the calibration oftransport models and pass through this interrogative lens the available literature on agent-based
passenger mobility modeling; (2) highlighting the important challenges and issues that need tobe addressed as future research and provide a guide for the research community on how to tackle
these challenges with the suggestion of some perspective avenues. We reviewed over 50 articles
in the target area and synthesized promising research directions for open calibration challenges
from the vast literature on computer science and social sciences.

2REVIEW METHODOLOGY

The calibration concept has been defined in transportation management since the seminal workof Greenshields et al. (1935) and the simulation calibration is defined by Wigan (1972) in theurban transportation system. While agent-based modeling dates backs to 1970’s (Nagel & Axhausen, 2016), simulation tools are introduced to the transportation field at the beginning of the
21st century (cf. e.g. Mahmassani (2001); Ben-Akiva et al. (2001)).The review process starts
with the collection of relevant publications during the last two decades from scientific databases
including Scopus and Web of Science. These sources have been searched for articles in scientificjournals and international conferences.
To systematically consider all relevant studies, we use the following search expression basedon keywords: One keyword combination reflects the type of transport model within the scope of
this review as “(‘agent-based’ OR ‘multi-agent’ OR disaggregate) AND (passenger OR ‘passengerdemand’ OR ‘passenger flow’ OR ‘travel demand’ OR transport OR traffic OR mobility) AND(model OR simulation)”. The next keyword reflects the problem in transport modeling, whichis the subject of this review and belongs to the set (calibration OR estimation OR ‘parameter
adjustment’ OR tuning). Specific concerns have also been searched in the calibration literature,such as (uncertainty OR stochasticity) to identify publications that take into account these issues
in their calibration process.
We collected 53 references from the mentioned sources that address the calibration problem
for agent-based mobility simulations. Figure 1 depicts the taxonomy of the calibration problem.
The problem is defined at the intersections of data collection and processing with agent-based
simulation and/or decision-making models. Indeed, the parameters of the simulator and behavior
model need to be calibrated against the collected data. The references are divided into threemain groups based on the scope of the calibration problem. These three categories are comparedfollowing seven features that are defined for a calibration problem in Figure 1.
Furthermore, each study is screened across several categories of questions. This analysis
allows us to further categorize studies and identify weaknesses that need to be addressed inupcoming research. Table 1 presents 26 questions that we pose to each study. Five groups ofquestions are designed to investigate each study. The first three groups include methodologicalquestions to clarify the setting and characteristics of the calibration problem and its solution
method. The two last groups aim to evaluate and compare the performance of the calibration
models in practice. Thus an approximation can be provided to a reader regarding the applicationof each methodology in the literature. In addition, such a questionnaire allows us to revise theprocess of building an agent-based model of passenger mobility and identify the most significant
challenges and peculiarities of its calibration. Considering these questions, this research wasconducted using the systematic literature review method, following Kitchenham (2004).
TRISTAN XI Symposium

Original abstract submittal

T. Seregina, M. Ameli, N. Coulombel and M. Zargayouna

3Figure 1 – Taxonomy on the calibration of agent-based mobility simulation
Table 1 – The list of questions has been posed to each reference in the review process.
Transport model description
Q1 Which type of model is considered? / How does the study include agent-based simulation model?
Q2 What is the name and features of simulator? e.g, SUMO, SimMobility, MATSim, etc.
Charchteristics of the case study
Q3 What are the general characteristics of the study area?
Q4 What is the population size? What are the general characteristics of the demand profile?
Q5 What is the size of the transport network in terms of the number of nodes and links?
Q6 Which transport modes are considered?
Calibration problem
Q7 What is the scope of the calibration problem? (Figure 1)
Q8 What is the calibration strategy? direct calibration or surrogate-based?
Q9 Whether the problem is formulated as deterministic calibration or stochastic calibration?
Q10 What is the characteristics of model stochasticity in case of using stochastic settings?
Q11 How many parameters are calibrated?
Q12 What are the calibration parameters? Types?
Q13 Whether some sensitivity analysis is used to filter parameters prior to the calibration procedure?
Q14 Whether inter-dependencies are considered? If yes how?
Q15 What are the variables of interest, or performance measurements?
Q16 What (field) data is collected and used for calibration?
Q17 Which objective function is used?
Q18 Which optimization method, algorithm is used?
Q19 What additional performance control is used?
Q20 Whether calibration process is documented?
Transport model validation
Q21 Whether validation process is performed for the calibrated model?
Q22 What validation method is applied in case of performing validation process?
Computation timeQ23 How many evaluations of the objective function are made? with how many iterations each?
Q24 Which convergence criterion is applied?
Q25 Which computer resources are used?
Q26 What is the order of computation time for the calibration?

TRISTAN XI Symposium

Original abstract submittal

T. Seregina, M. Ameli, N. Coulombel and M. Zargayouna

34FINDINGS AND DISCUSSION

Analysis of the collected information from the questionnaire prepared categories for the classification and benchmark of the selected articles. The results of the literature review consist in twoparts: (i) Classification of the studies based on the scope of the agent-based model calibration
problem and identification of all the characteristics of each study (Figure 1); (ii) Identificationof current scientific challenges in the literature. For instance, in the second part, we investigated
how existing studies address the computational cost of the calibration process. However, theresults highlight that it remains an extremely serious issue for calibrating large-scale multi-agent
models with a high level of detail. Another angle of exploitation of the existing calibration experiences is the problem of a large number of parameters in agent-based transport models. We
draw from the available studies different approaches addressing this challenge to select the relevant parameters for calibration and how the inter-dependency between calibration parameters istaken into account. We can also mention different challenges such as data issues for calibration,modeling of emerging transportation modes and services, stochasticity modeling, and observed
variability that are dissected and will be presented by this study.
Moreover, we proposed future research directions, including several solutions from otherfields of application of agent-based models to address the calibration challenges. We found that
despite the existence of rich data from emerging technologies, their use in the calibration processof agent-based transport models is still in an evolving stage. Evaluating and improving accessissues, pre-processing, and treatment issues of the data sets improve the model reliability. Thus
these have to be considered for further investigation. In addition, by the wide application ofmachine learning (ML) models to different types of agent-based models, leveraging ML andartificial intelligence techniques to calibrate agent-based mobility simulation is an interesting
research direction. From the optimization point of view, decomposition approaches can be applied
to the models in order to benefit from parallel computing during the calibration process. The
combination of decomposition approaches and search algorithms or meta-heuristics may overcome
the curse of dimensionality of the calibration problem.

References
Ben-Akiva, M., Bierlaire, M., Burton, D., Koutsopoulos, H. N, & Mishalani, R. 2001. Network state
estimation and prediction for real-time traffic management. Networks and spatial economics, 1(3),293–318.
Bonabeau, E. 2002. Agent-based modeling: Methods and techniques for simulating human systems.
Proceedings of the national academy of sciences, 99(suppl 3), 7280–7287.
Djavadian, S., & Chow, J. 2017. An agent-based day-to-day adjustment process for modeling ‘Mobility as
a Service’ with a two-sided flexible transport market. Transportation research part B: methodological,
104, 36–57.
Flötteröd, G., Bierlaire, M., & Nagel, K. 2011. Bayesian demand calibration for dynamic traffic simulations. Transportation Science, 45(4), 541–561.
Greenshields, B. D, Bibbins, J.R, Channing, W.S, & Miller, H.H. 1935. A study of traffic capacity. In:
Highway research board proceedings, vol. 1935. NRC (USA), Highway Research Board.
Kitchenham, B. 2004. Procedures for performing systematic reviews. Keele, UK, Keele University,33(2004), 1–26.
Mahmassani, H. S. 2001. Dynamic network traffic assignment and simulation methodology for advanced
system management applications. Networks and spatial economics, 1(3), 267–292.
Nagel, K., & Axhausen, K. W. 2016. Some history of MATSim. Ubiquity Press.
Wigan, M R. 1972. The fitting, calibration, and validation of simulation models. Simulation, 18(5),188–192.
Zargayouna, M., Othman, A., Scemama, G., & Zeddini, B. 2020. Multiagent simulation of real-time
passenger information on transit networks. IEEE Intelligent Transportation Systems Magazine, 12(2),50–63.

TRISTAN XI Symposium

Original abstract submittal