# An Overview of Agent-Based Models for Transport Simulation and Analysis

Authors: Jiangyan Huang; Youkai Cui; Lele Zhang; Weiping Tong; Yunyang Shi; Zhiyuan Liu
Journal: Journal of Advanced Transportation
 (2022)

---

## Abstract

This article presents an overview of the agent-based modeling and simulation approach and its recent developments in transport fields, with the purpose of discovering the advantages and gaps and encouraging more valuable investigations and applications of agent-based models. We clarify the agent-based model from agents, the background of development, and the basic structure applied in transport systems. Then, the agent-based transport modeling toolkits are discussed. The applications of agent-based models in transport systems are reviewed in three time scale models followed by an additional discussion of hybrid modeling approaches. The extensive modeling of the beliefs, desires, learning, and adaptability of individuals and the optimization problems using agent-based models are explored. Besides, we point out some limitations in terms of calibration and validation procedure, agents’ behavior modeling, and computing efficiency. In conclusion, some recommendations are given and suggest potential and insightful directions such as Big Data and Digital Twin for future research.


---

## Body

## 1. Introduction

An appropriate transport model features prominently in characterizing travel demand, related traffic/passenger flows, and their dynamic performance. It serves as a useful tool for transportation planning and management in terms of alleviating traffic congestions, fulfilling travel needs, and creating a better transport environment. However, it is very challenging to describe such a highly complex, dynamic, and stochastic transport system with analytical mathematical models. Simulation models/methods, by contrast, are capable of modeling the complex traffic dynamics in a real-time manner and therefore have been attracting more and more research efforts [1].Agent-based models are not limited to modeling and simulating the mechanical movement but also introduce the autonomous decision of the agent. Usually, the agent in the systems does not “play by the script” but performs in real-time. It starts from the local rules of agents to the more complex adaptive and emerging behaviors formed by the interaction of the neighborhood agents, thus yielding the system dynamics in the environment. In transport systems, such as transportation management systems and transportation control systems, there exist a large number of independent entities that behave in heterogeneous and inherently complex manners. This usually requires the structural framework of the combination of the distributed subsystems representing the agents with local plans or strategies based on knowledge and rule. The overall system performance is achieved by the cooperative and even learning the logic of agents, which are the vital components to perform a global improvement [2–7].In general, the autonomy, collaboration, and reactivity of agent-based models are the significant advantages to use for modeling different behaviors and interactions such as perception, reasoning, and decision-making, therefore performing different policies from the traffic planning, control, and management perspective. For instance, in transportation study areas, agent-based approaches have been widely used in traffic management frameworks [8–11], congestion management [12], traffic policy [13], traffic control, and especially traffic signal control [3, 14–19] in order to explore the complexity, interdependencies, and systematic structural evolution pattern under complex urban traffic environment [20].In summary, this article is motivated by the increasingly promising potential of agent-based models applied in transport system studies. It reviews the recently emerged development of agent-based transport modeling and analysis, aiming to explore the inherent features of agent-based models and identify their advantages and the current gaps of their applications in the transport domain.The contribution of this review lies in the following three aspects:(1)
It provides a summary of the basic structure for agent-based models and popular modeling and simulation tools in transport system modeling and analysis. The general agent-based model structure for the transport system is proposed by summarizing the related studies. Preanalysis, agent definition, rule modeling, and model tuning are four fundamental components in an agent-based model. Agent-based models are proved unique in their ability to integrate heterogeneous entities and investigate the emergent dynamics of the transport systems by modeling all the interactive behaviors involved. The agent-based modeling and simulation tools provide a shortcut for developing such an agent-based model, which aims to avoid the cumbersome and time-consuming programming work from the ground up, but some limits still need to be breached.(2)
It reviews the ideas of developing and improving agent-based model applications in the transport domain. We categorize the previous research contributions into the short-term, medium-term, and long-term agent-based models based on the time horizon in which the agent behaviors are considered, and we also identify the gaps in the existing studies about modeling and analysis. In general, the design and implementation of agent-based models used in traffic studies are still immature and require further investigation. On the one hand, incorporated with heterogeneous agents in terms of types, decisions, purposes, etc., agent-based models often model the communication strategies or cooperative rules in more detail to describe the global dynamics. On the other hand, the hybrid modeling approach, which can combine the short-, medium-, and long-term models, integrating microscopic, mesoscopic, and macroscopic simulations, is proved to effectively achieve a joint and multiscale system to meet multidimensional simulation requirements. Instead of making a unified choice of one single microscopic modeling approach, components in different levels of resolution should be incorporated according to the characteristics of the targeted transport system and the modeling and simulation purpose. This modeling approach can save a large number of computational resources and allow the simulation to be more effective.(3)
It provides promising research directions for agent-based model studies. As for the insights for future research, we point out that one of the less-developed aspects of the agent-based model approach is modeling the adaptability of the individuals by embedding the knowledge learning abilities in agents’ attributes. Additionally, combining optimization algorithms to solve problems like routing, mode choice, or fleet allocation with simulation is a promising research direction. Moreover, the agent-based simulation can be incorporated into the simulation-based optimization framework as an iterative component to provide intermediate input to objective function in every iteration [21]; Zhang and Xu [22]. The use of the agent-based approach not only opens a channel for searching the optimal solution in an optimization program but also facilitates more realistic and reliable policy-making results.The article is organized as follows. Section2 demonstrates a brief review of the agent description and the agent-based model introduction, including the background, key properties, and the model structure applied for the transport system. Section 3 is devoted to the comparison in terms of their functions, strengths, and limitations of some popular agent-based model tools. The applications of agent-based models in transport systems are thoroughly researched through previous studies and limitations of these applications are also discussed in Section 4. Finally, Section 5 provides some recommendations followed by insights and challenges for future research.


## 2. Agent-Based Models

### 2.1. Agents

#### 2.1.1. Definition and Features of Agents

There is no consensus on the definition of the term “agent” since the diversity in the application or environment makes it difficult to extract consistent and concise features. Nevertheless, the essence is that an agent, the basic unit in the agent-based model, can be a representation of any type of autonomous entity, autonomously working within an environment in pursuit of its agenda or goals and actively interacting with other agents if necessary [23]. A typical agent in the agent-based model can hopefully possess the following features: (a) autonomy: operating without direct external intervention; (b) sociability: cooperating with the environment and/or other agents to achieve its task(s); (c) reactivity/perceptibility: perceiving external influence and responding to changes; (d) proactivity: exhibiting goal-directed behavior on its initiative; (e) adaption/learning: learning knowledge from experience and environment and adjust yourself to the environment and simulation objectives [2, 24–26]. It should be noted that the modeling of agents is flexible, and the features listed are not exhaustive or exclusive. Within a simulation experiment, there might exist various types of agents that possess different characteristics with different perceived importance [27]. For example, an agent-based simulation of a demand-responsive transport system encompasses different types of agents. The clients can be taken as autonomous agents who constantly generate trip requests. They are also be designed to be proactive and adaptive, learning to alert and manage the choices/plans for the reproduction of trip requests. Vehicle agents monitor the status of the real vehicles and react to the clients’ requests with prior knowledge supplied. Besides, the planner agents can interface between clients and vehicles through processing trip requests and handling bids from available vehicle plans.


#### 2.1.2. Basic Functionalities of Agents

An agent can be regarded as an intelligent synthesis in possession of explicit and identifiable separate function parts. The basic functionalities of an agent involve six parts: effector, sensor, communication, intention, motivation, and cognition [28]. These functionalities can be divided into four different modules: (a) effector, sensor, and communication module: making contact and transmitting and receiving information with the external environment and other agents; (b) intention module: comprising the short-/long-term goal(s) of the agent; (c) motivation module: implementing the strategic intentions of the agent; (d) cognition module: consisting of the agent’s knowledge base, reasoning, and deciding component and protocol execution component. The last module has the functions of evaluating the current situation and selecting, executing, and monitoring the final actions of the agent. The functionalities of agents and interactions between multiple agents and with the environment are depicted in Figure 1.Figure 1
Functionalities and interactions with agents and the system environment.


### 2.2. Background and Concept of the Agent-Based Model

Agent-based model, which is different from the term activity-based model, is used to simulate the behaviors and interactions of autonomous agents, thus assessing their overall impact on the system [29, 30]. It is unique in the ability to integrate heterogeneous and dynamically changing processes of complex systems composed of autonomous entities, intending to investigate the emergent and collective effects on the system [31].In recent years, there has been an increasing prevalence of agent-based models in transport studies. The reasons are threefold: first, the transport system is a very complex (and adaptive) system. The key characteristics such as multimodality, heterogeneity, complexity, and adaptivity in drivers’ behavior are too challenging to be fully and accurately modeled. Through agent-based modeling, one can formulate a more realistic model considering specific characteristics of heterogeneities and interdependencies among road users and the transport system. Second, new technologies (such as AV, vehicle-to-vehicle (V2V), and vehicle-to-everything (V2X)) expand the scope of research, add complexity to the transport system, and bring more challenges to the modeling process. The technology development also brings a large amount of detailed data, which are very important for calibrating agent-based models. Finally, the computational power has been booming following Moore’s law, which enables exploring transport system processes through constituent elements no matter how microscopic they are.


### 2.3. Agent-Based Model Structure for Transport Systems

The agent-based model can be tailored for various problems, objectives, and scenarios. On the one hand, the applications in the transport domain cover specific systems or phenomena, such as ridesharing, bike-sharing, autonomous taxi, and high-speed railway models to capture only the most salient features connected with the goals of the simulation. On the other hand, large-scale simulation of a certain region or city can also be realized through an agent-based approach to solving real-world policy problems. Nevertheless, agent-based models for different traffic and transport problems share a similar structure with the following four parts, as shown in Figure2.Figure 2
Agent-based model structure.Step 1.
Preanalysis: it determines the application scenario, the purpose(s) of the model, the scale, and the resolution of the simulation system/environment. The application and purpose of the agent-based model are the grand plans for future work, regarding what problem to be solved by applying the agent-based model; what results to achieve; whether to describe, observe, and analyze a specific transport system or predict its unknown emergent phenomena. Then, the scale of the system restricts the range of the moving and interactive environment of agents and the quantities of agents involved. As for the simulation resolution, it should be determined as per the characteristics of the system in terms of agents and the environment. The model can feature agents’ motion level modeling and simulation like gap acceptance and lane change to study the traffic dynamics, detailed flow changes, and resulting impacts. Alternatively, it can be designed to focus on the modeling of immobile behaviors, decisions, and feedback mechanisms. In this case, the travel motion does not require a precise description and could even be presented at a link level. Moreover, more macroscopic and long-term perspective models can be chosen, which involve activities or behavioral patterns like the choice of housing location and job location [32].Step 2.
Agent definition: it identifies and classifies the heterogeneous entities (agents) in the transport system environment in terms of types. DeLaurentis [33] classified agents into eight basic types based on their behavioral rules, mobility, and information accessibility (see Figure 3). Agents can vary in property and mutate into different entities with different functions when applied in different agent-based models to achieve different objectives. For example, in a traffic control system, reactive agents represent the role of policemen who enforce a set of laws on the road. Various disrupters like weather and accidents can be modeled as autonomous agents. Drivers are adaptive, mobile, and utility agents who may avoid roads that are known to be congested and would like to minimize their drive times or distances. Determining the types of agents and their general characteristics appropriately will pave the way for the following behavioral rules-modeling process, making it clear to define the specific characteristics further and set up detailed behavioral rules.Figure 3
Types of agents. Source: DeLaurentis [33].Step 3.
Rule modeling: it models the behavior and interaction of agents. Typically, published literature, expert knowledge, and data analysis can all be the major sources of rules. On the one hand, they are generally based on “if-else” statements that the actions of agents are triggered by some external stimulus. On the other hand, rules can also be put into action directed by goals like the signal controller agent [34]. The rules are reflected by the behavior and interaction of agents, so they are usually influenced by each other and can provide positive and negative feedback to the environment. Many agent-based models focus on the precise motion-related behaviors that deal with agents’ movement description and performance in the physical perspective [35, 36], and they adopt microscopic models like car-following (CF) and lane changing (LC) models as their mobility rules to facilitate the simulation presenting reality. However, this sacrifices computational proficiency, which is particularly for large-scale scenarios. Therefore, one of the challenges of agent-based models is the tradeoff between the complete complexity of the inner logic of the models and computational efficiency. Liu and Meng [37] showed that the allocation of the processor in the distributed computing system performed well in the computational time reduction. Each simulation trial is conducted in parallel and in turns. This study enlightens researchers to utilize parallel computing approaches to overcome the high nonlinearity and huge computational cost in agent-based simulation.
Sometimes, the event-based mechanisms can be combined seamlessly with the agent-based simulation method. Traditionally, the entities in the discrete event-based simulation are defined to follow a sequence of processes, which are similar to the definition of the local behaviors in agent-based modeling and simulation. Some agent-based modeling tools allow the environment for multiparadigm modeling with agent behavior, agent environment, and event-based description of the occasional or time-certain occurrences included. The hierarchical, sequential, or parallel simulation structure can also be realized with different modules designed based on agent-based models and other simulation methods.Step 4.
Model tuning: model verification, calibration, and validation. Overall, the calibration and validation deal with building theright model, while the verification deals with building the model right. The calibration process aims to examine whether the agent-based model fits well with the empirically supported data. The model evaluation with respect to calibration and validation varies due to different modeling purposes [24]. In general, the development of an agent-based model should include a calibration process and/or a validation process to demonstrate that the agent-based model presents the system in validity and accuracy.


## 2.1. Agents

### 2.1.1. Definition and Features of Agents

There is no consensus on the definition of the term “agent” since the diversity in the application or environment makes it difficult to extract consistent and concise features. Nevertheless, the essence is that an agent, the basic unit in the agent-based model, can be a representation of any type of autonomous entity, autonomously working within an environment in pursuit of its agenda or goals and actively interacting with other agents if necessary [23]. A typical agent in the agent-based model can hopefully possess the following features: (a) autonomy: operating without direct external intervention; (b) sociability: cooperating with the environment and/or other agents to achieve its task(s); (c) reactivity/perceptibility: perceiving external influence and responding to changes; (d) proactivity: exhibiting goal-directed behavior on its initiative; (e) adaption/learning: learning knowledge from experience and environment and adjust yourself to the environment and simulation objectives [2, 24–26]. It should be noted that the modeling of agents is flexible, and the features listed are not exhaustive or exclusive. Within a simulation experiment, there might exist various types of agents that possess different characteristics with different perceived importance [27]. For example, an agent-based simulation of a demand-responsive transport system encompasses different types of agents. The clients can be taken as autonomous agents who constantly generate trip requests. They are also be designed to be proactive and adaptive, learning to alert and manage the choices/plans for the reproduction of trip requests. Vehicle agents monitor the status of the real vehicles and react to the clients’ requests with prior knowledge supplied. Besides, the planner agents can interface between clients and vehicles through processing trip requests and handling bids from available vehicle plans.


### 2.1.2. Basic Functionalities of Agents

An agent can be regarded as an intelligent synthesis in possession of explicit and identifiable separate function parts. The basic functionalities of an agent involve six parts: effector, sensor, communication, intention, motivation, and cognition [28]. These functionalities can be divided into four different modules: (a) effector, sensor, and communication module: making contact and transmitting and receiving information with the external environment and other agents; (b) intention module: comprising the short-/long-term goal(s) of the agent; (c) motivation module: implementing the strategic intentions of the agent; (d) cognition module: consisting of the agent’s knowledge base, reasoning, and deciding component and protocol execution component. The last module has the functions of evaluating the current situation and selecting, executing, and monitoring the final actions of the agent. The functionalities of agents and interactions between multiple agents and with the environment are depicted in Figure 1.Figure 1
Functionalities and interactions with agents and the system environment.


## 2.1.1. Definition and Features of Agents

There is no consensus on the definition of the term “agent” since the diversity in the application or environment makes it difficult to extract consistent and concise features. Nevertheless, the essence is that an agent, the basic unit in the agent-based model, can be a representation of any type of autonomous entity, autonomously working within an environment in pursuit of its agenda or goals and actively interacting with other agents if necessary [23]. A typical agent in the agent-based model can hopefully possess the following features: (a) autonomy: operating without direct external intervention; (b) sociability: cooperating with the environment and/or other agents to achieve its task(s); (c) reactivity/perceptibility: perceiving external influence and responding to changes; (d) proactivity: exhibiting goal-directed behavior on its initiative; (e) adaption/learning: learning knowledge from experience and environment and adjust yourself to the environment and simulation objectives [2, 24–26]. It should be noted that the modeling of agents is flexible, and the features listed are not exhaustive or exclusive. Within a simulation experiment, there might exist various types of agents that possess different characteristics with different perceived importance [27]. For example, an agent-based simulation of a demand-responsive transport system encompasses different types of agents. The clients can be taken as autonomous agents who constantly generate trip requests. They are also be designed to be proactive and adaptive, learning to alert and manage the choices/plans for the reproduction of trip requests. Vehicle agents monitor the status of the real vehicles and react to the clients’ requests with prior knowledge supplied. Besides, the planner agents can interface between clients and vehicles through processing trip requests and handling bids from available vehicle plans.


## 2.1.2. Basic Functionalities of Agents

An agent can be regarded as an intelligent synthesis in possession of explicit and identifiable separate function parts. The basic functionalities of an agent involve six parts: effector, sensor, communication, intention, motivation, and cognition [28]. These functionalities can be divided into four different modules: (a) effector, sensor, and communication module: making contact and transmitting and receiving information with the external environment and other agents; (b) intention module: comprising the short-/long-term goal(s) of the agent; (c) motivation module: implementing the strategic intentions of the agent; (d) cognition module: consisting of the agent’s knowledge base, reasoning, and deciding component and protocol execution component. The last module has the functions of evaluating the current situation and selecting, executing, and monitoring the final actions of the agent. The functionalities of agents and interactions between multiple agents and with the environment are depicted in Figure 1.Figure 1
Functionalities and interactions with agents and the system environment.


## 2.2. Background and Concept of the Agent-Based Model

Agent-based model, which is different from the term activity-based model, is used to simulate the behaviors and interactions of autonomous agents, thus assessing their overall impact on the system [29, 30]. It is unique in the ability to integrate heterogeneous and dynamically changing processes of complex systems composed of autonomous entities, intending to investigate the emergent and collective effects on the system [31].In recent years, there has been an increasing prevalence of agent-based models in transport studies. The reasons are threefold: first, the transport system is a very complex (and adaptive) system. The key characteristics such as multimodality, heterogeneity, complexity, and adaptivity in drivers’ behavior are too challenging to be fully and accurately modeled. Through agent-based modeling, one can formulate a more realistic model considering specific characteristics of heterogeneities and interdependencies among road users and the transport system. Second, new technologies (such as AV, vehicle-to-vehicle (V2V), and vehicle-to-everything (V2X)) expand the scope of research, add complexity to the transport system, and bring more challenges to the modeling process. The technology development also brings a large amount of detailed data, which are very important for calibrating agent-based models. Finally, the computational power has been booming following Moore’s law, which enables exploring transport system processes through constituent elements no matter how microscopic they are.


## 2.3. Agent-Based Model Structure for Transport Systems

The agent-based model can be tailored for various problems, objectives, and scenarios. On the one hand, the applications in the transport domain cover specific systems or phenomena, such as ridesharing, bike-sharing, autonomous taxi, and high-speed railway models to capture only the most salient features connected with the goals of the simulation. On the other hand, large-scale simulation of a certain region or city can also be realized through an agent-based approach to solving real-world policy problems. Nevertheless, agent-based models for different traffic and transport problems share a similar structure with the following four parts, as shown in Figure2.Figure 2
Agent-based model structure.Step 1.
Preanalysis: it determines the application scenario, the purpose(s) of the model, the scale, and the resolution of the simulation system/environment. The application and purpose of the agent-based model are the grand plans for future work, regarding what problem to be solved by applying the agent-based model; what results to achieve; whether to describe, observe, and analyze a specific transport system or predict its unknown emergent phenomena. Then, the scale of the system restricts the range of the moving and interactive environment of agents and the quantities of agents involved. As for the simulation resolution, it should be determined as per the characteristics of the system in terms of agents and the environment. The model can feature agents’ motion level modeling and simulation like gap acceptance and lane change to study the traffic dynamics, detailed flow changes, and resulting impacts. Alternatively, it can be designed to focus on the modeling of immobile behaviors, decisions, and feedback mechanisms. In this case, the travel motion does not require a precise description and could even be presented at a link level. Moreover, more macroscopic and long-term perspective models can be chosen, which involve activities or behavioral patterns like the choice of housing location and job location [32].Step 2.
Agent definition: it identifies and classifies the heterogeneous entities (agents) in the transport system environment in terms of types. DeLaurentis [33] classified agents into eight basic types based on their behavioral rules, mobility, and information accessibility (see Figure 3). Agents can vary in property and mutate into different entities with different functions when applied in different agent-based models to achieve different objectives. For example, in a traffic control system, reactive agents represent the role of policemen who enforce a set of laws on the road. Various disrupters like weather and accidents can be modeled as autonomous agents. Drivers are adaptive, mobile, and utility agents who may avoid roads that are known to be congested and would like to minimize their drive times or distances. Determining the types of agents and their general characteristics appropriately will pave the way for the following behavioral rules-modeling process, making it clear to define the specific characteristics further and set up detailed behavioral rules.Figure 3
Types of agents. Source: DeLaurentis [33].Step 3.
Rule modeling: it models the behavior and interaction of agents. Typically, published literature, expert knowledge, and data analysis can all be the major sources of rules. On the one hand, they are generally based on “if-else” statements that the actions of agents are triggered by some external stimulus. On the other hand, rules can also be put into action directed by goals like the signal controller agent [34]. The rules are reflected by the behavior and interaction of agents, so they are usually influenced by each other and can provide positive and negative feedback to the environment. Many agent-based models focus on the precise motion-related behaviors that deal with agents’ movement description and performance in the physical perspective [35, 36], and they adopt microscopic models like car-following (CF) and lane changing (LC) models as their mobility rules to facilitate the simulation presenting reality. However, this sacrifices computational proficiency, which is particularly for large-scale scenarios. Therefore, one of the challenges of agent-based models is the tradeoff between the complete complexity of the inner logic of the models and computational efficiency. Liu and Meng [37] showed that the allocation of the processor in the distributed computing system performed well in the computational time reduction. Each simulation trial is conducted in parallel and in turns. This study enlightens researchers to utilize parallel computing approaches to overcome the high nonlinearity and huge computational cost in agent-based simulation.
Sometimes, the event-based mechanisms can be combined seamlessly with the agent-based simulation method. Traditionally, the entities in the discrete event-based simulation are defined to follow a sequence of processes, which are similar to the definition of the local behaviors in agent-based modeling and simulation. Some agent-based modeling tools allow the environment for multiparadigm modeling with agent behavior, agent environment, and event-based description of the occasional or time-certain occurrences included. The hierarchical, sequential, or parallel simulation structure can also be realized with different modules designed based on agent-based models and other simulation methods.Step 4.
Model tuning: model verification, calibration, and validation. Overall, the calibration and validation deal with building theright model, while the verification deals with building the model right. The calibration process aims to examine whether the agent-based model fits well with the empirically supported data. The model evaluation with respect to calibration and validation varies due to different modeling purposes [24]. In general, the development of an agent-based model should include a calibration process and/or a validation process to demonstrate that the agent-based model presents the system in validity and accuracy.


## 3. Agent-Based Transport Modeling Tools

An agent-based model can be constructed using an object-oriented language, which is largely facilitated by the development of modeling tools. The development of agent-based modeling toolkits aims to avoid cumbersome and time-consuming programming work from the ground up. From the reviewed articles, the agent-based software toolkits can be categorized and distinguished based on the scalability, execution speed, programming language, primary application domain and key features [25, 38] (see Table 1 in Appendix A.1).Table 1
Comparison of agent-based software toolkits.
PlatformScaleProgramming languagePrimary application domainKey featuresAnyLogicLarge-scaleJavaGeneral purpose, distributed simulation (e.g., urban dynamics, pedestrian dynamics)-Multiparadigm modeling-Statistical tools-Good exportability-Animation capabilitiesNetLogoDesktop computingNetLogoSocial and natural science-Multiple built-in agent types-Extensive model libraries-Exploration of interface elements-Flexible model building and modificationMASONLarge-scaleJavaSocial complexity, physical modeling, machine learning-Complete separation of models and visualization-Dynamical migration across platforms-2D and 3D visualizationSwarmLarge-scaleObjective-C; JavaGeneral purpose-Model building, analyzing, displaying-Hierarchical modeling approaches-Object-oriented libraries of reusable componentsRepastLarge-scaleJava; Python; C++Social science-Object-oriented-Concurrent discrete event scheduler-Dynamic modeling and modification-Result logging and graphing tools-Built-in libraries for different algorithmsSources: Macal and North [25]; Zheng et al. [39]; Railsback et al. [40].There are several agent-based tools specially designed for transport systems, which assist the development of agent-based models in a simplified way with desired attributes, features, and rules. Table2 in Appendix A.1 summarizes and compares some most commonly used agent-based simulation tools. The agents in these platforms refer to various transport participants, drivers, travelers, and vehicles. In general, when working with relatively large-scale simulation, most of the existing agent-based transport modeling platforms share the similarity of creating an integrated system by combining the travel activity and network loading components. The frameworks are often based on individual models and rooted in activity-based models. Figure 4 shows one typical type of modeling structure. The system generates demand and activity plans of the population based on demographic and infrastructure characteristics. Each agent then generates individual travel mode and route choice. The plans of agents are fed into a microsimulation, which will produce certain traffic performance. The overall performance feeds back to the decision-making process of activity plan and mode and route choice decisions so that the plans can be revised and improved to be ready for the next iteration of the simulation.Table 2
Comparison of agent-based transport software toolkits.
PlatformProgramming languagePublic accessibilityKey featuresMATSimJavaOpen source-Activity-based framework-Two functional layers: physical layer and mental layer-Queue model for traffic simulation-Scoring and replanning iteration-Coevolution algorithmTRANSIMSC++; pythonOpen source-Activity-based approach-CF and LC models for simulating vehicle movements-Traffic condition and related impacts analysisGAMAGAMLOpen source-GIS and data-driven models-Intuitive agent-based language-Research domain including transport and transportation, urban planning, environment, epidemiologyOpenAMOSROpen source-Activity-based, travel-demand model-Four integrated components-Population generator and urban simulation-Activity-travel demand and dynamic traffic assignment-Activity-travel pattern estimationSACSIMC#Open source-Regional travel forecasting model-Activity-based, disaggregate econometric model-Analytical framework combined with the traffic assignmentSources: Macal and North [25]; Zheng et al. [39]; Taillandier [41]; Bradley et al. [42] and [43]; Kitamura et al. [44].Figure 4
Model structure of an agent-based platform in transport applications. Source: Zheng et al. [39].Note that many of them exhibit a similar theory of simulation-based dynamic traffic assignment (DTA) in agent-based transport modeling. It should be noted that simulation-based DTA and agent-based transport modeling share some similarities and dissimilarities. On the one hand, the route choices can be optimized by using the historical travel time information in both methods and this processes iteratively to accomplish the optimal performance. On the other hand, simulation-based DTA focuses more on the trip of individuals in the simulation, so that the travel time information is usually only provided for route choice decision of travelers, while the feedback information can be fed into multiple dimensions of decisions like activity location and schedule in addition to route choice domain. In addition, the heuristic approaches used in agent-based models can sometimes converge to the equilibrium faster. In terms of the internal models and algorithms, the existing platforms exhibit a certain difference in the design process, data structures, resolution levels, and emphasis on activities of agents to some extent.Several agent-based tools aim for specific realms of transport research like traffic control and management or route guidance. For example, Cooperative Traffic Management and Route Guidance System (CTMRGS) can be applied in traffic management [10]. TRACK-R (TRaffic Agent City for Knowledge-based Recommendation) can be used to deal with the dynamic route guidance problem [45]. Agent-Based Dynamic Activity Planning and Travel Scheduling (aDAPTS) can be used to study traffic signal control [46].More and more studies have applied agent-based simulation tools to traffic and transport systems, and different platforms and tools can be chosen to serve the research purpose. For mobility simulation, some platforms are aimed at presenting microscopic traffic flow dynamic characteristics, which simultaneously lead to longer computation times, particularly for large-scale scenarios. In contrast, others adopt the faster microscopic or mesoscopic models like the spatial queue model to measure traffic dynamics, pushing the limits of computational feasibility. Nevertheless, many platforms are becoming more and more flexible to provide many connectors or fully open-source to allow the researcher to write extensions to solve such problems and explore the potential functions.


## 4. Applications in Transport Systems

Compared to other microscopic simulations, the agent-based model has inherent advantages in modeling multimodal traffic by incorporating different transport modes and multiple entities of operators and users into traffic flows. Table3 in Appendix A.2 summarizes the existing studies of the agent-based model methods in the transport domain and compares/highlights their main objectives, agent types, method features, strengths, and potential improvements. How the agent-based model method is applied and constructed is determined by the objectives of the studies. Most paradigms focus on traffic control and management, route choice or guidance, and carsharing or ridesharing. Accordingly, the agents in these models usually represent components like drivers, passengers, and vehicles. The most common goals are developing effective strategies, novel systems, and algorithms. In addition, some psychological characteristics can be embedded to enrich the agent models and make their behavior close to real human beings, which might be favorable in evacuation simulation [63, 64].Table 3
Summary of agent-based models applied in transport studies.
StudyObjectiveAgent types and method featuresStrengthsUnhandled issues/potential improvement[47]Dynamic ridesharing-Decentralized optimization-Driver-passenger agents-Autonomous agents-Passenger agents-Driver agents-Centralized and optimization algorithms-Ride-matching algorithm-Multipassenger matching approach underestimates passenger bids or overestimates driver bids and thereby reduces system reliability[48]Traffic control-Dynamical agents-Autonomous agents-Cooperation tracking for vehicles-Novel distributed adaptive collaborative control strategy-Lyapunov–Krasovskii approach-No multiagent platform for simulation[49]Traffic management and route guidance-Autonomous agents-Authority agents-Intersection agents-Case study on real network (Brisbane western corridor)-Artificial neural network techniques-Need detailed route choice field surveys-Incorporating more complex BDI agent frameworks[50]Traffic control and management, route guidance-Autonomous agents-Driving agents-Centralized/decentralized coordination-Development of an autonomous driving system-Not specified if the lateral control is automated or a simulation of a human driver[51]Traffic control-Driver agents, car agents-Enhanced traffic system’s performance-Genetic algorithm-Computational difficulties[52]Route choice behavior-Human agents-Decentralized coordination-Sampling and weighting algorithm-Static allocation designs like “queue”-Limited consideration of homogenous bounded-rational agents[53]Transportation networks analysis-Traveler agents-Cooperation of system-Differential greedy algorithm-Diffusive load balancing algorithm-Not consider multimodal agent-based traffic simulators[54]Traffic control and management, route guidance-Driving agents-Centralized/decentralized coordination of the platoon-Hierarchical driving architecture-Centralized and decentralized coordination models-Single type of agents-Longitudinal guidance system needs to be improved[55]Vehicle assignment-Vehicle agents-Traveler agents-Centralized/decentralized coordination-Case study on a real network (Oakville)-Comparison between the one-sided and two-sided market-No reliable individual elasticity measures-Need real data for validation[56]Dynamic traffic management-Traveling agents-Centralized coordination-Stochastic programming-Lagrangian relaxation-based heuristic approach-Single type of agents-Not consider variables such as random incident duration and compliance rate[57]Mixed traffic management-Car agents-Pedestrian agents-Tram agents-Coordination of signals-Real traffic network (Okayama)-Multiagent-based traffic simulator-A small number of scenarios tested[58]Traffic control and management-Lower-level agents-Communication network-Simplified management of the traffic flows-Decision-making algorithm-Single type of agents[59]Traffic control and managementRoute guidance-Driver-vehicle agents-Cooperation of vehicles-Coordination of signals-Realistic flow-speed relationships during congested situations-Single type of agents[10]Route guidance and traffic management-Driver agents-Cooperation of network managers and travelers-Improved dynamic routing and traffic management-No multiagent platform for simulation[60]Traffic control-Driver agents-Road segment agents-Real-world proof-of-concept case study (London)-A model of roadway traffic dynamics-More detailed representation of road network should be considered[61]Traffic network reorganization, traffic control, and management-Intersection control agents-Vehicle agents-Traffic manager agents-Context-aware intelligent (Bellifemine et al.) vehicle agents-The traffic reorganization operations improve evacuation time-Only full road reversal operations are implemented[62]Emergency management, traffic control-Vehicle agents, driver agents-Support emergency evacuation planning, showing realistic agents’ driving behaviors during emergency situations-Need further study of psychological characteristics affecting drivers’ behaviors


### 4.1. Simulation of Transport Systems

In this section, previous research contributions of agent-based models differentiated by rule-based behaviors of agents at the different temporal scales are reviewed. Some studies may select the traditional categorization of the “micro-, meso-, macroscopic” models according to the distinction of spatial information. However, this classification has the limitation in providing insights when the agent-based simulation system to be built have different requirement of the timespan [65]. Moreover, we cannot have a clear mind about the system-related agent-based modeling difference for developing a specific agent-based transport system only based on another categorization of “strategic, tactic, operational” types of modeling. Therefore, this article intends to provide a review of agent-based modeling methods from the perspective of the time horizon of simulation. We adopt a categorization of the models into short-, medium-, and long-term types, comparing the different agent-based modeling techniques and algorithms used in different frameworks. It not only reveals another aspect of information of the traditional one in terms of “strategic, tactic, operational” to some extent but also corresponds to the ideas of microscopic, mesoscopic, and macroscopic simulations in some ways. This categorization is beneficial to provide a reference for exploring agent-based systems at different levels.Short-term movement control models handle the within-day movement of agents at a microscopic granularity. Detailed driving and travel behaviors under the impact of other moving agents or traffic control/management are described. Mid-term models function at the design/operational scheduling: simulating the day-to-day activity and travel patterns of agents, which are often generated by choice model, behavior-based demand model, etc. Long-term planning models, with the view of capturing land use and economic activities, are commonly applied to year-to-year transportation planning studies. The discussions of these three levels of agent-based models are followed by an investigation of the hybrid models, which merge two or all the three different scale time models into a single simulation process. Figure5 presents the important key points of three different models, respectively, to show their difference. Note that for some agent-based models embedding the event-based mechanisms, the time scale might also depend on the event mechanism in the model, and the time between two events may range from seconds to days in the same model.Figure 5
Key points of different models.


#### 4.1.1. Short-Term Models

Short-term models are usually developed at a microscopic level and on the temporal scale at a very fine resolution of the second level and some even up to 100 ms (e.g., in pedestrian flow or traffic flow models). As the agents involved in transport systems are basically mobile agents (not necessarily in other domains), the behavior and the relationships with other agents and the external environment for their interaction in the short-term models are usually related to mobility and are commonly used to simulate traffic movement from the physical perspective [66].The mobility of vehicle agents in traffic flow simulation usually involves LC, CF, collision avoidance actions, etc. They can be modeled to function at sensing and perceiving the conditions and states of their environment around. The timely and prompt reaction to the infrastructure equipment such as signboards, traffic signals, and traffic laws can be incorporated. Moreover, the pedestrian agents’ walking behavior modeling also caters to the short-term agent-based models, which involve the modeling of walking direction and relative speed based on the condition of both the pedestrian and road traffic around the current position. Similar researches include the microscopic movement of pedestrians during an evacuation or the movement of cars on the street. Various agent models facilitate the application of short-term agent-based models in mixed traffic flows that refer to the mixture traffic of two or more pedestrians, motorcycles, cars, buses, and trams. The study of such complex vehicular and pedestrian flow patterns should be started from the individual’s behavior pattern.The type of agent and behavior modeling process in these short-term agent-based models vary with different objectives. Yang et al. [67] established the model of pedestrians’ road-crossing behavior to study the pedestrian flow at different traffic volumes. Explicit descriptions of the pedestrians’ road-crossing and decision-making behavior like whether to run through the red light and the followers’ violative behavior are required. Some studies involve other vehicle agents such as motorcycles and public transport. In Fujii et al. [57], a framework for modeling and simulating the mixed traffic flows composed of cars, trams, and pedestrians was constructed to test the control and management strategies. The pedestrian agents have the behavior of free walk, collision avoidance, momentary stop, overtaking, and the following mobility with car and tram agents. The car agents can plan the route and determine acceleration, and the tram agents are extended from the car agents, excluding the LC and route-searching functions. The mobility of agents was expanded with more detail and flexibility in that the existing 1D CF model for cars can be extended to a 2D discrete choice model for pedestrians.


#### 4.1.2. Medium-Term Models

Medium-term models refer to the (mesoscopic) simulations that involve mode and route choice, activity schedules, or en route diversion at the temporal scale of minutes or even longer. Such multidimensional travel behavior simulation is effective in analyzing or predicting behavioral changes and the dynamics of the traffic condition [68, 69].An agent-based simulation is frequently used in sharing mobility research to capture the feedback and information-transferring processes. The study of sharing mobility is closely related to the problems of the optimal fleet size dispatch and the passenger-passenger (and driver-passenger) matching. Agent-based models specifically pertaining to sharing mobility phenomenon are grouped into medium-term models. For mobility simulation in these models, the agents can still be modeled at the individual level in terms of mode or route choice process. The attributes and behaviors of agents are related to making decisions about travel plans based on their cognitive information (knowledge) and returning effective feedback of the matching information.Medium-term agent-based models perform well in dynamic ridesharing (also known as on-demand ridesharing and dynamic carpooling), and the model usually involves a service operator dynamically matching potential driver and passenger agents with similar trips and allowing them to travel together and sharing the costs [70]. A binary integer programming was combined with the dynamic auction-based multiagent optimization algorithms to solve a dynamic ridesharing problem [47]. The decentralized agent-based model allows the matching condition of drivers to passengers, which is proved to be flexible in solving ridesharing schemes. In some cases, the medium-term agent-based models contribute to studying multimodal transport systems in that the explicit mode choice and split condition can be presented from an individual perspective. Shen et al. [71] developed an agent-based system combining on-demand and shared AV service and public transportation (PT) to identify the synergy effect. The passengers’ searching, choosing, quitting, waiting, and recalling processes are considered. In this case, all the occupied AVs will be searched, and then the passengers will agree to share the ride according to their sharing preference. As for sharing AVs, shared rides are matched subject to certain constraints like enough available seats and detour constraints.Further study of psychological characteristics affecting the medium-term behaviors of drivers can be conducted to the incorporated preference of agents like passengers’ sharing preference of riding alone or with other riders under certain conditions. Notably, the agent environment in medium-term models is not as specific as that in short-term models with continuous or grid cell tangible interaction space for micromovement particularly related to geospatial and spatial modeling and simulation. Instead, the interactive environments in medium-term models are relatively virtual and fuzzy as they are designed not to deal with physical and social behavior like moving in different directions and at different velocities within their environment.


#### 4.1.3. Long-Term Models

Long-term models, which are usually at the temporal scale of months to years, do not deal with the simulation of the behavior or state of agents at the microscopic or mesoscopic level. Agents in such models also can be modeled microscopically at the individual level to represent the evolution in the long run, which is widely used in the demography, household relocation, housing market, travel demand, and assignment research field. For instance, explicit modeling of the individual decision process is used to predict the land use evolution and property development closely connected with life-cycle decisions of agents in the demand-side [72]. Research efforts are made to study the fundamental connection between land use patterns and travel needs or activity agendas. The ILUTE (Integrated Land Use, Transportation, Environment) system targets the simulation of agent activities evolving in the long term [73]. The activities/behaviors of person agents can be simulated in terms of birth and death, in-migration and out-migration, the property (e.g., car ownership) change, housing relocation, job location choice, etc. [39, 74].The long-term agent-based models are usually integrated with the models in other levels mentioned above to provide the synthetic population information along with land use, agent’s socioeconomic characteristics, vehicle ownership, etc. [75]. This can be used to analyze the activity schedule, daily plan, and other lower-level choices made at the mid-term and even short-term time scale. The point of the integration approach will be discussed further and extended with more details in the next section.


#### 4.1.4. Hybrid Models

The entities in transport systems like a person, traveler, driver, and vehicle or driver-vehicle unit exhibit diverse behavior patterns at different levels. Models at a simplex temporal scale cannot capture the behavior patterns ranging from mesoscopic mode choice, route choice, and rescheduling to the microscopic simulation of lane changing, gap acceptance, acceleration, deceleration, and braking.In some studies, the short-term models are integrated with the medium-term ones to achieve hybrid agent-based systems, though the difference in emphasis may lead to different internal modeling structures [76]. Galland et al. [77] proposed a computational agent-based model for the carpooling problem, which focused on simulating the interaction of autonomous agents. As Figure 6 shows, the daily travel agenda (schedule) of each agent is predicted by the FEATHERS module, which is a developed activity-based model based on traffic demand [78]. On the one hand, modeling the medium-term individual behaviors is dominant during the carpooling (decision) process, such as activity pattern, mode selection, partner-matching, plan negotiation, and feedback to all concerned agents. On the other hand, the trip/plan execution activity in the carpooling model involves simulating driving agents on the road network. The behavior of a driver activity primarily involves path planning and collision avoidance in the network.Figure 6
An agent-based model for carpooling. Source: Galland et al. [77].Due to the different objectives or focuses of the studies, the structure and the combination of functionalities and/or policy decisions of the hybrid agent-based model may vary substantially. For example, suppose that researchers focus on capturing interaction and policy effects between land development and transportation. Then the model of long-term system evolution in terms of the components like migration, location choice, and auto ownership should be embedded to explore their influences on agents’ decisions in lower time scale models.A hybrid model (simulator or platform) can be composed of long-term and medium-term models. The long-term one is designed to provide land use information by synthesizing a set of households, persons, jobs, etc., while the medium-term one is responsible for the daily activity-travel plan of individuals. In the SimAGENT (Simulator of Activities Greenhouse Emissions, Networks, and Travel) system, the long-term agent-based model modules, PopGen and CEMSELTS (Comprehensive Econometric Microsimulator for Socioeconomics, Land Use, and Transportation System), deal with the generation of a synthetic population and demographic attributes for persons and households, respectively [79]. The CEMDAP (Comprehensive Econometric Microsimulator for Daily Activity travel Patterns) serves as a medium-term simulation component, and an additional traffic assignment component is used to achieve a traveler’s route choice process. Such hybrid modeling approaches can realize the simulation of lower-level decisions like activity participation agenda or daily travel behavior influenced by those at a higher level like residential mobility. Similarly, the daily travel behavior is qualified as a measurement to feed back the decisions in terms of residential mobility, location choice, and economic decisions.


### 4.2. Simulating Adapting and Learning Behaviors

In some agent-based models, various models such as logit models might be applied to better capture individual characteristics of selecting an alternative based on preferences, habits, and prejudices. Some cognition-oriented behavior modeling approaches are also developed to capture the uniqueness, such as motivation, emotion, and risk avoidance that make us human (driver). The information source for adaptation behaviors comes from the agent environment, where positive and negative feedback are introduced. Xiong et al. [17] conducted a study in which the drivers’ behavior was modeled and represented under the effect of the psychological factors. Apart from the threshold-based rules (a behavior or action is triggered when a related parameter exceeds a predetermined threshold, for example, decelerating to keep a safe distance and avoid collision with the car forward and stopping at the intersection with the red light on), more abstract concepts are embedded in individual behavior modeling to represent the agents’ learning and adapting hallmarks.Instead of using mathematical approaches, complex agents can be designed based on the Belief-Desire-Intention (BDI) paradigm. It is developed on cognitive frameworks and embeds beliefs, desires, and intentions into individual agents [80]. The BDI architecture regarding the decision-making mechanism indicated that the decision-making behaviors of agents, such as the perception of information towards the current system state, desirable and motivational state, and final act of agents, greatly influenced the model output [81]. Similarly, PECS (Physis, Emotion, Cognition, Social Status), a multipurpose reference model, was developed to simulate human behavior with the four main factors [82].The knowledge learning ability allows an agent to save information on previous experiences, and the learned knowledge is used to generate a better scheme through this feedback mechanism. Bayesian models are important learning methods to be used in agent-based models. The Bayesian approach provides an excellent conduit to rule the definition of agents in the context of a large number of alternatives and complex internal relationships where agents conduct joint selection behavior. In a Bayesian model, each agent’s spatial and temporal knowledge used for travel-related choices is updated according to the previously learned experience. Probabilities can be modeled into the agent-based model by using Bayesian methods—specifically, a unique probability can be created by an agent or agent population at each point of time or space to represent the decision-making process based on learning and perceived knowledge from the environment. Zou et al. [83] proposed an agent-based model with an iterative search shown in Figure 7 to predict travelers’ choices of mode and departure time. Agents of travelers can make use of information generated in the traffic environment, such as traffic management and control policies, to learn and accumulate travel experience. For mode choice in the Bayesian learning process, a quantitative relationship between the quantified knowledge vector and the individual’s subjective beliefs vector is established based on individuals’ prior beliefs [84]. The learned beliefs are usually used to search for new alternatives of travel decisions, such as mode, departure time, and route, and new plans or strategies are fed into the agent-based model for the next iteration of the simulation.Figure 7
Flowchart of an agent-based choice model. Source: Zou et al. [83].In mimicking the actual human (learning) behavior, reinforcement learning (RL) appears to be a good fit in agent-based models. Such a machine learning paradigm can be fed into modeling agents who seek to maximize cumulative reward by developing a state-action policy through repeated interaction with both agents and the environment [30, 85]. RL often deals with finding the best solution(s) to achieve optimal control or optimal travel behavior. Therefore, RL agents can be equipped with the ability to learn how to survive and optimize their behavior by updating the knowledge base and inference rules in an unknown, complex environment, especially when prior knowledge is quite limited.


### 4.3. Integrating Agent-Based Approaches in Optimization Problems

The agent-based model can be combined with optimization techniques in two ways: (1) embedding an optimizing method into the physical behavior of agents (e.g., by translating optimization algorithms like route-search into agents’ behavior) [86]; (2) utilizing the agent-based model in a simulation-based optimization framework to obtain the simulation results, which are designed as an intermediate component to provide input data iteratively [87].One primary application of the agent-based model in optimization problems is transport management and control. To achieve the objectives such as the optimal allocation of vehicles or the optimal schedule of flights, the optimization strategies can be developed by embedding the agent-based models to the model the process from pursuing the local goals to the global solution. Böcker et al. [88] illustrated a multiagent train coupling and sharing (TCS) scheduling system, which included the planning of train units and the optimization of solutions to reduce the track allocation cost and increase the capacity of tracks. The optimization technique was achieved by a negotiation protocol, which was a process of peer matching: finding the agents that share the same routes and making them share the respective routes by coupling together and splitting up afterward. In the traffic signal control optimization research field, agent-based technology can effectively handle multiple traffic signal systems. It allows the massive information distributed among agents responsible for controlling each intersection, which is influenced by the traffic demand at intersections [89]. In the study of [18], the traffic signal timing optimization problem was solved by integrating a multiagent traffic control component into a mathematical program. Additionally, the multiple signal coordination was developed using the initial typical coordination scheme. It further dealt with the coordination scheme through information exchange among agents controlling different intersections.


### 4.4. Limitations

There are some limitations of the existing agent-based model studies in the following two aspects.


#### 4.4.1. Incomplete and Limited Calibration and Validation Procedures

We can easily notice that the techniques of developing agent-based models for all the different applications in the transport systems are intensively explored. However, the ideological framework cannot be successfully and securely implemented for the lack of explicit calibration and validation varying from different problem scales. Although some attempts have been made to evaluate the proposed agent-based models, there are no acknowledged, standard guidelines formed to conduct the model calibration or validation. For instance, some studies have weakened the calibration of the individual parameters in the large-scale transport system [60, 90]. Some articles did explicitly mention the role of calibration, verification, and validation, but their effects on the final results were not clearly discussed, while other studies just tagged the validation to be the future work [24]. Due to the various applications of the agent-based model approach in transport, it is understandable that there are no specific unified methods/steps of the calibration, verification, or validation process. Nevertheless, different methods like comparing with analytical models should be explored to increase the credibility of the results of agent-based models.


#### 4.4.2. Limitations in Agents’ Behavior Modeling

Although the previous studies have investigated the new ideas of agent modeling and the use of ABMs, agent modeling, especially the complex and adaptive behavior, is still one of the notorious and common difficulties to solve. Human behavior usually needs to be understood concerning intelligent perspectives like preference and memory. Frameworks like PECS and BDI are applied in some studies, but in most cases, they are not completely embedded in the context of agent-based transport systems. Many of those systems are still developed based on simple threshold-based rules, which are not realistic enough. Therefore, one potential improvement to the agent-based model approach is to consider how to merge the behavioral realism in individual-based models to improve the maturity level of the model. The development of AI technology can also provide more and more insights to improve the previous modeling approaches by optimization methods like model training of machine learning.


#### 4.4.3. Limitations in Innovative Approaches to Improve Computing Efficiency

Within many conventional agent-based models, the movement of individual agents is modeled in great detail. The explicit simulation of agent interactions and moving dynamics increase the model complexity, which causes large usage of computational memory and subsequent reduction of processing speeds, in particular for large-scale transport networks consisting of a large number of entities. Therefore, the motivation falls in finding the solutions of some modeling approaches, which offer an answer to protect a certain level of modeling resolution and remain acceptable computational efficiency, seeking the balance of behavioral realism and computational efficiency. For example, Manley et al. [60] studied a hybrid agent-based model framework, which integrates a detailed description of driving behavior with a macroscopic traffic flow model. The agents are granted cognitive abilities in the short-term agent-based model, and a macroscopic-level traffic dynamic model of traffic flow is employed to constrain the aggregated movement of agents. Their hybrid approach allows analyzing drivers’ behavior and interactions without scarifying computational efficiency. However, such innovative approaches are relatively sparse.


## 4.1. Simulation of Transport Systems

In this section, previous research contributions of agent-based models differentiated by rule-based behaviors of agents at the different temporal scales are reviewed. Some studies may select the traditional categorization of the “micro-, meso-, macroscopic” models according to the distinction of spatial information. However, this classification has the limitation in providing insights when the agent-based simulation system to be built have different requirement of the timespan [65]. Moreover, we cannot have a clear mind about the system-related agent-based modeling difference for developing a specific agent-based transport system only based on another categorization of “strategic, tactic, operational” types of modeling. Therefore, this article intends to provide a review of agent-based modeling methods from the perspective of the time horizon of simulation. We adopt a categorization of the models into short-, medium-, and long-term types, comparing the different agent-based modeling techniques and algorithms used in different frameworks. It not only reveals another aspect of information of the traditional one in terms of “strategic, tactic, operational” to some extent but also corresponds to the ideas of microscopic, mesoscopic, and macroscopic simulations in some ways. This categorization is beneficial to provide a reference for exploring agent-based systems at different levels.Short-term movement control models handle the within-day movement of agents at a microscopic granularity. Detailed driving and travel behaviors under the impact of other moving agents or traffic control/management are described. Mid-term models function at the design/operational scheduling: simulating the day-to-day activity and travel patterns of agents, which are often generated by choice model, behavior-based demand model, etc. Long-term planning models, with the view of capturing land use and economic activities, are commonly applied to year-to-year transportation planning studies. The discussions of these three levels of agent-based models are followed by an investigation of the hybrid models, which merge two or all the three different scale time models into a single simulation process. Figure5 presents the important key points of three different models, respectively, to show their difference. Note that for some agent-based models embedding the event-based mechanisms, the time scale might also depend on the event mechanism in the model, and the time between two events may range from seconds to days in the same model.Figure 5
Key points of different models.


### 4.1.1. Short-Term Models

Short-term models are usually developed at a microscopic level and on the temporal scale at a very fine resolution of the second level and some even up to 100 ms (e.g., in pedestrian flow or traffic flow models). As the agents involved in transport systems are basically mobile agents (not necessarily in other domains), the behavior and the relationships with other agents and the external environment for their interaction in the short-term models are usually related to mobility and are commonly used to simulate traffic movement from the physical perspective [66].The mobility of vehicle agents in traffic flow simulation usually involves LC, CF, collision avoidance actions, etc. They can be modeled to function at sensing and perceiving the conditions and states of their environment around. The timely and prompt reaction to the infrastructure equipment such as signboards, traffic signals, and traffic laws can be incorporated. Moreover, the pedestrian agents’ walking behavior modeling also caters to the short-term agent-based models, which involve the modeling of walking direction and relative speed based on the condition of both the pedestrian and road traffic around the current position. Similar researches include the microscopic movement of pedestrians during an evacuation or the movement of cars on the street. Various agent models facilitate the application of short-term agent-based models in mixed traffic flows that refer to the mixture traffic of two or more pedestrians, motorcycles, cars, buses, and trams. The study of such complex vehicular and pedestrian flow patterns should be started from the individual’s behavior pattern.The type of agent and behavior modeling process in these short-term agent-based models vary with different objectives. Yang et al. [67] established the model of pedestrians’ road-crossing behavior to study the pedestrian flow at different traffic volumes. Explicit descriptions of the pedestrians’ road-crossing and decision-making behavior like whether to run through the red light and the followers’ violative behavior are required. Some studies involve other vehicle agents such as motorcycles and public transport. In Fujii et al. [57], a framework for modeling and simulating the mixed traffic flows composed of cars, trams, and pedestrians was constructed to test the control and management strategies. The pedestrian agents have the behavior of free walk, collision avoidance, momentary stop, overtaking, and the following mobility with car and tram agents. The car agents can plan the route and determine acceleration, and the tram agents are extended from the car agents, excluding the LC and route-searching functions. The mobility of agents was expanded with more detail and flexibility in that the existing 1D CF model for cars can be extended to a 2D discrete choice model for pedestrians.


### 4.1.2. Medium-Term Models

Medium-term models refer to the (mesoscopic) simulations that involve mode and route choice, activity schedules, or en route diversion at the temporal scale of minutes or even longer. Such multidimensional travel behavior simulation is effective in analyzing or predicting behavioral changes and the dynamics of the traffic condition [68, 69].An agent-based simulation is frequently used in sharing mobility research to capture the feedback and information-transferring processes. The study of sharing mobility is closely related to the problems of the optimal fleet size dispatch and the passenger-passenger (and driver-passenger) matching. Agent-based models specifically pertaining to sharing mobility phenomenon are grouped into medium-term models. For mobility simulation in these models, the agents can still be modeled at the individual level in terms of mode or route choice process. The attributes and behaviors of agents are related to making decisions about travel plans based on their cognitive information (knowledge) and returning effective feedback of the matching information.Medium-term agent-based models perform well in dynamic ridesharing (also known as on-demand ridesharing and dynamic carpooling), and the model usually involves a service operator dynamically matching potential driver and passenger agents with similar trips and allowing them to travel together and sharing the costs [70]. A binary integer programming was combined with the dynamic auction-based multiagent optimization algorithms to solve a dynamic ridesharing problem [47]. The decentralized agent-based model allows the matching condition of drivers to passengers, which is proved to be flexible in solving ridesharing schemes. In some cases, the medium-term agent-based models contribute to studying multimodal transport systems in that the explicit mode choice and split condition can be presented from an individual perspective. Shen et al. [71] developed an agent-based system combining on-demand and shared AV service and public transportation (PT) to identify the synergy effect. The passengers’ searching, choosing, quitting, waiting, and recalling processes are considered. In this case, all the occupied AVs will be searched, and then the passengers will agree to share the ride according to their sharing preference. As for sharing AVs, shared rides are matched subject to certain constraints like enough available seats and detour constraints.Further study of psychological characteristics affecting the medium-term behaviors of drivers can be conducted to the incorporated preference of agents like passengers’ sharing preference of riding alone or with other riders under certain conditions. Notably, the agent environment in medium-term models is not as specific as that in short-term models with continuous or grid cell tangible interaction space for micromovement particularly related to geospatial and spatial modeling and simulation. Instead, the interactive environments in medium-term models are relatively virtual and fuzzy as they are designed not to deal with physical and social behavior like moving in different directions and at different velocities within their environment.


### 4.1.3. Long-Term Models

Long-term models, which are usually at the temporal scale of months to years, do not deal with the simulation of the behavior or state of agents at the microscopic or mesoscopic level. Agents in such models also can be modeled microscopically at the individual level to represent the evolution in the long run, which is widely used in the demography, household relocation, housing market, travel demand, and assignment research field. For instance, explicit modeling of the individual decision process is used to predict the land use evolution and property development closely connected with life-cycle decisions of agents in the demand-side [72]. Research efforts are made to study the fundamental connection between land use patterns and travel needs or activity agendas. The ILUTE (Integrated Land Use, Transportation, Environment) system targets the simulation of agent activities evolving in the long term [73]. The activities/behaviors of person agents can be simulated in terms of birth and death, in-migration and out-migration, the property (e.g., car ownership) change, housing relocation, job location choice, etc. [39, 74].The long-term agent-based models are usually integrated with the models in other levels mentioned above to provide the synthetic population information along with land use, agent’s socioeconomic characteristics, vehicle ownership, etc. [75]. This can be used to analyze the activity schedule, daily plan, and other lower-level choices made at the mid-term and even short-term time scale. The point of the integration approach will be discussed further and extended with more details in the next section.


### 4.1.4. Hybrid Models

The entities in transport systems like a person, traveler, driver, and vehicle or driver-vehicle unit exhibit diverse behavior patterns at different levels. Models at a simplex temporal scale cannot capture the behavior patterns ranging from mesoscopic mode choice, route choice, and rescheduling to the microscopic simulation of lane changing, gap acceptance, acceleration, deceleration, and braking.In some studies, the short-term models are integrated with the medium-term ones to achieve hybrid agent-based systems, though the difference in emphasis may lead to different internal modeling structures [76]. Galland et al. [77] proposed a computational agent-based model for the carpooling problem, which focused on simulating the interaction of autonomous agents. As Figure 6 shows, the daily travel agenda (schedule) of each agent is predicted by the FEATHERS module, which is a developed activity-based model based on traffic demand [78]. On the one hand, modeling the medium-term individual behaviors is dominant during the carpooling (decision) process, such as activity pattern, mode selection, partner-matching, plan negotiation, and feedback to all concerned agents. On the other hand, the trip/plan execution activity in the carpooling model involves simulating driving agents on the road network. The behavior of a driver activity primarily involves path planning and collision avoidance in the network.Figure 6
An agent-based model for carpooling. Source: Galland et al. [77].Due to the different objectives or focuses of the studies, the structure and the combination of functionalities and/or policy decisions of the hybrid agent-based model may vary substantially. For example, suppose that researchers focus on capturing interaction and policy effects between land development and transportation. Then the model of long-term system evolution in terms of the components like migration, location choice, and auto ownership should be embedded to explore their influences on agents’ decisions in lower time scale models.A hybrid model (simulator or platform) can be composed of long-term and medium-term models. The long-term one is designed to provide land use information by synthesizing a set of households, persons, jobs, etc., while the medium-term one is responsible for the daily activity-travel plan of individuals. In the SimAGENT (Simulator of Activities Greenhouse Emissions, Networks, and Travel) system, the long-term agent-based model modules, PopGen and CEMSELTS (Comprehensive Econometric Microsimulator for Socioeconomics, Land Use, and Transportation System), deal with the generation of a synthetic population and demographic attributes for persons and households, respectively [79]. The CEMDAP (Comprehensive Econometric Microsimulator for Daily Activity travel Patterns) serves as a medium-term simulation component, and an additional traffic assignment component is used to achieve a traveler’s route choice process. Such hybrid modeling approaches can realize the simulation of lower-level decisions like activity participation agenda or daily travel behavior influenced by those at a higher level like residential mobility. Similarly, the daily travel behavior is qualified as a measurement to feed back the decisions in terms of residential mobility, location choice, and economic decisions.


## 4.1.1. Short-Term Models

Short-term models are usually developed at a microscopic level and on the temporal scale at a very fine resolution of the second level and some even up to 100 ms (e.g., in pedestrian flow or traffic flow models). As the agents involved in transport systems are basically mobile agents (not necessarily in other domains), the behavior and the relationships with other agents and the external environment for their interaction in the short-term models are usually related to mobility and are commonly used to simulate traffic movement from the physical perspective [66].The mobility of vehicle agents in traffic flow simulation usually involves LC, CF, collision avoidance actions, etc. They can be modeled to function at sensing and perceiving the conditions and states of their environment around. The timely and prompt reaction to the infrastructure equipment such as signboards, traffic signals, and traffic laws can be incorporated. Moreover, the pedestrian agents’ walking behavior modeling also caters to the short-term agent-based models, which involve the modeling of walking direction and relative speed based on the condition of both the pedestrian and road traffic around the current position. Similar researches include the microscopic movement of pedestrians during an evacuation or the movement of cars on the street. Various agent models facilitate the application of short-term agent-based models in mixed traffic flows that refer to the mixture traffic of two or more pedestrians, motorcycles, cars, buses, and trams. The study of such complex vehicular and pedestrian flow patterns should be started from the individual’s behavior pattern.The type of agent and behavior modeling process in these short-term agent-based models vary with different objectives. Yang et al. [67] established the model of pedestrians’ road-crossing behavior to study the pedestrian flow at different traffic volumes. Explicit descriptions of the pedestrians’ road-crossing and decision-making behavior like whether to run through the red light and the followers’ violative behavior are required. Some studies involve other vehicle agents such as motorcycles and public transport. In Fujii et al. [57], a framework for modeling and simulating the mixed traffic flows composed of cars, trams, and pedestrians was constructed to test the control and management strategies. The pedestrian agents have the behavior of free walk, collision avoidance, momentary stop, overtaking, and the following mobility with car and tram agents. The car agents can plan the route and determine acceleration, and the tram agents are extended from the car agents, excluding the LC and route-searching functions. The mobility of agents was expanded with more detail and flexibility in that the existing 1D CF model for cars can be extended to a 2D discrete choice model for pedestrians.


## 4.1.2. Medium-Term Models

Medium-term models refer to the (mesoscopic) simulations that involve mode and route choice, activity schedules, or en route diversion at the temporal scale of minutes or even longer. Such multidimensional travel behavior simulation is effective in analyzing or predicting behavioral changes and the dynamics of the traffic condition [68, 69].An agent-based simulation is frequently used in sharing mobility research to capture the feedback and information-transferring processes. The study of sharing mobility is closely related to the problems of the optimal fleet size dispatch and the passenger-passenger (and driver-passenger) matching. Agent-based models specifically pertaining to sharing mobility phenomenon are grouped into medium-term models. For mobility simulation in these models, the agents can still be modeled at the individual level in terms of mode or route choice process. The attributes and behaviors of agents are related to making decisions about travel plans based on their cognitive information (knowledge) and returning effective feedback of the matching information.Medium-term agent-based models perform well in dynamic ridesharing (also known as on-demand ridesharing and dynamic carpooling), and the model usually involves a service operator dynamically matching potential driver and passenger agents with similar trips and allowing them to travel together and sharing the costs [70]. A binary integer programming was combined with the dynamic auction-based multiagent optimization algorithms to solve a dynamic ridesharing problem [47]. The decentralized agent-based model allows the matching condition of drivers to passengers, which is proved to be flexible in solving ridesharing schemes. In some cases, the medium-term agent-based models contribute to studying multimodal transport systems in that the explicit mode choice and split condition can be presented from an individual perspective. Shen et al. [71] developed an agent-based system combining on-demand and shared AV service and public transportation (PT) to identify the synergy effect. The passengers’ searching, choosing, quitting, waiting, and recalling processes are considered. In this case, all the occupied AVs will be searched, and then the passengers will agree to share the ride according to their sharing preference. As for sharing AVs, shared rides are matched subject to certain constraints like enough available seats and detour constraints.Further study of psychological characteristics affecting the medium-term behaviors of drivers can be conducted to the incorporated preference of agents like passengers’ sharing preference of riding alone or with other riders under certain conditions. Notably, the agent environment in medium-term models is not as specific as that in short-term models with continuous or grid cell tangible interaction space for micromovement particularly related to geospatial and spatial modeling and simulation. Instead, the interactive environments in medium-term models are relatively virtual and fuzzy as they are designed not to deal with physical and social behavior like moving in different directions and at different velocities within their environment.


## 4.1.3. Long-Term Models

Long-term models, which are usually at the temporal scale of months to years, do not deal with the simulation of the behavior or state of agents at the microscopic or mesoscopic level. Agents in such models also can be modeled microscopically at the individual level to represent the evolution in the long run, which is widely used in the demography, household relocation, housing market, travel demand, and assignment research field. For instance, explicit modeling of the individual decision process is used to predict the land use evolution and property development closely connected with life-cycle decisions of agents in the demand-side [72]. Research efforts are made to study the fundamental connection between land use patterns and travel needs or activity agendas. The ILUTE (Integrated Land Use, Transportation, Environment) system targets the simulation of agent activities evolving in the long term [73]. The activities/behaviors of person agents can be simulated in terms of birth and death, in-migration and out-migration, the property (e.g., car ownership) change, housing relocation, job location choice, etc. [39, 74].The long-term agent-based models are usually integrated with the models in other levels mentioned above to provide the synthetic population information along with land use, agent’s socioeconomic characteristics, vehicle ownership, etc. [75]. This can be used to analyze the activity schedule, daily plan, and other lower-level choices made at the mid-term and even short-term time scale. The point of the integration approach will be discussed further and extended with more details in the next section.


## 4.1.4. Hybrid Models

The entities in transport systems like a person, traveler, driver, and vehicle or driver-vehicle unit exhibit diverse behavior patterns at different levels. Models at a simplex temporal scale cannot capture the behavior patterns ranging from mesoscopic mode choice, route choice, and rescheduling to the microscopic simulation of lane changing, gap acceptance, acceleration, deceleration, and braking.In some studies, the short-term models are integrated with the medium-term ones to achieve hybrid agent-based systems, though the difference in emphasis may lead to different internal modeling structures [76]. Galland et al. [77] proposed a computational agent-based model for the carpooling problem, which focused on simulating the interaction of autonomous agents. As Figure 6 shows, the daily travel agenda (schedule) of each agent is predicted by the FEATHERS module, which is a developed activity-based model based on traffic demand [78]. On the one hand, modeling the medium-term individual behaviors is dominant during the carpooling (decision) process, such as activity pattern, mode selection, partner-matching, plan negotiation, and feedback to all concerned agents. On the other hand, the trip/plan execution activity in the carpooling model involves simulating driving agents on the road network. The behavior of a driver activity primarily involves path planning and collision avoidance in the network.Figure 6
An agent-based model for carpooling. Source: Galland et al. [77].Due to the different objectives or focuses of the studies, the structure and the combination of functionalities and/or policy decisions of the hybrid agent-based model may vary substantially. For example, suppose that researchers focus on capturing interaction and policy effects between land development and transportation. Then the model of long-term system evolution in terms of the components like migration, location choice, and auto ownership should be embedded to explore their influences on agents’ decisions in lower time scale models.A hybrid model (simulator or platform) can be composed of long-term and medium-term models. The long-term one is designed to provide land use information by synthesizing a set of households, persons, jobs, etc., while the medium-term one is responsible for the daily activity-travel plan of individuals. In the SimAGENT (Simulator of Activities Greenhouse Emissions, Networks, and Travel) system, the long-term agent-based model modules, PopGen and CEMSELTS (Comprehensive Econometric Microsimulator for Socioeconomics, Land Use, and Transportation System), deal with the generation of a synthetic population and demographic attributes for persons and households, respectively [79]. The CEMDAP (Comprehensive Econometric Microsimulator for Daily Activity travel Patterns) serves as a medium-term simulation component, and an additional traffic assignment component is used to achieve a traveler’s route choice process. Such hybrid modeling approaches can realize the simulation of lower-level decisions like activity participation agenda or daily travel behavior influenced by those at a higher level like residential mobility. Similarly, the daily travel behavior is qualified as a measurement to feed back the decisions in terms of residential mobility, location choice, and economic decisions.


## 4.2. Simulating Adapting and Learning Behaviors

In some agent-based models, various models such as logit models might be applied to better capture individual characteristics of selecting an alternative based on preferences, habits, and prejudices. Some cognition-oriented behavior modeling approaches are also developed to capture the uniqueness, such as motivation, emotion, and risk avoidance that make us human (driver). The information source for adaptation behaviors comes from the agent environment, where positive and negative feedback are introduced. Xiong et al. [17] conducted a study in which the drivers’ behavior was modeled and represented under the effect of the psychological factors. Apart from the threshold-based rules (a behavior or action is triggered when a related parameter exceeds a predetermined threshold, for example, decelerating to keep a safe distance and avoid collision with the car forward and stopping at the intersection with the red light on), more abstract concepts are embedded in individual behavior modeling to represent the agents’ learning and adapting hallmarks.Instead of using mathematical approaches, complex agents can be designed based on the Belief-Desire-Intention (BDI) paradigm. It is developed on cognitive frameworks and embeds beliefs, desires, and intentions into individual agents [80]. The BDI architecture regarding the decision-making mechanism indicated that the decision-making behaviors of agents, such as the perception of information towards the current system state, desirable and motivational state, and final act of agents, greatly influenced the model output [81]. Similarly, PECS (Physis, Emotion, Cognition, Social Status), a multipurpose reference model, was developed to simulate human behavior with the four main factors [82].The knowledge learning ability allows an agent to save information on previous experiences, and the learned knowledge is used to generate a better scheme through this feedback mechanism. Bayesian models are important learning methods to be used in agent-based models. The Bayesian approach provides an excellent conduit to rule the definition of agents in the context of a large number of alternatives and complex internal relationships where agents conduct joint selection behavior. In a Bayesian model, each agent’s spatial and temporal knowledge used for travel-related choices is updated according to the previously learned experience. Probabilities can be modeled into the agent-based model by using Bayesian methods—specifically, a unique probability can be created by an agent or agent population at each point of time or space to represent the decision-making process based on learning and perceived knowledge from the environment. Zou et al. [83] proposed an agent-based model with an iterative search shown in Figure 7 to predict travelers’ choices of mode and departure time. Agents of travelers can make use of information generated in the traffic environment, such as traffic management and control policies, to learn and accumulate travel experience. For mode choice in the Bayesian learning process, a quantitative relationship between the quantified knowledge vector and the individual’s subjective beliefs vector is established based on individuals’ prior beliefs [84]. The learned beliefs are usually used to search for new alternatives of travel decisions, such as mode, departure time, and route, and new plans or strategies are fed into the agent-based model for the next iteration of the simulation.Figure 7
Flowchart of an agent-based choice model. Source: Zou et al. [83].In mimicking the actual human (learning) behavior, reinforcement learning (RL) appears to be a good fit in agent-based models. Such a machine learning paradigm can be fed into modeling agents who seek to maximize cumulative reward by developing a state-action policy through repeated interaction with both agents and the environment [30, 85]. RL often deals with finding the best solution(s) to achieve optimal control or optimal travel behavior. Therefore, RL agents can be equipped with the ability to learn how to survive and optimize their behavior by updating the knowledge base and inference rules in an unknown, complex environment, especially when prior knowledge is quite limited.


## 4.3. Integrating Agent-Based Approaches in Optimization Problems

The agent-based model can be combined with optimization techniques in two ways: (1) embedding an optimizing method into the physical behavior of agents (e.g., by translating optimization algorithms like route-search into agents’ behavior) [86]; (2) utilizing the agent-based model in a simulation-based optimization framework to obtain the simulation results, which are designed as an intermediate component to provide input data iteratively [87].One primary application of the agent-based model in optimization problems is transport management and control. To achieve the objectives such as the optimal allocation of vehicles or the optimal schedule of flights, the optimization strategies can be developed by embedding the agent-based models to the model the process from pursuing the local goals to the global solution. Böcker et al. [88] illustrated a multiagent train coupling and sharing (TCS) scheduling system, which included the planning of train units and the optimization of solutions to reduce the track allocation cost and increase the capacity of tracks. The optimization technique was achieved by a negotiation protocol, which was a process of peer matching: finding the agents that share the same routes and making them share the respective routes by coupling together and splitting up afterward. In the traffic signal control optimization research field, agent-based technology can effectively handle multiple traffic signal systems. It allows the massive information distributed among agents responsible for controlling each intersection, which is influenced by the traffic demand at intersections [89]. In the study of [18], the traffic signal timing optimization problem was solved by integrating a multiagent traffic control component into a mathematical program. Additionally, the multiple signal coordination was developed using the initial typical coordination scheme. It further dealt with the coordination scheme through information exchange among agents controlling different intersections.


## 4.4. Limitations

There are some limitations of the existing agent-based model studies in the following two aspects.


### 4.4.1. Incomplete and Limited Calibration and Validation Procedures

We can easily notice that the techniques of developing agent-based models for all the different applications in the transport systems are intensively explored. However, the ideological framework cannot be successfully and securely implemented for the lack of explicit calibration and validation varying from different problem scales. Although some attempts have been made to evaluate the proposed agent-based models, there are no acknowledged, standard guidelines formed to conduct the model calibration or validation. For instance, some studies have weakened the calibration of the individual parameters in the large-scale transport system [60, 90]. Some articles did explicitly mention the role of calibration, verification, and validation, but their effects on the final results were not clearly discussed, while other studies just tagged the validation to be the future work [24]. Due to the various applications of the agent-based model approach in transport, it is understandable that there are no specific unified methods/steps of the calibration, verification, or validation process. Nevertheless, different methods like comparing with analytical models should be explored to increase the credibility of the results of agent-based models.


### 4.4.2. Limitations in Agents’ Behavior Modeling

Although the previous studies have investigated the new ideas of agent modeling and the use of ABMs, agent modeling, especially the complex and adaptive behavior, is still one of the notorious and common difficulties to solve. Human behavior usually needs to be understood concerning intelligent perspectives like preference and memory. Frameworks like PECS and BDI are applied in some studies, but in most cases, they are not completely embedded in the context of agent-based transport systems. Many of those systems are still developed based on simple threshold-based rules, which are not realistic enough. Therefore, one potential improvement to the agent-based model approach is to consider how to merge the behavioral realism in individual-based models to improve the maturity level of the model. The development of AI technology can also provide more and more insights to improve the previous modeling approaches by optimization methods like model training of machine learning.


### 4.4.3. Limitations in Innovative Approaches to Improve Computing Efficiency

Within many conventional agent-based models, the movement of individual agents is modeled in great detail. The explicit simulation of agent interactions and moving dynamics increase the model complexity, which causes large usage of computational memory and subsequent reduction of processing speeds, in particular for large-scale transport networks consisting of a large number of entities. Therefore, the motivation falls in finding the solutions of some modeling approaches, which offer an answer to protect a certain level of modeling resolution and remain acceptable computational efficiency, seeking the balance of behavioral realism and computational efficiency. For example, Manley et al. [60] studied a hybrid agent-based model framework, which integrates a detailed description of driving behavior with a macroscopic traffic flow model. The agents are granted cognitive abilities in the short-term agent-based model, and a macroscopic-level traffic dynamic model of traffic flow is employed to constrain the aggregated movement of agents. Their hybrid approach allows analyzing drivers’ behavior and interactions without scarifying computational efficiency. However, such innovative approaches are relatively sparse.


## 4.4.1. Incomplete and Limited Calibration and Validation Procedures

We can easily notice that the techniques of developing agent-based models for all the different applications in the transport systems are intensively explored. However, the ideological framework cannot be successfully and securely implemented for the lack of explicit calibration and validation varying from different problem scales. Although some attempts have been made to evaluate the proposed agent-based models, there are no acknowledged, standard guidelines formed to conduct the model calibration or validation. For instance, some studies have weakened the calibration of the individual parameters in the large-scale transport system [60, 90]. Some articles did explicitly mention the role of calibration, verification, and validation, but their effects on the final results were not clearly discussed, while other studies just tagged the validation to be the future work [24]. Due to the various applications of the agent-based model approach in transport, it is understandable that there are no specific unified methods/steps of the calibration, verification, or validation process. Nevertheless, different methods like comparing with analytical models should be explored to increase the credibility of the results of agent-based models.


## 4.4.2. Limitations in Agents’ Behavior Modeling

Although the previous studies have investigated the new ideas of agent modeling and the use of ABMs, agent modeling, especially the complex and adaptive behavior, is still one of the notorious and common difficulties to solve. Human behavior usually needs to be understood concerning intelligent perspectives like preference and memory. Frameworks like PECS and BDI are applied in some studies, but in most cases, they are not completely embedded in the context of agent-based transport systems. Many of those systems are still developed based on simple threshold-based rules, which are not realistic enough. Therefore, one potential improvement to the agent-based model approach is to consider how to merge the behavioral realism in individual-based models to improve the maturity level of the model. The development of AI technology can also provide more and more insights to improve the previous modeling approaches by optimization methods like model training of machine learning.


## 4.4.3. Limitations in Innovative Approaches to Improve Computing Efficiency

Within many conventional agent-based models, the movement of individual agents is modeled in great detail. The explicit simulation of agent interactions and moving dynamics increase the model complexity, which causes large usage of computational memory and subsequent reduction of processing speeds, in particular for large-scale transport networks consisting of a large number of entities. Therefore, the motivation falls in finding the solutions of some modeling approaches, which offer an answer to protect a certain level of modeling resolution and remain acceptable computational efficiency, seeking the balance of behavioral realism and computational efficiency. For example, Manley et al. [60] studied a hybrid agent-based model framework, which integrates a detailed description of driving behavior with a macroscopic traffic flow model. The agents are granted cognitive abilities in the short-term agent-based model, and a macroscopic-level traffic dynamic model of traffic flow is employed to constrain the aggregated movement of agents. Their hybrid approach allows analyzing drivers’ behavior and interactions without scarifying computational efficiency. However, such innovative approaches are relatively sparse.


## 5. Conclusion and Discussion

The agent-based model can be used to study the systems consisting of agents that simultaneously interact and affect each other. The transport systems, which are complex, dynamic, stochastic, and heterogeneous, conform to these characteristics and hence are ideal to be modeled and analyzed using agent-based models. This article explains the thriving applications of the agent-based model in the transport research domain, such as traffic management, dynamic route, traffic signal control, mode choice, environmental/economic analysis, and transportation infrastructure and policy development. The advantages of the agent-based model approach in transport modeling and analysis can be summarized as follows: (1) consideration of the interactive features and rule settings in the study of heterogenicity and emergent dynamics in the form of different time scale models; (2) exploration of modeling the adaptability, beliefs, and desires of individual entities to establish a more robust system; (3) capability of being integrated with optimization to achieve realistic and reliable results.


### 5.1. Recommendations

Agent-based modeling and simulation approaches are widely applied for transport systems with mixed traffic, mobility adjustments, dynamically personalized routing guidance, cooperative behavior. The differences in the predefined purpose lead to different agent-based models. This article made a comparison of the agent-based models developed in the literature by categorizing them into three different types as per time scale in behavior (rule) simulation. The hybrid modeling approach is discussed further with refinement in model establishment patterns. The short-term, medium-term, and hybrid agent-based models are the most commonly used in transport research.The short-term agent-based models can be used for systems that focus on detailed motion level simulation (vehicle model, pedestrian model, information-sharing rules, mobile coordination, etc.) for analyzing the traffic condition or assessing the local impact of some proposed extension plan for a real city. However, the high-resolution simulation demands considerable computer memory if the agent number and the network size reach the scale of a city, thus slowing down the model’s execution speed. For general cases, we recommend covering a scale of hundreds of thousands of agents within a specific area. Otherwise, some simplifications should be made to the agent models.In (on-demand) sharing systems, due to the complex and discontinuous characteristics or the discrete protocols for interaction related to decision-making behavior, it is impossible to deal with them by nonlinear analytical methods. Medium-term agent-based models are well suited to deal with the great complexity of the systems under study and assess the performance of some proposed services or plans in various scenarios. The feedback information from interaction among different agents directly influences the following decision-making process. In addition, positioned at the strategic level of simulation, medium-term agent-based models can capture the transformation pattern of agents’ choice (e.g., due to an incident) that affects the system performance.In practice, the hybrid agent-based models are gaining wide consideration within transport system simulation areas as the models allow explicit modeling of the individual decision process from microscopic to macroscopic levels and demand to supply sides. Incorporating the short-, medium-, and long-term models, the hybrid modeling approaches integrate microscopic, mesoscopic, and macroscopic simulation to achieve joint and multiscale systems that meet multidimensional simulation requirements. In some cases, if short-term models are omitted or simplified properly, like searching for very fast microscopic (or mesoscopic) mobility models to speed up the computation, these hybrid models can be applied across relatively large regions like a whole transport system in a city with millions of people. Moreover, developing the simulation system in a single type of model may lead to a waste of computational power if the models are only designed in a short-term way or the failure of an accurate description of the phenomenon or systems otherwise. Using hybrid models, components in different levels of resolution can be grouped according to the characteristics of the targeted transport system and the modeling and simulation purposes. Therefore, hybrid models have advantages in terms of computational power savings by focusing on different simulation parts or evolving stages of the whole transport system.As an important application domain of agent-based models, the intelligence, adaptation, and learning behavior simulation can be further applied in AVs [91–93]. The key features of intelligent and autonomous vehicles (IAVs) compared to humans are that they are more likely to cooperate, be fast in processing communication information, and make more precise operations and decisions. In the domain of AVs, collaborative driving is a key developing direction of ITS (Intelligent Transportation System).


### 5.2. Insights for Future Research

For future research of agent-based models in the transport domain, one important aspect is researching how to develop efficient and accurate individual behavioral models. This is hard to achieve by simply introducing more probabilistic descriptions. It is necessary to figure out how to identify and characterize the key behavior of stakeholders such as drivers in real-life transport systems and feed them (directly) into the representations of behavior in agents. The large-scale simulation, like city-scale or even country-scale, is another research gap of the agent-based model, which requires increased support in computer processing power and adequate and accurate data sources. Besides, the credibility of the agent-based models and their results need to be increased, which involves extracting useful and valid data needed for model calibration and validation [94].The agent-based modeling and simulation have formed a relatively perfect theory, method, and technology system, which will provide a solid foundation and favorable support for the research and application of Digital Twin (DT) technology in transport systems. Considering the disruptive forces that are transforming the mobility ecosystem, transportation industries have to manage the social and economic impacts of traffic growth and provide more “intelligence” components to manage future challenges. The key elements, the rules of the agents’ movement, and the interactions of the “twin” system can be clearly defined close to the real-world based on agent-based modeling. Therefore, in combination with agent-based models, DT can be used to develop an upgraded “Dynamic Information Physical System” closely integrated with the actual traffic system and eliminate the effects of some inevitable deviations of the accurate matching and real-time performance of data in simulation. The digital and structured data can be obtained through the process of model reconstruction and accurate perception on the computing platform, which has twin functions established with these agent-based rules in place. These calculated real-time data, applied to all traffic services, and can be used to quickly realize the representation and detection of traffic behavior and the deduction and automatic analysis and prediction of traffic state [95, 96].The gaps in the context of the different transportation settings in developing economies vs. developed economies may also lead to more challenges and insights for future research. Industry digitalization has become the key leading force driving the development of the global digital economy. The transportation settings like the infrastructures are more complete under the developed economies and more promising to implement the agent-based transportation scheme. Applying the agent-based model framework in a real-world complex system requires a number of critical issues like collecting and integrating multisource data that is readily used to construct a complete business platform. The higher the level of economic development, the higher the digitalization of the industry. Therefore, based on the advantages of artificial intelligence and Big Data technology, the developed economies better promote the successful implementation of agent-based technologies and achieve the objectives of alleviating urban traffic congestion. Specifically, no matter in the developing economies or the developed economies, there can be significant challenges to achieve the accurate perception of traffic events and traffic flow based on holographic data, traffic knowledge maps establishment to mine historical and real-time portraits of road network supply capacity, and travel characteristics of people and vehicles and the development of the terminal of the intelligent transportation operation architecture.


## 5.1. Recommendations

Agent-based modeling and simulation approaches are widely applied for transport systems with mixed traffic, mobility adjustments, dynamically personalized routing guidance, cooperative behavior. The differences in the predefined purpose lead to different agent-based models. This article made a comparison of the agent-based models developed in the literature by categorizing them into three different types as per time scale in behavior (rule) simulation. The hybrid modeling approach is discussed further with refinement in model establishment patterns. The short-term, medium-term, and hybrid agent-based models are the most commonly used in transport research.The short-term agent-based models can be used for systems that focus on detailed motion level simulation (vehicle model, pedestrian model, information-sharing rules, mobile coordination, etc.) for analyzing the traffic condition or assessing the local impact of some proposed extension plan for a real city. However, the high-resolution simulation demands considerable computer memory if the agent number and the network size reach the scale of a city, thus slowing down the model’s execution speed. For general cases, we recommend covering a scale of hundreds of thousands of agents within a specific area. Otherwise, some simplifications should be made to the agent models.In (on-demand) sharing systems, due to the complex and discontinuous characteristics or the discrete protocols for interaction related to decision-making behavior, it is impossible to deal with them by nonlinear analytical methods. Medium-term agent-based models are well suited to deal with the great complexity of the systems under study and assess the performance of some proposed services or plans in various scenarios. The feedback information from interaction among different agents directly influences the following decision-making process. In addition, positioned at the strategic level of simulation, medium-term agent-based models can capture the transformation pattern of agents’ choice (e.g., due to an incident) that affects the system performance.In practice, the hybrid agent-based models are gaining wide consideration within transport system simulation areas as the models allow explicit modeling of the individual decision process from microscopic to macroscopic levels and demand to supply sides. Incorporating the short-, medium-, and long-term models, the hybrid modeling approaches integrate microscopic, mesoscopic, and macroscopic simulation to achieve joint and multiscale systems that meet multidimensional simulation requirements. In some cases, if short-term models are omitted or simplified properly, like searching for very fast microscopic (or mesoscopic) mobility models to speed up the computation, these hybrid models can be applied across relatively large regions like a whole transport system in a city with millions of people. Moreover, developing the simulation system in a single type of model may lead to a waste of computational power if the models are only designed in a short-term way or the failure of an accurate description of the phenomenon or systems otherwise. Using hybrid models, components in different levels of resolution can be grouped according to the characteristics of the targeted transport system and the modeling and simulation purposes. Therefore, hybrid models have advantages in terms of computational power savings by focusing on different simulation parts or evolving stages of the whole transport system.As an important application domain of agent-based models, the intelligence, adaptation, and learning behavior simulation can be further applied in AVs [91–93]. The key features of intelligent and autonomous vehicles (IAVs) compared to humans are that they are more likely to cooperate, be fast in processing communication information, and make more precise operations and decisions. In the domain of AVs, collaborative driving is a key developing direction of ITS (Intelligent Transportation System).


## 5.2. Insights for Future Research

For future research of agent-based models in the transport domain, one important aspect is researching how to develop efficient and accurate individual behavioral models. This is hard to achieve by simply introducing more probabilistic descriptions. It is necessary to figure out how to identify and characterize the key behavior of stakeholders such as drivers in real-life transport systems and feed them (directly) into the representations of behavior in agents. The large-scale simulation, like city-scale or even country-scale, is another research gap of the agent-based model, which requires increased support in computer processing power and adequate and accurate data sources. Besides, the credibility of the agent-based models and their results need to be increased, which involves extracting useful and valid data needed for model calibration and validation [94].The agent-based modeling and simulation have formed a relatively perfect theory, method, and technology system, which will provide a solid foundation and favorable support for the research and application of Digital Twin (DT) technology in transport systems. Considering the disruptive forces that are transforming the mobility ecosystem, transportation industries have to manage the social and economic impacts of traffic growth and provide more “intelligence” components to manage future challenges. The key elements, the rules of the agents’ movement, and the interactions of the “twin” system can be clearly defined close to the real-world based on agent-based modeling. Therefore, in combination with agent-based models, DT can be used to develop an upgraded “Dynamic Information Physical System” closely integrated with the actual traffic system and eliminate the effects of some inevitable deviations of the accurate matching and real-time performance of data in simulation. The digital and structured data can be obtained through the process of model reconstruction and accurate perception on the computing platform, which has twin functions established with these agent-based rules in place. These calculated real-time data, applied to all traffic services, and can be used to quickly realize the representation and detection of traffic behavior and the deduction and automatic analysis and prediction of traffic state [95, 96].The gaps in the context of the different transportation settings in developing economies vs. developed economies may also lead to more challenges and insights for future research. Industry digitalization has become the key leading force driving the development of the global digital economy. The transportation settings like the infrastructures are more complete under the developed economies and more promising to implement the agent-based transportation scheme. Applying the agent-based model framework in a real-world complex system requires a number of critical issues like collecting and integrating multisource data that is readily used to construct a complete business platform. The higher the level of economic development, the higher the digitalization of the industry. Therefore, based on the advantages of artificial intelligence and Big Data technology, the developed economies better promote the successful implementation of agent-based technologies and achieve the objectives of alleviating urban traffic congestion. Specifically, no matter in the developing economies or the developed economies, there can be significant challenges to achieve the accurate perception of traffic events and traffic flow based on holographic data, traffic knowledge maps establishment to mine historical and real-time portraits of road network supply capacity, and travel characteristics of people and vehicles and the development of the terminal of the intelligent transportation operation architecture.



---

*Source: 1252534-2022-02-27.xml*