CBLab: Scalable Traffic Simulation with Enriched Data Supporting
Authors: Chumeng Liang
Zherui Huang
Yicheng Liu
Zhanyu Liu
Guanjie Zheng
Hanyuan Shi
Yuhao Du
FULIANG LI
Zhenhui Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: Infrastructure
Traffic Policy
Traffic Simulation
Large-scale Dataset

Abstract: Traffic simulation provides interactive data for the optimization of traffic policies. However, existing traffic simulators are limited by their lack of scalability and shortage in input data, which prevents them from generating interactive data from traffic simulation in the scenarios of real large-scale city road networks. 

In this paper, we present \textbf{C}ity \textbf{B}rain \textbf{Lab}, a toolkit for scalable traffic simulation. CBLab is consist of three components: CBEngine, CBData, and CBScenario. CBEngine is a highly efficient simulator supporting large-scale traffic simulation. CBData includes a traffic dataset with road network data of 100 cities all around the world. We also develop a pipeline to conduct a one-click transformation from raw road networks to input data of our traffic simulation. Combining CBEngine and CBData allows researchers to run scalable traffic simulations in the road network of real large-scale cities. Based on that, CBScenario implements an interactive environment and several baseline methods for two scenarios of traffic policies respectively, with which traffic policies adaptable for large-scale urban traffic can be trained and tuned. To the best of our knowledge, CBLab is the first infrastructure supporting traffic policy optimization in large-scale urban scenarios. The code is available on GitHub:~\url{https://github.com/CityBrainLab/CityBrainLab.git}.

Under review as a conference paper at ICLR 2023
CBLAB: SCALABLE TRAFFIC SIMULATION WITH EN- RICHED DATA SUPPORTING
Anonymous authors Paper under double-blind review
ABSTRACT
Traffic simulation provides interactive data for the optimization of traffic control policies. However, existing traffic simulators are limited by their lack of scalability and shortage input data, which prevents them from generating interactive data from traffic simulation in the scenarios of real large-scale city road networks.
In this paper, we present City Brain Lab, toolkit for scalable traffic simula- tion. CBLab consists of three components: CBEngine, CBData, and CBSce- nario. CBEngine is a highly efficient simulator supporting large-scale traffic simulation. CBData includes a traffic dataset with road network data of 100 cities all around the world. We also develop pipeline conduct a one-click transformation from raw road networks to input data of our traffic simulation. Combining CBEngine and CBData allows researchers run scalable traffic simulations in the road network of real large-scale cities. Based on that, CB- Scenario implements an interactive environment and several baseline methods for two scenarios of traffic control policies respectively, with which traffic con- trol policies adaptable for large-scale urban traffic can be trained and tuned. To the best of knowledge, CBLab is the first infrastructure supporting traffic policy optimization in lcrge urban scenarios. The code is available on qit.
GitHub: pectps://qithh: com/ cttm:/Ciyysaaaaaa/CCCttt.
CBLab
Data Tool
Simulator
Enviroment
Action
Deploy
CBData
CBEngine
CBSoenario
Store
Input Data
Observation
Tman
Raw Traffic ata
Traffic Policies
Figure I: An overview of CBLab.
INTRODUCTION
Well-crafted traffic control policies, such as traffic signal control and congestion pricing, are expected improve the efficiency of urban transportation. In recent years, many studies have been conducted to optimize the traffic control policies according to real-time traffic data HHarnoja et al. (2018); Wei al. (2018) 2019a); Zheng al. (201); Zhang al. (2020); Chen et al. (2020a); Zang et al. (2020); Oroojlooy et al. 2020); Wu et al. 2022 b): Zhang et al. (2022); Mirzaei et al. (2018) Qiu et al.
(2019) These policies depend on data generated by interaction with the traffic environment where
they explore to make good decisions under different consequences.
However, real-world urban traffic cannot provide enough interactive data train these policies, because the exploration of the policy may have toxic impact on the urban traffic e.g. provoke severe congestion. Traffic simulators are therefore born as alternatives to provide traffic environments for traffic control policies interact with. These simulators Lopez et al. (2018): : Zhang et al. (2019); Chen et al. (2020b) simulate the microscopic evolution of the urban traffic. For each time step, they describe the traffic state, obtain a traffic action from the decision of traffic control policies and make it happen in the simulation. traffic control policies can then learn from how the traffic evolves under certain actions and improve decision making.

---

Under review as a conference paper at ICLR 2023
While existing traffic simulators help hatch various traffic control policies successfully, they still come with drawbacks. Current simulators, as they were designed primitively, support simulation in road networks smaller than one hundred intersections and cannot scale to city-level traffic, which involves thousands of intersections. Due to limits in efficiency and mechanisms, these simulators either not able conduct city-level simulation in a feasible time set to prevent masses of vehicles from coming in the traffic.
Another concern lies in the shortage of input data for large-scale traffic simulation. Although the map data of main cities in the world is now completed roughly and being refined, there is an absence of infrastructure for convenient access the map data and a pipeline to transform it into simulation inputs. Therefore, inputs for traffic simulation only come from manual work and are limited to small set of road networks Wei et al. (2019c by; Zheng al. 2019); Xu et al.) (202T) whose scales are often dozens of intersections (e.g. 4x3 or 4x4) much less than real urban road networks.
To overcome two aforementioned drawbacks, we propose City Brain Lab, novel toolkit for scalable traffic simulation. CBLab consists of three components: microscopic traffic simulator CBEngine, a data tool CBData, and a traffic control policy environment CBScenario. CBEngine of high efficiency which benefits from well-designed parallelization. With ordinary computing hardware, CBEngine is capable of running the traffic simulation on the scale of 10,000 intersections and 100,000 vehicles with real-simulation time ratio of 1:4. CBData includes accessible dataset that contains raw road networks of 100 main cities all around the world. A pipeline is prepared to automatically transform the raw data into input data for traffic simulation. Combining CBEngine and CBData, users can easily start traffic simulation on real city-level road networks. Based on the scalable traffic simulation, we further implement CBScenario as an environment for common traffic control policies: traffic signal control and congestion pricing. Users can design, develop, and train traffic control policies in the framework of CBScenario. To the best of our knowledge, we are the first to provide infrastructure for large-scale traffic control policy optimization.
Our contribution can be summarized as follows.
We develop a scalable traffic simulator CBEngine which supports city-level microscopic traffic simulation for the first time.
We develop data tool CBData to provide input data for large-scale traffic simulation. Based on CBEngine and CBData, we implement an interactive environment CBScenario for two common traffic control policies under a large-scale setting.
2 CBENGINE: CITY-SCALE TRAFFIC SIMULATION ENGINE
In this section, we introduce the design of CBEngine. We demonstrate the modeling of urban traffic and conduct extensive experiments show the efficiency and scalability of CBEngine.
2.1 THE TRAFFIC MODEL OF CBENGINE
The objective of the simulator is to describe the interaction between the road network and the traffic flow (vehicles in the traffic). As shown in Figure 2 the road network involves the interaction through one of its components: the traffic signal lights, which control the passing of vehicles at intersections. When the simulation starts, vehicles in the traffic flow set out from their origins, travel down the routes, and finally arrive at their destinations.
Specifically, traffic signals and vehicles interact as follows. Each traffic signal may change the signal phase (controlling the traffic direction allowed to move) as time changes. Vehicles move on, accelerating or decelerating according to the current speed, the traffic signal the movement of nearby vehicles, and other circumstantial factors. Meanwhile, they may also change their routes accordingly. For each time step, traffic signals, and vehicles observe from their views and make decisions on their next action a (driving and routing). The simulator will then conduct these actions. This forwards the system from state s, time to new state $1+5t at time t + ôt. One such step can be concluded by [ Due to the page limit, concrete modeling of the road network, traffic signals and traffic flows is discussed in Appendix C.
St + + S1+61
(1)
2

---

Under review as a conference paper at ICLR 2023
CBEngine Model
Traflic Signal
Velicle
Roaling
Driving
stop
step
step
action change signal phose from 1o
action change the change acceloration
Figure 2: The Traffic Model of CBEngine
2.2 VEHICLE MODELS: DRIVING AND ROUTING
Behaviors of vehicles are controlled by vehicle models in the traffic simulation. Towards simulating the behaviors of vehicles, researchers have proposed various models Krauß (1998); Yuan et al. (2010) 201T). These models can be summarized into two categories: driving models and routing models. Driving models determine how vehicles move the road while routing models make choices on routes.
Following Cityflow Zhang et al. (2019). widely-used traffic simulator, the default driving model of CBEngine is modification version the model proposed by Stefan Krauß Krauß [998). The key idea of this model is to drive as fast possible subject to safety regularization, with nearby vehicles, traffic signals, and cutting-in considered. Due to the page limit, we discuss the details the driving model in Appendix C.
An important function provided in CBEngine vehicle model customization. In the simulator, both the routing model and the driving model are black boxes with definite inputs and outputs. Hence, we modularize two models as an independent C++ class in our implementation, respectively. Users customize models by only editing the class without modification to other parts of CBEngine. Our documentation in Appendix A provides detailed instructions. To the best of our knowledge, CBEngine the first traffic simulator to support easy-to-use customization of vehicle models.
2.3 EFFICIENCY AND SCALABILITY
The major bottleneck of supporting city-wide traffic control policy training is the limit in efficiency and scalability of existing simulators, of which extensive efforts have been made in CBEngine for improvement. In the following part, we illustrate a design adapted by CBEngine that greatly promotes the efficiency of CBEngine. Then, we conduct extensive experiments to validate the efficiency and scalability of CBEngine. More implementation details CBEngine are discussed in Appendix D.
Parallel Design in Computing the Vehicle Behavior The future state in the simulation deter- mined by the present state and the action. Therefore, computing the action is the main computational job. CBEngine, we carefully design the parallelization process of computing the action for vehicles. The comparison of our design and that of other simulators is shown in Figure 3
The process to compute actions for vehicles can divided into two stages: getting the Status and getting the Action. In the first stage, the simulator collects information for computing the action of vehicle. The set of information is denoted as the Status. In the second stage, the simulator computes the action according Status. Existing simulators parallelize these two stages respectively. Vehicle objects in the simulator need to keep two data members: the Status and the Action. Two members are updated sequentially by two parallelized methods: Get VetVehiclSStatusO and GetVehicleActtinn..
In CBEngine, we assemble two stages in one parallelized method: Compute(). We implement this by adjusting data dependencies and reconstructing the parallel architecture. This design benefits
3

---

Under review as a conference paper at ICLR 2023
Other Simulator(s):
Vehicle
Vehicle
Vehicle
GetVehicle Status)
GetVehicle
Status
Action
Status
Action
Status
A ction
A Attinn)
Vehicle
Vehicle
Vehicle
Thread
Status
Action
Thread
Status
A ction
Vehicle
Thread
Vehicle
Thread
Vehicle
Status
A Ation
Status
Action
Status
A ction
C CBEngine:
Vehicle
Vehicle
Compute(
Action
Action
Thread
Vehicle
GetStatus
Status
GetAction
A Attion
Vehicle
Action
Action
Thread
Vehicle
GetStatus
Status
GetAction
A ctiion
Vehicle
Action
Action
Figure 3: Comparison between CBEngine and other simulators on parallelization design of computing actions for vehicles. The gray data member indicates not updated, while the black one is updated.
V2, V3: three vehicles processed by one thread. The
memory
Compute the status of the vehicle
Compute the action of the vehicle
and V2 Is stored ed in the same page, with that of V in another page.
page to cache
Load Momery
Vi Leed Monsar
Canpute
V2 Load Morsary
V2 Lead Menoey
Computt
V3
uud Momary
V3 Laad Menory
Compote
Compate
tteer Simulator(s) (s)
Cione 07
CBEngine
63
time
Figure 4: A schedule case of computing vehicle action. The length of the bar indicates the time the corresponding task consumes.
efficiency from two perspectives. First, the space cost is lower. We bind the Status data the Thread object rather than on the Vehicle objects. This is because the number of threads is quite smaller than that of vehicles. Second, by merging two stages, the CPU is more likely access existing pages in the memory when getting the action, because those pages are recently loaded to the memory when getting the status. Since this design reduces the number of times that the CPU loads pages from the memory, it decreases the number of cache misses and the time cost.
Figure raises case comparing the scheduling of thread processing vehicles in other simulators to that in CBEngine. For each vehicle, the thread needs to compute its status and then its action accordingly. The page where the vehicle is stored is required be loaded in memory for access to the vehicle. Assume that the cache can only store one page. For other simulators, the processing is divided into two stages, while each stage loads two pages from memory. This because the thread needs to access all three vehicles in each stage. By contrast, processing in CBEngine combines two stages and does not access new vehicles until the job on the vehicle is finished. This design helps reduce the operation of loading pages and saves processing time.
To evaluate the efficiency and scalability of our simulator, we compare it with two widely-used open-source microscopic traffic simulators, SUMO Lopez et al. (2018) and Cityflow Zhang al. (2019). We compare these simulators in three aspects, running time, road network scalability, and traffic flow scalability, All the experiments are conducted on a Ubuntu20.04 system with a 40-core CPU and 128GB RAM. The unit of time cost is second for all three experiments. More details of the experimental setup are given in Appendix B.
Experimental Setup
Experiment 1: Efficiency We run a one-hon hour traffic simulation on urban traffic cases of six cities with distinct scales and compare the time cost of baselines and our simulator. Road networks of
4

---

Under review as a conference paper at ICLR 2023
Dataset
Nanchang
Changchun
JiNan
Shenzhen
Hangzhou
Shanghai
Intersection Num
1506
2228
2314
3427
3434
8474
Vehicle Num
-25000
-50000
-50000
|
-70000
~70000
-120000
Time Cost
1239.93
2091.60
2151.01
3103.58
3199.14
6173.51
SUMO
(+-3.58
(+7.84)
(+70.64)
(+110.08)
(+70.87)
(+75.277
164.08
243.22
242.47
289.31
310.98
664.18
Cityflow
(+,,30)
(11)))
(#21.34)
(+1.744
(+18.22)
(#21.01)
104.39 (+:)5
169.36 (=:1600)
174.29
243.26 (+1.75
245.01
472.07
CBEngine
(=193)
(122116
(+22900
Table Comparison results of efficiency experiments in six real-world cities.
these cases are obtained and cleaned from OpenStreetMap Haklay Weber (008). Traffic flows are generated according the scale of the road network. Results are displayed in Table
We can observe that CBEngine achieves significant improvement simulation efficiency (usually 30%-40% compared with Cityflow and more than 90% compared with SUMO). The stability of CBEngine distinctly better than that of baselines. Furthermore, the gap between baselines and ours grows with the scale of traffic cases, indicating that CBEngine adapt well to large-scale cases.
Road Network Scalability
Traffic Flow Scalability
logiStep Time Cost]+1
2
SUMO
log(Step Time Cost)+1
SUMO
Cityflow
Cityflow
CBEngine
CBEngine
2
3
3
6
Ig(# intersections)
lg(## of vehicles)
Figure 5: Comparison results of two experiments of scalability.
Experiment 2: Scalability on Road Networks Simulators with high scalability on road networks run the simulation efficiently large-scale road networks. To explore the scalability of baselines and CBEngine on road networks, we run a traffic simulation on road networks of different scales. We select five regions from real road networks with {10, 102 10 10a 10..5) intersections. The upper bound is set as 101.5 hecause this is the largest road network for single city in OpenStreetMap. For each setting, we repeat the experiment three times. We report the time cost of single-step simulation for baselines and CBEngine as well as the range.
The results are visualized in Figure 5 (left). CBEngine outperforms two baselines in time on road networks with all scales selected. Take the experiment setup under the road network with the largest scale as a quantitative example. The average single step time cost of CBEngine is 0.2670 seconds, while that of SUMO Lopez et al. (2018) and Cityflow Zhang et al. (2019) are 9.1832 seconds and 1.0343 seconds, respectively.
Experiment 3: Scalability on Traffic Flows With high scalability on traffic flows, the simulator keeps efficient under heavy traffic. Similar to Experiment 2, we conduct experiment to evaluate the scalability of traffic flows of baselines and CBEngine. We generate five traffic flows with {10?, 103. 10 103, 108} on on-way vehicles. For each setting, repeat the experiment three times. We report the time cost of a single step for baselines and our simulator as well as the range.
The results visualized in Figure 5 (right). CBEngine outperforms two baselines under different scales of traffic flows. To give a quantitative example, the average single step time cost of CBEngine under the traffic flow of 1 vehicles is 0.2610 seconds, while that of SUMO Lopez et al. 2018 and
5

---

Under review as a conference paper at ICLR 2023
User Query
Cily
Map
Raw Graphh
Rechand Graph
Road Network
CRFlggne
Delact
Preprcuso
Flow Generator
QgarSStrorhPapp
Trafflic Flow
Figure 6: The simulation input data supporting pipeline CBData.
Cityflow Zhang et al. 2019)) are 8.9111 seconds and 1.1058 seconds, respectively. Specifically, two baselines are not able run the case with 1,000,000 vehicles. We discuss the reason in Appendix D.
3 CBDATA: TRAFFIC DATA NETWORK CONNECTED TO CBENGINE
In this section, introduce data tool CBData. CBData serves to provide enriched input data supporting large-scale traffic simulation. The support is achieved by a dataset with raw road networks of 100 main cities all around the world and the transformation pipeline shown in Figure Moreover, CBData includes two other pipelines that help the simulator learn from other traffic data.
3.1 PIPELINE: SIMULATION INPUT DATA SUPPORTING
Despite existing traffic simulators being capable of simulating the evolution of urban traffic, the application traffic simulation is vastly limited by the shortage of input data. Specifically, to start up simulation, the simulator takes road networks and traffic flows as input data. At present, there is no convenient access to these two kinds of data, although road networks of almost all main cities in the world can be extracted from open source map data Haklay Weber (200).
To disentangle this problem, we implement a pipeline to bridge open source map data and input data for simulation. This pipeline provides one-click service offer enriched input data for large-scale traffic simulation, solving the shortage of input data.
As shown in Figure the pipeline consists of dataset, preprocessor, and a flow generator.
We obtain the map data of the whole world from OpenStreetMap Haklay & Weber (2008). We extract the road network data of 100 main cities and store the data our dataset. The dataset now available on Google Drive (See Appendix A). Users can directly download the data and pick up their interested road networks. Details of the dataset are given in Appendix E.
Dataset:
Preprocessor:
The preprocessor first constructs the road network as raw graph by matching and connecting edges and nodes. The raw graph is then cleared to remove the redundant nodes and graphs. This is necessary because redundancy is common in open-source map data. After removing redundancy, reduced graph transformed into road network in the standard format.
The flow generator generates the traffic flow for road network. Given the total number of vehicles, the generator assigns origins and destinations for these vehicles, respectively, which distribute averagely as possible. The default route for each pair of the origin and the destination is the shortest path. However, the route can be changed by the routing model of the traffic simulator when running the simulation.
Flow Generator:
3.2 PIPELINE: LEARNING TO SIMULATE FROM TRAFFIC DATA
In addition to map data, there a lot of other traffic data with the potential to enhance the plausibility of traffic simulation. In CBData, propose two paradigmatic pipelines to illustrate how to leam to simulate from traffic data. Note that we are not to propose effective methods but provide a paradigm for learning simulate.

---

Under review as a conference paper at ICLR 2023
3.2.1 LEARNING TO SIMULATE DRIVING
The driving model determines how drivers accelerate and decelerate according their observation of the circumstance, The driving behavior in different traffic or different cities can be distinctly different. Therefore, learning the driving parameters of the traffic simulator from the traffic data is sound for traffic simulation. It helps the simulator to behave more plausibly like the local drivers.
The goal of learning to drive is find set of driving model parameters that can minimize the gap between the traffic data and the simulator, e.g. that between the observed speed in the real data and the observed speed in the simulator, with the same traffic flow, As mentioned Section 2.1] the driving model of CBEngine is easy modify. Here, adopt the default model (See Appendix for more details) and select three parameters from the model as the parameters to be optimized: acceleration maximum, deceleration maximum, and speed limit.
We use black-bon optimization toolkit OpenBox Li et al. (202T) as the optimization tool. Note that users
any other optimization toolkit according to their needs. OpenBox searches for parameters to fill the gap between simulation observation and the ground truth. The 1-hour GPS trajectory data of vehicles two roads in Shenzhen, China used search the parameters. The observation interval is 1 minute. For every 20 minutes, the traffic distribution will change. We expect that acceleration parameters can be continuously learned when the data changes.
The loss curves of the learning process of these two avenues are displayed in Figure 7 The gap between the observed speed average and the ground truth is decreasing time changes. After the traffic changes at the 20th and the 40th minute, the driving model can not fit the new traffic data. Hence, we observe a high loss immediately. After a few minutes, the gap continuously decreases. Note that, the difference still maintains a certain positive value. This implies the potential to change the driving module to pursue a better performance of driving module correction.
Leaming Procoss, Xinyu Road
Learning Process. Zhanggur Road
Data injacton
New Data Ivectton
155 (000e
Lone values
Time step
Time step
Figure 7: Loss curves during learning the driving module on two roads.
3.2.2 LEARNING TO SIMULATE ROUTING
The routing model determines how vehicles route themselves, given the origin and destination. A data-dependent routing model can be formulated as a route generator, which generates route for certain origin and destination based on real trajectories. However, the studies in this field are limited and we exploit the Recurrent Neural Networks (RNNs) as our route generator Choi et al. (202T).
We use part of vehicle trajectory dataset in Shenzhen, China. This dataset contains 22 different routes in total. We train the RNN (Noii et al. (222I) to Iearn the distribution of routes and conduct routing for vehicles in the simulator. Figure 8 shows the result of loss curves and two trajectory generating metrics, BLEU PPapineni et al.] (2002) and METEOR Banerjee & Lavie (2005). The loss converges to 0 at the first twenty iterations, while BLEU and METEOR get close This indicates that at this stage, routes generated overlap at least one route in the real trajectory data.
In addition, the routing result visualized Figure 9 Compared with an untrained generator, the trained generator recognizes the main distribution pattern of real trajectories. Meanwhile, it tends to ignore some infrequently-appeeee routes.
CBSCENARIO: ENVIRONMENT FOR LARGE-SCALE TRAFFIC CONTROL POLICIES
In this section, we introduce CBScenario, an interactive environment for large-scale traffic control policies. CBScenario benefits from the large-scale traffic simulation supported by CBEngine and
7

---

Under review as a conference paper at ICLR 2023
0.8
Lo55
4-
Bleu
0.6
Metnor
0.8
2-
0.4
0.2
0.4
20
Iteration
Iteration
Iteration
Figure 8: Curves of loss, BLEU, and METEOR during learning the routing module.
Original Data
Generted Deta (Untrainol Mockl)
Gemerted Data (Toined Modks)
Figure 9: Visualization of the routing result. The deeper red is, the more frequent the route is picked.
CBData and capable of training traffic control policies for city-level traffic. Concretely, CBScenario includes benchmarks for two traffic control policies: "Traffic Signal Control" and "Congestion Pricing". We conduct experiments on our environment show the plausibility of the traffic simulation.
Traffic Signal Control
Congestion Pricing
CBEngine
Reward
CBEngine
Reward
Yelidee
Tike
Threphost
Cloose
State
Inpel
State
Agpet
the plaer
Throplpe
Valids
Set prin
Yeaide
Tree
Interface
Teali
Trattc
A cllon
Action
$1.00
Prisr
Oxtyrn
Clange De
Pric
Figure 10: Illustration of two scenarios: traffic signal control and congestion pricing.
4.1 TRAFFIC CONTROL POLICY 1: TRAFFIC SIGNAL CONTROL
The traffic signal control problem Wei et al. (0199c tries to improve the performance of urban traffic by carefully choosing the phase of traffic signals at intersections. An ideal traffic signal control policy can capture the global and local traffic dynamics and allocate more passing time to the phase with higher traffic pressure. Figure (left) shows the problem setting of traffic signal control.
In consideration the Markov nature traffic signal control, the traffic signal control can be formulated Markov Decision Process (MDP):
State: Intersection-level and road-level observation and statistics of observation, e.g. the number of waiting vehicles on the road, historical average vehicle throughput of different phases at the intersection.
Action: Decide which directions pass.
Reward: Metrics measuring the performance of urban traffic. We provide two widely-used metrics: the total number waiting vehicles at the intersections and the average waiting time during action interval.
We implement several baseline algorithms to justify the plausibility of our simulation. We consider two metrics to evaluate the performance of these algorithms: arriving vehicle throughput and average travel time of vehicles. More details of the experiment given in Appendix B. The results
8

---

Under review as a conference paper at ICLR 2023
shown in Table Two transportation methods, MaxPressure Varaiya
(003)) and Self Organized
Traffic Light (SOTL) Cools et al. (2013), show a degree of advantages in inrrassing throughput and reducing travel time. However, learning based traffic control policy, DQN Mnih et al. (2015);
Wei et al. 2018) performs even better.
Dataset
Hangzhou
Manhattan
Metrics
Throughput
Travel Time(s)
Throughput
Travel Time(s)
FixedTime
2184
1478.01
2894
1309.88
MaxPressure
3336
700.66
3364
805.
SOTL
1122
305.79
137
488.62
DQN
3573
309.10
3926
375.51
Table 2: Performance of baseline algorithms on traffic signal control.
4.2 TRAFFIC CONTROL POLICY 2: CONGESTION PRICING
Congestion pricing reroutes vehicles by dynamically assigning prices to different routes and guiding vehicles to drive on the route of the lowest price. Good pricing methods tend to allocate heavy traffic on different routes with a trade-off of the distance and the capacity, therefore enhancing traffic efficiency. Figure TO (right) shows the setting of congestion pricing.
Similar to traffic signal control, we can define congestion pricing as an MDP:
State: Road-level observation and statistics of observation. For the observation common used, we have the vehicle number and the average speed of vehicles on the road.
Action: Price roads in the road network.
Reward: Metries measuring the performance of urban traffic. A common reward is the average travel distance of vehicles during an action interval.
Congestion pricing has been researched in the transportation field for while yet few studies data-driven methods. We implement two transportation-bae based methods, Random and Deltatoll al. (2017), and an RL algorithm, EBGtoll Qiu et al. (2019). More details of the experiment given in Appendix B. Results are shown in Table 3 Random outperforms No-change which keeps original routes for vehicles. This is plausible because random rerouting allocates traffic averagely on all available routes. Deltatoll and EBGtoll behave similarly and outperform Random in both evaluation metries.
Dataset
Hangzhou
Manhattan
Metrics
Throughput
Travel Time(s)
Throughput
Travel Time(s)
No- Do-Change
2176
1455
2911
1328
Random
3008
644.00
3459
1120.69
Deltatoll
3186
604.00
3670
960.17
EBGtoll
2803
310.18
3494
1019.85
Table 3: Performance of baseline algorithms on congestion pricing.
5 CONCLUSION
In this paper, we present CBLab, a toolkit for scalable traffic simulation. CBLab provides the first simulator to support real-time simulation of large-scale cities with more than 10,000 intersections. data tool is implemented to supply large-scale input data for the simulation. An interactive environment and benchmarks for two common traffic policies have been built and prove that the simulation is plausible. To the best of our knowledge, CBLab is the first infrastructure supporting the training of large-scale traffic policies.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved
correlation with human judgments. In IEEvaluation ACL., pp. 65-72, 2005. URL https
/aaclantooooge http:/hooogo...ooo/ooooooo wwwiiioooppppree 0000:00000000000 9999//00 0909
Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng. Ming Yang. Yuanhao Xiong, Kai Xu, and Zhenhui Li. Toward thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 2020a.
Hao Chen, Ke Yang, Stefano Giovanni Rizzo, Giovanna Vantini, Phillip Taylor, Xiaosong Ma, and Sanjay Chawla. Qarsumo: A parallel, lungestion-optimizze traffic simulator. In Proceedings of the 28th International Conference on Advances in Geographic Information Systems, pp. 578-588, 2020b.
Seongjin Choi, Jiwon Kim, and Hwasoo Yeo. Trajgail: Generating urban vehicle trajectories using generative adversarial imitation learning. Transportation Research Part C: Emerging Technologies, 128: 103091, 2021.
Seung-Bae Cools, Carlos Gershenson, and Bart D'Hooghe. Self-organizing traffic lights: A realistic simulation. In Advances in applied self-organizing systems, pp. 45-55. Springer, 2013.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference machine learning, pp. 1861-1870. PMLR, 2018.
Mordechai Haklay and Patrick Weber. Openstreetmap: User-generated street maps. IEEE Pervasive computing, 7(4):12-18, 2008.
Stefan KrauB. Microscopic modeling of traffic flow: 1998.
Investigation of collision free vehicle dynamics.
Yang Li, Yu Shen, Wentao Zhang, Yuanwei Chen, Huaijun Jiang, Mingchao Liu, Jiawei Jiang, Jinyang Gao, Wentao Wu, Zhi Yang, al. Openbox: A generalized black-box optimization service. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2209-3219, 2021.
Pablo Alvarez Lopez, Michael Behrisch, Laura Bicker Walz, Jakob Erdmann, Yun-Pang Flötteröd, Robert Hilbrich, Leonhard Lücken, Johannes Rummel, Peter Wagner, and Evamarie Wießner. Mi- croscopic traffic simulation using sumo. In The 2 IEEE International Conference on Intelligent Transportation Systems. IEEE, 2018. URL BtRLbttps://ee
de/124092/
Hamid Mirzaei, Guni Sharon, Stephen Boyles, Tony Givargis, and Peter Stone. Enhanced delta- tolling: Traffic optimization via policy gradient reinforcement learning. 2018 2 International Conference Intelligent Transportation Systems (ITSC), pp. 47-52. IEEE, 2018.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 51 5187754)::29-5333 2015.
Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. Attendlight: Universal attention-based reinforcement learning model for traffic signal control. Advances in Neural Information Processing Systems, 33:4079 4090, 2020.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL, pp. 311-318, 2002. URL URLhttp: httt:://ttp..ct wwwww
org/ oxt/aanthccooggy/22--OOoo pdf
Wei Qiu, Haipeng Chen, and Bo An. Dynamic electronic toll collection via multi-agent deep reinforcement learning with adge-based graph convolutional networks. In IJCAI, pp. 4568 -574, 2019.
10

---

Under review as a conference paper at ICLR 2023
Guni Sharon, Michael W Levin, Josiah P Hanna, Tarun Rambha, Stephen D Boyles, and Peter Stone. wide adaptive tolling for connected and automated vehicles. Transportation Research Part C: Emerging Technologies, 84:142-157, 2017.
Pravin Varaiya. Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies, 36: 177-19,, 2013.
Hua Wei, Guanjje Zheng, Huaxiu Yao, and Zhenhui Li. Intellilight: A reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496-2508 2018.
Hua Wei, Chacha Chen, Guanjie Zheng, Kan Wu, Vikash Gayah, Kai Xu, and Zhenhui Li. Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1290-1298, 2019a.
Hua Wei, Nan Xu, Huichu Zhang, Guanjie Zheng, Xinshi Zang, Chacha Chen, Weinan Zhang, Yanmin Zhu, Kai Xu, and Zhenhui Li. Colight: Learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1913-1922, 2019b.
Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. A survey on traffic signal control methods. arXiv preprint arXiv: I 1904.094.11110 17, 2019c.
Libing Wu, Min Wang, Dan Wu, and Jia Wu. Dynstgat: Dynamic spatial-temporal graph attention network for traffic signal control. In Proceedings of the 30th ACM International Conference Information & Knowledge Management, pp. 1150-2159, 2021.
Bingyu Xu, Yaowei Wang, Zhaozhi Wang, Huizhu Jia, and Zongqing Lu. Hierarchically and cooperatively learning traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 669-677, 2021.
Jing Yuan, Yu Zheng, Chengyang Zhang, Wenlei Xie, Xing Xie, Guangzhong Sun, and Yan Huang. T-drive: driving directions based on taxi trajectories. In Proceedings of the 18th SIGSPATIAL International conference on advances in geographic information systems, pp. 99-108, 2010.
Jing Yuan, Yu Zheng, Xing Xie, and Guangzhong Sun. Driving with knowledge from the physical world. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 316-324, 2011.
Xinshi Zang, Huaxiu Yao, Guanjie Zheng, Nan Xu, Kai Xu, and Zhenhui Li. Metalight: Value-based meta-reinforcemmnt learning for traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 1153-1160, 2020.
Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The world wide web conference, pp. 3620-3624, 2019.
Huichu Zhang, Chang Liu, Weinan Zhang, Guanjie Zheng, and Yong Yu. Generalight: Improving environment generalization of traffic signal control via meta reinforcement learning. In Proceedings of the 29th ACM International Conference Information Knowledge Management, pp. 1783- 1792, 2020.
Liang Zhang, Qiang Wu, Jun Shen, Linyuan LU, Bo Du, and Jianqing Wu. Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control. In International Conference on Machine Learning, pp. 26645 -2654.. PMLR, 2022.
Guanjie Zheng, Yuanhao Xiong, Xinshi Zang, Jie Feng, Hua Wei, Huichu Zhang, Yong Li, Kai Xu, and Zhenhui Li. Learning phase competition for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1963-1972, 2019.
11

---

Under review as a conference paper at ICLR 2023
A
KEY INFORMATION OF CBLAB
A.1
LICENSING
CBLab uses the MIT license.
A.2
CODE
The code of CBLab is available on GitHub.
A.3
DOCUMENTATION
The documentation of CBLab is available.
attps://eBaab-dccmmetaaaa readthedocs recthhoossssscaccccccooooooo.. htwph//hdooo..ooo/aaaeeeeeeeeeee
A.4
ROAD NETWORK DATASET
The road network dataset is available on Google Drive:
A.5 LIMITATIONS & FUTURE WORKS
CBLab aims to provide efficient and data-driven traffic simulation for research in traffic policies. Although provide interfaces for using data to optimize the simulator, cannot obtain all real traffic data from different cities to conduct complete optimization on our simulator. We will invite future contributors to make to optimize the simulator and run the simulation.
traffic data compatible with these interfaces that they can be used
scenarios for traffic policies, e-g- traffic restriction. Also, aim to include more powerful algorithms in these scenarios, e.g. Intellilight and FRAP for traffic signal control. It is our hope that CBLab can serve an online benchmark for various scenarios with state-of-the-arr algorithms in addition to supporting powerful traffic simulation.
Moreover, plan implement
A.6 POTENTIAL NEGATIVE SOCIAL IMPACTS
All traffic simulators will suffer from the gap between simulated and simulated observations. This gap may lead to biased observations and traffic policies based on the simulation. CBLab releases the first traffic simulator open for users to optimize so that tends to perform as the real traffic. However, the traffic patterns from different places are different. It is worth considering how to ensure the reality of our traffic simulation when simulating the traffic from different places. Hence, we are collecting more traffic data and studying the common patterns among the data, with the hope of trying to resolve this problem.
B DETAILS OF EXPERIMENTAL SETUP
B. EFFICIENCY AND SCALABILITY (SECTION 2.3)
Baselines Setup In our experiment, use the default setting of SUMO (TRACI) and Cityflow. Note that our simulator a microscopic one. Therefore, we select two open source microscopic traffic simulators not relevant if routing parallelization is used. In the experiment presented in our paper, internal links between intersections are used.
baselines. The routing model is not activated for all three simulators is
Considering that CBEngine simplifies links between intersections, a question may raised whether the internal link has decisively negative impact the performance of the baseline. To answer this question, we conduct new efficiency experiments on SUMO with no internal links and the same setting otherwise. The comparison result is demonstrated in Table
12

---

Under review as a conference paper at ICLR 2023
Dataset
Nanchang
Changchun
JiNan
Shenzhen
Hangzhou
Shanghai
Time Cost
SUMO (with internal links)
1239.93
2091.60 (#7.84)
2151.01
3103.58
3199, 14
6173.51
(#3.58)
(#70.64)
(#110.08)
(#70.87)
(#75.27)
SUMO (no internal links)
1218.53
1987.12
2046.63
2973.50
3020.16
6083.37
Table 4: Comparison results of efficiency experiments between two setups of SUMO.
According to the result, removing internal links for SUMO does improve the simulation efficiency. However, SUMO's efficiency still not considerable compared with that of CBEngine. This is because CBEngine deploys other optimization (e.g. an optimized parallelization architecture) to improve efficiency.
Datasets The efficiency experiment is conducted on the traffic data from six cities of different scales: Nanchang. Changchun, Jinan, Shenzhen, Hangzhou, and Shanghai. We obtain the road network data from our dataset of CBData. Traffic flows are generated according to the scale of the road network. Two scalability experiments are conducted on real-world road networks from our dataset with scales close to the selected road network size ((10, 102, 10 102. 101.5) intersections). Traffic flows are generated according to the scale of the road network. All road networks and traffic flows are available in our code provided on Google Drive.
r?usp=sharing
We also provide a reproducing instruction to help reproduce our experrmmeaallli results.
https://gtthb..o
Reproduciigg__sstrrree
md
Computing Resources All the experiments are conducted on a Ubuntu20.06 system with 40-core CPU and 128GB RAM.
The number of threads is chosen as 20 to stay consistent with the number of used cores. Note that, using fewer more threads will lead to worse efficiency for both Cityflow and CBEnninee
Hyperparameters
Error Bars Error bars of the experiment are shown in Table 1 and Figure 3.
B.2 LEARNING FROM REAL TRAFFIC DATA (SECTION 3.2)
In these experiments, we aim to provide demonstrations show the possibility to use real-world traffic data optimize the traffic simulator. Hence, there might be further room for improvement if the parameters are tuned carefully.
B.2.1 LEARNING TO SIMULATE DRIVING
Datasets The road network of Shenzhen is obtained from dataset. We obtain the traffic flow (as the input) and the observation of speed (as the ground truth) from the GPS trajectory data of cars in Shenzhen, China for day. The data covers 123,481 trajectories and comes from personal data providers with consensus. We are working on releasing a processed version by removing sensitive data.
Optimization Details
We use OpenBox as a toolkit to search for parameters. The code can be com/ PW/./PUU-AAAAAAA www.:///pppbbb...o qit under the MIT license. The
found at /ttttts:///iiattae start-up hyperparameters are as follows. For the maximum acceleration and deceleration. we set the default value (value where the search starts) 2.0m/s² and 5.0m/s², respectively. For the speed
13

---

Under review as a conference paper at ICLR 2023
limit, we set the default value 11.1m/s. The number of rounds is set 20. We use the surrogate type auto and optimizer type auto.
B.2.2 LEARNING TO SIMULATE ROUTING
Datasets We demonstrate how learn the routing module trajectories data of Shenzhen,
China for one day. We collect the trajectories with the origin of {Latitude: 22.5405 N, Longitude: 113.967 }} and the destination of (Latitude: 22. 22.6164 N, Longitude: 113.853' E}. The origin is the bus station of the Window of the World, famous scenic spot in Shenzhen. The destination is bus station on the highway from Shenzhen Guangzhou. This rrgginddetinationn pair aggregates the most number of different routes in our dataset, while routes of other origin-destination pairs are quite unified. The total number of different routes is 22 and that of trajectories is 118.
Optimization Details We use an RNN- model as the trajectory generator. The code can be found https:://g htt///tttbbbbbb com/ www.bheinnnhheeeeeee Tra jGAIL. git under MIT license. We follow the setting of hyperparameters in the original paper Choi et al. (2021) except for the iteration number since our trajectories are complicated for the generator learn from. We set the iteration number 100.
B.3 TRAFFIC SIGNAL CONTROL AND CONGESTION PRICING (SECTION 4.2 AND 4.3)
The goal of these two experiments to provide possible benchmarks for algorithms. Here, we only use several typical algorithms to validate these scenarios. Providing comprehensive baseline methods comparison is out of the scope here. Hence, people are welcome to provide more advanced algorithms for these scenarios.
Datasets We use two real-world datasets to validate CBScenario: Hangzhou and Manhattan. Both datasets are transformed from the traffic data u/https://traffi gna - coootroo) github. ic/ open datasets which serves as a widely used benchmark for traffic signal control. We use part of CBData to transform them to the format suitble for CBEngine. The goal of the experiment is to validate the plausibility of our traffic simulation. Therefore, we refer to the widely used benchmark rather than picking up novel cases not being evaluated yet.
Hyperparameters of Traffic Signal Control The traffic in one episode lasts for 1800 seconds. The DQN method is trained for 50 episodes and the batch size is set as 64. Other hyperparameters are listed in T
Hyperparameter Memory size
Value network updating interval
?
Value
5000
0.9
0.95
Hyperparameter Learning rate
Target network updating interval Emin
Decay of Emin
Value
0.005
20
0.2
0.995
Table 5: Hyperparameters of DQN in traffic signal control.
Hyperparameters of Congestion Pricing The traffic in one episode lasts for 10800 seconds. Actions are taken every 540 seconds. We use the fixed time policy the default traffic signal control methods, we evaluate them in one episode. For the training of
policy. For transportation-baal EBGtoll, the number of episodes is 200 and the batch size is set as 32. Other hyperparameters are listed T
C TRAFFIC MODELING OF CBENGINE
C.I ROAD NETWORK
In CBEngine, Road Network is the topological network where vehicles drives. It consists of two components: roads and intersections. Road models the road segment in the real-world road network.
14

---

Under review as a conference paper at ICLR 2023
Hyperparameter Memory size
Value network updating interval
Policy learning rate
Value
2000
0.001
Hyperparameter
T
Target network updating interval
Critic learning rate
Value
0.125
10
0.0005
Table 6: Hyperparameters of EBGtoll in congestion pricing.
A road may include multiple lanes. Each lane holds one or more vehicles. Intersection is the nexus of different roads. Via lane links in the intersection, lanes of different roads connect to each other. Traffic signal light is another key element in the intersection, which assigns true-or-false signal for each lane link. Vehicles can only move from one road another through available lane links with true signal.
C.I.1 MODEL DESIGN
The default driving model of CBEngine a modified version the driving model used in Cityflow, originating from the driving model proposed Stefan Krauß. The key idea is that: the vehicle will drive as fast as possible subject to safety regularization. Specifically, the maximum speed of the vehicle is subject to several static or dynamic speed constraints. Vehicles will accelerate or decelerate to the speed with the maximum acceleration or deceleration. In the implementation, the maximum acceleration (deceleration) is hyperparaaettrr and is open to users to tune with API. The considered speed constraints are listed and discussed respectively as follows:
Road speed limit
Collision-free following & leading speed
Cutting-in collision-free speed
Traffic-signal safe speed
Road speed limit Each road has its own speed limit. This is static speed constraint.
Collision-free following & leading speed To avoid collisions, vehicles need to adapt their speed to the speed of their following and leading vehicles. We use the collision-free following speed to model the max speed constrained by the leading vehicle. Following the driving model in Cityflow, compute these two constraints with Eq It takes current speed of the vehicle, Ur current speed of the leading vehicle, d maximum deceleration of the vehicle, d,, maximum deceleration of the leading vehicle, /) the current distance between two vehicles, inderval the length of each time step as parameters to compute the collision-free following speed 8efs-
1
2.d interval
2
(2)
v interval
07
D
2
2. 2.d
bt vb
4ac
Sefs
2
To compute the collision-free leading speed 8ats, we use a mirror equation of Eq 2 We use the dr the maximum deceleration of the following vehicle and Up the speed of the following vehicle to replace dp and er in the Eq 2 The collision- -free following and leading speed constraints can be summarized
as Eq 3
Sela 5 v S Sefe
15

---

Under review as a conference paper at ICLR 2023
Cutting-in collision-free speed CBEngine supports self-adaptive lane changing within the road (Cutting-in). This asks for cutting-in collision-fres following and leading speed, avoiding collisions with the leading and following vehicles in the target lane. We compute this constraint using Eq 2 with wz, dz, and D given by the leading vehicle in the target lane and vy, dp, and D given by the following vehicle in the target lane.
An exception happens when the vehicle needs to conduct an emergent cutting-in. This takes place when the vehicle very close the intersection but still in the wrong lane (e.g. The vehicle in the go-straight lane but needs to turn left). On this occasion, the vehicle tries to stop to wait until is able to change the lane. Therefore, the maximum speed constraint equals zero and the vehicle will decelerate to zero with its maximum deceleration.
Tranfl safe speed Vehicles heading for an intersection are subject to two constraints determined by the traffic signal. First, the current speed can be decelerated zero within the remained passing time of the traffic signal. Second, the driving distance cannot exceed the distance to intersection, under the decelerating process defined in the first constraint.
C.1.2 EFFECTIVENESS
Although CBEngine provides convenient driving model customization, which decreases the impact of the default driving model, conduct two experiments evaluate its effectiveness. First, compare the average speed of vehicles in CBEngine and SUMO under the same simulation setups over 600 seconds. The experiments are finished on three of the experimental setups used in our efficiency experiments: Nanchang, Changchun, and JiNan. The result is visualized in Figure [T Overall tendencies of two average speeds fit each other approximately, while that CBEngine more volatile. This can be explained by the difference in the driving model and will not have visible impacts on traffic flow statistics, which the main factor in learning traffic control policies.
Comparison of average speed (left to right: Nanchang Changchun (iNan)
CBEnginee
CBEngine
CBEngine
SUMO
SUMO
SUMO
time
Figure 11: Comparison of the average speed of vehicles, CBEngine and SUMO
The second experiment focuses on the speed distributions of vehicles in CBEngine and SUMO. We compare the speed distributions in 150 seconds, 300 seconds, and 450 seconds under the experimental setup Changchun used in our efficiency experiments. The comparison visualized in Figure [2] Normalized Wasserstein distance between two distributions is shown in Table 7 The two distributions are similar to each other roughly but differ in details. The difference may not influence the overall performance of the simulated traffic in the level of traffic statistics. Also, the driving model customization supported by CBEngine makes it possible for users to use driving models according to their needs, which greatly improves the effectiveness of CBEngine, compared to SUMO and Cityflow.
Time Step
150
300
450
Normalized Wasserstein Distance
0.0141
0.0124
0.0089
Table 7: Normalized Wasserstein distance between speed distribution of CBEngine and SUMO
C.2 APIS FOR TRAFFIC CONTROL
CBEngine provides various APIs for users to develop novel traffic control policies. The functions supported by these APIs are listed follows:
16

---

Under review as a conference paper at ICLR 2023
Speed distribution vehicles, 150 step, Changchun
CBEngine
SUMO
010
the percentage of vehiclos
02-
02-
01-
speed
speed
Speed distribution of vehicles, 300 step, Changchun
CDCngine
SUMO
the percentage of vehicles
00
67
66-
96-
05
64-
speed
speed
Speed distribution vehicles, 450 step, Changchun
CDCngine
SUMO
1.0
the percentage of vehicles
03
03-
63
0.1
speed
speed
Figure 12: Comparison of speed distributions of vehicles, CBEngine and SUMO
Changing the phase traffic signal light
Modifying the speed limit of road
Changing the route of vehicle
D IMPLEMENTATION DETAILS OF CBENGINE
Except for the parallel design mentioned in Section 2.3, two mechanisms of CBEnngnee also contribute to its efficiency.
Lane Changing in Driving Model Lane changing is driving action. Drivers may change the lane they feel the current lane is too congested. However, lane changing is hard to simulate in the traffic simulation. Cityflow, of our baselines, omits all lane changing except those happening the
17

---

Under review as a conference paper at ICLR 2023
intersection. The lane of the vehiele is determined by the direction it will turn. This simplifies the implementation but leads to poor plausibility because vehicles on the road cannot make full use of all lanes they do in real urban traffic. They have to stay the current lane and may wait a long time, although their neighbor lane clear. Furthermore, to avoid collision on this occasion, Cityflow does not allow new vehicles to come in until the lane is relatively unblocked. This limitation severely impacts the scalability of Cityflow and explains the fact in our experiment of scalability that Cityflow cannot hold 1,000,000 vehicles.
In CBEngine, we implement a driving model allowing lane changing. Vehicles are put in a random lane when they get into the road. They will try lane changing according to the direction they are to turn to. But if that lane is in congestion, they will keep going in the current lane which is relatively clear. This design achieves higher plausibility and provides stronger scalability for CBEngine.
Intersection Links Another key mechanism of the traffic simulator is the intersection link. SUMO and Cityflow track the behavior of vehicles inside the intersection. However, due to the limit in the implementation, this design may lead to deadlocks very frequently in the practice, especially when running large-scale traffic simulations. This because the track of some vehicles may block that of others. Therefore, conduct simplification here avoid such deadlocks. When vehicle passes the intersection, the intersection will hold it for while and then send it to the target road. We believe that the effect of the intersection link can be simulated by the holding time. We also discuss the impact of intersection links on our baselines in Appendix B. To the best of our knowledge, our design avoids all such deadlocks in practice.
E DETAILS OF ROAD NETWORK DATASET
The road network dataset in CBData includes raw road networks of 100 main cities around the world. The list and the range of these road networks are given on Github:
https://gi thub. CommCCityrrra CeCCttyyraanaaCCCCCc.c Salaab/CityBi wtw.a///llb/aann///oo
citylist..... CSv We obtain the data from OpenStreetMap, an open-source map database. Note that bounding boxes these road networks have to be a rectangle thus not strictly consistent with the boundary of the city. These road networks can be transformed into road network inputs for traffic simulation with our pipeline in CBData. Here, we visualize six road networks used in our experiment for example to give scratch of our road network data in Figure B3
18

---

Under review as a conference paper at ICLR 2023
Figure 13: Visualization of road networks of six cities used in our experiment: Nanchang (top left), Changehun (top right), JiNan (middle left), Shenzhen (middle right), Hangzhou (bottom left), Shanghai (bottom right).
19


Reviews:
[{"review_id": "e_rlc61VbT0", "reviewer": "Reviewer_JNCH", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work is based on two concerns: 1. Current simulators cannot handle city-level simulation with thousands of intersections; 2. Current map data cannot be easily accessed and transformed into simulation inputs. In CBEngine, the authors design a paralleled module to improve computing efficiency. The experiments show that CBEngine has achieved significant progress compared to SOTA simulators. In CBData, the authors construct a pipeline to treat map data, and introduce new modules to adjust simulation parameters via traffic data. At last, two experiments are shown to provide the plausibility of the traffic simulation.", "summary_of_the_review": "This work is interesting and meaningful, the writing is good and the overall paper is well-organized. I believe it can be helpful to both cs and transportation community to verify models or policies in a large scale.\n\n", "strength_and_weaknesses": "This work proposes a novel simulator for urban transportation. The writing and logic are good. Some minor issues:\n-\tSome typos can be found in the manuscript, e.g. page 4: load -> loaded\n-\tFor CBScenario, I wonder if the authors conduct traffic light control and congestion pricing with a singe agent RL or multi-agent RL, and if the agent merely controls a single intersection or road?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "S0cULFpeskL", "reviewer": "Reviewer_VNgb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The article describes the development of a traffic simulation and analysis framework named CBLab. Experiments are executed to show the effectiveness of the framework. The framework is said to perform better than the state-of-the-art. The framework provides all the tools needed for analyzing the traffic of a large city, including a simulation engine, a data engine, and a scenario builder. There are also capabilities provided that enable a user to introduce custom traffic and driver behaviors.  Although the work is much needed, but there are almost no fundamental improvements within the theoretical domain. ", "summary_of_the_review": "The paper assists the researchers in conducting research on traffic analysis using simulation. The paper is also well-written, and the results are explained nicely. Apart from one contradiction that is listed above, I could not find any discrepancies. \n\nNevertheless, the simulation of parallel and distributed systems is a large domain in itself, and the paper seems not to address that topic at all. There is no discussion on the type of model of simulation they have used, for example, event-driven or fixed time step, although one can imagine that they must have used event-driven simulation. Then comes the questions of parallelism, how they have managed parallel events, and whether there was a single event queue or multiple event queues for disjoint sets of simulation. All these fundamental questions are unanswered.", "strength_and_weaknesses": "\nThe paper is well-written and organized in a manner that is easy to grasp. A thorough explanation of the experiments is provided. The graphs are used to show the results that are explained thoroughly. The paper is well structured and well organized. \n\nDespite its importance, the paper seems to be an account of a development project that ended well. The experiments show the success of the framework and its interfaces, but there is not much fundamental scientific contribution. The framework will assist the scientists and will surely spur the research in the traffic analysis domain, but does the paper offer a new paradigm or a new algorithm? I am afraid the answer is no. The only contribution that can be associated with a conceptual level is the change in the architecture of the simulation engine that decreases page faults. \n\nEven that is not significant as a reduction in page faults will only affect when the computer memory is low or the size of the simulation is very large. For all other cases, the change in architecture does not make much of a difference. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "B6NQmDY0hi", "reviewer": "Reviewer_4dnh", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper design a useful tool for scalable traffic simulation, which would benefit RL training. This system is very comprehensive, and large-scale including three modules,  CBEngine, CBData and CBScenario. ", "summary_of_the_review": "It seems to be a good tool that may facilitate more research. But I have some concerns as mentioned in weakness part. ", "strength_and_weaknesses": "### Strength:\nLarge-scale traffic simulation is a crucial topic and designing such a tool is useful. \n\n### Weakness:\n1. This paper lacks some key related works, such as:\nChen, H., Yang, K., Rizzo, S. G., Vantini, G., Taylor, P., Ma, X., & Chawla, S. (2020). QarSUMO: A Parallel, Congestion-optimized Traffic Simulator. GIS: Proceedings of the ACM International Symposium on Advances in Geographic Information Systems, 578–588. https://doi.org/10.1145/3397536.3422274\n2. In Figure 3, why do other models divide the process into two: status and action?\n3. How to ensure the authenticity of the road network (the paper didn't mention, such as the number of lanes, and connection relationship);\n4. How to ensure the authenticity of traffic?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3UrDXd6hl5", "reviewer": "Reviewer_9smp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new traffic simulator to support large scale scenarios that include thousands of intersections and vehicles. Compared to prior work, the simulator is implemented in a more efficient parallelized fashion to enable such large simulations. The paper also proposes a new dataset of 100 traffic networks processed from OpenStreet map along with benchmarks for various traffic control tasks such as traffic signal control and congestion pricing. Experiments show the simulator is more efficient than prior works and that baseline methods for traffic control tasks can indeed be run in the simulator ", "summary_of_the_review": "Overall, I can see why a large scale simulator like the one presented could be useful for several aspects of traffic modeling, but the current paper does not do enough to convince this and has several issues in terms of presentation and clarity. Moreover, I don’t think this is a great fit for ICLR considering the limited contribution on the technical/learning side. \n\n\n================= After Author Response ========================\n\nMy initial review focused on the paper's connection to traffic behavior modeling and not necessarily the problem of traffic flow control, which I now understand is the focus here. The authors have clearly detailed the key differences between these two, why low-level behavior is not as important for traffic flow, and what makes simulation for traffic flow control unique. I think it's good they changed terminology to \"traffic control policies\" although I think \"traffic flow control policies\" would be even more clear.\n\nAfter re-considering the work in the context of traffic flow control, I have raised my score to a borderline accept.\n\nWith the newly added information in the paper on the behavior model, I think it makes the context and organization more clear. Moreover, the added experiments showing similarity of CBEngine traffic to SUMO indicating it is reasonable enough for traffic flow applications; and the behavior can be extended with their API.\n\nI am still concerned about the fit with ICLR seeing how the technical novelty is extremely limited; the contribution here is purely one of a simulation framework and dataset. Also, while the paper shows previous traffic flow methods can be successfully applied at new scales in the proposed simulator, there is no demonstrated empirical advantage to doing so (e.g. showing that training at a larger scale improves performance compared to smaller scale). However, previous works in this field have demonstrated the desire for operating at such city scales (Chen et al., 2020a; Zhang et al., 2022), so I do believe the proposed simulator and dataset would be a benefit to the community. In the end, I think it would be more beneficial to for this framework to be published and made available than not. And the authors have already shown a commitment to this by releasing code and detailed documentation.", "strength_and_weaknesses": "Strengths:\n* A very large scale and realistic traffic simulator could be useful for things like learning and evaluating traffic behavior models, especially in the context of reinforcement learning.\n* The proposed CBData with 100 road networks across the world is an interesting dataset that may be useful for several tasks in the community.\n* Experiments show the proposed simulator can handle very large scale simulations (thousands of intersections and vehicles) more efficiently than relevant previous simulators. Also, that the traffic behavior model can to some extent be tailored to a specific map through black-box optimization of parameters using GPS data.\n* Extensive code and clear documentation is provided which would greatly aid reproducibility and use of the simulator.\n\nWeaknesses:\n* It is not clear until the very end of the intro that the traffic “policies” considered here are things like traffic signal control and not the driving policy that controls traffic behavior. This really should be made clearer both in the abstract and early in the intro. But I’m wondering why the simulator is only shown for high-level traffic flow tasks? As a microscopic simulator couldn’t it be directly used for learning behavior models as well?\n* The paper does not mention or cite nuPlan (https://www.nuscenes.org/nuplan), l5kit (https://woven-planet.github.io/l5kit), or Carla (https://carla.org/) which are comparable traffic simulators. nuPlan and l5kit also include large scale real-world data for both maps and traffic that can be used in simulation. Though these simulators are not meant for tasks like RL, they do include detailed real-world maps along with real traffic on these maps (unlike the proposed simulator, which simply evenly distributes traffic across the map to start simulation). Moreover, at least some of them support custom traffic behavior models so it's not true that the proposed CBEngine is \"the first traffic simulator to support easy-to-use customization of vehicle behavior models.\"\n* In general, the paper does not do enough to convince that the need for 1000s of intersections and vehicles compared to 100s already supported by prior simulators. Sec 4 shows it’s possible to run baseline methods using the simulator, but I don’t think it actually “justifies the plausibility of our simulation” since metrics are only reported for the proposed simulator. It would be better to compare metrics on the proposed simulator to those in a prior simulator (like SUMO) that’s already accepted as plausible. It would also be more convincing to show an application like training an RL traffic driving model that truly requires the large-scale parallelism the simulator provides.\n* The organization and presentation of the method is often confusing and unclear. For example, Sec 2.1/2.2 are high level and vague, and no details of the behavior model are given until much later in Sec 3.2.1. Also, experiments are scattered throughout the description of the methods, making them sometimes hard to follow since the full context of the method has yet to be described. \n* For the simulator to be useful it’s important that traffic behavior be realistic, but currently this is not sufficiently supported. Firstly, the actual traffic model used is only very briefly described as “similar to IDM” and is only evaluated by deviation of average speed to ground truth which is a very coarse metric. A more comprehensive metric that compares the full distribution of speed and accelerations for individual vehicles would be more informative (e.g. see [BITS: Bi-level imitation for traffic simulation, Xu et al., 2022]) in evaluating realism. It’s also not clear does the simulator support more recent learned traffic behavior models (e.g. [TrafficSim, Suo et la., CVPR 2021])? Many of these are implemented in Python – is this supported by the simulators C++ interface? I would also appreciate some qualitative visualizations of the simulated traffic.\n* It’s not clear what is the granularity of the road networks provided in CBData. Does this have lane-level information or is it just the coarse road network?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject