VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture
Authors: Maonan Wang
Yirong Chen
Aoyu Pang
Yuxin Cai
Chung Shue Chen
Yuheng KAN
Man On Pun
Venue: NeurIPS 2025 poster ()
Keywords: traffic signal control
urban systems
multimodal language model
Primary area: other
TL;DR: VLMLight leverages vision-language scene understanding and dual-branch reasoning to achieve safe and efficient traffic signal control, especially in critical scenarios.

Abstract: Traffic signal control (TSC) is a core challenge in urban mobility, where real-time decisions must balance efficiency and safety. Existing methods—ranging from rule-based heuristics to reinforcement learning (RL)—often struggle to generalize to complex, dynamic, and safety-critical scenarios. We introduce \textbf{VLMLight}, a novel TSC framework that integrates vision-language meta-control with dual-branch reasoning. At the core of VLMLight is the first image-based traffic simulator that enables multi-view visual perception at intersections, allowing policies to reason over rich cues such as vehicle type, motion, and spatial density. A large language model (LLM) serves as a safety-prioritized meta-controller, selecting between a fast RL policy for routine traffic and a structured reasoning branch for critical cases. In the latter, multiple LLM agents collaborate to assess traffic phases, prioritize emergency vehicles, and verify rule compliance. Experiments show that VLMLight reduces waiting times for emergency vehicles by up to 65% over RL-only systems, while preserving real-time performance in standard conditions with less than 1% degradation. VLMLight offers a scalable, interpretable, and safety-aware solution for next-generation traffic signal control.

VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture
Maonan Wang1.2 Yirong Chen Aoyu Pang"
Yuxin Cai
Chung Shue Chen* Yuheng Kan Man-On Pun
The Chinese University of Hong Kong, Shenzhen, China
SShangaai AI Laboratory, Shanghai, China
Nanyang Technological University, Singapore Nokia Bell Labs, Paris, France
Fourier Intelligence, Shanghai, China
(maonanwang, aoyupang>@link, (link. cuh.. edu. cn chenyirong@pjlab org. caiy0039ce ntu. edu sg chung_shue whwnenooka--elllllaeeeeeeee com
kanyuheng@gmail. com com simonpun@cuhk. edu. cn
Abstract
Traffic signal control (TSC) is a core challenge in urban mobility, where real-time decisions must balance efficiency and safety. Existing methods-ranginn from rule- based heuristics to reinforcement learning (RL) often struggle to generalize to complex, dynamic, and safety-critical scenarios. We introduce VLMLight, novel TSC framework that integrates vision-lamm laiiunla meta-control with dual-branch reasoning. At the core of VLMLight is the first image-based traffic simulator that enables multi-view visual perception at intersections, allowing policies to reason over rich cues such vehicle type, motion, and spatial density. A large language model (LLM) serves as a safety-prioritized meta-controller, selecting between a fast RL policy for routine traffic and structured reasoning branch for critical cases. In the latter, multiple LLM agents collaborate to assess traffic phases, prioritize emergency vehicles, and verify rule compliance. Experiments show that VLMLight reduces waiting times for emergency vehicles by up to 65 % over RL-only systems, while preserving real-time performance in standard conditions with less than 1 1% degradation. VLMLight offers scalable, interpretable, and safety-aware aware solution for next-generation traffic signal control.
Introduction
Efficient management of urban traffic is a critical global challenge, with congestion leading to significant economic losses, environmental damage, and decreased quality of life [ Traffic signal control (TSC) at intersections plays an essential role in regulating traffic flow and reducing delays (2. Traditional TSC methods, such Webster's method (3) MaxPressure control 44) and Self-Organizing Traffic Lights (SOTL) 5] rely on predefined rules and domain-specific heuristics. While reliable under steady conditions, these rule-based approaches cannot cope with the dynamic, stochastic, and non-stationary realities of modern road networks. As a result, they struggle to respond unexpected changes in traffic patterns, where real-time, safety-critical decision-making is required to prevent delays and ensure public safety.
"qqual contribution
t *Coresponding author: simonpun@cuhi. edu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To overcome the rigidity of cule-based approaches. recent studies have explored reinforcement learning (RL) as a more flexible paradigm for TSC 7 [0. By interacting with the environment and learning from feedback, RI. agents can dynamically adjust signal phases in response to changing traffic conditions, offering improved adaptability and long-term optimization. However, despite their success in routine scenarios, these methods still rely on simplified, vectorized state representations, such as queue lengths [1] [3] or intersection pressure [14 [5], and use static reward formulations
[16 17. Such abstractions omit the semantically rich cues needed for context-dependent decisions. In safety-critical situations, such as granting the right-of-way to ambulances, RL agents trained solely on minimizing delay or maximizing throughput can deviate from the real-world priorities. Although some research [18 19 has started to consider special vehicles, these methods still face two key challenges: first, determining the trade-off between emergency and regular traffic; second, integrating these priority mechanisms with existing RI.-based controllers that differ in agent architecture and reward formulation. Consequently, current RL. approaches still fall short when rapid, semantically informed intervention is required.
The emergence of large language models (LLMs) 20 has opened new possibilities for incorporating high-level reasoning and generalization into traffic signal control systems (21 Recent efforts have explored the use of LLMs to handle edge cases and long-tail scenarios that are difficult for RL-based policies to address 23 24 25]. However, current LLM- approaches often rely on templated or manually crafted textual descriptions of traffic scenes, which lack the richness and fidelity needed to capture complex, real-world dynamics, leading to significant information loss, especially in visually grounded or ambiguous situations. Inference speed poses another hurdle: multi- step reasoning typically makes LLMs orders of magnitude slower than RI controllers, rendering LLMs impractical as standalone controllers in traffic environments where both latency and precision are essential. Moreover, current TSC simulators, such as the widely used SUMO (26] and CityFlow 277, can only provide statistical data for traffic intersections and cannot render real-time images, limiting the possibility to fuse the visual domain with language-basse reasoning for traffic signal control.
To address the limitations of both RL- and LLM-based methods, we propose VLMLight, a safety- critical traffic signal control framework that integrates lisionlanguage meta-control with dual- dual-branch reasoning. At the core of VLMLight is a novel, custom-buil built traffic simulator that, for the first time, enables multiview image inputs at urban intersections, allowing policies to reason over rich visual cues such as vehicle types, spatial density, and dynamic motion patterns. Each intersection scene is first processed by VLM to produce interpretable, multi-directional traffic deseriptions. These structured scene summaries are then passed to LLM acting as a meta-controller, which determines whether the situation requires low-latency control or high-level reasoning based on semantic understanding of the junction status. For routine traffic. VLMLight invokes a pre-trained RL policy to ensure fast and efficient signal control. In contrast, when safety-critical condition is detected (c.g., an ambulance requiring priority passage), the system activates slow deliberative reasoning branch in which multiple specialized LLM agents collaborate through structured dialogue. These agents sequentially perform phase analysis, action planning, and rule compliance verification, simulating a human-like deliberation process. This architecture fuses the speed of learned policies with the interpretability and robustness of language- based reasoning. Empirical results show that VLMLight reduces emergency vehicle waiting time by up to 65 % compared to RL-only baselines. Meanwhile, maintains comparable performance in standard traffic with less than a 1 % degradation, demonstrating strong potential for trustworthy and scalable deployment.
The key contributions of this work are as follows:
1. We propose VLMLight, a novel traffic signal control framework that integrates vision- lision-language scene understanding for both routine traffic and safety-critich scenarios. The framework leverages the first image- based simulator supporting multi-viww visual inputs at intersections, enabling real-time, context-aware decision-making and enhancing flexibility beyond traditional handcrafted traffic states.
2. Guided by an LLM-based meta-controller, VLMLight dynamically selects between a fast RL policy and a structured reasoning module based on real-time junction context. This dual-branch architecture ensures both high efficiency in standard traffic and deliberate, high-level reasoning for safety-critica events, such as prioritizing emergency vehicles.
3. Experimental results show that VLMLight reduces the waiting time for emergency vehicles (e.g., ambulances) by up to 65% compared to RL-only baselines, while maintaining comparable performance in standard traffic with less than 1 1% degradation.
2

---

2 Preliminaries
2.1 Traffic Signal Terminology
Figure ! Dillustrates typical four-way intersection. Each intersection comprises two types of ap- proaches: incoming approach with varying number income lanes (4n),, which carries traffic into the intersection, and outgoing approach with a varying number of outgoing lanes (lout), which the vehicles can leave the intersection. movement, denoted as m, refers to vehicles going from an incoming approach
For example, movement my includes i and 2 To regulate traffic flow safely, movements are grouped into phases, each representing a set non- conflicting movements that receive a green light simultaneously. For instance, Phase 1 is defined as p1 = [m,, m2). The complete set of feasible signal phases is denoted as P = {pI. P2, Ps: p4}.
outgoing
2.2 Vision-Based TSC Simulator
To overcome the limitations of con- ventional traffic simulators. we intro- duce the first vision-enabled traffic signal control simulator that supports multi-view visual inputs at intersec- tions. As illustrated in Figure [ the simulator allows configurable cam- placement to replicate real-world monitoring setups, including a bird's- eye view for monitoring overall traffic flow (left) and directional views simu- lating roadside perspectives from each approach (right).
North
Track
m2
West
South
Plase-1
Phase
Phase-4
Figure I: Illustration of a four-way intersection with four sig- nal phases. The simulator supports multi-view visual inputs, including a bird's bird's-cyee view (left) and directional views from each approach (right), enabling lane-level observation of ve- hicle movements. this example, the North- facing camera captures fire truck traversing the intersection, highlighting the simulator's ability to support safety-criticn reasoning through perceptually grounded traffic understanding.
These views form the perceptual ba- sis for VLMLight. For example, the North-facing camera clearly captures a fire truck navigating through the in- tersection, and the lane markings pro- vide additional guidance for tracking its trajectory. These visual cues sup- port fine- fine-graine -oa--graied scene understanding, which is essential for adaptive and saaety-aware traffic signal control. By enabling real-time observation of dynamic intersection behavior in complex and realistic traffic scenarios, the simulator enhances the system's 's ability to make informed and context-aware signal decisions under both routine and high-stakes scenarios.
3 Method
We propose VLMLight, a vasion-language traffic signal control framework that integrates the fast decision-making capabilities of RL with the generalization and structured reasoning strengths of LLMs. The core insight behind VLMLight that routine traffic scenarios can be efficiently managed by pre-trained RL policy. In contrast, emergency vehicles, require interpretable and high-level reasoning that goes beyond the RI. training distribution. As shown in Figure VLMLight consists of four stages: (1) Scene Understanding, where a VLM processes multi-view intersection images into natural language descriptions; (2) Safett-Prioritize Meta-Control, where an LLM agent analyzes the scene and determines whether to activate the fast RL branch (for routine control) or the deliberative LLM reasoning branch (for complex or critical events); (3) Routine Control, where pre-trained RL policy selects traffic signal actions based on agents collaborate through structured dialogue to assess traffic priorities and verify rule compliance.
safety-critical situations, such the presence of
features; and (4) Deliberative Reasoning. where multiple LLM
By expressing perception, reasoning, and control processes in natural language, VLMLLight provides a transparent interface that facilitates interpretability and post-hoc analysis across decision stages, thereby bridging low-level control and high-level reasoning in unified framework. This hybrid
3

---

Scene Understanding
1:M ofure
Meta-Control
Collabomative Reasoning
Alevvtionn
awere Aronch scfectios.
MMa directions to traffic
LLM Agar-el
planer and summarize each phaze's & staöas
Phase Reasoning
"A0l phaner are month only Phase- bax yueiaal vriteis
Vález Newwerk O-O
"Devide signal
veliele prionty and traffic rolamee.
o
Signal Planning
Fast Side
Trijp
ooooo
Policy Netwok a 0+
plase compuiie valid control options.
"Verif' of selerted
"Priatttic gresn fight Phare due to special vebtcii."
1.M.Ager of
Verification
Deliberative Reasoning Policy
Phoje and complies with control rales.
Routine Control Policy
Ravironment
Rr
Matt Step Henonn
Figure 2: VLMLight architecture. Multi-view intersection images are first parsed by VLM agent for scene understanding, after which a safety-prioritized LLM meta-controller interprets the scene and selects either fast RL policy (orange) for routine traffic flow collaborative reasoning policy (blue) for safety-critical scenarios. A team of LLM Pcents-Phase Reasoning, Signal Planning, and Rule Verification- sequentially assess traffic phases, vehicle priority, and rule compliance to determine the final action (t, giving real-time control and robust handling of complex events.
architecture enables VLMLight to maintain real-time responsiveness in typical scenarios while ensuring trustworthy and explainable decisions under safety-critical conditions.
3.1 Scene Understanding via VLM
Effective TSC requires an accurate and interpretable understanding of intersection conditions. Prior LLM-based approaches often rely on fixed, templated descriptions of traffic state (c.g., queue lengths average delays), which are limited in expressiveness and fail to capture rich visual semantics, such as vehicle types, spatial configurations, or the presence of special vehicles. To address this, we build a closed-looy traffic simulator that provides real-time image observations from multiple directions of an intersection for perceptual grounding. Based on these visual inputs, a dedicated Scene Understanding Agent (Agentsceme) leverages pretrained VLM to process these inputs and generate natural language traffic scene descriptions.
Given directional images (T, 12,.... ID} from D camera viewpoints (e.g., North, East, South, West), the agent produces a set textual summaries:
Tb} Agentsoonc (( 42,,, ID}),
where each T, is textual description of the traffic conditions in direction i, including lane-level semantics, congestion level, and whether any emergency or special vehicles (c.g., ambulances) are present. These free-form yet structured descriptions decision modules. As illustrated Figure 3 this process is demonstrated on a T-junction with three incoming directions. For each J, the Agentsceme generates corresponding T, that captures actionable traffic semantics, forming the perceptual basis for mode selection and phase reasoning in later stages.
as interpretable inputs for downstream
3.2 Cafety-Priorittize Meta-Control
Once the directional scene descriptions the appropriate decision-makee strategy. RL-based policies are efficient for routine traffic but are limited by fixed reward structures, making them unsuitable for unexpected or safety-criticn events. In contrast, HLLM-based reasoning offers greater flexibility but incurs high latency, which is impractical for real-time control.
Tp} are generated, the system must determine
To balance responsiveness with adaptability, we introduce a implemented as a language model agent (AgentMeddsSeeennx) Rather than issuing control actions directly, AgentModssoloto interprets the set of textual scene descriptions and decides whether to route
Meta- Peta-Controoler,

---

Phose Description P
Seene Description T
Frufic Plase
1. First Lavs on the Lef. Designated for going instlet
-Sratghe Lone in Y2. Moderate congestion 12 regaior wheries,
Congentio Maderott, regular vehislez Vehiclo
Strlight Lave in Ty; Madiruie to high congention veliclou. -Speral loticles Nons sithrr lane.
Secoud ase the Left Draggnted for nurnnng igf
2.1 Congertion Level: Madicoate, wilh regalar velinles quosed. Speciat Vchiolos Novr.
Tragic Placend
Tarn in T.. Maderate congertion (3 regalar veliciss.. -iihht Thonn is Madlrntte in high congastion 12 rchidlos) Soccial Febicles: None ctttrr iose.
Z
Scene Description Tz
2. Firsi Lave on Left Desggaatd nerning right
Conpestion Level: Moderatt high. Three vetacle exii.
1.2 Sperial Volbetoe All reglar vebieee
Right da congestion regalar vokiele).
Left Tarn in in T.. congestion (1 regalar velicio
Second Lone the Lef Designoted for going stronght
SSucial Hédiclas: None stherr baye.
Congeation Leset Law Only vekidder Special
rogular
Phase-1
Scene Description T
Right- Turn Lone
No roagestion lour
only révv single velicle
1.2 Special Feltctios Nome.
Phase-2
2. Left Tuon Lane (Second line on the ke0))
biue regular volible lune, Fohectio Nine.
Phase-3
Figure 3: Illustration of Agentseene VLMLight. Given three-view images from a T-junction (left), a VLM--based Scene Description Agent generates directional-level textual summaries (Ti, T2. Ta), describing lane semantics, congestion, and special vehicle presence. These summaries are then aggregated into phase-level descriptions (P) based on predefined signal phase mappings.
the system to the fast RL routine policy or the deliberative LLM reasoning module. If the scene is consistent with the assumptions of the RL training objective, such as the absence of special vehicles or traffic accidents, the system proceeds with low-latency control via the RL branch. However, if the description includes critical elements such emergency vehicle detection, conflicting traffic goals, the meta-controlle activates the structured reasoning path. This conditional routing allows VLMLight respond efficiently in common scenarios while retaining the capability to reason explicitly under rare, high-stakes conditions.
3.3 Routine Control Policy
Under normal traffic conditions, VLMLight engages fast RL decision branch designed for low- latency, high-throe teththhroughput control. The current intersection state is constructed using recent statistics across up to 12 traffic movements (e.g., vehicle flow, occupancy, signal status), aggregated over the past five time steps to form a spatio-temporal input tensor.
This input is encoded by a lightweight Transformer-based module that first captures spatial dependen- cies among concurrent movements and then models temporal dynamics across frames. The resulting representation is used to select traffic signal phase from predefined discrete action set A.
The RL policy is trained using Proximal Policy Optimization (PPO) (28] with reward function designed to minimize intersection-leww congestion and delay. This routine path enables fast, reactive control without invoking high-level reasoning, making it well-suited for the majority of non-critical traffic scenarios. Implementation details, including the agent design, state encoder, and PPO objective, provided in Appendix A.1
3.4 Deliberative Reasoning Policy
When traffic conditions deviate from routine patterns, such as the presence of emergency vehicles, confficting priorities, or abnormal congestion, VLMLight activates its slow, deliberative reasoning branch. This path engages a team of specialized LLM agents that collaborate through structured natural language dialogue to generate context-aware signal decision. The process unfolds in three stages: Phase Reasoning. Signal Planning, and Rule Verification.
The first step to convert directional scene descriptions into phase-level traffic summaries aligned with the TSC control action space. As shown in Figure 3 given directional-level
Phase Reasoning
5

---

descriptions { the Agentphase reorganizes the information using the predefined mapping from traffic movements to signal phases. For instance, Traffic Phase-1 will combine straight-going lanes from 7i and 72. The agent aggregates relevant semantic lane-level details (e.g., vehicle types, congestion levels) form coherent descriptions {P:} for each candidate phase. This transformation bridges low-level visual perception and high-level traffic control, enabling the system to make decisions directly in the discrete space of traffic phases instead of raw directional inputs.
Signal Planning Next, Agentrian evaluates the candidate phase descriptions {P;} in light of the current control objectives, such as minimizing delay. prioritizing emergency vehicles, or balancing directional load. Based on this reasoning. it selects the most appropriate phase a.L.M A and generates textual explanation justifying the decision. This rationale provides interpretability and enables auditability of the agent's decision-making process.
Rule Verilication Finally, Agentcheck verifies whether the chosen action af complies with the current feasible phase set Ar. If the action is valid, the system proceeds with execution. If not, the agent selects an alternative from A that best aligns with the original intent while maintaining safety and consistency. The final decision is then formatted as a JSON object for downstream execution.
Experiments
4.1 Experimental Setup
Experiment Settings All experi- ments are conducted in our custom- built TSC simulator, which inte- grates SUMO [26] to simulate ve- hicle dynamics and supports multi- view image rendering for vision- based perception. Each traffic episode adheres to standard urban signal timing, including a green phase (minimum duration of 10 s), 3-second yellow phase, and red phase. A minimum headway 2.5 meters is enforced to reflect safe urban driving behavior. For vision-language processing, Qwen2.5-VL-322B (29 to generate structured traffic scene descriptions from multi-view images. In safety- critical scenarios, high-level reason- ing is performed by Qwen2.5-72B [30 via structured multi-agent di- alogue among three LLM agents. While Qwen is the primary back- bone for our experiments, VLM- Light is modular and compatible with alternative VLM and LLM models. We analyze the impact model choices in Appendix C.2
me
(c)
Figure 4: Three real-world intersections, each shown with three image modalities: (a) Songdo (South Korea), (b) Yau Ma Tei (Hong Kong), and (c) Massy (France). For each site, the satellite view on the left, SUMO simulation in the middle, and our simulator rendering on the right.
Dataset We evaluate VLMLight on traffic data collected from three real-world intersections located in Songdo (South Korea), Yau Ma Tei (Hong Kong), and Massy (France), as illustrated in Figure These locations represent diverse urban settings. Songdo, a newly developed urban district, features larger intersections with up to five lanes per direction. Yau Ma Tei, situated in dense urban core, has narrower roads and movement

---

decision-making making within restrictions (e.g., no left or right turns in certain directions). Massy provides a contrasting layout with a T-junction and distinct lane configurations. The diversity of intersection iypes-including crossroads (Songdo and Yau Ma Tei) and a T-junction (Massy)-as well as differences in size and movement patterns, enables more comprehensive evaluation of the method's generalizability across varied traffic scenarios. Each intersection is equipped with multi-direction minutes of traffic data. The first 20 minutes are used for training the RL policy, while the remaining 10 minutes are reserved for testing. This setup allows to evaluate the generalizability of VLMLight across varying topologies, lane geometries, and movement constraints. Additional dataset details are provided in Appendix B.2
cameras capturing 30
Evaluation Metrics We evaluate TSC system performance using four metries that jointly assess traffic efficiency and emergeney vehicle handling. For overall traffic flow, we report the Average Travel Time (ATT), defined as the mean duration for vehicles to reach their destinations, and the Average Waiting Time (AWT), which represents the average duration that vehicles remain nearly stationary (speed < 0.1 m/s), typically due to signal-induced delays. To specifically evaluate emergency vehicle treatment, we measure Average Emergency Travel Time (AETT) and Average Emergency Waiting Time (AEWT). These reflect the system's 's responsiveness to high-priority traffic. Together, these metrics capture both global intersection efficiency and the framework's s ability to prioritize safety-critical scenarios.
To evaluate the performance of VLMLight, compare it against both traditional and RL-based TSC methods. The traditional baselines include FixTime, Webster [3]. and MaxPressure 4 which rely handcrafted timing rules pressure-based heuristics. For RL-based approaches, we consider IntelliLight (I]) UniTSA 19. A-CATs (6]. 3DQN-TSCC (7]. and CCDA [IO), which learn policies based vectorized traffic state representations. Since VLMLight is the first framework that supports image-based input and leverages vision-language models to perform real-time traffic signal control, there are no existing VLM-based methods available for direct comparison. Therefore, to provide a more targeted comparison, we additionally include baseline named Vanilla-VLn which directly uses VLM-generated descriptions for decision-make maainng without VLMLight's 's safety-critical meta-control and dual-brande reasoning architecture. A detailed description of these methods is provided in Appendix B.3
Compared Methods
4.2 Performance under Routine and Safety-Critical Traffic Scenarios
We evaluate VL.MLgght across three real-world intersection datasets and compare with rule-based, RL-based, and VLM-hased baselines. As shown in in both routine and emergency scenarios, demonstrating a robust tradeoff between control efficiency and safety-aware decision-making
Table VLMLight achieves strong performance
Routine Traffic Efficiency VLMLight maintains near-optimal performance under standard traf- fie conditions. Compared to the best-performing RL-based methods, the increase in ATT is marginal-less than 1% across all datasets. For example, in the Songdo intersection, ATT increases slightly from 86,80 (UniTSA) to 87.14 s (VLMLigght) Similarly, average waiting time (AWT) increases from 39.53 s to 39.73 s. These minor differences reflect the effectiveness of VLMLight's meta-controller, which defers to the fast RL branch in routine settings, avoiding unnecessary LLM overhead. By contrast, VanillaVVLM, which relies solely on vision-language reasoning, performs significantly worse due to the semantic burden of integrating perception and decision a single monolithic pipeline.
Emergency Vehicle Prioritization In safety-critical scenarios, VLMLight activates its deliberative reasoning branch, leading to substantial improvements in emergency vehicle handling. Across all datasets, VLMLight reduces both AETT and AEWT by over 60% relative to RL-only baselines. For example, in the Songdo dataset, AEWT drops from 22.0 to just 7.48% s. While Vanilla-VI. also benefits from explicit scene reasoning, its lack of structured phase mapping and decision modularity leads to frequent action errors, especially in complex intersections like Yau Ma Tei. This underscores the importance decomposing the TSC task into modular reasoning steps. By assigning specialized roles to collaborating I.M agents' reasoning, VL.Light ensures correct, auditable actions and maintains real-time responsiveness even in safety-critical scenarios. Detailed analysis and examples of the intermediate reasoning steps generated by VLMLight are provided in Appendix D
7

---

Table Performance comparison on the three intersections. The top three results are marked with (best), (second), and (third).
South Korea, Songdo
Category
Method
ATT
AWT +
AETT 4
AEWT +
Fix
111.73 = 5.11
59.68 # 1.95
108.66 68.688 + 7.20
49.53 : 2.08
Rulc-based
Webster
3
102.89 = 4.20
50.02
# 2.75
82.77 3.94
26.60 = 1.62
MaxPressure 4
93.65 3.41
43.71
1.61
79.38
4 2.53
35.38
IntelliLight
87.12 + 5,10
39.68
# 1.98"
70.00
2.36
22.12 # 1,27
UniTSA 99
86,80 4.89*
39.53
1,97
69.74 3.80
22.04 = 0.77
RL-based
A-CAT's
88.23 + 6.01
:: 41.61
70.08
+: 4.40
22.15
+ 0.95
3DQN-TSCE
99.09 + 5.68
45.13
+ 2.20
79.62
+ 3.25
25.17
# 1.54
CCDA 10
89.32 6.21
40.68
+ 2.48
71.76 + 4.82
22.68
1.25
Vanilla- VLM
105.48 +: 17.28
48.09
:: 8,98
60.38
11.78
11.05
1.73
VLM-based
VLMLight (Ours)
87.14 + 4.98*
39.73 1.71
49.88 2,42"
7.48
0.45*
Hongkong, Yau Ma Tei
Category
Method
ATT
AWT
AETT 4
AEWT
Fix
67.63
40.00 ::
82.67 : 3.23
53.17 2.14
Rule-based
Webster
3
56.26 # 3.39
28.62 # 1.23
59.67 : 4.11
30.83 = 1.79
MaxPressure
41.36
2.22
13.33 + 0.40
36.17 + 2.39
8.83 0.27
IntelliLight
11
38.07
= 2.52
10.28
+ 0.54
33.17
1.54
5.17 + 0.18
UniTSA
38.10
:: 1.28
10.29
+ 0.50
33.19 + 1.92
5.17
0.35
RL-based
A-CAT' 6
42.64
+ 1.43
11.51
+ 0.71
37.14
+ 1.67
5.79
0.34
3DQN-TSCC
46.20
12.47
0.54
+: 40.25
6.27
CCDA 10
41.60
: 2.02
11.23
+ 0.45
36.24
+ 1.45
5.65
0.31
Vanilla- VLM
62.45 8.76
17.62
4: 2.45
27.79
3.79
5.86
0.69
VLM-based
VLMLight (Ours)
39.80 +: 1.653
11.85
+ 0.60
13.50 + 0.56
2.17
0.12
France, Massy
Category
Method
ATT +
AWT 1
AETT
AEWT
FixTime
75.84 4.46
28.19 1.58
73,60 4.62
27.80 +: 1.06
Rule-based
Webster
68.92 2.15
20.89 0.79
65.20
= 3.04
19.80 + 0.61
Max PaePsusue
64.82 4.01
15.25
0.86
72.40
2.83
22.40
4: 0.92
Intellil.ight
57.73 3.51
9.80
0.67*
54.80 + 2.45
7,81
0.39
UniTSA 9
57.84
1.91
9.82 + 0.541
64.90
2.28
9.81 + 0.41
RL-based
A-CAT' 66
62.44
3.24
10.60
0.35
59.27
2.72
7.35 + 0.30
3DQN-TSCC 7
69.10
2.86
13.73
0.52
65.59
2.80
8.14 + 0.49
CCDA 10
62.83
2.42
11.19
0.38
58,80
2.40
7.20 + 0.49
Vanilla-VLA VLM
81.18
11.91
13.44
1.91
53.11
4: 8.03
3.04
0.61
VLM-based
Vgh (Ours)
60.84 # 2.63
11.49
+ 0.75
45,40
+ 2,88
2.60
0.10
4.3
Ablation Study
We conduct ablation studies to quantify the impact each agent module the Deliberative Reasoning branch. As shown in Table 2 disabling Agentpaase leads notable increase in AEWT from 48s to 58s, indicating its essential role in the reasoning pipeline. This agent plays a crucial role by abstracting low-level directional features into structured traffic phase representations. enabling downstream reasoning to operate over a discrete and legally grounded action space. Without this abstraction, the reasoning agent must handle raw directional information directly, which leads to error- prooe or inapplicable decisions in intersections with complex phase- movemmet mappings.
In contrast, removing Agentcheck, which verifies the legality of the final decision has limited effect on overall performance, as invalid actions are rare in most cases. These results highlight the impor- tance of structured semantic grounding and explicit planning in traffic signal control. VLMLight's

---

Table Ablation results on Songdo, impact of structured reasoning components and corresponding inference time.
Modules
Metrics
Agene
Agentrlan
Agentcheck
AETT
AEWT :
Time (J. s)
X
58.93
9.03
7.10
49.96
8.12
10.97
V
V
48.88
7.48
11.48
modular agent design not only improves emergency response but also supports transparent, auditable decision-making in safety-critical environments.
We further evaluate the inference latency introduced by the structured reasoning branch under the Songdo intersection. As reported in the last column of Table we compare different configurations of VLMLight modules to measure their computational overhead. Even with all modules enabled, the total decision latency remains 11.48 seconds. This latency is well within the acceptable range for real-time deployment, as it fits the typical signal phase buffer (10s green + 3s yellow). Detailed latency breakdowns and further analysis are provided in Appendix C.3
Related Work
Traffic Signal Control TSC methods fall into three categories: rule-based, RL.-based, and L.M-- based. Traditional approaches like Webster's method 3] SOTL [5], and MaxPressure [4] use fixed heuristics and perform well under steady traffic, but struggle with real-time adaptability 22 RL-based
(35] improve responsiveness by learning from interaction, but often overlook safety-critical events due to simplified states and static rewards [36] 37 38 39]. Recent
methods 66 [3 9
32
33
34
11
LLM-based approaches [2] 23 22 [25]
introduce high-level reasoning for rare or long-tail cases, but rely on hand-crafted text inputs and lack true visual grounding. Moreover, they underperform in routine traffic control compared to specialized RL agents.
Simulators for TSC Most TSC simulators, such as SUMO 266 CityFlow 27 and LibSignal
40), provide only structured traffic states (e.g., vehicle counts. signal phases) without visual outputs, limiting their use in assion-grounde reasoning. High-fidelity driving simulators such as CARLA 41 and MetaDrive [42] offer photorealistic rendering, but they are primarily designed for autonomous driving tasks rather than intersection control, and they often lack native support for signal scheduling or require extensive customization. To address this gap, SynTraC [43) introduces an image-based dataset for TSC built upon CARLA, providing intersection images annotated with traffic states, signal phases, and reward information under diverse weather and lighting conditions. Although SynTraC represents significant step toward visual perception in TSC, remains limited to single-intersection scenarios and does not support user-defined or multi-view configurations, constraining its scalability and applicability to broader traffic management research.
Conclusion
We present VLMLight, vision-language TSC framework that unifies fast policy execution with structured semantic reasoning. By dynamically selecting between a reinforcement learning policy for routine traffic and deliberative reasoning branch for safety-critical cases, VLMLight adapts to diverse intersection scenarios with both efficiency and robustness. Empirical experiment results demonstrate that our method reduces emergency vehicle waiting time by up to 65%, while maintaining comparable performance to RL-only baselines in standard conditions. Importantly, the full inference process completes within 11.5 seconds, which fits comfortably within the typical 13 s signal phase buffer used in urban deployments, indicating practical feasibility for real-time use. In addition, we release the first vision-based TSC simulator with support for multi-view image inputs and dynamic scene rendering. This simulator enables richer visual understanding of intersection conditions and provides a foundation for future research into perceptually grounded and interpretable traffic control strategies.

---

Limitation VLMLight has two primary limitations. First, our simulator currently lacks diverse weather and lighting conditions, limiting the evaluation of visual robustness under challenging envi- ronments. Second, all experiments are conducted on single intersections. Extending the framework to multi-intersection settings requires further study on the scalability and coordination of vision-language reasoning across broader traffic network.
Broader Impacts VLMLight provides a vision- lssounla framework for TSC that enhances safety and efficiency through real-time visual reasoning. The open-source simulator offers valuable tool for future research on perception-based traffic systems.
Acknowledgments and Disclosure of Funding
This work was supported in part by the Guangdong Science and Technology Department under Grant 2025SF0001.
References
[1] Amudapuram Mohan Rao and Kalaga Ramachandra Rao. Measuring urban traffic congestion-a review. International Journal for Traffic & Transport Engineering. 2(4), 2012.
[2] Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. A survey on traffic signal control methods. arXiv preprint arXiv:1904 1904.09119911119 08110,811771111 2019.
[3] Fo Vo Webster. Traffic signal settings. Technical report, 1958.
[4] Pravin Varaiya. Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies, 36:177-195, 2013.
[5] Beung-Bae Cools, Carlos Gershenson, and Bart D' D'Hooghe. Self-organizing traffic lights: A realistic simulation. Advances applied self-organizing systems, pages 41-50, 2008
[6] Mohammad Aslani, Mohammad Saadi Mesgari, and Marco Wiering. Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events. Transportation Research Part C: Emerging Technologies, 85:732-752. 2017.
[7] Xiaoyuan Liang. Xunsheng Du, Guiling Wang, and Zhu Han. A deep reinforcement learning network for traffic light cycle control. IEEE Transactions on Vehicular Technology, 68(2):1243 1253, 2019.
[8] Afshin Oroojlooy, Mohammadreza Nazari, Davood Hajinezhad, and Jorge Silva. Attendlight: Universal attention- based reinforcement learning model for traffic signal control. Advances in Neural Information Processing Systems, 33:4079 4090, 2020.
[9] Maonan Wang. Xi Xiong, Yuheng Kan, Chengeheng Xu, and Man-On Pun. UniTSA: universal reinforcement learning framework for v2x traffic signal control. IEEE Transactions on Vehicular Technology, 2024.
[10] Maonan Wang, Yirong Chen, Yuheng Kan, Chengcheng Xu, Michael Lepech, Man- On Pun, and Xi Xiong. Traffic signal cycle control with centralized critic and decentralized actors under varying intervention frequencies. IEEE Transactions on Intelligent Transportation Systems, 25(12):20085-2014 2024.
[11] Hua Wei, Guanjie Zheng, Huaxiu Yao, and Zhenhui Li. IntelliLight: A reinforcement learn- ing approach for intelligent traffic light control. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2496-2505 2505. ACM, 2018.
[12] Aoyu Pang, Maonan Wang, Yirong Chen, Man-On Pun, and Michael Lepech. Scalable rein- forcement learning framework for traffic signal control under communication delays. IEEE Open Journal of Vehicular Technology, 2024.
10

---

[13] Hanyang Chen, Yang Jiang, Shengnan Guo, Xiaowei Mao, Youfang Lin, and Huaiyu Wan. Difflight: a partial rewards conditioned diffusion model for traffic signal control with missing data. Advances in Neural Information Processing Systems, 37:123353-12337 2024.
[14] Hua Wei, Chacha Chen, Guanjie Zheng. Kan Wu, Vikash Gayah, Kai Xu, and Zhenhui Li. Presslight: Learning max pressure control coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1290-1298, 2019.
[15] Chacha Chen, Hua Wei, Nan Xu, Guanjie Zheng, Ming Yang, Yuanhao Xiong, Kai Xu, and Zhenhui Li. Toward a thousand lights: Decentralized deep reinforcement learning for large- scale traffic signal control. In Proceedings of the AAAI conference on artificial intelligence. volume 34, pages 3414-3421, 2020.
[16] Zian Ma, Chengcheng Xu, Yuheng Kan, Maonan Wang, and Wei Wu. Adaptive coordinated traf- fic control for arterial intersections based on reinforcement learning. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 2562-2567, 2021.
[17] Hao Mei, Junxian Li, Bin Shi, and Hua Wei. Reinforcement learning approaches for traffic signal control under missing data. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI '23, 2023.
[18] Haoran Su, Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. EMVLight: A decentralized reinforcement learning framework for efficient passage of emergency vehicles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4593-4601, 2022.
[19] Miaomiao Cao, Victor OK Li, and Qiqi Shuai. A gain with pain: Exploring intelligent traffic signal control for emergency vehicles. IEEE Transactions on Intelligent Transportation Systems, 23(10):17899-1793
2022.
[20] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
[21] Yiqing Tang, Xingyuan Dai, Chen Zhao, Qi Cheng, and Yisheng Lv. Large language model- driven urban traffic signal control. In 2024 Australian & New Zealand Control Conference (ANZCC), pages 67-71, 2024.
[22] Siqi Lai, Zhao Xu, Weijia Zhang, Hao Liu, and Hui Xiong. LLMLight: Large language models as traffic signal control agents. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD '25, page 2335 2346, New York, NY, USA, 2025. Association for Computing Machinery.
[23] Maonan Wang, Aoyu Pang, Yuheng Kan, Man-On Mnn-On Pun, Chung Shue Chen, and Bo Huang. LLM-Assisted Light: Leveraging large language model capabilities for Human--imetti traffic signal control in complex urban environments. arXiv preprint arXiv:2403.0833 2024.
[24] Aoyu Pang, Maonan Wang, Man-On Pun, Chung Shue Chen, and Xi Xiong. iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement. arXiv preprint 240:::24000000255 2024.
[25] Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang, and Yuxuan Liang. Traffic-Rl: Reinforced Ilms bring human-like reasoning to traffic signal control systems. arXiv preprint arXiv:2508. 02344, 2025.
[26) Pablo Alvarez Lopez, Michael Behrisch, Laura Bieker-Wal Jakob Erdmann, Yun-Pang Flöt- teröd, Robert Hilbrich, Leonhard I Lücken, Johannes Rummel, Peter Wagner, and Evamarie Wießner. Microscopic traffic sinulation using sumo. In The 21 IEEE International Conference on Intelligent Transportation Systems. IEEE, 2018.

---

[27] Zheng Tang, Milind Naphade, Ming- Yu Liu, Xiaodong Yang, Stan Birchfield. Shuo Wang, Ratnesh Kumar, David Anastasiu, and Jenq-Neng Hwang. Cityflow: A city-scale benchmark for talget multi-camera vehicle tracking and re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8797-8806, 2019.
[28] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint SEGiv:17077063444 2017.
[29] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang. Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL. technical report. arXiv preprint wrXv:2500.139223 2025.
[30] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang. Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.15113 2024.
[31] Tianshu Chu, Jie Wang, Lara Codecà, and Zhaojian Li. Multi-agent deep reinforcement learning for scale traffic signal control. IEEE transactions intelligent transportation systems, 2109):1086-1095, 2019.
[32] Minshuo Chen, Yu Bai, H Vincent Poor, and Mengdi Wang. Efficient rl with impaired ob- servability: Learning to act with delayed and missing state observations. Advances in Neural Information Processing Systems, 36:46390 46418, 2023.
[33] Wenlu Du, Junyi Ye, Jingyi Gu, Jing Li, Hua Wei, and Guiling Wang. Safelight: A reinforcement learning method toward collision-free traffic signal control. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 14801-14810. 2023.
[34] Yin Gu, Kai Zhang, Qi Liu, Weibo Gao, Longfei Li, and Jun Zhou x-light: Programmatic interpretable reinforcement learning for resource-limited traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21107-211 2024.
[35] Qinchen Yang. Zejun Xie, Hua Wei, Desheng Zhang, and Yu Yang. MalLight: Influence- aware coordinated traffic signal control for traffic signal malfunctions. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 2889, 2024.
[36] Kok-L.im Alvin Yau, Junaid Qadir, Hooi Ling Khoo, Mee Hong Ling, and Peter Komisarczuk. A survey on reinforcement learning models and algorithms for traffic signal control. ACM Computing Surveys (CSUR), 50(3):1-38, 2017.
[37] Syed Shah Sultan Mohiuddin Qadri, Mahmut Ali Gökçe, and Erdinç Öner. State-of-art review of traffic signal control methods: challenges and opportunities. European transport research review, 12:1-23 2020.
[38] Hua Wei, Guanjie Zheng, Vikash Gayah, and Zhenhui Li. Recent advances in reinforce- ment learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD explorations newsletter, 22-2):12-18, 2021.
[39] Haiyan Zhao, Chengeheng Dong, Jian Cao, and Qingkui Chen. A survey on deep reinforce- ment learning approaches for traffic signal control. Engineering Applications of Artificial Intelligence, 133: 138:108100 1000000,00000000 2024.
[40] Hao Mei, Xiaoliang Lei, Longchao Da, Bin Shi, and Hua Wei. Libsignal: an open library for traffic signal control. Machine Learning, 51308):5235-527,, 2024.
12

---

[41]
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings the 1st Annual Conference on Robot Learning, pages 1-16, 2017.
[42]
Quanyi Li, Zhenghao Peng, Lan Feng, Qihang Zhang, Zhenghai Xue, and Bolei Zhou. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 45(3):3461-347) 2022.
[43] Tiejin Chen, Prithvi Shirke, Bharatesh Chakravarthi, Arpitsinh Vaghela, Longchao Da, Duo Lu, Yezhou Yang, and Hua Wei. SynTraC: A synthetic dataset for traffic signal control from traffic monitoring cameras. In 2024 IEEE 27th International Conference Intelligent Transportation Systems (ITSC), pages 2386-2391. IEEE, 2024.
[44] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 148922-34915 Curran Associates, Inc., 2023.
[45] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arxiv preprint arXiv:2407.21783, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Abstract and Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section 6 where two primary limitations are discussed.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
We do not have theory assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[Yes]
Justification:
See the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppi.ccppulli//uuuiu wwwiiicgguie///eeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section and Appendix
B
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: See Section where standard deviation is reported for the experiment results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section 4.1 and Appendix, B.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http::::///rips...ee http:/p......cuutttttttccccciaa.
Answer: [Yes]
Justification: This research follows the NeurlPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section 6
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
17

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not suffer from this risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example, by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswipteeeee whworee.wthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
18

---

Answer: [NA]
Justification:
The paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve research with human subjects,
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
19

---

Justification:
See Section 3
The backbone used for the experiment is Qwen as disclosed in
Section[4.1
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non utansstndard components.
Please refer to our LLM policy hattps://neurips 200t:///e///coofeeeeee////2///// for what should or should not be described.
20

---

Appendix
In this appendix, we provide supplementary material to further elaborate on VLMLight:
Additional method details, including the routine control policy, full algorithm, and prompt
templates in Section
Full experimental setup, including datasets and compared baselines in Section B
Extended results, including RL convergence curves, LLM model comparisons, and inference times in Section C
Three case studies illustrating the decision-making process of VLMLight in Section D
In-depth discussion limitations and broader impact Section
A Method
A.1 Details of Routine Control Policy
In standard traffic scenarios, VLMLight adopts an Reinforcement Learning (RL) policy trained within Markov Decision Process (MDP) framework. At each timestep the intersection state is encoded where each m IR represents the status of the i-th movement at the intersection. The vector m includes a combination of traffic flow, movement, and signal-related features. The traffic characteristics are captured through the average vehicle flow Fi,, maximum occupancy O and mean occupancy O since the last control action. The movement-speciffc features consist of an indicator I E {0, 2, that reflects whether the movement straight, left, or right, and the lane count [4. Signal status described by two binary indicators: T&st for whether the current phase is green, and L'g indicate whether the minimum green duration requirement has been satisfied. The complete feature vector written as:
as J, [mj ...... m121,
mf = F , Oud T Li, Th.t Td.1] max
(2)
To ensure consistent input size, intersections with fewer than 12 movements are zero-padded. For example, at the Yau Ma Tei intersection, the absence of a left-turn movement from north to south is represented by a zero vector. An illustration the zero-pase zero-padding scheme shown in Figure [ To capture temporal dynamics, the agent receives input from the current and previous four timesteps. forming a 5-frame observation window:
S: = = Jr 2, E 5x12x7
At each timestep, the agent selects an action a; E P, where P denotes the set of available traffic signal phases. The reward r, is defined the negative average queue length, encouraging smoother traffic flow.
To extract expressive representations from S,, we use a Transformer-based encoder that processes both spatial and temporal dimensions. In the spatial encoding stage, the individual movement vectors mf E R² for each frame Jj E 1l2x7 are projected to d-dimensional embeddings h via shared linear layer:
h W.m + be, 1= 12,
producing token matrix h' = [hi...., h E !2xd This matrix is processed by Transformer block composed of layer normalization (LN), multihead self-attention (MSA), and feed-forward network (MLP):
E ML. (LN (h' + MSA(LN(h')) E R 12xd
(5)
We then apply mean pooling across the 12 movement embeddings to obtain compact spatial summary s.:
1 12
S, 12
E E Rd.
(6)
21

---

padding
Figure 5: Example of zero- sero-padding at the Yau Ma Tei intersection.
In the second stage, process the five temporal embeddings S, [st [s St] E R5xd using another Trranformer encoder:
Z, MLP (LN (S, + MSA (LN(S.)))) RRxd
where each row Zr, Rdd corresponds the enriched representation of time step k: To obtain fixed-length state embedding for decision-making, we apply average pooling over the temporal axis:
f,
Z k E Rd
(8)
After obtaining the spatio-temporal representation h, we pass it into both the policy and value networks. The policy head outputs a probability distribution (a, hy: over actions, and the value head estimates the expected return V V(h; o). We optimize the policy using the Proximal Policy Optimization (PPO) algorithm, which maximizes the following surrogate objective:
(0) E, min (r,, 1ipp,,(0),,-,,1+)))))
where r. (0) sala, lb.)
Tegp (a, ]b,] is the policy ratio and A, is the advantage estimate. The clipping parameter is used to bound policy updates and improve training stability. The value network is trained by
minimizing the squared error between predicted values and empirical returns:
21 Cralue (Φ) = E, ((V(bL: R.) }
(10)
where Rg denotes the bootstrap return. This hierarchical design enables the routine control policy to leverage both short-term dynamics and long-term patterns for efficient traffic management. The full training objective combines the above terms:
Lioaa (0, の) - Epolicy (0) + (o),
(11)
where A is a hyperparameter that balances value learning. This Transformer-bart hicrarchical design allows the fast RL policy effectively reason fine-grained spatio-temporal signals for efficient traffic control in routine scenarios.
A.2 Algorithm for VLMLight
VLMLight employs modular set of collaborative agents that together enable perception-aware, safery-critical traffic control. Table[]]summarize the responsibilities of each agent. The architecture is designed to interleave fast decision-making (via RL) with high-level reasoning (via LLM agents)
22

---

Table 3: Summary of agent roles in VLMLight.
Agent Name
Function
Agents
Converts multi-view images into directional text descriptions T; Selects control mode: fast RL policy structured LLM reasoning Aggregates T, into phase-level descriptions P.
Agen ModeSelector
Arane Phase
Agen
Selects optimal action a. and explains the rationale
AgentCheck
Validates action fcasibility against current legal phase set A
under unified meta-control mechanism. Each agent operates on structured inputs-cithan either visual, textual, phase-level representations- and outputs either decision an intermediate semantic representation used by downstream agents.
Algorithm Algorithm for VLMLLight
Require: Maximum simulation time Taax- legal phase set A, phase-to-lane mapping Mph-lunes maximum check attempts Nebeck, control interval At.
1: Initialize 1 <-
2: while < <Taxx do
3: Obtain multi-view images (41, 1,, 1D} from simulator
4: {Ti, , TD} + ACENTScINo((413)
5:
m < MOENTTTnwSSL
6:
if m then
7:
01 -- AGENTRL (s:)
8:
else
9:
[P). P2. PR} + AGENTPHASE((ET) Mph. lane)
10:
LLLM + AGENTPLAN
for
N Neheck do
12:
<- (GENTCHECx(C! L.M ILM A.))
a; E A, then
14:
break
end if end for
16:
0 & A, then - (s))TR
18:
19:
end end if
20:
21:
Execute a, in simulator
22:
t + At
23:
end while
Algorithm Noutlines the inference procedure of VLMLight. At each decision interval, the system receives multi-view images from the simulator and invokes the Agentscene to generate directional scene descriptions. A safety-prioritized meta-controller AgentMideSelesum then determines whether proceed with the fast RI. policy or activate the structured reasoning branch. In routine conditions, lightweight RL agent issues control action based the current traffic state. In contrast, for safety- critical scenarios, three LLM agents collaborate sequentially: the Agentphase module transforms scene descriptions into phase-level summaries using predefined phase-to-lang mapping Mph-lane; the Agentplan agent proposes candidate action aligned with system objectives; and the Agen agent verifies the action's 's legality against the current feasible phase set A., the verification fails after Neheek attempts, the system falls back to the action proposed by AgentRL to ensure continued operation. This strict validation pipeline, coupled with the fallback mechanism, safeguards against potential noise reasoning errors, ensuring that only feasible and safety-compliant actions are executed. The selected action then executed in the simulator, and the simulation clock advances by fixed interval At. This loop continues until the maximum simulation time T max is reached.
23

---

A.3 Prompt Templates
This section presents the prompt templates used by the five agents in Vig Agent uses a VLM convert intersection images into textual descriptions, as shown in Figure 6 The other four agents are based on LLMs:: MuinEtue determines the control mode (Figure 7 Agent generates phase-level descriptions based on the scene context (Figure Agentplan selects the optimal and AgentCheck verifies rule compliance (Figure
phase (Figure 9)
Agentsceme scene
You are TrafficVision Vision, an traffic analyst. Based on intersection image below from fixed surveillance camera, provide an accurate and concise description of the scene:
Assess traffic congestion level.
Identify special vehicles (e.g.. ambulances, police cars, trucks) only if clearly visible.
Avoid speculation report only what is verifiable.
[[Image]]
Figure 6: Prompt template for Agent
Agent Modeseletor
You are in a role play game. The following roles are available:
Routine Control Agent: Handle normal traffic using RL-based decisions optimize flow and ensure safety. Reasoning Agent: Take over when special vehicles appear or unusual conditions arise, ensuring their priority while keeping traffic orderly. Please read the dialogue history and choose the next suitable role to speak.
When user indicates to stop chatting or when topic should terminated, please return [[TOP]]
Only return the role name from [[agent_names]]] or "[TOP]]. Do not reply any other content.
Figure 7: Prompt template for Agen
Agent phantphasee
You are a traffic phuse analyst. The intersection has [dirrection number]] directional deseriptions, each representing different view. The following is the phase-to-lane mapping for this junction: [[phase-tollaee]]
Please summarize each traffic phuse by extracting:
1) Congestion level
2) Confirmed special vehicles (ambulance, police car, truck only clearly visible)
3) Any notable traffic events
Use the seene descriptions provided for each direction:
[juuetion__dscciipiion-diiii dddescrpppiionnddeccion....
Figure 8: Prompt template for Agentphase
Agentpıan
You roleplaying traffic police officer managing real-time intersection. You have received the description for each traffic phase: [[phase_descripttee description)].
Please make decisions by:
Prioritizing confirmed emergency vehicles (ambulances, police cars, fire trucks)
Otherwise, adjusting signal timings to minimize congestion and maximize overall traffic efficiency
Note: You must choose only from the following available actions: [(available
[[available_actioos]]] actions)]
Figure Prompt template for Agentplan-

---

Agentcneck
You evaluator responsible for verifying whether traffic control decision is valid.
valid decision must select an action strictly from the following set: [sselfavailable | If the decision is compliant, return a JSON object with exactly two keys:
"decision": the selected Traffic Phase IDD
"explanation"; rationale for the decision based on the traffic phase description
No other keys are allowed in the output. Example output:
"decision": "Phase-2",
"explanation": "The image depicts basic road intersection scenario with no special vehicle markings;
Figure 10: Prompt template for AgentCheck.
B Experiments Setup
B.1 Experiment Settings
In this section, describe the experimental setup used to evaluate the performance of VLMLight, our proposed traffic signal control framework. The experiment involves the integration self- developed traffic simulator, the deployment of large models for dision-languee understanding, and the training of an RL policy to assess VLMLight's 's adaptability in dynamic traffic scenarios, especially in safety-critical situations.
The experiments are conducted using a self- delf-developed traffic simulation environment built on top of the SUMO (Version 1.22) framework. The simulated roads feature maximum speed limit of 13.9 m/s (approximately 50 km/h) to model typical urban traffic conditions. The simulator includes three specialized types of vehicles-police cars, ambulances, and fire trucks-to evaluate the system's s performance emergency response scenarios. The vehicle speed distribution is modeled as a Gaussian distribution with mean of 10 m/s and variance of 3, reflecting typical urban traffic flow patterns.
To ensure high-performance traffic signal control, we deploy both VLMMs and LLMs locally on high-performance computing system. The system is equipped with an Intel Xeon 6738P CPU, 256 GB of RAM, and five A100 GPUs, all running Ubuntu 20.04 L.T.. This setup allows us to run multiple VLMs in parallel, significantly improving the speed and efficiency of scene understanding. The use of multiple VLMs is essential for rapidly processing the visual inputs from various intersection views and generating natural language descriptions that capture the complex dynamics of the scene. This configuration ensures that VLMLight can make real-time decisions based on a thorough understanding of the current traffic situation.
For the RL-based components of VLMLight. we use the PPO (28 algorithm, implemented via the Stable Baselines3 library. speed up training and improve the exploration of different traffic conditions, we deploy 30 parallel processes, each interacting with separate instance of the simulation. The total number of environment steps set 3e5 and the batch size is configured to 64. The learning rate follows linear schedule, starting at le 3 and gradually decreasing as the number of training steps increases. Additionally, the trajectory memory size is set to 3000.
B.2 Dataset Details
To evaluate the generalizability of VLMLight, we construct an evaluation suite based three real- world intersections with varying topologies and traffic conditions, located in Songdo (South Korea), Yau Ma Tei (Hong Kong), and Massy (France). Each site was selected to represent distinct urban forms: Songdo features large-scale grid intersections with high traffic throughput; Yau Ma Tei is situated in densely populated downtown with constrained geometry and restricted turning rules; and Massy contains a suburban T-junction with lighter traffic and fewer lanes. These variations allow us systematically test VLMLight across broad range of physical layouts and flow intensities.
For each intersection, multi-directional cameras were deployed capture minutes of continuous traffic footage, with all approaches covered. As shown in Figure the first column in each subfigure
25

---

(b)
(c)
Figure 11: Multi-view camera observations three real-world intersections. Top-down layouts are shown on the left; directional inbound views follow. (a) Songdo, (b) Yau Ma Tei, and (c) Massy.
presents the top-down intersection layout, while subsequent images show direction-specific views from each inbound lane. These direction-wise visual inputs are fed into the VLMLight perception module for downstream reasoning. Table summarizes the directional traffic flow statistics. Vehicle counts and arrival rates vary considerably across sites: Songdo shows the heaviest traffic load, with arrival rates exceeding 2 vehicles/s in certain directions, while Massy represents the lightest scenario with sub-1 vehicle/s flow. Emergency vehicles were sparsely but consistently present across all sites, ensuring meaningful evaluation of safety-critical reasoning. This comprehensive setup enables reproducible and diverse benchmarking for vision laroo--lngugg--bre traffic signal control.
B.3 Compared Methods
In this section, we introduce the methods compared to our VLMLight framework, which includes three traditional baselines, five RL-based approaches, and one VLM-based method. These methods are evaluated highlight the advantages of VLMLight in terms of traffic efficiency and safety.
Traditional Methods.
We adopt three traditional approaches in experiments follows:
FixTime: Fixed-time control assigns predetermined cycle and phase durations, which are most effective in steady traffic conditions. We consider the FixTime-30 variant, where each phase duration is fixed at 30 seconds.
The Webster method adjusts cycle lengths and phase splits based on traffic volumes, optimizing travel time in uniform traffic. In this study, we use for adjusting traffic lights based on real-time traffic flow.
Webster 3:
The MaxPressure method prioritizes phases with the highest traffic demand, optimizing the flow by minimizing congestion. This approach known for its simplicity and effectiveness in maximizing intersection throughput.
MaxPressure 4
26

---

Table 4: Traffic flow statistics for each approach direction at the three intersections, #Veh: total vehicles; #Emerg: emergency vehicles.
Arrival Rate (vehicles/s)
Network
Dir
#Veh
#Emerg
Mean
Std
Min
Max
3780
9
2.10
0.31
1.60
2.67
3740
4
2.08
0.32
1.58
2.78
Songdo
2993
4
1.66
0.23
1.20
2.20
2932
5
1.63
0.21
1.35
2.03
2556
7
1.42
0.20
1.05
1,70
1916
4
1.06
0.17
0.78
1.37
Yau Ma Tei
1927
4
1.07
0.17
0.75
1.35
2346
2
1.30
0.20
1.00
1.65
1216
3
0.68
0.09
0.45
0.85
Massy
626
2
0.35
0.06
0.25
0.47
1079
2
0.60
0.10
0.42
0.78
RL-Based Methods.
We examine five RL-based methods, each offering distinct strategies for TSC:
IntelliLight uses DQN-based approach to select the best traffic phase from available options, addressing data imbalance by maintaining balanced data buffer for each phase. In this study, decisions are made every 5 from available phases.
IntelliLight Mi
UniTSA introduces junetion matrices, which enable it adapt to different intersection layouts. The method also leverages state augmentation, ensuring the agent encounters diverse intersection types and traffic volume during training.
UniTSA 19:
A-CATs employs an actor-critic approach to train TSC agents, where the output phase duration is adjusted within range of 10 40 seconds. This method provides continuous learning for phase duration optimization based on traffic conditions.
A-CATs 6
3DQN-TSCC 77 3DQN-TSCC applies DQN adjust phase durations in small increments, focusing on stabilizing the signal light transitions. In this method, the phase duration is modified by fixed of values 0,
CCDA 10 CCDA introduces a centralized critic and decentralized actor framework, ensuring stability in phase duration changes. The method adjusts all phase durations in smaller steps 6-6, -3, 0, 3, 6) s and decisions are made every 10 to ensure stability.
VLM- Based Method. We also consider VLM-based approach, Vanilla- ManillaVLLM, which directly utilizes VLM for scene understanding and generates a textual description of the traffic situation, which is then used by LLM to make decisions without the involvement of RL policies in regular scenarios.
Camulativ Deward
Cussillttvv Roward
10300
Cunzalattve Newarr
-5000
-50000
-00000
-7500
artellllagh
Justiaale
UhiTSA
-3000
A-CATT aDQU-TSC
12500
A-CATH
100000
CCDA
INIQN-TO
-0000
CCDA
19200
SDQN-TSC
CCDA
080
800
350
320
(0]
500
3500
420
(00
300
Epiesde
Egisode
Epiaode
(a)
(b)
(c)
Figure 12: Training reward curves of five RL-based TSC methods across three real- world intersections: (a) Songdo (South Korea), (b) Yau Ma Tei (Hong Kong), and (c) Massy (France).
27

---

C Additional Experiments Results
C.1 Additional Performance Analysis of RL Methods for TSC
In this section, describe five RL-based TSC methods: IntelliLight, UniTSA, A-CATs, CCDA, and 3DQN-TSC. Figure shows the reward curves for each method three scenarios, where the x-axis represents training episodes and the y-axis represents cumulative rewards. Among the methods, Intellil.ight and UniTSA achieve the highest cumulative rewards and exhibit rapid convergence. Their superior performance stems from their design: both adopt direct phase-switching actions, allowing for timely and responsive adjustments to dynamic traffic conditions.
A-CATs and CCDA exhibit competitive, though slightly inferior performance. A-CATs decomposes the multi-phase scheduling task into sequential single-phase adjustments, which increase the learning difficulty but eventually result in near-optimal control once convergence is reached. CCDA extends this idea by enabling simultancous updates of all phases with a larger action space, leading to slower convergence but similar final performance.
Disignn-immited Finally, 3DQN-TSC performs the worst across scenarios. Its restrictive action design-limitn limited to modifying single phase per cycle cycle-constrail constrains its ability to optimize phase allocations holistically. As result, accumulates fewer rewards and struggles to match the performance of other methods.
Table 5: Performance comparison of different VLMs for scene understanding. The top two results are marked with (best), (second).
Scene
Metries
VLMLight
Qwen2.5-VL-7B
LLava-7B
LLava-13B
GPT-4o
Songdo Yau Ma Tei
87.14*
92.68
95.25
91.96
87.461
ATT
39.80*
41.79
48.54
41,27
41.12t
Massy
60.84"
65.62
70.54
66.44
63.72
Songdo Yau Ma Tei
49.881
76.13
80.45
60.86
48.06'
AETT
13.50
30.34
27.55
19.71
13.04
Massy
45.40
62.63
65.97
55.78
45.17*
Table 6:
Performance comparison of different I.LMs for mode selection.
Scene
Metrics
VL.MLight
Qwen2.5-7B
Qwen2.5-32B
Llama3-70B
GPT-4o
Songdo
87.14t
87.18
86.28*
89.53
88.57
Yau Ma Tei
ATT +
39.801
41.01
41.14
38.89*
40.19
Massy
60.84
62.46
62.73
62.67
60.79*
Songdo
49.88
49.811
50.67
51.25
49.70*
Yau Ma Tei
AETT
13.50
13.36
13.91
13.19
12.96
Massy
45.40*
46.81
48.61
45.44
46.76
Table 7:
Performance comparison of different LLMs reasoning policy.
Scene
Metries
VLMLight
Qwen2.5-7B
Qwen2.5-32B
Llama3-70B
GPT-4o
Songdo
87.14*
90.59
88.59
87.401
87.99
Yau Ma Tei
ATT |
39.80
38.501
40.57
38.23*
39.93
Massy
60.84
63.71
62.56
59.11*
61.11
Songdo Yau Ma Tei
49.88
51.85
49.71
49.451
49.37*
AETT L
13.501
13.34
13.97
13.76
13.14*
Massy
45.40
53.68
46.68
50.54
44.61*
28

---

C.2 Additional Ablation Study on different LLMs
In this section, we analyze the impact of different LLM models on the performance of VLMLight across various modules, including scene understanding, mode selection, and reasoning policy. We evaluated multiple models, including Qwen2.5-Ve VL.7B (2). LLaVA-7B, LLaVA-13B 44,, and
GPT-4o [20] for the scene understanding agent (Agentscene). and Qwen2.5-7B. Qwen2.5-322 (30),
Llama3, 1-7BB [45], and GPT-4o [20] for the AgentMoocSclette and reasoning policy agents. The
results are presented in Tables 5 6 and showcasing the effectss of model selection on the respective
modules.
The results indicate that the scene understanding module (Agentsceme) is the most sensitive to model changes. As shown [a the performance of the model significantly affects both the Average Travel Time (ATT) and Average Emergency Travel Time (AETT), especially in identifying special vehicles. For instance, switching from Qwen2.5-VL-322. to DW results notable increases in the waiting time and travel time of special vehicles, likely due to missed recognition of emergency vehicles. Moreover, the performance drop in model accuracy also leads to longer travel times for regular vehicles, normal vehicles may be mistakenly treated as special vehicles, causing unnecessary green lights to be given. This highlights the importance of reliable and accurate scene understanding for the overall system performance, inaccurate scene descriptions propagate errors in the subsequent peciisin-makingg stages.
In contrast, the Agent and reasoning policy agents, which involve simpler textual pro- cessing tasks, show more resilience model changes. As indicated in Table6]anm Table 7 even smaller models like Qwen2.5-7B maintain similar performance to larger models, with only marginal differences ATT for regular vehicles. However, special vehicles may still experience slight delays due to inaccurate sane-to-phasse mappings in the reasoning process. Overall, the most critical take- away is that the scene understanding module has the greatest impact on VLMLight's performance, particularly in ensuring timely prioritization of special vehicles. Thus, using high-perfonme setth-eeffrmanc model for scene understanding is essential for maintaining the system's 's ability to handle complex, safety-critichs scenarios effectively.
C.3 Additional Ablation Study on Inference Time
In this section, we analyze the inference time of VLMLight across three distinct environments. The results, shown T demonstrate that VLMLight achieves inference times well below 13 in all three environments, falling within acceptable range for real-world deployment. This is particularly notable considering the minimum green light duration of 10 s and the additional s for yellow lights.
As illustrated in Table the majority of the inference time is spent scene understanding, while mode selection and deliberative reasoning stages require considerably less time. Overall, the results indicate that VLMLight is suitable for deployment in practical settings, with its architecture optimized for both speed and safety.
Table 8: Inference time for each stage of VLMLLggh across three environments.
Stage
Songdo
Yau Ma Tei
Massy
Agentscene
5.12
5.15
4.79
Agen MoieSeeeeeelettrr
0.75
0.95
0.77
Agentplase
3.87
1.95
2.24
Agentrian
1.23
0.86
1.21
AgentCheck
0.51
0.45
0.34
Total
11.48
9.36
9.35
D Case Study
To showcase VLMLight in action, we present three representative case studies. Each example covers a complete TSC cycle from time step + demonstrating how different agents collaborate under both routine and safety-critical scenarios. For each case, describe the visual inputs, the decision
29

---

made by each agent. and the resulting traffic outcome. This offers insight into how VLMLight dynamically selects between the fast RL branch and the deliberative LLM branch as circumstances demand.
T
Scene Understanding
Deseribe the traffic condition in this direction.
Outpur
Direction-1: Moderate traffic with regular vehicles and no emergency presence.
Direction-2: Light traffic with few rogular vehicles clear intersection.
Direction-3:
Minimal traffic with spaced regular vehicles
delays.
Direction-4: Moderate flow with six regular vehicles and emergency vehicles.
Safety-Priorine Meta-Control
Ippur: Return role from [Routine Control Agent] [Reasoning Agent].
Outpur: Routine Control Agent
T T+1
Routine Control Policy
Output
"decision": "Phase-2". "explanation": "Under routine traffic conditions, implement the decisions recommended by the reinforcement learning system to optimize traffic flow."
Figure 13: Routine Control in Yau Ma Tei.
T
Scene Understanding
Input:
Deseribe traffic condition in this direction.
Outpuf:
Direction-l: Light traffic... special vehicles detected.
Direction-2: Moderate congestion... ambulance detected in center lane, immediate
green light recommended.
Direction-3: Steady flow... emergency vehicles, delay green phase needed.
Input: Safety-Prioritized Meta-Control
[Routine Control Agent] [Reasoning Agent].
Outpur: Reasoning Agent
Deliberative Reasoning Policy
Input: Please summarize the traffic condition for each phase. Output:
Phase-1: Lightest congestion... priority for green light.
Phase-2: Light to moderate traffic gmbulance detected, switch to green immediately.
Phase-3:
Moderate congestion... no special vehicles, can be delayed briefly.
Inpur:
Please select an appropriate traffic phase.
Output: Phase- ambulance approaching. immediate green light needed safe passage.
Impur Please check if the current action (e.g., Phose-1, Phase-2, ...) is valid. so, return the result JSON format.
Outpur
"decision": "Phasc-1".
"explanation": : "An ambulance is detected in Phase-1... no special vehieles in other phases, prioritizing Phase-1."
Figure 14: Deliberative Reasoning policy for complex traffic in Massy.
30

---

Scene Understanding
Ippur: Describe the trfffi condition in this direction.
Output
truc in the struight lane. Direction-2: Moderate congestion; emergency vehicles seen.
Direction-1: I: Moderate traffic
Direction-3: Vehicles queued both lanes; special vehicles.
Direction 4: Steady flow with only regular vehicles.
Safety-Prioritized Meta-Control
Input: Return the role name from [Routine Control Agent] or [Reasoning Agent].
Output. Reasoning Agent
Deliberative Reasoning Policy
Imput: Please summarize the traffic condition for each phase.
Outpur
Phase-O: Moderate traflic Several regular cars from different directions
T+1
Phase- 1: Moderate congestion no special vehicles.
Phase-2: Moderate flow A fire appronching.
Phase-3: Moderate congestion
standard present.
Input: Please select an appropriate traffic phase.
Output: Traffic Phase-2 is prioritized... Aftre truck is approaching. This ensures safe, quick passage and meets priority rules for special vehicles.
EEEER
Input: Please check the the resalt in JSON format. Output
(e.g., Phase-1, Phase-2, valid.
"decision": "Phaso-2",
"explanation", : "Phase-2 prioritized allow approaching pass safely, required for special vehicles."
Figure 15: Deliberative Reasoning policy for complex traffic in Songdo.
D.1 Example 1: Routine Control in Yau Ma Tei
Figure 13 presents routine scenario the Yau Ma Tei intersection. The Agentscene first transforms multi-view traffic images into structured language descriptions. Finding no anomalies or priority vehicles, the AgentModeSdeecrr routes the control to the RL branch. Based on the traffic density, the AgentgL selects Phase-2 (westbound) for the green signal. The transition from T to + confirms that the westbound queue clears once Phase is activated.
D.2 Example Complex Scenario in Massy
Figure 14 showcases special case at the Massy intersection, where an ambulance is detected on the west approach. The Agentscene detects the emergency vehicle from the image inputs and generates descriptive observations. Recognizing a priority event, the AgentMoleSeteaor subsequently triggers the Deliberative Reasoning branch. The Agentphase agent maps the scene to candidate signal phases, Agentplan recommends Phase-1 for green signal, and Agentclock verifies compliance with emergency-priorts rules. By the time T + 1, the ambulance has cleared the intersection through the northbound turn.
D.3
Example 3: Complex Scenario in Songdo
Figure 15 presents complex case the Songdo intersection. Similar to the previous Massy case, the Agent identifies key cues (a fire truck approaching), and AgentMistSslecuee activates the Deliberative Reasoning branch. The sequence of Agentphase- Agentplan and AgentClook ensures
31

---

compliant and safe control action. By the time T + 1, the fire truck moves through the intersection without interruption.
E
Disscussion
VLMLight introduces novel vision- vision-language framework for TSC, combining real-time visual reasoning with safety and efficiency improvements. As the first open-source vision-based simulator in the TSC domain, VLMLight is compatible with RL-based TSC algorithms, offering valuable resource for future research on perception-drive
traffic systems. This framework enables enhanced scene understanding through multi-view visual perception and structured reasoning. ensuring both fast peeriiiiommakinn for routine traffic and reliable handling of critical scenarios like emergency vehicles.
However, VLMLight has several limitations. Firstly, the current simulator lacks diverse weather and lighting conditions, limiting its evaluation of visual robustness in challenging environments. Secondly, the absence of pedestrians, bicycles, and other real-world elements makes the simulated environment less realistic; incorporating more diverse models is necessary for future iterations. Third, all experiments have been conducted on single intersections, and extending the framework to muutii-ntersection scenarios requires further research scalability and coordination in broader traffic networks. Lastly, VLMLight's performance is closely tied to VLM capabilities, with optimal results requiring models with numerous parameters. Future work will focus fine-tuning smaller models to improve both traffic scene recognition accuracy and inference speed.
32


Reviews:
[{"review_id": "I1r8ZMCBhd", "reviewer": "Reviewer_ubqN", "summary": "The paper presents VLMLight, a traffic signal control framework that integrates vision-language meta-control with dual-branch reasoning. At the core of VLMLight is an image-based traffic simulator that enables multi-view visual perception at intersections, allowing policies to reason over rich cues such as vehicle type, motion, and spatial density. A large language model serves as a safety-prioritized meta-controller, selecting between a fast RL policy for routine traffic and a structured reasoning branch for critical cases. Experimental results show that VLMLight reduces the waiting time for emergency vehicles (e.g., ambulances) by up to 65% compared to RL-only baselines, while maintaining comparable performance in standard traffic with less than 1% degradation.\n\nThe article contains an introduction, including the traffic signal control, a description of the methodology, setup, and results of experiments, including ablation results. It also contains separate sections presenting the limitations, related works, and broader impacts. The supplementary materials contain the code.", "questions": "-", "limitations": "There is a separate section describing the identified limitations. I believe they are addresses adequately.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The idea is quite original\n- The results of the experiments are very good, the article has the potential to be significant\n- The paper is well-written, it has a good structure, and all necessary elements\n- There is a good ablation study and discussion of limitations\n- The code is available to verify the results\n- There is also an analysis of the inference time, suggesting that the method can be applied to real-time traffic signal control\n\nWeaknesses:\n- I did not identify any significant weaknesses", "was_revised": true, "final_justification": "After reading the other reviews and responses, it seems that the issues raised by the other reviewers have been adequately addressed. Therefore, I am maintaining my initial score.", "rebuttal": "We sincerely thank the reviewer for their positive evaluation and thoughtful recognition of VLMLight's contributions. As you noted, VLMLight is the first image-based simulator for traffic signal control, enabling multi-view visual perception at intersections and allowing policies to reason over rich visual cues such as vehicle type, motion, and spatial density, an important step beyond traditional RL- or statistics-only approaches.\n\nWe also wish to highlight that our vision-language meta-control framework adopts a dual-branch design (fast RL + structured LLM reasoning), enabling the system to manage routine traffic efficiently while reliably handling safety-critical scenarios (e.g., emergency vehicle prioritization) with greater interpretability. This dual-branch reasoning addresses real-world challenges that RL-only methods struggle to manage.\n\nWe deeply appreciate your recognition and encouragement, which further reinforces our belief that VLMLight provides a scalable, interpretable, and safety-aware foundation for next-generation traffic signal control."}, {"review_id": "RPLhu1nhXQ", "reviewer": "Reviewer_biB6", "summary": "The paper presents a vision-language model-based approach for traffic signal control. A novel custom-built traffic simulator is first proposed, and multiple LLM agents are introduced and used as controllers to reduce the waiting time of emergency vehicles.", "questions": "1. Maybe I understand the results wrong - I am wondering whether the authors have reported the real-time performance of the proposed approach. Given that the approach involves vision-language model, and multiple LLM agents, are all LLMs deployed in local and the inference time can be ignored? Or should the approach wait for the responses of the LLMs? If so, the overall computation time of the approach cannot meet the real-time requirement in real-time traffic control application.\n\n2. Is it possible that we directly use the embeddings along the process instead of using explicit texts, as texts sometimes can be misleading and inappropriate representations can affect the model performance?\n\n3. How can we guarantee that the LLM agents (controllers) can always give safe instructions in order to prioritize the emergency vehicles? I understand that if the actions are not safe, we will select an alternative, but do the authors conduct any operation to guarantee that there will be an alternative that is feasible?\n\n4. The overall framework, while unseen before in my opinion, is still a bit trivial. Basically we just let the VLM tell us what happen and LLM agents respond to prioritize the emergency vehicles. Also, the experimental results only support that the approach is better when there is emergency vehicle in the intersections. Somehow this feels limited and cannot be generalized to other scenarios. Please correct me if I am wrong.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. A new traffic simulator\n2. A meaningful and practical problem setting involving emergency vehicles.\nWeakness:\n1. The reliance of scene description and phase description in terms of text seems a bit unnecessary, which may even introduce noise since text representations can be misleading sometimes.\n2. While important, the improvement only for emergency vehicles is still limited.", "was_revised": true, "final_justification": "I have read the rebuttal and I think the responses have addressed some of my concerns. I have raised my scores accordingly.", "rebuttal": "We thank the reviewer for the thoughtful and insightful comments. We respond to each of the points as follows:\n\n> [W1]. Use of textual scene/phase descriptions may introduce noise\n\nWe appreciate the reviewer's point and agree that text representations require careful design. **Our intent for using textual descriptions is to ensure interpretability, traceability, and trustworthiness-critical factors for deployment in real-world traffic systems.** In real-world traffic signal control, these aspects are indispensable. Textual descriptions allow every decision made by the system to be easily reviewed, audited, and justified by traffic authorities, capabilities that are difficult to achieve with latent embeddings alone. Prior work [1] has also emphasized the importance of interpretability in TSC, reinforcing our design choice.\n\nCrucially, all final actions are subject to a rule-based compliance check that strictly enforces traffic regulations and safety constraints. This mechanism ensures that even if noise were introduced at the textual stage, only safe and legally valid traffic signal actions can be executed, thereby eliminating the risk of unsafe outcomes from LLMs.\n\n> [W2]. Improvement only on emergency vehicles feels limited\n\nWe agree that the current experiments focus on emergency vehicle scenarios, but we respectfully emphasize that emergency vehicle handling is a core safety-critical issue in traffic signal control research, as demonstrated by prior work such as EMVLight [2]. Our approach enables real-time emergency vehicle prioritization **directly from intersection camera feeds**, without relying solely on pre-collected statistical data.\n\nAdditionally, we open-sourced the **image-based traffic signal control simulator**, shifting TSC research from purely statistical inputs to realistic vision-based scenarios, which we believe will benefit a broader range of future studies and applications.\n\n> [Q1]. Real-time feasibility of multi-agent LLM reasoning\n\nWe agree that real-time performance is crucial. VLMLight addresses this by design:\n1. Minimum green time inherently provides reasoning time. In TSC, **pedestrian safety** necessitates a minimum green light interval. Based on standard calculation formulas cited in [3,4], the minimum pedestrian green times for our three experimental intersections are 25s, 23s, and 17s (corresponding to crosswalk widths of 30m with a median island, 25m, and 17m respectively). As shown in Table 2, VLMLight's reasoning latency, around 10s (even when both VLM and LLM are engaged) remains well below these green time thresholds. Moreover, government guidelines (e.g., Government of South Australia [5], Toronto [6]) stipulate minimum pedestrian green times of 9s for all users (including those using mobility aids). Adding the standard 3s yellow phase, there is a guaranteed 12s buffer for reasoning, which comfortably covers VLMLight's inference time.\n2. Multi-agent reasoning is only triggered in critical scenarios. The RL branch handles routine traffic control, while the multi-agent LLM reasoning branch is activated only in safety-critical cases (e.g., emergency vehicles, obstructions). This design means that most cycles involve only the VLM scene perception and mode selector decision, avoiding multi-round LLM reasoning in the majority of situations.\n3. To further confirm real-world deployability, we evaluated VLMLight using Gemma3-4B [7], a lightweight vision-language model optimized for edge devices. As shown below, Gemma3-4B's inference time is only ~1.5s in normal cases and <3s even in safety-critical scenarios, easily within operational limits.\n\n**Inference Latency (seconds)**\n|           |         | Songdo | Yau Ma Tei | Massy |\n|:---------:|---------|:------:|:----------:|-------|\n| Gemma3-4b | Normal  |  1.61  |    1.49    | 1.45  |\n|           | Safety-critical |  2.38  |    2.29    | 2.21  |\n|  VLMLight | Normal  |  5.87  |    6.10    | 5.56  |\n|           | Safety-critical|  11.48 |    9.36    | 9.55  |\n\nPerformance comparisons (Table below) show Gemma3-4B maintains nearly identical efficiency to VLMLight in handling special vehicles, with only a slight decrease for normal vehicles due to occasional false emergency-vehicle detections (a known hallucination issue).\n\n**Performance Comparison (seconds)**\n|           |      | Songdo | Yau Ma Tei | Massy |\n|:---------:|------|:------:|:----------:|-------|\n| Gemma3-4b | ATT  |  89.81 |    41.38   | 64.79 |\n|           | AETT |  47.54 |    14.57   | 47.01 |\n|  VLMLight | ATT  |  87.14 |    39.80   | 60.84 |\n|           | AETT |  49.88 |    13.50   | 45.41 |\n\nThese measures collectively ensure VLMLight meets real-time requirements.\n\n> [Q2]. Could embeddings be used instead of explicit text\n\nWe chose text for its explainability and transparency, which is critical for system audits and adoption by traffic authorities. We believe the benefit of human-readable justifications outweighs the potential risk of occasional misleading phrasing-again mitigated by the rule-based action validation step.\n\n> [Q3]. Guaranteeing safety of LLM-issued actions\n\nSafety in VLMLight is ensured through a multi-layer safeguard mechanism:\n\n1. We have a dual verification of actions. As illustrated in Figure 2, the `Rule Verification Agent` first validates whether the `Plan Agent`'s proposed action complies with traffic regulations. Before execution, a second hard rule check confirms that the action is within the predefined available action set. If it is not, the system automatically falls back to the fast RL branch, ensuring only safe and valid actions are issued.\n2. Even if an LLM outputs a suboptimal choice, the consequence is limited to minor delays (e.g., longer waiting time), never unsafe outcomes.\n3. Table 1 demonstrates that LLM agents make correct decisions with high reliability, further supporting their operational safety.\n\n> [Q4]. Framework seems trivial / limited generalization\n\nWe clarify that VLMLight is more than a VLM+LLM combination. Its dual-branch architecture merges (i) a high-speed RL policy for standard flow and (ii) a reasoning-based branch for safety-critical cases-mimicking how human traffic officers manage both routine and exceptional situations. \n\nFurthermore, the **open-source simulator is a core contribution that transitions TSC research** from simplified statistics to realistic vision-based control, enabling future generalization to diverse traffic scenarios beyond emergency vehicles.\n\nReference:\n\n[1] Gu, Yin, et al. \"π-light: Programmatic interpretable reinforcement learning for resource-limited traffic signal control.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 19. 2024.\n\n[2] Su, Haoran, et al. \"EMVLight: A multi-agent reinforcement learning framework for an emergency vehicle decentralized routing and traffic signal control system.\" Transportation Research Part C: Emerging Technologies 146 (2023): 103955.\n\n[3] Koonce, Peter. \"Traffic signal timing manual. No. FHWA-HOP-08-024.\" United States. Federal Highway Administration, 2008.\n\n[4] Iryo-Asano, Miho, Wael KM Alhajyaseen, and Hideki Nakamura. \"Analysis and modeling of pedestrian crossing behavior during the pedestrian flashing green interval.\" IEEE Transactions on Intelligent Transportation Systems 16.2 (2014): 958-969.\n\n[5] Government of South Australia, \"Traffic Signal Standard: Signal Timings - TS001\"\n\n[6] City of Toronto, \"Standard Operating Practice: Pedestrian Timing at Signalised Intersections.\" Transportation Services\n\n[7] Team, Gemma, et al. \"Gemma 3 technical report.\" arXiv preprint arXiv:2503.19786 (2025)."}, {"review_id": "9yfZjdnd5p", "reviewer": "Reviewer_6Mxr", "summary": "VLMLight introduces a vision-language framework for traffic signal control that addresses part of the limitations of traditional rule-based and RL-only methods. The framework leverages a custom traffic simulator that supports multi-view visual inputs for real-time perception of vehicle types, spatial density, and dynamic motion patterns. LLM and VLM agents act as a meta-controller to dynamically switch between a fast RL policy and a deliberative reasoning branch for safety-critical scenarios. Experimental results demonstrate a feasible reduction in emergency vehicle waiting times while maintaining comparable efficiency in standard conditions.", "questions": "- **Justification for Latency**: The authors state that the system’s 11.5-second end-to-end runtime falls within an acceptable 13-second urban signal buffer. However, no citation or empirical evidence is provided to support this threshold. How was the 13-second buffer determined? Please provide a reference or data-driven justification to substantiate this claim.", "limitations": "See weakness and questions", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- **Safety-Critical Adaptability**: The dual-branch architecture ensures fast execution for routine traffic while activating dedicated decision-making for emergencies. This hybrid approach reduces the waiting time for emergency vehicles. The author successfully proposed an effective approach to addressing this gap in prior TSC systems.\n- **Open-Source Simulator for Vision-Based TSC**: The release of a vision-based simulator with dynamic multi-view rendering enables future research on perception-driven traffic control, fostering community innovation. While there are alternatives around [1], the authors may over claim their contribution as the **first** vision-based simulator.\n- **Innovative Integration of Vision and Language Models**: VLMLight is the first to combine visual scene understanding with LLM reasoning in TSC, enabling richer semantic analysis of traffic dynamics compared to traditional RL-based state representations.\n\n## Weakness\n\n1. **Scalability to multi-intersection networks**: The experiments are confined to single intersections. While VLMLight reduces emergency vehicle waiting time locally, its performance in coordinated multi-intersection scenarios—critical for real-world emergency response efficiency—remains untested. The framework’s reliance on high-fidelity visual inputs and LLM deliberation may introduce latency bottlenecks when scaled to city-wide deployments.\n2. **Justification for vision-language overhead**:\n   - **Redundancy in perception**: The paper does not include ablation studies comparing vision-language models (e.g., Qwen-VL, LLaVA) with simpler object detection approaches (e.g., YOLO) for emergency vehicle recognition. While VLMs offer richer semantic understanding, their computational overhead—approximately 5.12 seconds per inference (Table 8)—is significantly higher than detection-only models. It remains unclear whether such complexity is necessary, given that traditional object detectors may be sufficient for accurate emergency vehicle identification.\n   - **Cost-effectiveness**: The paper lacks discussion on the deployment costs associated with maintaining dual VLM/LLM stacks, potentially requiring high-end GPUs (e.g., A100) per intersection. Rule-based or reinforcement learning methods with emergency prioritization heuristics could offer comparable performance with substantially lower resource requirements. Additional experiments and rationale are needed to justify the proposed approach's cost and design complexity.\n\n[1] Jin, Zhijing, Tristan Swedish, and Ramesh Raskar. \"3d traffic simulation for autonomous vehicles in unity and python.\" *arXiv preprint arXiv:1810.12552* (2018)", "was_revised": true, "final_justification": "concerning the effectiveness-efficiency tradeoff of current model, I maintain my score", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive comments. Below, we respond point by point.\n\n> [W1] Scalability to Multi-Intersection Networks\n\nWe fully agree that multi-intersection coordination is an essential challenge in real-world TSC. Our decision to begin with single-intersection settings was driven by three reasons:\n- **Foundational Importance**, as highlighted by prior surveys [1,2], single-intersection control is the fundamental unit of TSC research and remains a crucial benchmark before scaling to larger networks.\n- **Modular Design for Generalization**, VLMLight is intentionally built as a modular framework: the vision-language meta-controller and dual-branch reasoning can be plugged into any multi-intersection coordination algorithm (e.g., Max-Pressure or MARL approaches). In fact, our experiments already include three distinct intersection geometries, demonstrating that the framework adapts to varied layouts without retraining.\n- We also acknowledge the reviewer's latency concern when scaling to larger networks. Recent studies [3,4,5] show that LLMs can now operate efficiently on edge devices, and our validation with the lightweight Gemma3 [6] model further confirms this. VLMLight maintains real-time performance with minimal degradation even under resource constraints, as shown below:\n\n**Performance Comparison (seconds)**\n|           |      | Songdo | Yau Ma Tei | Massy |\n|:---------:|------|:------:|:----------:|-------|\n| Gemma3-4b | ATT  |  89.81 |    41.38   | 64.79 |\n|           | AETT |  47.54 |    14.57   | 47.01 |\n|  VLMLight | ATT  |  87.14 |    39.80   | 60.84 |\n|           | AETT |  49.88 |    13.50   | 45.41 |\n\n> [W2] Justification for Vision-Language Overhead\n\nWe sincerely thank the reviewer for raising this important point. To address the concern about vision-language overhead, we provide a two-part justification: first explaining why a VLM is necessary, then clarifying how VLMLight minimizes reasoning and deployment costs.\n\n**(a) Why not rely on simpler object detection approaches (e.g., YOLO)?**\nOur objective extends beyond merely detecting emergency vehicles, we aim for scene-level reasoning to support safety-critical decisions. Traditional detectors like YOLO require lane-specific annotations and manual direction labeling for every intersection [7], making them brittle when adapting to new layouts. In contrast, VLMLight's vision-language module:\n- Adapts seamlessly to unseen intersection geometries without re-labeling;\n- Understands contextual anomalies, such as a fallen bicycle blocking a lane or an ambulance stopped in a non-standard position, reasoning that goes beyond binary \"is this an emergency vehicle?\" detection.\n\n**(b) Deployment cost and dual-stack complexity.**\nWe fully agree that efficiency is crucial for real-world deployment. VLMLight addresses this through its dual-branch architecture:\n- The fast RL branch handles the vast majority of routine traffic decisions, ensuring real-time performance.\n- The structured reasoning branch is only activated in rare, safety-critical scenarios (e.g., emergency vehicle prioritization), minimizing computational overhead.\n\nMoreover, VLMLight's modular design allows the VLM/LLM components to be swapped for lighter alternatives (e.g., Gemma3 or other models) as hardware constraints demand. Our released image-based simulator further empowers future researchers to experiment with different model sizes, balancing cost vs. accuracy without redesigning the entire framework.\n\n> [Q1] Latency Justification \n\nWe thank the reviewer for highlighting this important issue. To clarify why VLMLight's latency is acceptable for real-world deployment, we address it in three parts:\n\n**(a) Safety standards create a natural timing buffer.**\nPedestrian safety regulations require a minimum green interval to ensure safe crossing. Using the standard calculation formula from [8,9] (assuming a walking speed of 1.2m/s), the required pedestrian green times for our three test intersections are 25s, 23s, and 17s (based on crosswalk widths of 30m, 25m, and 17m). In addition, government guidelines (e.g., South Australia [10], Toronto [11]) specify a minimum 9s green phase plus 3s yellow, effectively providing at least a 12s reasoning buffer before any signal phase change.\n\n**(b) Latency mainly applies to rare, complex scenarios.**\nVLMLight's 11.5 s latency occurs only in the infrequent, safety-critical cases that require multi-agent deliberation. In the majority of traffic flow situations, the fast RL + VLM branch is used, which operates significantly faster.\n\n**(c) Lightweight model substitution further reduces latency.**\nVLMLight is modular, meaning the perception and reasoning agents can be replaced with lighter models. For example, when we substituted the VLM/LLM components with Gemma3-4B, inference latency shows in the following table. These results confirm that even in the most demanding scenarios, VLMLight stays well within the available safety margin, while in regular cases inference time is typically under 2s with lightweight models.\n\n**Inference Latency (seconds)**\n|           |         | Songdo | Yau Ma Tei | Massy |\n|:---------:|---------|:------:|:----------:|-------|\n| Gemma3-4b | Normal  |  1.61  |    1.49    | 1.45  |\n|           | Safety-critical|  2.38  |    2.29    | 2.21  |\n|  VLMLight | Normal  |  5.87  |    6.10    | 5.56  |\n|           | Safety-critical|  11.48 |    9.36    | 9.55  |\n\nThis means VLMLight remains well within safety margins, and we have added these references and calculations to the manuscript.\n\nReference\n\n[1] Wei, Hua, et al. \"A survey on traffic signal control methods.\" arXiv preprint arXiv:1904.08117 (2019).\n\n[2] Zhao, Haiyan, et al. \"A survey on deep reinforcement learning approaches for traffic signal control.\" Engineering Applications of Artificial Intelligence 133 (2024): 108100.\n\n[3] Xu, Jiajun, et al. \"On-device language models: A comprehensive review.\" arXiv preprint arXiv:2409.00088 (2024).\n\n[4] Friha, Othmane, et al. \"Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness.\" IEEE Open Journal of the Communications Society (2024).\n\n[5] Yao, Yuan, et al. \"Efficient GPT-4V level multimodal large language model for deployment on edge devices.\" Nature Communications 16.1 (2025): 5509.\n\n[6] Team, Gemma, et al. \"Gemma 3 technical report.\" arXiv preprint arXiv:2503.19786 (2025).\n\n[7] Lin, Jia-Ping, and Min-Te Sun. \"A YOLO-based traffic counting system.\" 2018 Conference on Technologies and Applications of Artificial Intelligence (TAAI). IEEE, 2018.\n\n[8] Koonce, Peter. Traffic signal timing manual. No. FHWA-HOP-08-024. United States. Federal Highway Administration, 2008.\n\n[9] Iryo-Asano, Miho, Wael KM Alhajyaseen, and Hideki Nakamura. \"Analysis and modeling of pedestrian crossing behavior during the pedestrian flashing green interval.\" IEEE Transactions on Intelligent Transportation Systems 16.2 (2014): 958-969.\n\n[10] Government of South Australia, Traffic Signal Standard: Signal Timings - TS001\n\n[11] City of Toronto, Standard Operating Practice: Pedestrian Timing at Signalised Intersections. Transportation Services"}, {"review_id": "xJZOyWj4xw", "reviewer": "Reviewer_Srph", "summary": "This paper introduces VLMLight, a Traffic Signal Control (TSC) framework that integrates vision-language meta-control with dual-branch LLM-based reasoning. In particular, it first uses  VLM to process multi-view images to detect the existence of a special vehicle to route to either a fast RL policy for routine traffic or a structured LLM-based reasoning branch for critical cases. The authors evaluate VLMLight against rule-based and RL-based TSC on data collected from three real-world intersections and show the reduction of waiting times for emergency vehicles by up to 65% over RL-only systems with a performance degradation in standard conditions for less than 1%.", "questions": "-can the method be adjusted to not degrade the regular traffic waiting time while still decreasing the special vehicles waiting time to some extent?\n\n-how do the use of VLMs impact the results?", "limitations": "yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Pros\n\n-well-written and clear\n\n-effectively reduce the waiting of special vehicles\n\nCons\n\n-the proposed method (using VLM to select between RL-based TSC and LLM-based TSC) is practical regarding the evaluated metrics (waiting time for regular vehicle/special vehicle) but relatively incremental.\n\n-the practicalness/impact of the proposed method can also be potentially limited as in practice 1% decrease of waiting time of regular vehicles can be significant due to their absolute majority.", "was_revised": true, "final_justification": "I am not fully convinced by the author's response and still have some concerns regarding practicalness of the current approach because of the 1% compromise on regular traffic (which takes the most majority of the traffic) and the lack of evaluation in multi-intersection networks.", "rebuttal": "We sincerely thank the reviewer for their constructive feedback and address each of the concerns and questions below.\n\n> [W1] On the concern that the method is \"relatively incremental.\"\n\nWe understand your concern. However, we would like to emphasize that VLMLight introduces two key innovations:\n- The first **image-based traffic simulator for TSC** and support multi-view intersection perception, enabling policies to reason about vehicle type, motion, and spatial density.\n- A vision-language meta-control framework that generalizes beyond emergency vehicles. While emergency vehicle priority is used as a primary case study, the same mechanism can handle other safety-critical events such as pedestrian safety, public transport lanes.\n\nMoreover, the impact is substantial: emergency vehicle waiting times dropped from 52s to just 2s (Table 1), while maintaining real-time performance. This goes well beyond what rule-based or RL-only approaches have demonstrated.\n\n> [W2] On the impact of a ~1% increase in regular vehicle waiting time.\n\nWe acknowledge that VLMLight introduces a small ~1% increase in waiting time for regular vehicles when emergency vehicles are present. This is an expected trade-off, as the system must temporarily reprioritize traffic flow to ensure that emergency vehicles can pass safely and quickly [1].\n\nImportantly, VLMLight maintains RL-only performance for regular traffic when no emergency vehicles are present. In addition, prior studies [2,3] focused only on emergency vehicle delay without reporting the impact on regular vehicles. In contrast, our work evaluates both sides, offering a transparent view of this trade-off.\n\nWe also note that the dual-branch design is modular, the fast RL branch can be replaced with an improved policy in future work, potentially further reducing this minor impact.\n\n> [Q1] Can the method be adjusted to avoid degrading regular traffic?\n\nOur current system already ensures that in the absence of special vehicles, VLMLight relies solely on the RL branch, achieving state-of-the-art regular traffic performance. When emergency vehicles are detected, a slight compromise (1%) is made for regular vehicles to enable rapid clearance of priority vehicles. This trade-off is not only inevitable but consistent with safety priorities in real-world traffic management.\n\nThat said, our architecture is designed for easy upgrade: future RL policies, heuristic scheduling, or hybrid optimization methods could be plugged into the \"fast branch,\" allowing improvements in regular vehicle flow while preserving the core safety benefits.\n\n> [Q2] How do VLMs impact the results?\n\nVLMs are central to VLMLight's success. They provide multi-view semantic understanding of intersections, detecting special events like the arrival of an emergency vehicle.\n\nAs shown in Appendix Table 3, larger VLMs significantly outperform smaller ones in scene interpretation accuracy, directly improving decision quality. This highlights that VLM-powered perception is not merely a detector replacement; it enables structured, interpretable reasoning over complex traffic scenes that conventional computer vision pipelines cannot achieve.\n\nReference\n\n[1] Nguyen, Dang Viet Anh, et al. \"Robustness of Reinforcement Learning-Based Traffic Signal Control under Incidents: A Comparative Study.\" arXiv preprint arXiv:2506.13836 (2025).\n\n[2] Zhong, Li, and Yixiang Chen. \"A novel real-time traffic signal control strategy for emergency vehicles.\" IEEE Access 10 (2022): 19481-19492.\n\n[3] Cao, Miaomiao, Victor OK Li, and Qiqi Shuai. \"A gain with no pain: Exploring intelligent traffic signal control for emergency vehicles.\" IEEE Transactions on Intelligent Transportation Systems 23.10 (2022): 17899-17909."}]

Author rebuttal:

Meta-review: After the reviews, rebuttal, and discussion two reviewers lean towards reject, two towards acceptance. The reasons for acceptance are:

- A new (open-source) simulator
- Original idea
- A good evaluation of the core ideas

Issues:

- Scalability, performance overheads
- Trade-offs in performance across vehicle types

The authors rebutted both issues well, and the AC sees no major remaining issues that would prevent the paper from being published.
Decision: Accept (poster)