RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Authors: Mingyuan Li
Jiahao Wang
Guangsheng Yu
Xu Wang
Qianrun Chen
Wei Ni
Lixiang Li
Haipeng Peng
Venue: ICML 2025 poster (2025)
Keywords: reinforcement learning
diffusion
traffic signal control
Primary area: reinforcement_learning
TL;DR: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control

Abstract: Reinforcement Learning (RL) optimizes Traffic Signal Control (TSC) to reduce congestion and emissions, but real-world TSC systems face challenges like adversarial attacks and missing data, leading to incorrect signal decisions and increased congestion. Existing methods, limited to offline data predictions, address only one issue and fail to meet TSC's dynamic, real-time needs. We propose RobustLight, a novel framework with an enhanced, plug-and-play diffusion model to improve TSC robustness against noise, missing data, and complex patterns by restoring attacked data. RobustLight integrates two algorithms to recover original data states without altering existing TSC platforms. Using a dynamic state infilling algorithm, it trains the diffusion model online. Experiments on real-world datasets show RobustLight improves recovery performance by up to 50.43\% compared to baseline scenarios. It effectively counters diverse adversarial attacks and missing data. The relevant datasets and code are available at Github.

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Mingyuan Li!
Jiahao Wang'
Guangsheng Yu² Xu Wang? Qianrun Chen 3 Wei Ni Lixiang Li
Haipeng Peng'
Abstract
to conventional expert systems in improving vehicular traf- fic flow (Wei et al., 2018). Deploying TSC algorithms in real-world settings typically involves using sensors, such as cameras and radar, to monitor traffic states, including vehi- cle counts and speeds. Sensors exposed in public areas are susceptible to noise attacked (Chen et al., 2018; Chowdhury et al., 2023b), leading to potential adversarial attacks. More- over, in extremely adverse weather conditions, these sensors are prone to physical damage (Laszka et al., 2016b; Lee & Park, 2012). This study delves into these sensor security challenges real-world TSC systems.
Reinforcement Learning (RL) optimizes Traffic Signal Control (TSC) to reduce congestion and emissions, but real-world TSC systems face chal- lenges like adversarial attacks and missing data, leading to incorrect signal decisions and increased congestion. Existing methods, limited to offline data predictions, address only one issue and fail to meet TSC's dynamic, real-time needs. We propose RobustLighh, a novel framework with an enhanced, plug-and-play diffusion model improve TSC robustness against noise, missing data, and complex patterns by restoring attacked data. Robustl.ight integrates two algorithms to re- cover original data states without altering existing TSC platforms. Using a dynamic state infilling algorithm, it trains the diffusion model online. Experiments on real-world datasets show Robust- Light improves recovery performance by up 50.43% compared to baseline scenarios. It ef- fectively counters diverse adversarial attacks and missing data. The relevant datasets and code are available at GitHub.
Adversarial attacks on the sensors can range from Gaussian noise and uniformly distributed random noise (U-rand) to more sophisticated strategies, including maximum action- difference attack (MAD) and minimum Q-value attack (MinQ) (Tang et al., 2016). When testing a TSC algo- rithm, its performance can significantly decline (Gershen- son, 2004; Chen et al., 2020; Zhang et al., 2022b) within the CityFlow (Zhang et al., 2019) simulation environment under various attacks. Such attacks compromise sensors, leading to inaccurate state observations and consequent mal- functions in TSC systems. When the sensors are intruded on or compromised, traffic disruptions could potentially occur, even leading to traffic safety accidents and substantial eco- nomic losses. To this end, it is crucial to develop innovative defense algorithms to mitigate the impact of sensor-related anomalies for TSC.
1. Introduction
1.1. Motivation
Improving traffic efficiency through Traffic Signal Control (TSC) has been established as an effective strategy (Wei et al., 2019b). Traditional TSC systems depend heavily on static, predefined expert system controls, lacking the flexibility of dynamically responding to fluctuating traf- fic conditions (Lowrie, 1990; Hunt et al., 1982; Webster, 1958). Recently, by integrating the methods of Reinforce- ment Learning (RL), TSC has demonstrated its superiority
1.2. Challenges
Various solutions have been proposed in the context of on- line applications address security issues faced by sen- (Sun al., 2021). However, these methods typically only address a single type of attack. For instance, Zhang et al. (Zhang et al., 2021) proposed a method to address an adversarial attack, and it significantly decreases perfor- mance as noise levels increase, especially in complex, high- dimensional state environments (Yang et al., 2022). Lin al. (Lin al., 2017) employed model-based methods, e.g.. Multi-Layer Perceptron (MIP) for data prediction to ad- dress data loss problem. Similarly, Mei al. (Mei et al., 2023) advocated for interpolation techniques during training
BBijjnng University of Posts and Telecommunications *University of Technology Sydney 'Lanzhou University. Cor- respondence to: Haipeng Peng <penghaipeng catphaappppn@uuu.duuu......
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
to solve missing data problem. Yang et al. (Yang al., 2023) proposed offline method to solve the security prob- lem of robotics. However, in TSC system that requires real-time decision based on traffic volume, the above meth- ods need to collect a large amount of data offline, which cannot meet the demand real-time dynamic change of TSC data. The challenges faced by the existing TSC systems can be summarized as follows:
Normal
Derrege
(b) Sensors
(c) Movement
Recent TSC algorithms demonstrate significant per- formance degradation when undergoing adversarial attacks or sensor damage, often resulting in traffic con- gestion, indicating lack of security resilience.
(4) Intersection
(d) Phase
Existing defense methods address only one or two types of attacks and lack comprehensive framework to address multiple security issues holistically.
Figure I. Definition of the TSC.
2. Preliminary
Current offline methods typically rely on the collection of large datasets to train static models, which struggle to make accurate decisions when faced with untrained real-time data.
2.1. Definitions of TSC
We use a four-way intersection, as shown Figure 1, to illustrate the concepts and summarize the definition of TSC.
Intersection. Each road network consists of multiple in- tersections, each with multiple, e.g., N,, road segments, denoted by (Inter1,..., Intern
1.3. Contributions
Diffusion models have recently achieved great results in image generation and RL control (Janner et al., 2022; Wang et al., 2022; Yang et al., 2023). The training of diffu- sion model consists of the noise addition and denoising processes. The denoising process has the potential to evade noise attacks undergone by TSC systems. Moreover, the diffusion models have a strong data generation ability that has the potential to solve the missing data problem in TSC systems. Our contributions in addressing these challenges can be summarized as follows:
Traffic perception. Each intersection has four directional sensors (e.g., cameras, radars) monitoring vehicles in three lanes, with different security states indicated by colors: or- ange for noise attacks, red for sensor damage, and green for normal operation, as shown in Figure 1(b).
Traffic movement. The vehicle passes through the inter- section from entering the lane (lane,,) to leaving the lane (lanCout). This traffic movement is represented as TM- = (luneeny laneoat), as shown in Figure ((c).
We propose RobustLight, a framework to enhance the robustness of online TSC systems, consisting of TSC agent and dynamic state filling (DSI) agent. The DSI agent uses a model-free RI. algorithm with an enhanced diffusion model to recover TSC data in real- time, ensuring optimal strategies without altering the original TSC algorithms. This architecture prevents data missing without changing the TSC algorithms.
Traffic signal phase.
Two movements i.c.. (TM, M,, * j) form phase, represented as pio = MT, TMy), as depicted in Figure 1(d).
2.2. Adversarial Attack and Physical Sensor Damage
We introduce the concepts of four classic adversarial attacks and the definition of TSC sensor attacks.
Gaussian noise attack. The Gaussian noise attack adds Gaussian random noise A within the scale or intensity k: to the state 8, represented as 5, = 8t + k NNu, o²)
The RobustI framework adopts two algorithms, de- noise and repaint, which leverage the trained diffusion model of DSI to defend against adversarial attacks and handle missing data TSC systems.
U-rand attack. The uniform random noise (U-rand) attack adds U-rand noise 4 within the intensity k to the state s, i.e., 8, 8, + k: U(I where / is the identity matrix.
Experiments show that RobustLight improves the aver- age travel time of existing TSC algorithms by 50.43% under various adversarial attacks and missing data see- narios, with state recovery closely matching pre-attack distributions, demonstrating its enhanced robustness.
MAD attack. The maximum action-difference (MAD) attack selects noise within a given range h: to maxi- mize the difference between two policies T-(sss) and TT(:18) in the policy space, denoted as &t 8t +
2

---

RobustLight:: Improving Robustness via Diffusion Reinforeement Learning for Traffic Signal Control
arg max (-1ss || (-))),
where Ba(s, k) is the 60 ball centered state s with radius k.
3. RobustLight
In this section, we elaborate on the proposed Robust framework, which integrates a new dynamic state infilling (DSI) algorithm, and two new processes, namely, the no- attack training process and the attacked testing process, into any type of the existing TSC algorithms to protect from data false injection and missing data. Specifically, we propose DSI to train an improved diffusion model. The denoise and repaint algorithms use the trained diffusion model of DSI to solve adversarial attacks and missing data problems. The integration of these algorithms improve the robustness of TSC in unreliable real-time traffic environments.
MinQ attack. The Minimum Q-value (adversarial) attack selects the minimum Q-value within certain range it and adds it as noise to the original state s, denoted as & St + arg mingg Q T (-13)).
Physical sensor damage. For each intersection, sensor dam- age due to weather or human factors causes state dimensions to be unobserved, represented as 8, Mask
2.3. Diffusion Process and Guided Diffusion
The diffusion model (Ho et al., 2020b) is probabilistic deep learning model that generates samples through diffusion and inverse diffusion processes.
3.1. TSC Algorithm
A TSC algorithm implements the basic functions of signal control, serving TSC agent interact with the traffic environment. This allows for the use of various TSC algo- rithms, including both traditional and RL.-based algorithms.
Forward process. In this process, the data is deformed by introducing random noise, which makes the data gradu- ally lose structured information and eventually turn into random noise. The forward diffusion chain gradually adds noise to the data zo (sampled from the distribution ((xr)) over T steps using pre-defined variance schedule 3e {3 (0, 1)3 given by
In what follows, we elaborate on the TSC algorithms (namely, MaxPressure (Cools et al., 2013), Advanced- Maxpressure (Zhang et al., 2022b), Colight (Wei et al., 2019a), Advanced-Colight (Zhang et al., 2022b), Mp- light (Chen et al., 2020), and Advanced-Mplight (Zhang et al., 2022b)). The specific definitions of state, action, and reward are given by
q (xx x1) = N (xx; / Vārxo, (1 σt. I),
where a, 1- and ā, I' 0g. Equation (1) al- lows for the use of reparametrization (Gu et al., 2022) to directly obtain the noisy data corresponding to spe- cific timestep t from the initial data without the need for multi-step iteration.
State: The Efficient Pressure (EP) (Wu et al., 2021) and Running Vchicle (RV) are the input state s.
Action: The traffic signal phase is action a.
Reward: The negative of the queue length is the reward r.
Reverse process. This process is assumed to follow Gaus- sian distribution probability, as given by
We represent the TSC state trajectory 7 as sequence, as given by
Po (x 1 xx) = / (x (xe, (): Eg (xe, ()), (2)
- (81, 82, 8: 8,
(4)
where ł indicates the RL timestep. We can update the RL- based TSC agent according to the Bellman function. Then, the Q-value in the Rl.-based TSC evolves as
where M4 (Xe, f) represents the mean of the Gaussian dis- tribution that needs to be predicted by the neural network, and Eg (xt,0 !) is the pre-defined variance. After training, the neural network predicts the noise term. We sample data from an isotropic Gaussian noise and run the reverse diffusion process from f T to ===0 O.
Q(s,a) - a(s,a)+a r max Q's'',a') - a'Ea
where a represents the learning rate, y is the discount factor, and Q'(s', a') denotes the target Q-value for the next state- action pair. For the TSC algorithms based on RI., a replay buffer is used to store tuples (s, s'),s')
Guided Diffusion (Kim & Oh, 2022) extends diffusion by introducing external guidance during the reverse process. It controls the generation of specific content. Classifier-fie free guidance (Ho & Salimans, 2022) is a widely considered type of guided diffusion, which adds condition c to the 1s0 network in the diffusion reverse process, as given by
3.2. Dynamic State Infilling (DSI) Algorithm
We design DSI algorithm using an improved diffusion model as the policy to reduce noise in online model-free RL. The TSC agent's 's replay buffer is used to update the policy and adapt to real-time environments. The state, action, and reward of the DSI agent are defined as follows:
guided 1
(Iz, t, c) = ( t) + ww (Ze, t, (,c Mo t)),
(3)
where w is a weighting coefficient that controls the trade-off between conditional and unconditional generations.
3

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Online no attack training process
Online attacked testing process
TSC algorithm
TSC Agent
Action
Apent
Action
Runsing velidde
Update
City flow envirunment
Noise atfack Sensor
Attaclied sessurs
(5,0,1,8)
Sample for update
Store
Local cffective
Replay buffer
Recovered state
Denoise algorithm
attach
Agent
Stado represetios
Update
Detect
(S,.42
Dynamic state in infilling algorithm (DSI Agent)
Critie network (Linet)
Repaint algorithm
denoise
Sensor damaage
Fereder
x
Musio
Update
(p(
DSI training samples
Actor nenwork(Unee)
(S
Mask
F-11 (5g. -
Recover missing data
moise
Diffusiio model
Figure Robustl.ggh uses a two-process framework with four algorithms: the and DSI algorithms train on normal data to enhance efficiency and robustness, while the Denoise and Repaint algorithms mitigate noise and infer missing data during attack testing.
State. We represent the DSI agent's state as S. (o,-1, 7/-1), where a,-1 is the previous TSC action and 7 = $1-1) is the TSC state trajectory.
where fa(A,) VarÅ, and Io A S.
can be modeled as Gaussian distribution, as follows:
Action. The DSI denoises the noisy or missing state & to the recovered state åt. Hence, the action time / can represented A, št.
N (A. (A :14 S..1.)) Eg ((1:))
(9)
According to (Ho et al., 2020b), the denoising process for each step in the diffusion is expressed as
Reward. The reward is the negative of the queue length. The DSI agent consists of an actor and critic network.
Critic network. In the critic network parameterized by y,, a U-Net architecture (Ronneberger et al., 2015) identical to that of the actor network employed. The final layer of the critie network produces the Q value. The network is updated using the Bellman error, as given by
4 Vag
3
/ /,c. (10)
A 1
co Co St.
Vas(1- - ā.)
To handle denoising tasks with improved diffusion, we use the beta schedule (Zhihe & Xu) 3; = a; गमाक a I a, and 3, اسااا B, to address small to medium-scale noises, where en is the noise network. Unlike other beta (Xiao et al., 202 Nichol & Dhariwal, 2021a; Ho et al., 2020a) schedules that generate data from pure noise, our approach employs the U-Net architecture and a conditional generative diffusion model for noise pre- diction.
L, = r + (S A+t1) (S,,,,
(6)
where À is the discount factor.
Actor network. For the actor network, we use action gradi- ascent update the control policy (Yang et al., 2023)
A = A; + 7/ AS.
(7)
Lemma 3.1.
The lemma basis of diffusion for denoising to minimize the sum entropy of the denoised data:
where y is the learning rate the action gradient ascent. We employ an improved diffusion model denoise and predict the original state Å, 8, through the reverse process:
Lg = Σ EqANI- log po (
(11)
1=1
Ag (A(k = fa()) II (A) 1 (8)
which can be optimized using the variational lower bound

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Algorithm I The training process of RobustLight
Algorithm 2 Repaint algorithm of RobustLiggh
Initialize: DSI algorithm Qu critic network, Qa ac- tor network, Replay buffer and target network
1: Input Ai. 1,
2:
for i to K do
Q.- 20-09---
3: for
to U do
1: for w 1 E do
4:
<~ Ao( if > 1, else = 0
2: Get init state So from Cityflow
5: Get A known by Equation (13)
7:-1
3: for to T do
2 z~ () if 1, else 2 =
4: Execute TSC algorithms, including Colight,
Get Á
Lunknnon by Equation (14)
Advanced-Colight, Maxpressure,... etc.
8:
Get recovered A by Equation (15)
5: Training the TSC agent by Equation 5.
9:
if u < U and i > then (
6: Sample batch 5 = (T", a) and A = from R.
10:
A ~ ~
7:
Update critic network Qp by Equation 6.
11: end
8:
for n = 1 to N do
12: end for
9:
Execute gradient ascend by Equation 7. end for
13:
end for
14:
Return A, = A4
11:
Update actor network Qo by Equation 12.
12:
Update target critic network by:
13:
& mip + (1 77)pp
sion timestep. We perform the denoising process in each i timesteps to predict the original state &f using Equation (8).
14:
Update target actor network by
15:
A- = r70 + (1 77)6
16:
end for
3.4. Repaint Algorithm
17:
end for
Return Actor network Q9
The concept of utilizing diffusion for image repaint is in- spired by (Lugmayr et al., 2022). The core idea is to the known sensor data to infer the unknown sensor data. We adapt and refine the image repaint process to the TSC and introduce the repaint algorithm to interpolate the damaged or missing TSC state, as illustrated in Figure 2.
(VLB). For the detailed derivation process, please refer to (Zhihe & Xu; Ho et al., 2020a).
Expanding Lg, we obtain the following Non-Markov loss to update the actor network:
We use well-trained conditional denoising diffusion prob- abilistic model, based on DSI (distinct from (Zhihe & Xu)), to effectively restore the original state Å 8, where the unknown part is represented by m O Á with m being the mask matrix, and the known part is (1 m) @ A'-1 stands for unmask matrix. We note that the reverse pro- cess diffusion from Á' to A depends solely on A', as long as we maintain the correctness properties the corre- sponding distribution. Therefore, we update the known state (1 m) @ A According to Equation (1), sample the known state at any diffusion timestep i, and use Equation (2) for the unknown state. We use the following expression for one reverse step:
Lg
Nur IED,
++M-1
(oo(A), St-1,)
-c(ls
ci|l= Σ
+
(o i) cm
(12)
where Sm-1 (am 1) and - 1 is the predicted state trajectory. This loss function Lg balances the trade-off between the current RL timestep and future RL timesteps, aiming to minimize the accumulated error over time. De- tailed training algorithm summarized in Algorithm 1.
Á-11 tnnown = Vos VT,,At,,oonn + vi- - 046, (13)
3.3. Denoise Algorithm
In the testing phase, we may unintentionally expose the sen- sors within the TSC algorithm to various attacks, including Gaussian, MAD, U-rand, and MinQ attacks.
1 "uunknown
Vas Lunknown
Var(1 - (i))
co(Ai, St-1,, + 1
Biz, (14)
In the denoise algorithm, we use the trained actor diffusion model from the DSI algorithm to recover the original TSC state A, = 8, We use the diffusion reverse process to de- noise. As described in Equation (10), we input (S,, A.) and the diffusion time step i into the actor diffusion net- work. In each diffusion timestep, the data currently under a noise attack denoised obtain the data for the next diffu-
At-1 m 71-1 knnown + + m)
A unknown (15)
Thus, A/-1 is sampled using the known state in the given state m @ A and Å'-1 unkknown is sampled from the condition model given the previous iteration Á'. These are then com- bined into the new sample A'-1 The repaint algorithm is summarized in Algorithm 2.
5

---

RobustLight: Improving Robustness via Diffusion Reinforeement Learning for Traffic Signal Control
Table Performance of ATT in JiNan, HangZZou. "." implies that traditional algorithms are not adapted to MinQ and MAD attacks. Our RobustLight recovers the state of traditional and RL-based TSC algorithms to evaluate the performance.
Dataact
Notic Type
Noise
FiseTTiee
MuxPeesae Ribs
Advunced MuxPessuee
Advanced MpLiipht
Colight
Advunced CoLight
Robssiliiht
RebunLight
Robusdigh
Robustiide
Gassdon
35
301 301.9782.14 14
246, 01s2.15
40..411133 52
19.75
302, 26
287.7661.55
282.35e3.90
330,57:2.6
294.5762.02
4,0
305.8562.11 92
245, 9522.69
250 25e2. 17
289. T1al.59
377 59.50
314.97226.20 28
205.0722.27
338, 13.5233
298 26
U-adl
3.5
325.0982.65 65
319 27.17
312.27e2.92
92
306.12e1.79
558.60 56 69
349 499.9922.50
400 41:24.5
312. 172.27
449, 122112211110
358 722002
Jono
4.0
438. 10.000
531.58
322.93:2.67
319.18a2.91
301.32:1.97
364.90
552.6347885
460982:4779
9.5
487
MAD
4.0
146244444
287 187.04:2.99
555 11664
279.1548.22
500.0967.11
24
MinQ
3.5
313.73611
291.5NeM.13
083. 51
277.9363.47
344, 277
323.25e30055
40
295:1226.06 06
716.594134.92
281.1341.96
394.63347.55
351.52=10.44
Gassson
3.5
275 126132
270, 1647.7
767 7660 91
463-414588
272.1232:25
209 288.0/41002
256.8342.06
40
279.6941MM
374.18:1:25 25
372.1
26
271.8762:03
550.17e195M6
379.
293.55+1.00 300
261.63+3.66
461 461.57+129.55
294
U-rand
3.5
259 222.3442.22
286.71=2.56
293.40e1
289.0662.00
454,32=1 18.56
400,39950061
300.6627.67
987704442.0 02
561 001.02252.33
JcNons
4.0
383.01:0.00
300 07407422.0 96
291,1542.8
796
294.642.442
500.65 122
36.00447..33
554,11480009
206.0346.99
1015-79-43.75
596.24667.4 AK
MAI>
3.5
195 MM 66
189.54 1.53
554,05907 67
2657742.557
451.77417.55
244.76-1.42 43
4.0
5H41.8 45
264.52-133 35
150,26491355
267.0044
367.0362.00 899
474.16417.21
MinQ
3.5
733 233.60
653,052210001 01
283.01
726
744.21
226.85=262.56 56
267.34 65.33
273.1364.52
515.344167.85
329-51-14500 167.39
300 6...5.9.0
Gasedon
355
352.0122.01
532,45e1.255
564:00=105.25
356.3366.3 29
62..2255.9 96
4000 33
45
4,0
335.9762.49 49
326.5921.36
331.98a1.57
339, 139.35
490.72a90..4
24
366.62250.35 59.39
37L.97812.44
337.58
137.5845.60 68
510.58a25.17 17
331 31-66
U-ndl
3.5
356.51a3.72 72
338.4241.74
354.913331
297.34x1.30
373 373.85634005
325.18:3.17
647.64254.89
435 33 20.02
717.12470008
473 473.85=33..8
HongShow
4,0
495.5240.00
361 5824.7
341.1612.21
359.6123.34
350.8613.12
764.2524
330,47:3.11
475.49t20.78
441.33:17.05
738.51663.8
487.74:31144
1.5
309.2410.94
MAI
4.0
782.250121105
320.91-2275
419.07618.76
331,4768.42
091 991.69939976
352.586871
MisQ
3.5
77808478899
420.66287.75
317.3623.96 466.14470.37
449.54e822 22
406.3266.41
4,0
76488827.52
435.53269.87
460.29662.45
331.9363.91
482.9627.25
430.5168.49
4. Experiments
Dlwaaccdd-axxpressuree (Zhang et al., 2022a), which uses running and waiting vehicles to choose the phase; and Max- pressure (Gershenson, 2004), which uses waiting vehicles to choose the phase.
We conduct experiments with real-world datasets to evaluate RobustLight's generalization and efficiency on NVIDIA P100 hardware, using the hyperparameters in Table 10.
RL-based Methods: For RL benchmarks, we consider Co- Light (Wei et al., 2019a), which uses waiting and neigh- boring vehicles to select the signal phase; Advanced- CoLight (Zhang et al., 2022a), which employs waiting and running vehicles with a graph attention neural network; and Addanccd-Mpllige (Zhang et al., 2022a), which uses the FRAP (Zheng et al., 2019) model for signal phase selection.
4.1. Datasets
We use real-world traffic flow and road topology datasets for our experiments, with Cityflow (Zhang et al., 2019) as the simulator to evaluate Average Travel Time (ATT) and exit points with a simulation time of 60 minutes for all vehicles. The datasets include vehicle start and end points, following fixed motion model. Seven traffic datasets from three citics JiNan and HangZhou (China) and New York (Zheng et al., 2019) (USA) are used.
RobustLight Method: RobustLight integrates the base methods of traditional and RL-based TSC algorithms to recover data in real-time, and then evaluates the ATT under different sensor noise attacks and sensor damage. Results are presented as the average of ten independent runs.
JiNan Datasets: The JiNan road network consists of 12 intersections (in a 3 x 4 grid). It includes three traffic flow datasets: , JiN JiNang, ang, and Ji.ang.
4.3. Results
HangZhou Datasets: The HangZhou network encompasses 16 intersections (in 4 x grid) and features two datasets: HangZhou, and H ZanuZhou..
This subsection presents the results of our experiments, eval- uating RobustLight's performance under various conditions, including resilience to noise attacks and sensor damage, using ATT on real-world traffic datasets.
The New York network features a more complex structure with 192 intersections (28 x grid) and includes two datasets: N Newyorki and Newyorks.
New York Datasets:
4.3. 1. NOISE ATTACK ON STATE RESULTS
Table 15 summarizes our experimental results for the JiN an and HangZhou datasets based on the ATT met- ric. The noise scale range is based on the value of l for state noise attacks, as described in Section 2.2. The re-
4.2. Compared Methods
Traditional Methods: These methods include Fixed-
Time (Webster, 1958), which uses a fixed green phase time;
6

---

RobustLight: Improving Robustness via Diffusion Reinforeement Learning for Traffic Signal Control
Table ATT in JiNan and HangZhou: 25% refers to missing data sensorw, and 50% % refers sensorw and sensore. E.
Dutanct
Mask Scale
FixedTine
MaxPicsure
Advunced MaxPiesuure
Advunced MpLiphi
CoLigh
Advanced CoLight
hore
basd
KehentLigh
basd
Rebustigg Light
booe
Robost.agh
hase
Kotustl.gh
b9se
RobustLight
3Narl
25%
438.1 1.0000
586 74:0.00
324 224.31
553,0400.00
902.94:9.43
396 27..2766669 93
337.63:55.255
400
10.66
372.93:7.48
326. 12..52
298.35116.32
50%
798.90
613 23.234210 105.51
1061.9250000
548
1052 63
642.02e111
849.61
766.91 15
732.37
680.37497.32
JNaa2
300 200.00000000
272 5140.00
100000
701
250 269.16
06-86-665500 06
289.5647.99
83..00
1309:9760.00
985 04
130.94
3Naa3
25%g
363.01a0.00
289.8140.00
54005140100
124,356225644
16K.22116..77
367.9469933
303,86551.43
301.62438.22
45:000
574.97.69.0
1100,5740.00
212.45
947,84+190.54
567.37:75.16
362
156.57416..00
407.68%78.39
HangZhoul
25%
405.57:00.00
369:7760000
350.8:20.22
513.1550.00
371.64425.35
510.37:92.75
372.12621.49
450.36233.44
424.36213.22
401.87=448.31
325.26:12.67
50%
722.4143100
714.12:76991
119% 100000000000
752.66639986
117097671.88
515.77618192
756322:90.99
542.91=10K22
442,244255.M
$41.9661122.6
HongZhon2
25%
406 466A5-0.00
572: 12:00.0
353.8563.666
405 27.0.0
366.6454.64
362.93:15.36
350.26:10.22
307.66:20.68
390.43:10.2 10,37
376 378.56:26686
275.64:22.332
50%
533.6140.00
447.91::70.34
781,5760.00
542.372161.07
639,524121445
532.922157.32
663.22:90.52
470.91:34382
508.92230.27
444.37661.24
sults reveal that the incorporation of Robustlight leads to improved performance across all methods, demonstrating its effectiveness in mitigating various types of noise attacks.
Table 4. Performance Comparison with DiffLight
Ditnact
Naiss/Maak
Rubusilight
PSNR
MAF
M[aan1
U-sand Noise
(3,5)
Advanced MaxPeesse
DetLight
Yes
No
7,30
6.71
5.42
6.26
304.34
310,92
4.3.2. SENSOR DAMAGE ON STATE RESULTS
HangZhotl
U-ied Noise (3.5)
Advasced Mure
D.mLight
Yes
0.85
7.71
5.12
7.95
297.34
361.31
18am1
Smer Darnays
(25%)
Advanced Golight
DillLight
Yes
No
9.38
7.66
0.89
1.25
304.13
366.05
We compare traditional and RL-based TSC algorithms, focusing on deliberate sensor attacks on sensorw and sensorg, with data from these directions masked to simu- late damage. We use the Repaint algorithm within Robust- Light for data completion, allowing us to evaluate the TSC algorithm's 's performance before and after this process, as shown in Table 2. Our findings lead to several key conclu- sions. Under noisy conditions, MaxPressure outperforms Advanced-MaxPreesuun M with an 11.6% average improve- ment across all datasets. Advanced-CoLighh surpasses Co- Light with an 18.4% improvement. The RobustLight al- gorithm enhances performance in nearly all methods, with Advanced-Col ightt showing a 10.3% improvement. Follow- ing sensor damage in sensorw and sensorg, g, all methods perform worse than FixedTime, but our algorithms effec- tively recover missing data and still outperform FixedTime with damage in one direction, highlighting the importance of addressing sensor damage in real-world TSC deployments.
Hong2houl
Sevor
(29))
Dim igh
You
22.96
18055
1.37
1.81
306.56
372.55
Figure 3. Ablation of RobustLight based on Advanced-Coligee in JiNan and HangZhous under Gaussian noise (scale 3.5).
To validate our approach with DiffLight (Chen et al., 2024), we setup involved randomly masking data from Kriging Missing (12.5%, monngl-inttesection-eeeee dom Missing (12.5%, full-intersection failure). As demon- strated in Table 4, our method effectively addresses data missing scenarios (Kriging and random missing) while also exhibiting robust performance under noisy data conditions.
failure) and Ran-
Table 3. Performance comparison with other henchmarks
Dusser
Noka
Nova Scale
Rchadiige
Lieer Bora
Diliuce
4.3.4. ABLATION EXPERIMENTS
Glassiz
2210443001
4.
32533366555
9141182157
We conduct ablation experiments on RobustLight to assess the impact of each component. "w/o linear beta" refers to the experiment without the linear beta schedule, "w/o markov- loss" excludes the Markov loss, "w/o omits the original diffusion model with Markov loss, and "w/o ori-dif-non seeef-non-markeee markov-loss" excludes the original diffu- sion model with non-Markov loss. Showing the effective- ness of RobustLight. 4.3.5. ROBUSTNESS RECOVERY ANALYSIS
Scnae
25%
35006117.322 32
1055399116659
Dasage
50%
730 37:52.55
188.7797.3
600 1233643 43
1068/775100.11
Mong
4.
50135.25.11
331.31.345
5111062234
134026472622
-01.57043.31
Dasge
50%
$43.99235.35
541.54 16113.6
351.44240116 16
163 22144
4.3.3. OTHER METHODS COMPARISON EXPERIMENTS
We compare the native Diffusion model with three Beta schedule methods (Xiao et al., 2021b; Ho et al., 2020b; Nichol & Dhariwal, 2021b) and LSTM model (Sun et al., 2021) on real-world datasets. As shown in Table 3, our RobustLight outperforms the native Diffusion model in all noise attack scenarios, demonstrating its effectiveness.
We analyze RobustLight's is effectiveness in recovering the original data using t-SNE (Van der Maaten & Hinton, 2008) plots. In the plots, yellow represents the original state distri- bution, blue represents the recovered states, and red shows
7

---

RobustLighht: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Nodia deattttion
Newyork!
Newyork2
Average Whit Tanvf))
Average Wait Tuand))
JINan
Guadd 000
Gsard
L40h Scoee Bompphhh
Figure 6. Performance of RobustLight based on Advanced- Maxpressure in Newyork transfer by JiNanj.
HangZhou
dataset and test it the Newyork dataset. As shown in Figure 6, transferring the model to Newyork effectively mitigates noise attacks and demonstrates strong state restora- tion in different urban traffic environment.
5. Discussion
Figure 4. State visulization of RobustLight based Advanced- Colight in HongZhou, and JiNan..
Q1. Is there any challenge apply the diffusion model? We evaluate the time consumption of the DSI algorithm (Ta- ble 17) to assess its computational costs and impact TSC algorithms. As the noise range increases, more diffusion steps are needed, resulting in longer recovery times. A key challenge in large-scale scenarios is slow inference, driven by both the diffusion steps and the increasing number of intersections. To address this, we plan to reduce diffusion steps using DDIM and divide intersections into smaller sub- regions, each with its actor diffusion model, to speed up inference. This discussion highlights the trade-off be- tween accuracy and speed in diffusion models for real-time TSC systems and suggests ways to improve efficiency in large-scale deployments.
JiNanl
HangZhoul
Figure 5. Violin figure of normal, noise and recovery data.
the distribution with Gaussian noise (scale 3.5). Figure shows that the blue distribution is closer to the original, demonstrating RobustLight's strong robustness. Addition- ally, violin plots in Figure 5 show that the recovered data distribution closely matches the normal data.
Q2. there any report about cyber attacks on traffic control systems? Traffic sensor attacks (Chen al., 2019; Chowdhury et al., 2023a; Laszka et al., 2016 demon- strated at DEFCON 22 by Cesar Cerrudo, highlight vulnera- bilities in traffic control systems, where hackers target street magnetic sensors. Discussions on platforms like Quora ("hack traffic lights") further emphasize concerns about sen- sor safety. As Al systems become more prominent, ensuring their security is crucial, especially given the additional risks from extreme weather interference.
To directly evaluate the recovery effect, we define the metric Edenoise = N E' s s). Where N the number of intersections, T the running count, is the recovered state, and represents the original state. is the original state. This metric measures the ability to restore data by the absolute difference between the original and reconstructed states. As shown in Table 5, RobustLight exhibits strong recovery performance.
Q3. How to detect noise and missing data? To detect noise and missing data in our study, we use a combination of attack detection models and statistical methods. We assume that attacks, caused by factors such as weather conditions and radio interference, have been pre-detected, with the at- tack detection model identifying noisy or missing data based abnormal sensor readings. Noise is detected through out- lier detection and variance analysis, while missing data is identified by checking for null or NaN values and analyz- ing missingness patterns across sensors. Once detected, we handle noise using denoise algorithm and missing data through repaint algorithm of RobustLight, ensuring data
4.3.6. MODEL GENERALIZATION
To assess generalization, we train diffusion model based on the Advanced-Maxppesssiee algorithm using the JiNan, Vanj
Table 5. Edenoise the smaller the value, the better.
Type
Colight
Col
hod
Bohheilight
Robantl iebs
JoNan,
Gazadan
35
399 15471311
284-16431.30 21.36
30% 16
116.1511..6
628-36113382
1814.56131.56
1186-43144.1
Mang
Unned
1194311-1337
515.67616.31
2417.446:9944

---

RobustLight: Improving Robustness via Diffusion Reinforeement Learning for Traffic Signal Control
in TSC vary from value-based (Abdulhai et al., 2003; Wei et al., 2018), policy-based (Mousavi et al., 2017), actor- critic frameworks (Aslani et al., 2018; Wu et al., 2022), with state and reward designs incorporating features like queue length (Varaiya, 2013; Wu et al., 2021; Li et al., 2025), ve- hicle counts (Wei et al., 2019a; Xu et al., 2021), or travel time (Zheng et al., 2019). Advanced-Colighe method (Zhang et al., 2022a) stands out, achieving state-of ootte--o--hhharrr results by using running and waiting vehicles model lane capacity relationships.
Table Defection Rate and Throughput Comparison
Detection Rate
Dataset
Base
ATT
Base
Throughput
J:Nani
487
297
5812
6154
80%
H ZangZhou
463
326
2888
2938
60%
J&Nami
487
328
5812
6131
ZangZhou
463
331
2888
2930
integrity and robustness in the presence of environmental disturbances.
Robust RL. In RL, disturbances cause errors, leading to Robust RL for improved reliability, split into "training- time" and "testing "resting-time" robustness. "Training-timer adds noise during training for adaptability (Zhang et al., 2022c; Ye et al., 2023). "Testing-time" trains in clean settings and tests under disruptions (Yang et al., 2022: Panaganti et al., 2022). Our RobustLiggh focuses on "testing-time robustness for real-world attack resilience. "Testing- -ime" robustness covers state, action, and transition/reward per- turbations. State perturbation uses neural networks and SA-MDP (Zhang et al., 2021; 2020) or conservative ac- tions (Yang et al., 2022). Action perturbation employs ad- versarial training (Tan et al., 2020) or optimal policies (Liu et al., 2023). Transition/reward perturbations use Markov games (Pinto et al., 2017; Gleave et al., 2019) or robust Bellman operators (Panaganti et al., 2022). We prioritize "testing-time" robustness for real-world attack resilience. Diffusion Model. Diffusion models, first used for image generation (Ho et al., 2020b), excel in RL control tasks. Diffusion RL splits into online and offline settings. On- line, Yang et al. (Yang et al., 2023) applied diffusion as policy with model-free control. Offline, Ajay et al. (Ajay et al., 2022) generated trajectories for decisions, and Wang et al. (Wang et al., 2022) introduced Diffusion-QI, merging TD3+BC (Fujimoto & Gu, 2021) with behavior cloning. Online TSC algorithms (Wei et al., 2019a; Chen et al., 2020; Zhang et al., 2022a) falter under state attacks and missing data, risking congestion. RobustLight, our proposed algo- rithm, enhances online TSC resilience by recovering data without altering existing systems, offering a novel solution for research and industry.
Q4. If the RobustLight rely on the accuracy of detec- tion? To evaluate its reliance on detection accuracy, we integrated TP-FDS (Sarteshnizi et al., 2023), method that detects anomalies by comparing new data distributions with historical data from the same period, achieving an AUC of 96% and an Fl score of 76%. TP-FDS identifies anomalies through multi-sensor cross-referencing (c.g., cameras and radar) or rule-based methods, such as detecting queue increase from 3 to 5 during a north-south green light. Minor fluctuations, like queue changes from 3 to 5, minimally im- pact system efficiency. To assess Robustl.ggh 's dependence on detection accuracy, we conducted experiments simulating real-world scenarios with detection rates of 80% and 60%, as shown in Table 6. The results show that RobustLight has robust performance in most detection rates.
Q5. What are the practical considerations and potential barriers to real-world implementation? Our framework supports both distributed and centralized deployments: the distributed approach employs a federated learning architec- ture with central training and edge-basse based parameter updates, while the centralized solution addresses scalability through nigh-performance servers and algorithmic optimizations like DDIM (Song et al., 2020) acceleration. For comprehen- sive coverage of intersection failures (Kriging) and sensor- specific issues (Random missing), we recommend a hybrid approach, implementing centralized data missing recovery algorithms alongside edge-based noise reduction using cost- effective hardware for accelerated denoising. This balanced strategy ensures robustness, real-time performance, and at- tack resilience across all deployment scenarios.
7. Conclusion.
6. Related Works
In this paper, we introduced RobustLight, designed ad- dress abnormal TSC tasks. By leveraging the denoising properties of the diffusion model, RobustLight effectively handles noise interference and incomplete state information in real-world environments. Empirical results demonstrate its robustness and effectiveness, significantly enhancing the security of RL-based TSC strategies and strengthening tra- ditional TSC algorithms, thereby improving the safety and integrity of TSC systems.
TSC Algorithms. Since the introduction of static fixed- time plans in 1958 (Webster, 1958), TSC systems have evolved significantly. Systems like SCOOT (Hunt et al., 1982) and SCATS (Lowrie, 1990) rely on expert-designed plans with predefined thresholds, lacking dynamic adapt- ability to changing traffic conditions. The advent of RL methods has marked a paradigm shift, leveraging real-time traffic data to optimize signal management through trial- and-error, outperforming traditional approaches. RL models
9

---

RobustLight:
Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Acknowledgements
traffic signal intrusion detection. Sensors, 23(10):4646, 2023a.
We sincerely thank our collaborators for their invaluable feedback and meticulous proofreading of this manuscript. This work was supported in part by the National Key R&D Program of China (Grant No. 2024YFB2906503) and the National Natural Science Foundation of China (Grant No.
Chowdhury, A., Karmakar, G., Kamruzzaman, J., Das, R., and Newaz, S. S. An evidence theoretic approach for traffic signal intrusion detection. Sensors, 23(10):4646, 2023b.
62032002).
Cools, S.-B., Gershenson, C., and D D'Hooghe, B. Self- Organizing Traffic Lights: Realistic Simulation, pp. 45-55. 2013. ISBN 978- 778-1-4471-5113-5.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Fujimoto, S. and Gu, S. minimalist approach to offline
reinforcement learning- Advances in neural information processing systems, 34:201 32-20115,, 2021.
Gershenson, C. Self-organizing traffic lights. arXiv preprint nlin/041 1066, 2004.
References
Abdulhai, B., Pringle, R., and Karakoulas, G. J. Rein- forcement learning for true adaptive traffic signal control. Journal of Transportation Engineering-axce, 129:278- 285, 2003.
Gleave, A., Dennis, M., Wild, C., Kant, N., Levine, S., and Russell, S. Adversarial policies: Attacking deep reinforcement learning, arXiv preprint 190iv 2019.
Gu, S., Chen, D., Bao, J., Wen, F., Zhang, B., Chen, D., Yuan, L., and Guo, B. Vector quantized diffusion model for text-to-image synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10696-10706, 2022.
Ajay, A., Du, Y.. Gupta, A., Tenenbaum, J., Jaakkola, T., and Agrawal, P. Is conditional generative model- ing all you need for decision-making arXiv preprint arXiv:2211. 15657, 2022.
Aslani, M., Mesgari, S., Seipel, S., and Wiering, M. Devel- oping adaptive traffic signal control by actor-critic and direct exploration methods. Proceedings of the Institution of Civil Engineers- Transport, 172:1-25, 01 2018. doi: 10. 10.1 10.0//r/tr... 1...n.1....0 17.00085.
Ho, and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-68511 6851, 2020a.
Chen, C., Wei, H., Xu, N., Zheng, G., Yang. M., Xiong, Y., Xu, K., and Li, Z. Toward thousand lights: Decentral- ized deep reinforcement learning for aarge-scale traffic signal control. Proceedings of the AAAI Conference on Artificial Intelligence, pp. 44114-322, 2020.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020b.
Hunt, P., Robertson, D., Bretherton, R., and Royle, M. C. The scoot on-line traffic signal optimisation technique. Traffic Engineering & Control, 23(4): 1982.
Chen, H., Jiang, Y., Guo, S., Mao, X., Lin, Y., and Wan, H. Difflight: partial rewards conditioned diffusion model for traffic signal control with missing data. Advances in Neural Information Processing Systems, 37:123353- 123378, 2024.
Janner, M., Du, Y., Tenenbaum, J. B., and Levine, S. Plan- ning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.0998 09091. 2022.
Chen, Q. A., Yin, Y., Feng, Y., Mao, Z. M., and Liu, H. X. Exposing congestion attack on emerging connected vehi- cle based traffic signal control. In NDSS, 2018
Kim, K. and Oh, S. J. Diffusion graph convolutional net- works with adaptive diffusion time. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI), 2022.
Chen, X., He, Z., and Sun, I.. bayesian tensor decomposi- tion approach for spatiotemporal traffic data imputation. Transportation research part C: emerging technologies, 98:73-84, 2019.
Laszka, A., Potteiger, B., Vorobeychik, Y., Amin, S., and Koutsoukos, X. Vulnerability of transportation networks to traffic-signal tampering. In 2016 ACM/IEEE 7th Inter- national Conference on Cyber-Physicae Systems (ICCPS), pp. 1-10. IEEE, 2016a.
Chowdhury, A., Karmakar, G., Kamruzzaman, J., Das, R., and Newaz, S. An evidence theoretic approach for
10

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Laszka, A., Potteiger, B., Vorobeychik, Y., Amin, S., and Koutsoukos, X. Vulnerabiliity of transportation networks to traffic-signal tampering. In 2016 ACM/IEEE 7th Inter- national Conference on SybeerPhyssien Systems (ICCPS), pp. 1-10. IEHE, 2016b.
Pinto, L., Davidson, J., Sukthankar, R., and Gupta, A. Robust adversarial reinforcement learning. In Interna- tional Conference Machine Learning, pp. 2817-2826. PMLR, 2017.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assister Incervention-MICCA/ 2015: 18th International Confer- ence, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234-241. Springer, 2015.
Lee, J. and Park, B. Development and evaluation of cooperative vehicle intersection control algorithm under the connected vehicles environment. IEEE transactions on intelligent transportation systems, 13(1):81-90, 2012.
Li, M., Wang, J., Du, B., Shen, J., and Wu, Q. Fuzzylight: A robust two-stage fuzzy approach for traffic signal control works in real cities. arXiv preprint arXiv:2501.15820, 2025.
Sarteshnizi, I. T., Bagloee, S. A., Sarvi, M., and Nassir, N. Traffic anomaly detection: Exploiting temporal po- sitioning of flow-density samples. IEEE Transactions on Intelligent Transportation Systems, 25(5):4166 4180, 2023.
Lin, Y.-C., Liu, M.-Y., Sun, M., and Huang. J.-B. Detecting adversarial attacks neural network policies with visual foresight. arXiv preprint urXiv:1710.000 1710.00814, 2017.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500, 2020.
Liu, G., Zhou, Z., Liu, H., and Lai, L. Efficient action robust reinforcement learning with probabilistic policy execution uncertainty. arXiv preprint arXiv:2307.07666 2023.
Sun, Y., Zheng, R., Liang, Y., and Huang, F. Who is the strongest enemy? towards optimal and efficient evasion attacks in deep rl. arXiv preprint arXiv:2106.05087, 2021.
Lowrie, P. R. Scats, sydney co-ordinated adaptive traffic system: A traffic responsive method of controlling urban traffic. Roads and Traffic Authority NSW, pp. 28, 1990.
Tan, K. L., Esfandiari, Y., Lee, X. Y., Sarkar, S., et al. Robustifying reinforcement learning agents via action space adversarial training. In 2020 American control conference (ACC), pp. 3959-3964. IEFE, 2020.
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., and Van Gool, I.. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 11461-11471, 2022.
Tang, B., Yan, J., Kay, S., and He, H. Detection of false data injection attacks in smart grid under colored gaussian noise. In 2016 IEEE Conference on Communications and Network Security (CNS), pp. 172-179. IEEE, 2016.
Mei, H., Li, J., Shi, B., and Wei, Reinforcement learning approaches for traffic signal control under missing data. arXiv preprint 2r04v:2304. 10722, 2023.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(1 I), 2008.
Varaiya, P. The Max- Max-Pressure Controller for Arbi- trary Networks of Signalized Intersections, pp. 27- 66. 2013. ISBN 978- 1 -4614-6244-9.
Mousavi, S. S., Schukat, M., and Howley, E. Traffic light and value-function- based reinforcement learning. IET Intelligent Transport Systems, 11(7):417-423, 2017.
control using deep
doi: 10.1007/
978 18811-4614-6243--9 2. URL /ttpttts:://i.oooooeo 0-0------------------3-
Nichol, A. Q. and Dhariwal, P. Improved denoising diffu- sion probabilistic models. In International conference machine learning, pp. 8162 1162-8171. P.LL, 2021a.
Wang, Z., Hunt, J. and Zhou, M. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.061
2022.
Nichol, A. Q. and Dhariwal, P. Improved denoising diffu- sion probabilistic models. In International conference on machine learning, pp. 8162-8171. PMLR, 2021b.
Webster, F. V. Traffic signal settings. Technical Report 39, 1958.
Wei, H., Zheng, G., Yao, H., and Li, Z. Intellilight: A reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496-2505, 2018
Panaganti, K., Xu, Z., Kalathil, D., and Ghavamzadeh, M.
Robust reinforcement learning using offline data. Ad- vances in neural information processing systems, 35: 32211-32 1-3224,, 2022.

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., and Li, Z. J. Colight: Learn- ing network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM '19 pp. 1913-1922, 2019a.
Zhang, H., Chen, H., Boning, D., and Hsich, C.-J. Robust reinforcement learning state observations with learned optimal adversary. arXiv preprint arXiv:2101.0845. 2021.
Zhang, L., Wu, Q.. Shen, J., Lü, L.,, Du, B., and Wu, J. Expression might be enough: representing pressure and demand for reinforcement learning based traffic signal control. In Proceedings of the 39th International Con- ference on Machine Learning, volume 162, pp. 26645- 26654, 2022a.
Wei, H., Zheng, G., Gayah, V., and Li, Z. A survey traffic signal control methods. arXiv preprint arXiv:1904.08//1 abs/1904.08 17, 2019b.
Wu, Q. Zhang, L., Shen, J., Lü, L., Du, B., and Wu, Efficient pressure: Improving efficiency for signalized intersections. arXiv preprint arXiv:2 12.0336, 2021.
Zhang. L., Wu, Q.. Shen, J., Lü, L., Du, B., and Wu, J. Expression might be enough: Representing pressure and demand for reinforcement learning based traffic signal control. In International Conference on Machine Learn- ing. pp. 26645-26654 PMLR, 2022b.
Wu, Q., Wu, J., Shen, J., Du, B., Telikani, A., Fahmideh, M., and Liang, C. Distributed agent-basse based deep reinforcement learning for large scale traffic signal control. Knowledge- Based Systems, 241: 103044 2022.
Zhang, X., Chen, Y., Zhu, X., and Sun, W. Corruption- robust offline reinforcement learning. In International Conference on Artificial Intelligence and Statistics, pp. 5757-5773. PMLR, 2022c.
Xiao, Z., Kreis, K., and Vahdat, A. Tackling the generative learning trilemma with denoising diffusion gans, arXiv 2021a.
preprint ar XrXiv:21
Zheng, G., Xiong, Y., Zang. X., Feng. J., Wei, H., Zhang, H., Li, Y., Xu, K., and Li, Z. Learning phase competition for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM '19, pp. 1963-1972, 2019.
Xiao, Z., Kreis, K., and Vahdat, A. Tackling the generative learning trilemma with denoising diffusion gans. arXiv preprint xrXiv:2112.078/0 2021b.
Xu, B., Wang, Y., Wang. Z., Jia, H., and Lu, Z. Hierarchi- cally and cooperatively learning traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pp. 6799-67772000 2021.
Zhihe, Y. and Xu, Y. Dmbp: Diffusion model base- predic- tor for robust offline reinforcement learning against state observation perturbations. In The Twelfth International Conference on Learning Representations.
Yang, L., Huang, Z., Lei, F., Zhong, Y., Yang, Y., Fang, C., Wen, S., Zhou, B., and Lin, Z. Policy representation via diffusion probability model for reinforcement learning.
arXiv preprint arXiv:2305.13122 2023.
Yang, R., Bai, C., Ma, X., Wang, Z., Zhang, C., and Han, L. Rorl: Robust offline reinforcement learning via con- servative smoothing. Advances in Neural Information Processing Systems, 35:23851 23866, 2022.
Ye, C., Yang, R., Gu, Q., and Zhang, T. Corruption robust offline reinforcement learning with general function ap- proximation. arXiv preprint arXiv:2310. 130.00500, 2023.
Zhang, H., Feng, S., Liu, C., Ding, Y., Zhu, Y., Zhou, Z., Zhang, W., Yu, Y., Jin, H., and Li, 7. Cityflow: A multi- agent reinforcement learning environment for large scale city traffic scenario. In The world wide web conference, pp. 3620-3624, 2019.
Zhang, H., Chen, H., Xiao, C., Li, B., Liu, M., Boning, D., and Hsieh, C.-J. Robust deep reinforcement learning against adversarial perturbations on state observations. Advances in Neural Information Processing Systems, 33: 21024-21037, 2020.
12

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
A. Appendix: Settings.
A.1. Unet Network Structure
The unct structure shown in Tables 7, and 9.
Table 7. TemporalUnet Structure
Layer
Layer (input size, outpui size)
Linear(state dim, hidden size)
eeceeee
Mish()
Lincar(hidden size, state.d statc.dim) dim)
Linear(action_din) _dim, hidden_size) Mish()
action.encoder
Linear(hidden size, action dim)
SimusidalPPoSEnntthiddenn size.hidden size)
Linear(hidden size, [idden_size*2] Mish()
time_mlp
Linear(hidden size*2, hidden size)
Resduua/PenpprriBlccckllddd.... hie..essiii.eiidddssis=>>>
ResiduulTTempprrllloocchhhhh size*2,hidde, size*2) Identity()
wwwvssampeeddhhddddeee 2,iiden size*2)
downs
Residual ReeichuiTemmpralBBccckhhceeee si =
ResidhalTTmmprriiloocchhhhha hiideenssiiiiei""
Identity()
Identity()
Residual Teanaaa/TTTmoolBBBeechhhheeeee hiidenn size*2)
Residual ResidualTeepprra (enpppralBBiccchhaadeeeiiieeeeee
ups
Identity()
Conv Cony Transpose ddhiddddeeddiiddenn.... wwwhiddennsieeee222222 hiddde 22z
Residda/TenporiBBlccckcccee.... http://idddeeiiie.eiiiiiaaeeeaea h.w.e/...iiddnneee #4)
mid block1
Residua//eemporalBlekk
mid attn
Identity()
mid block2
hoaoaa/@ooooooBBcccciieeee wwiieeedsdddddddeeeedeeeee hiiddenn hiiddeh.siieee/aeee size"4)
Conv dBBlockhiddeensiig 22 wwwhhhddddndse...
final conv
Conv I 2, size/4)
(out horizon"hidden size/4+
(hidden 3/2=*3/2+ size,hidden size*2)
Mish()
mid layer
Linear(hidden.size $ize*2, 22)
Mish()
Linear(hidden hidden_size*2)
Mish(
final final-laer
LinearChidden.sizze hidden_size/2)
13

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Table 8 Residual Block Structure
Layer
Details
Conv (inpot www.mooioouuuuu....ooo _dim)
blocks
(onok dim.outpur dim)
Mish()
ResiduaaTTemporaBllekk
time mip
Lincar(output dim, outpur.dimi dim
Rearranget batch batch 1')
residual conv
Conv Convid(input dim, output dim)
Table Convid Structure
Layer
Details
Conv ddiinput_ding cttpappuudiiouuuuuuddiiiii Kernel Size, Stride, Padding) Kere
Rearrange("baae batch channels horizon' + 'batch channels horizon' batch channels horizon' - 'batch channels horizon')
ConvidBlock
GroupNorm(output. dim, Group, eps, Affine: True) MishO
A.2. Hyperparameter
By effectively tnnng these hyperparameters, users can optimize RobustLight performance for their specific data recovery tasks, achieving better accuracy and robustness in handling missing or corrupted data. The detailed settings are summarized in Table 10.
Table 10. Hyperparameters
Diffusion policy
Hyperparameter type
Setting
Hyperparameter embed_die
64
UNet hyperparameter
state_dim
12/24
action_dimg
non nrl_makkovinnste
condition
beta schedule a, b, discount(7)
2.1 2.1190,25.06,22.544
0.99
target critic(r)
0.005
Diffusion training hyperparameter
diffusion timstep batch size buffer capacity optimizer
100
64
12000
Adam
learning rate
0.0003
epochs action gradient steps
20
discount(r)
0.8
target critic(r)
0.95
buffer capacity epochs batch_size
12000
100
TSC RL agent
20
training hyperparameter
learning rrate target update time normal factor loss function optimizer
0.001
20
MSE
Adam
14

---

RobustLight: Improving Robustness via Diffusion Reinforeement Learning for Traffic Signal Control
Table 11. Performance in terms of ATT. %25 means the sensorn, %050 means the sensory, N, sensorg e missing data.
Dussost
Mask Soal
FixedTime
Ioor
hoar
MasPicouue
Kabrel agle
Advanced AdhuncalMaaaPrsaae
MusPrsseure
Knhheel.gh
hese
Advasced MgLigla
Kabsod agh
baec
CoLight
Rehasd ight
hewr
Advanedd CoLigte
Rabued agh
INNrl
25%
438. 110.11
256 750000
1740.0
279.22:1.7
299.5662.88
27751-00
26536620
276.1341.83
3638721.09
324
270.46:5.09
3513664477
51676615524
51207669 91
321884501
276002344
251.81184
3136912889
304,1361144
S183113
ANaa2
308 700
27695
272.6733.52
2691600
2666741.58
250.7100.3
25
242.01225774
279.0166 46
271.442..44
3078556647
28231676
ANaa3
2516
50%
383.0120:00
25122200:0
273 32
270.6424.39
2500908.83
265.72000
288052300
263.7741.53
345
289022-003
255.55+1.54
343.01252.36
238.563333
288.546500 05
267178500
27388922221
253991.16
304.07426.17
17
268.829.00
239.161.77
HungZwuul
2556
90%
495.37a0:00
1662100000
592, 9:0.0
534 584.78
301.811.38
W7.12-00
S28.11:00
433.3613817
361.5344,84
426 426.11:42.96
25887944.2
14..84:10.38
M6.12:7.49
267.12
178.2+195
362.189977
10X 10.64
511.01156.7
375.96247.54
24012:7.84
Hang2hou2
406 406.0500:00
3521 00017700000
34981:1.19
3/89600
3468263.91
1521550055
549.262..6
3659781587
1621118816
341.51818.05
33401177322
452.73-00
434.6647.16
415.5147.82
444.82441
383,46:17.09
Table 12. Performance of in terms of ATT. %25 means the sensorg. 9:50 means the sensorg, sensors g missing data.
Mask Scale
basz
bost
KobesLight
KetuaLighr
Dase
KolusLige
ightt
RokesLggt
Ligh
oghe
JiNant
25%
595
1:000
708 672011
877.54811 141
19
113555000
35300000
11000866 85
3ML76274,35
K3KW67663
W0..621..1
35336477
953.85a1.26
2010814
638.06221.00 00
615.95:31.08 48
RNNa2
599
25%
368 Nich 80
255
758 94-00
66:376
239.6826.4
273,48500
11091100
009.77790.72
257.172L6
817.22223.993
107695/10622 23
64.771644.4
475.6646.36
356.7964.03
125.11337224
1844.8:16558
281.862..77
264.355555 53
65929199.5
600.122:9.64
25400554113
JiNaa3
5066
2515
363
658 316t01
261 325010
700,
260.451.35
76253.94
125457800
264
35
727 59-16661
11145766426
545
79..99
761.7611822
32541544777
305.4187.81
271.
643.5455.6 69
20010962.7
596.7651272
253/8164 76
HangZaual
2514
405.57:00 60
5-0.0
476.
3451 56
309.102.119
612 51..546551
X15,2641019
151 196366
506
$1-00
754.
33.55
1006
01100
781.8869.55
919.94999
4248:58.41
912.21661776
462.8137771
602 102.16108441
116,18:19.18
HangZhou22
50%
25R4
406.550000
$540220.0
552.24223.82
371-466628
718.9330.0
534/73:38.6
351.19:4.77
667.
106-12221.66
47114:1257
501.53648.07
446.97:11.96
Table 13. Performance in terms of ATT. %25 means the sensors. 9:50 means the sensorw. , sensors missing data.
Dinseet
Masi
PesediTime
Sase
buse
MaxPresure
RibsstLipa
Advanced MaxPhecure
haze
Acvonced Miti
Collggh
Ratuslials
Advanoed Colligh
Inhant
595
438 1140000
30709900.02
422.7650.0
537 52612.4
51851:911
20596-000
520
519.4465.6
31439970
326.
1566. 18
76399 76
377.0222584
451 91125555
801699
IiNan2
50%
25%
308 Nellu0
306.72x0.0
2615240.0
25K 1642
407 597.9
29471a0.0
208 0000000
25.4661A5
288.5564L1
300
298.4618.01
20.-3670004 04
240.7161.88
304.-22.8
342.822.6626
100. 3155.7
295.6726.56
260 26..62.85
86
307 141.51
258 47.2..0
S7T. 277.25428.18
257.176687
ANNart
255
SSS 01-000
323,31-0,0
270.4116.3
278 751751
459 5569
362350000
3065800000
286.
16
390,86922.35
451.8990219 19
537.5664.54
315
15.85
37131220
4369.399.1
261.913.6
Hangzhool
50%
57
430.52-0.0
369.0567.55
63361
460.54612.55
36882211.75
577.96200119
511.1243348
655 40429,0
514.43646.55
HongZhou?
50%
406 65-0.00
387.5860.00
461.1144.11
364.5122
646.460.0
36704494225
411.7865:05
458 458,86516.85
64:378
667.59H11
457.73630.48
411,47+13.55
396.8558.36
402.53417.42
993.6111279
419:9249.52
361.117.711
Table 14. Performance ATT comparison between the native diffusion model, LSTM model and our improved diffusion model.
Typo
Lines Daca
Croiao Bata
DiPaac Bets
LITT
Babardight
Samae Buts
Cod Bats
Swt
LSTM
303
260
351256130
Sleadd
28
40
9151420
10
45501418
75128-192.56
125 25-0220000 20
MAD
42
98157299212
Maa
11304277 83
225 2442443
190. 175.1511331
Sanor
400
1281 $1625.5
Danage
1255 142233
191223000
370 5141244 44
96
44110122.11 11
51121:115
1115676300
Toal
3.5
4353543422
16
73540 54
ZangZoon
MAD
419 3083331335
331474
250 15
130044711 71
464 14,3137
317364396
102754585 85
700 29 29
703-6429-13
Snowe
299
495 1::18
414 No5322
2000151. St.
AlL (16201
385 25
1122 18
4651555 4
1991 FN770
Derage
575
75633-41
50391111434
7768342533
1101 7541275
56 35
15-31476-3
(123444330
611 511551131777
109-354141
15

---

RobustLighh: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Table Performance ATT JiNan, HangZhou. " implies that traditional algorithms are not adapted to MinQ MAD attacks. Our RobustLight recovers the state of traditional and RL-based TSC algorithms to evaluate the performance.
Type
MusPeuuse
Kohusligt
Advanced Musheaue
base
MpLizte
Adtinodd
RocasiLiph
Gassdon
2190551155
282.9983
279 M1l1
279.061111 15
276. 564176
155.5te1
281 28-
190.
271 67
540 4154617.41
2900165.1
U-ran
MAD
4.0
4.0
3.5
3.5
368.7eet1.00
301 51
306 293309
294.3922 25.299
290.442.40
7947561 16
304/75e1.09
297.5863.19
302.721.91
5140111162
2550
50804656
207 07218.883
32504411.34
361
16..941111
19955
588.37647.11
56653279444 44
3MLA
35412453 A3
262 0541.80
263455 45145
7417661181.97
357
34
361722-3549
271 5821.52
MinQ
4.0
3.5
283 572009
289
161.8-113
17099-131
516.49 116.992:00.7
S4 17
266 1742.20
41
323.072775
MA 11
316.296.077
30.3115.44
Gazsiian
4,0
3.5
363.6442.7
360 5682
358.57:1.71
318.12l111
345
3460641142
342 5261:33
26
4184181244
371.71-3455
36
352 352
340 7565 13
351, 58
344 61.333
410
3N6Ale 59
368 27
371 S466.4
RongZlon,
Denal
MAD
3.5
4.0
1.5
4,0
406 (56.00
373.8963.09
371.2183.4
366.5222.44
27
162.11e2:3
366.92335
359.1762.02
289.2222.0
21383el1
949 6994842666
345 3462977771
6369777771
437.93:14.0
341522-429
34861::577
442.07226.22
49:15a15.66
478 178-25415:10
363.2799.41
356.7964.07
540.7742.54
3565334,33
353.6733.26
472.1122077
487.5321R.17
40% 17412 53
406-75:501
379.27:11.99
384.24-13.31
359.669..21
MinQ
3.5
25
54338-954
447.499445
484.32112.88
352.1362.72
39
40H (66.266677
372
Table 16. Performance Compared with DiffT.ghht.
Dataset
Noise Type
Noise Scale
DifiLight
Advanced-Colign-RRouuuse
Gaussian
3,5
289.4523.37
294.57=2.02
U-rand
3,5
311.5324.17
358.22=7.02
JiNon,
MAD
3,5
384.71=13.25
283.13#1.56
MinQ
3,5
321.98=6.37
323.25+20.54
Mask
25%
353.45:34.31
320.31+3.21
(Kriging and Randorn)
Gaussian
3,5
325.0542.58
327.98+2.45
U-rand
3,5
365.05=20.67
473.85=32.68
HongZhou
MAD
3,5
366. 14+5.34
309.24
MinQ
3,5
106.56
406.3226.41
Mask
25%
346.0 146.05+41.27
296.69+4.67
(Kriging and Random)
B. Appendix: Other Experiments
Table 17. Recover time comparison.
Advanced Colight
Advanced
Diffusion step
Dataset
Noise type
Noise scale
Maxpressure
Recover
Recover
time(ms)
time(ms)
1.0
20
376.16
343.06
Gaussian
0.5
202.98
178.95
1.0
262.89
233.45
U-rand
0.5
142.05
124.33
1.0
240
3792.56
JiNan on
MAD
0.5
120
1959.03
1.0
3779.34
MinQ
0.5
120
1976.44
Sensor
25%
200
2987.34
2765.36
damage
50%
200
2997.61
2768.24
1.0
417.26
447.84
Gaussian
0.5
221.38
233.92
1.0
16
281 281.60
302.86
U-rand
0.5
156.32
160.16
1.0
5220.45
HongZhowz
MAD
0.5
120
3581.67
1.0
240
5348.28
MinQ
0.5
120
2889.14
Sensor
25%
200
3232.13
3552.44
damage
50%
200
3223.55
3559.41
16

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
C. Appendix: Model Generalization Analysis
In this section, we analyze model's generalization ability. We transfer the algorithm trained on the JiNan dataset to other datasets to observe whether our RobustLight can effectively recover the data under various attacks.
Neepok22
Figure Performance RobustLight based on Maxpressure transfer by JiNanj.
23141
Juan]
Heeghhost
Figure 8. Performance of RobustLight based Advanced- transfer by JiNan.
17

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Hanghhout
Hanghedd
1004
e/c
Neayolkt
Newyork2
Figure 9, Performance of RobustLight based Advanced-Mplight transfer by JiNam.,
Hengshoat
Reegshas2
440
Figure 10. Performance of RobustLight based on Colight transfer by JiNan
120
100
122
Hampheul
Hanghoud
Sonparki
Nowyork
Figure 11. Performance of RobustLight based Advanced-Colight transfer by JiNan
18

---

RobustLight:
Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
D. Appendix: Robustness Analysis
In this section, we provide detailed visualizations the state t-SNE distributions and violin plots after dimensionality reduction under Gaussian noise attacks for different algorithms. As observed, for different algorithms, our RobustLight shows a significantly better recovery of the state distribution in most datasets, making more closely resemble the original state distribution compared the attacked state distribution.
Original stae disribution
Recover ctele dieribation
Noise etric dierisation
Smasl
Bnon]
Jaan
Hanghhal
Bangshos?
Figure 12. State t-SNE visulization of RobustI ight based on Maxpressure in HangZhon and JiNon. Compared to the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using algorithm. RobustL ight has the ability of state recovery.
19

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
44
Jusl
Figure State violin visulization of RobustLight based Maxpressure in HangZhou and JiNan. Compared the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using our algorithm. RobustI ight has the ability of state recovery.
Original slate
Recover state distnbutton
Noise sizte distibution
Jbae
Jaand
Jnn1]
Bangrhas
Hagghe
Figure 14. State t-SNE visulization of RobustLight based Maxpeeudr in HangZhou and JiNan. Compared to the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using our algorithm. RohustLight has the ability of state recovery.
20

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Jeaii
Figure State violin visulization RobustLight based on Mdxpesedaa HangZhou and JiNon. Compared state distribution after the noise attack, the state distribution is much closer to the original state distribution after using algorithm. RobustLight has ability of state recovery.
3an2
lana3
Hasqohoul
Hangshoa2
Figure 16. State t-SNE visulization of RobustLight based on Colight in HangZhou and JiNan. Compared to the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using algorithm. RobustLight has the ability of state recovery.
21

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
TyM
Ape
Rusi
Ikas
Hegieat
Figure 17. State violin visulization of RobustLight based Colight in Hon and JiNan. Compared the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using our algorithm. RobustLight has the ability of state recovery.
Original staic distnbutiun
Recevrr steic distribation
Nokez stee diatribation
Jnmn
Jane1
Hangzheul
Hanzcre
Figure 18. State t-SNE visulization of RobustLight based AdvancedColight in HangZhou and JiNan. Compared to the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using our algorithm. RobustLight has the ability of state recovery.
22

---

RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control
Jusi
Hangbwal
Figure 19. State violin visulization of Robust Light based AovincedColight in Hang and JiNan. Compared the state distribution after the noise attack, the state distribution is much closer to the original state distribution after using our algorithm. RobustLight has the ability of state recovery.
23


Reviews:
[{"review_id": "LY064O8TVU", "reviewer": "Reviewer_YPk8", "summary": "This paper focuses on a very interesting problem. For the data missing problem faced in traffic signal control, the authors use the diffusion model to complete and clean the data. The experimental results show that this method effectively improves the control performance of the reinforcement learning model in data missing or contaminated environments.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First and foremost, we sincerely thank you for pointing out the issues. Your suggestions are invaluable in enhancing the quality of this paper. Below is our answer to your questions.\n\n1. Experimental results on some datasets are not reported. \n\nDue to space constraints, data noise results for JN2 and HZ2 were omitted from the main text. The partial snapshot below shows RobustLight’s superior performance. Additional dataset results are included in the appendix and link.\n\n### Advanced-CoLight  Noise Scale 3.5\n| Dataset    | Noise Type | base| RobustLight |\n|------------|------------|--------------------------|-------------------------------|\n|     JN1       | MAD        | 325.87                   | **271.58**                    |\n|            | MinQ       | 323.07                   | **300.31**                    |\n|     JN2       | MAD        | 405.37                   | **359.03**                    |\n|            | MinQ       | 399.87                   | **372.84**                    |\n|      NY1      | Mask25\\%| 1246.85                   | **1086.77**                    |\n|      NY2      | Mask25\\%| 1430.49                   | **1290.79**                    |\n\n2. Other related data generation methods are introduced to reflect the innovation of the proposed data generation method.\n\nHINT[1] proposed using Transformer combined with GAN for data reconstruction, while CaPaint[2] employs diffusion for data generation. However, these methods require extensive offline data for pre-training and only address data missing, not data noise. DiffLight[3] tackles random missing and kriging missing but uses separate algorithms for each and underperforms SOTA on clean data. In contrast, our method employs a single model to simultaneously resolve data missing and noise, handling both random missing and kriging missing with a unified algorithm. Moreover, our approach acts as a plug-in, optimizing SOTA algorithms without altering their core structure.\n\n[1] HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention\n\n[2] Causal Deciphering and Inpainting in Spatio-Temporal Dynamics via Diffusion Model\n\n[3] DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data\n\n3. Missing baseline and quantitative analysis.\n\nWe fine-tuned the HINT model on an offline dataset and evaluated it on the TSC task, quantifying performance using PSNR (higher is better) and MAE (lower is better). Our algorithm surpasses HINT in data generation capability.\n\n### Data Generation Performance Comparison\n| Dataset | Method      | PSNR      | MAE      | ATT      |\n|---------|-------------|-----------|----------|----------|\n| JN1     | HINT        | 9.78      | 1.11     | 396.24   |\n|         | **RobustLight** | **10.46** | **0.66** | **298.35** |\n| JN2     | HINT        | 18.88     | 1.36     | 283.98   |\n|         | **RobustLight** | **24.07** | **1.20** | **259.56** |\n| JN3     | HINT        | 10.24     | 1.33     | 395.62   |\n|         | **RobustLight** | **11.71** | **1.15** | **301.63** |\n| HZ1     | HINT        | 11.90     | 0.88     | 371.74   |\n|         | **RobustLight** | **13.90** | **1.01** | **328.26** |\n| HZ2     | HINT        | 5.35      | 0.76     | 384.67   |\n|         | **RobustLight** | **6.53**  | **0.85** | **375.64** |\n| NY1     | HINT        | 16.31     | 1.67     | 1189.56  |\n|         | **RobustLight** | **17.25** | **1.51** | **1086.77** |\n| NY2     | HINT        | 13.39     | 1.53     | 1394.96  |\n|         | **RobustLight** | **15.65** | **1.29** | **1290.79** |\n\nTo validate our approach, we setup involved randomly masking data from **Kriging Missing (12.5\\%)** and **Random Missing (12.5\\%)**.  AMPR (Advanced-MaxPressure-Robustlight), ACR (Advanced-Colight-Robustlight).\n\n###  Performance Compared with DiffLight\n| Dataset | Method                  | Noise/Mask | PSNR      | MAE      | ATT      |\n|---------|-------------------------|------------|-----------|----------|----------|\n| JN1     | DiffLight               | U-rand     | 6.71      | 6.26     | 310.92   |\n|     | **AMPR**  |    | **7.20**      | **5.42**     | **304.34**   |\n| HZ1     | DiffLight               |     | 6.85      | 7.93     | 361.31   |\n|     |**AMPR** |     | **7.71**      | **5.12**     | **297.34**   |\n| JN1     | DiffLight               | 25%        | 7.66      | 1.05     | 366.05   |\n|    | **ACR** |      | **9.34** | **0.89** | **304.13** |\n| HZ1     | DiffLight               |     | 18.05      | 1.84     | 372.53   |\n|    | **ACR** |    | **22.96** | **1.17** | **306.56** |\n\n4. The difference with other diffusion model.\n\nKey modifications: (1)use a new beta schedule (2) optimized with new loss function. (3) employ new action gradient method to improve data noise and missing.\n\n5. New link: https://anonymous.4open.science/r/RobustLight-72B2/README.md\n\n6. More baseline results refer to qV9c and code link.\n\n7. Refer to Jqz8 and ACd7,  our theoretical explanation based Lemma 3.1", "claims_and_evidence": "i) Experimental results on some datasets are not reported. \nii) Lack of baseline comparison for data generation methods.", "methods_and_evaluation_criteria": "Traffic uses common standards in the field. There is a lack of quantitative standards for data denoising and completion.", "theoretical_claims": "This paper does not provide any theoretical proof.", "experimental_designs_or_analyses": "The comparison method is not perfect and some experimental results are not reported.", "relation_to_broader_scientific_literature": "Provides a new perspective for studying traffic signal control issues.", "essential_references_not_discussed": "The authors put the related work section in the appendix and did not discuss similar data generation methods.", "other_strengths_and_weaknesses": "Strengths:\ni) The authors have addressed a new issue\nii) Use the new diffusion model to complete missing data and denoise\n\nWeaknesses:\ni) The authors do not fully introduce other related data generation methods\nii) The experimental part lacks experimental results on some datasets\niii) There is no quantitative analysis of the quality of the generated data, nor is there any comparison with baseline methods.", "other_comments_or_suggestions": "i) Other related data generation methods are introduced to reflect the innovation of the proposed data generation method.\nii) Give complete experimental results\niii) Update the link to the code repository\niv) Provides quantitative analysis results of data generation methods and comparative experimental results of generation quality", "questions_for_authors": "i) What is the difference between the proposed diffusion model-based data generation method and the existing diffusion models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sRRwY9yo5i", "reviewer": "Reviewer_qV9c", "summary": "The paper introduces RobustLight, a novel framework designed to enhance the robustness of Traffic Signal Control (TSC) systems against adversarial attacks and missing data. The key contribution of RobustLight is the integration of an improved diffusion model into TSC, which enables real-time recovery of noisy or missing traffic data without altering the existing TSC algorithms. The framework consists of two main components: the Dynamic State Infilling (DSI) algorithm, which trains the diffusion model online, and two auxiliary algorithms, Denoise and Repaint, which leverage the trained diffusion model to address adversarial attacks and missing data, respectively.\n\n## update after rebuttal\nRebuttal acknowledged.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First and foremost, we sincerely thank you for pointing out the issues, as your suggestions are invaluable in enhancing the quality of this paper.\n\nW1:  https://anonymous.4open.science/r/RobustLight-72B2/README.md\n\nW2：Traffic movements and TSP are defined in Figure 1 and Section 2.1, with cyan coloring used for aesthetic design.  D() represents the norm distance.\n\nW3&W6:\n\nWe conduct the experiments for Difflight[1] and MissingLight[2]. [1] uses offline RL and diffusion models for two missing-data cases: **random missing** (single-intersection-sensor failure) and **kriging missing** (full-intersection failure), requiring separate treatments. To validate our approach, we setup involved randomly masking data from Kriging Missing (12.5%) and Random Missing (12.5%). \n\n\n### Data Missing , Our is Based On Advanced-Colight\n\n| Method                     | JN1             | HZ1             |\n|----------------------------|-----------------|-----------------|\n| [2]     | 354.73  | 348.68  |\n| [1]                 | 353.45     | 346.05  |\n| **Our** | **320.31** | **296.69** |\n\n\n### Data Noise Scale 3.5\n\n| Method                     | JN1             | HZ1             |\n|----------------------------|-----------------|-----------------|\n| [1]-MinQ           | 321.98  | 426.14  |\n| **Our** | **283.13** | **397.32** |\n| [1]-MAD            | 384.71  | 366.14  |\n| **Our** | **323.25** | **309.24** |\n\nAs can be seen, our method not only effectively handles data missing scenarios in both (**kriging missing and random missing**) but also performs well in data noisy condition. \n\n### W6：Without Data Missing and Data Noise\n| Dataset | DiffLight | Advanced-Colight |\n|---------|-----------|------------------|\n| JN1     | 268.43    | **245.73**       |\n| HZ1     | 283.92    | **270.45**       |\n\n\nUnder clean data, RobustLight preserves the optimal performance of the TSC algorithm without unnecessary denoising, it simplifies integration and enhancing system stability and reliability.\n\nCompared to [3], which uses adversarial pre-training to improve robustness of RL, but it cannot repair corrupted inputs and risks overfitting to specific attacks, our diffusion model handles diverse disturbances without attack-specific training, directly denoising states for robust RL inputs as a plug-and-play module.\n\n\n[1] DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data. NIPS 2025\n\n[2] Reinforcement Learning Approaches for Traffic Signal Control under Missing Data. IJCAI 2023\n\n[3] Explainable Deep Adversarial Reinforcement Learning Approach for Robust Autonomous Driving. TIV 2024\n\nW4: \n\nOur framework could leverage DDIM and high-performance GPUs for accelerated inference. Denoising supports decentralized deployment with centralized training, while data missing scenarios use centralized inference. DDIM acceleration times on one A100:\n\n| Deployment          | Scenario          | JN1   |  NY1   |\n|---------------------|-------------------|-------|-------|\n| Decentralized       | U-rand noise| 0.098s|  0.096s|\n| Centralized         | Data missing| 0.123s|  0.81s |\n\nAs shown, the total processing time is less than 1 second meet real-time control. Further deployment discussion can be found in reviewer Jqz812.\n\nW5:\n\nOur RobustLight addresses challenges in TSC deployment, where TSC RL inputs often suffer from noise or missing values, impacting control. To handle the dynamic nature of traffic flow, we propose an online training framework using diffusion models as the upper level RL policy during clean data periods. When noise or missing data is detected, our trained DSI agent employs denoising and repainting algorithms to restore data quality, improving lower level TSC RL performance.\n\nW6&W7：\n\n [1] introduced TP-FDS, which identifies anomalies by comparing new data distributions with historical data from the same period, achieving an AUC of 96% and an F1 score of 76%. Minor changes like 3-5 have minimal impact on system efficiency. Anomalies can also be detected by cross-referencing data from multiple sensors, such as cameras and radar. Rule-based methods are another option; for example, a queue increase from 3 to 5 during a north-south green light would signal an anomaly. To simulate real-world scenarios, we conducted experiments with detection rates of 80% and 60%: \n\n### MAD Scale 3.5 \n| Detection Rate | Base   | ATT | Base| Throughput | Dataset |\n|-----------------|-----|-----|-----|------------|----------|\n| 80%             | 487| **297**      | 5812  |**6154**   | JN1      |\n|            |  463   | **326**   |2888 | **2938**         | HZ1      |\n| 60%            | 487|  **328**   | 5812|**6131**         | JN1      |\n|             |     463   |**331**  |2888| **2930**          | HZ1      |\n\nIt sustains high performance with different detection rates.\n\nRefer to Jqz8 and ACd7, our theoretical explanation based Lemma 3.1\n\n[1] Traffic Anomaly Detection: Exploiting Temporal Positioning of Flow-Density Samples", "claims_and_evidence": "The claims made in the submission are generally supported by clear and convincing evidence, particularly through extensive experimental results and visualizations. However, this paper mentioned online training and real-time implementation, but it does not provide sufficient theoretical nor experimental results to validate these claims. While the paper demonstrates the effectiveness of RobustLight in recovering noisy or missing data and improving traffic signal control performance, it does not explicitly show the complexity of the framework or how the framework performs in a real-time, online setting.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria in the paper make sense for the problem of enhancing the robustness of Traffic Signal Control (TSC) systems against adversarial attacks and missing data.", "theoretical_claims": "This paper focuses on experimental results and algorithmic contributions rather than theoretical proofs. This paper provides a detailed description of the proposed methods, including the Dynamic State Infilling (DSI) algorithm, Denoise algorithm, and Repaint algorithm. This paper does not include formal theoretical claims or proofs. Instead, the paper relies on experimental results to demonstrate the effectiveness of the proposed RobustLight framework.", "experimental_designs_or_analyses": "The experiments are well-designed and provide convincing evidence to support the claims made in the paper.", "relation_to_broader_scientific_literature": "The key contributions of the paper are related to and build upon several areas of the broader scientific literature, including Traffic Signal Control (TSC), Reinforcement Learning (RL), Robust RL, and Diffusion Models.", "essential_references_not_discussed": "This paper lacks comparison with other recent methods. The latest method compared in the paper is proposed in 2022. Other TSC methods (e.g., GNN-based or Transformer-based approaches) or robust RL methods (e.g., adversarial training, self-supervised learning) can be compared and discussed (e.g., Explainable Deep Adversarial Reinforcement Learning Approach for Robust Autonomous Driving).", "other_strengths_and_weaknesses": "Strengths:\n- This paper proposes a novel framework, \"RobustLight.\" It introduces the Diffusion Model into the field of traffic signal control (TSC) to deal with data noise and missing value problems. Diffusion model has been widely used in the field of image generation, but it is an innovative attempt to apply it to TSC system to enhance robustness, especially in real-time online processing.\n- RobustLight is not only able to handle a single type of attack (such as Gaussian noise), but also multiple complex attack types (such as MAD and MinQ attacks), and is able to deal with the problem of missing data caused by sensor corruption. This integrated defense capability is rare in current TSC systems, demonstrating the broad usage and robustness of the framework.\n- The experiments are extensive. They are conducted on multiple real-world data sets to verify the effectiveness of RobustLight. Also, the visualization methods such as t-SNE visualization and violin diagram are used to demonstrate the effectiveness of state recovery, which enhances the reliability of experimental results.\n\n\nWeaknesses:\n- The code is not available. \n\n-  Though the authors discuss the detection method,  the performance in a normal situation without an adversarial attack is not presented.  The authors discussed the detection of outliers in the paper. However, if the noise attack changes the queue length from 3 to 5, how can this be detected with outlier detection?", "other_comments_or_suggestions": "- It is suggested that related works be added back to the main body of the paper to provide background knowledge; now, it appears in the Appendix.", "questions_for_authors": "1. Where is the correct code link?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2qk0JHlOtZ", "reviewer": "Reviewer_ACd7", "summary": "This paper point out the current challenge in the TSC systems, which include significant performance degration, limitation of existing defense methods and lack of online ability. To address these issues, authors propose RobustLight, a framework to enhance the robustness of online TSC systems, consisting of a TSC agent and a dynamic state filling (DSI) agent. This frame work contains 2 algorithms, denoise and repaint to defend against missing data and adversarial attack. Extensive experiments show the effectiveness of their framework.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First and foremost, we sincerely thank you for pointing out the issues, as your suggestions are invaluable in enhancing the quality of this paper.\n\n1. For the experiment part, my concern is whether this method is still effective under some potential adaptive attacks, such as attacks effective on Diffusion Models.\n\nDiffusion Models are generally robust due to their iterative denoising process, which inherently introduces noise and reduces the impact of adversarial perturbations. However, like other deep learning models, they are not immune to adaptive attacks specifically designed to exploit their weaknesses. \n\nThis situation resembles a Russian nesting doll, but based on our experience in TSC deployments, most TSC control systems are deployed within internal networks. While outdoor sensing devices are prone to noise interference and sensor damage, the TSC control module and our diffusion module can be deployed in either a distributed manner (with control devices at each intersection connected via an internal network) or a centralized manner (deployed within the internal network). This network isolation effectively protects the diffusion module from potential attacks.\n\n**Distributed Deployment**:  Specifically, if a distributed deployment is chosen, our diffusion model can be trained on a central server to fully leverage data from different intersections, while model parameter updates are performed on edge computing devices, similar to a federated learning architecture. \n\n**Centralized Deployment**: If a centralized deployment is preferred, scalability and real-time requirements for hundreds of intersections must be addressed. On the hardware side, we can procure high-performance servers or employ inference acceleration techniques, such as data partitioning and parallelized inference, to enhance the diffusion model's inference performance. On the algorithmic side, we can utilize techniques like DDIM [1] to accelerate the diffusion model's inference process.\n\nIn summary, by carefully selecting the deployment strategy and leveraging advanced hardware and algorithmic optimizations, we can ensure the robustness, scalability, and real-time performance of Robust TSC system while maintaining its security against potential attacks.\n\n[1] Denoising Diffusion Implicit Models.\n\n2.The latency issue of this method because of the usage of diffusion models and whether it can handle traffic problems in a real-time way.\n\n\nOur framework could leverage DDIM and high-performance GPUs for accelerated inference. Denoising supports decentralized deployment with centralized training, while data missing scenarios use centralized inference. DDIM acceleration times on one A100:\n\n| Deployment          | Scenario          | JN1   | HZ1   | NY1   |\n|---------------------|-------------------|-------|-------|-------|\n| Distributed       | U-rand noise| 0.098s| 0.097s| 0.096s|\n| Distributed       | Data missing| 0.129s| 0.115s| 0.131s|\n| Centralized         | U-rand noise| 0.364s| 0.431s| 2.43s |\n| Centralized         | Data missing| 0.123s| 0.164s| 0.81s |\n\nAs shown, the total processing time is less than 1 second meet real-time control.  However, it is important to note that in decentralized deployment scenarios, the repaint algorithm will not be able to address the **kriging missing (full-intersection failure)** issue within data missing. Distributed deployment significantly improves algorithm performance due to the reduction in the batch size of inference data.\n\n**We recommend deploying the centralized solution to address both kriging (full-intersection failure) and random missing (sensor-specific-direction-single-intersection-failure) data issues using data missing  algorithm, while employing data noise algorithms on the edge side using low-cost hardware to achieve accelerated denoising.**", "claims_and_evidence": "The effectiveness of RobustLight is shown clearly in the experiment results in different datasets and different attacks.", "methods_and_evaluation_criteria": "The benchmark used by their experiment session makes sense.", "theoretical_claims": "Although this paper focuses on application side, the theoretical explanation is clear and the Lemma 3.1 and algorithm boxes are clear.", "experimental_designs_or_analyses": "For the experiment part, my concern is whether this method is still effective under some potential adaptive attacks, such as attacks effective on Diffusion Models.", "relation_to_broader_scientific_literature": "I think the problem studied by this paper has a wide application in the real world traffic management.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "My another concern about this method, which is also stated in this paper, is the latency issue of this method because of the usage of diffusion models and whether it can handle traffic problems in a real-time way.", "other_comments_or_suggestions": "NA", "questions_for_authors": "My questions are 2 concerns mentioned in the previous sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PZo5pEQ2Zf", "reviewer": "Reviewer_Jqz8", "summary": "The paper introduces ​RobustLight, a novel framework designed to enhance the robustness of Traffic Signal Control (TSC) systems against adversarial attacks and missing data. The authors propose a ​plug-and-play diffusion model​ that integrates with existing TSC platforms to recover from noise attacks and restore missing data in real-time. The framework includes two key algorithms: ​denoise​ and ​repaint, which leverage a ​Dynamic State Infilling (DSI)​ algorithm to train an improved diffusion model online. The authors conduct extensive experiments on real-world datasets, demonstrating that RobustLight significantly improves the performance of TSC systems under various adversarial attacks and missing data scenarios, with up to ​50.43% improvement​ in average travel time compared to systems without RobustLight.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First and foremost, we sincerely thank you for pointing out the issues, as your suggestions are invaluable in enhancing the quality of this paper.\n\n1. Scalability: The paper mentions that the computational cost increases with the number of intersections. Could the authors elaborate on potential strategies to scale RobustLight for large-scale urban networks with hundreds of intersections?\n\nHere, we need to clarify that our method can be deployed in either centralized or decentralized scenarios. In the case of centralized deployment, the model inference speed may slow down as the number of intersections increases.\n\n**Distributed Deployment**:  Specifically, if a distributed deployment is chosen, our diffusion model can be trained on a central server to fully leverage data from different intersections, while model parameter updates are performed on edge computing devices, similar to a federated learning architecture. \n\n**Centralized Deployment**: If a centralized deployment is preferred, scalability and real-time requirements for hundreds of intersections must be addressed. On the hardware side, we can procure high-performance servers or employ inference acceleration techniques, such as data partitioning and parallelized inference, to enhance the diffusion model's inference performance. On the algorithmic side, we can utilize techniques like DDIM [1] to accelerate the diffusion model's inference process.\n\nIn summary, by carefully selecting the deployment strategy and leveraging advanced hardware and algorithmic optimizations, we can ensure the robustness, scalability, and real-time performance of our Robust TSC system while maintaining its security against potential attacks.\n\n[1] Denoising Diffusion Implicit Models.\n\n2. Real-World Deployment: While the experiments are conducted on real-world datasets, the paper does not discuss the challenges of deploying RobustLight in real-world TSC systems. What are the practical considerations and potential barriers to real-world implementation?\n\n\nOur framework could leverage DDIM and high-performance GPUs for accelerated inference. Denoising supports decentralized deployment with centralized training, while data missing scenarios use centralized inference. DDIM acceleration times on one A100:\n\n\n| Deployment          | Scenario          | JN1   | HZ1   | NY1   |\n|---------------------|-------------------|-------|-------|-------|\n| Distributed       | U-rand noise| 0.098s| 0.097s| 0.096s|\n| Distributed       | Data missing| 0.129s| 0.115s| 0.131s|\n| Centralized         | U-rand noise| 0.364s| 0.431s| 2.43s |\n| Centralized         | Data missing| 0.123s| 0.164s| 0.81s |\n\n\nAs shown, the total processing time is less than 1 second meet real-time control. However, it is important to note that in decentralized deployment scenarios, the repaint algorithm will not be able to address the **kriging missing (full-intersection failure)** issue within data missing. Distributed deployment significantly improves algorithm performance due to the reduction in the batch size of inference data.\n\n**We recommend deploying the centralized solution to address both kriging (full-intersection failure) and random missing (sensor-specific-direction-single-intersection-failure) data issues using data missing  algorithm, while employing data noise algorithms on the edge side using low-cost hardware to achieve accelerated denoising.**", "claims_and_evidence": "The claims made in the paper are ​well-supported by empirical evidence. The authors provide extensive experimental results on real-world datasets, including JiNan, HangZhou, and New York, to validate the effectiveness of RobustLight. The results show consistent improvements in TSC performance under various adversarial attacks (Gaussian noise, U-rand, MAD, MinQ) and sensor damage scenarios. The authors also compare RobustLight with traditional and RL-based TSC methods, demonstrating its superior robustness and recovery capabilities. The evidence is clear, and the results are statistically significant, with detailed metrics such as ​Average Travel Time (ATT)​ and ​state recovery performance​ provided.", "methods_and_evaluation_criteria": "The methods are ​well-designed and innovative, leveraging the strengths of diffusion models and reinforcement learning. The ​Dynamic State Infilling (DSI)​ algorithm is a key contribution, enabling real-time recovery of TSC data. The ​denoise​ and ​repaint​ algorithms are effectively used to handle adversarial attacks and missing data, respectively. The evaluation criteria are appropriate, focusing on ​ATT​ as the primary metric to measure the efficiency of TSC systems. The authors also use ​t-SNE plots​ and ​violin plots​ to visualize the state recovery performance, providing additional insights into the robustness of the proposed framework.", "theoretical_claims": "The theoretical claims are ​solid and well-grounded. The paper builds on the foundations of diffusion models and reinforcement learning, providing a clear theoretical framework for the proposed methods. The authors discuss the ​forward and reverse processes​ of diffusion models and how they can be adapted for TSC tasks. The theoretical basis for the ​denoise​ and ​repaint​ algorithms is also well-explained, with references to existing literature on diffusion models and adversarial attacks. The theoretical claims are supported by the experimental results, demonstrating the practical applicability of the proposed methods.", "experimental_designs_or_analyses": "The experimental design is ​rigorous and comprehensive. The authors use ​real-world datasets​ from three different cities (JiNan, HangZhou, and New York) to evaluate the performance of RobustLight under various adversarial attacks and sensor damage scenarios. The experiments are well-structured, with clear comparisons between RobustLight and traditional/RL-based TSC methods. The authors also conduct ​ablation studies​ to analyze the impact of different components of RobustLight, providing insights into the contribution of each component to the overall performance. The analysis is thorough, with detailed discussions on the results and their implications for real-world TSC systems.", "relation_to_broader_scientific_literature": "The key contributions of the paper are ​highly relevant to the broader scientific literature. The integration of diffusion models with reinforcement learning for TSC tasks is a novel approach that addresses the limitations of existing methods, which often fail to handle both adversarial attacks and missing data simultaneously. The proposed framework builds on recent advancements in diffusion models and RL, extending their applications to the domain of traffic signal control. The paper also contributes to the growing body of research on ​robust RL​ and ​adversarial defense​ in real-world systems, providing a practical solution for improving the security and reliability of TSC systems.", "essential_references_not_discussed": "There are no essential references not discussed in the paper.", "other_strengths_and_weaknesses": "All strengths and weaknesses are mentioned above.", "other_comments_or_suggestions": "No.", "questions_for_authors": "- Scalability: The paper mentions that the computational cost increases with the number of intersections. Could the authors elaborate on potential strategies to scale RobustLight for large-scale urban networks with hundreds of intersections?\n- Real-World Deployment: While the experiments are conducted on real-world datasets, the paper does not discuss the challenges of deploying RobustLight in real-world TSC systems. What are the practical considerations and potential barriers to real-world implementation?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a novel framework for utilizing a diffusion model to enhance the robustness of traffic signal control systems against adversarial attack and missing data. 

Among the paper's strengths are the following: (1) the framework makes novel use of diffusion models to address an important practical problem; (2) the paper's claims are supported by an extensive experimental study utilizing multiple real-world data sets which gives clear and convincing evidence; and (3) the approach is capable of handling multiple complex attack types. 

With respect to weaknesses: (1) there is some concern about the latency of the method and whether it can handle the real-time aspects of the traffic signal control domain, and (2) the significance of the results are weakened somewhat by the fact that diffusion models are not immune to adaptive attacks specifically designed to exploit their weaknesses, and (3) the paper lacks comparison with some more recent advances in diffusion models and reinforcement learning.

Overall, the paper makes a nice contribution and the consensus recommendation is that the paper be accepted. Please be sure to address all comments and suggestions made by the reviewers when preparing the final version of the paper.
Decision: Accept (poster)