Practical Adversarial Attacks on Spatiotemporal Traffic Forecasting Models
Authors: Fan Liu
Hao Liu
Wenzhao Jiang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Spatiotemporal traffic foresting
Adversarial attack

Abstract: Machine learning based traffic forecasting models leverage sophisticated spatiotemporal auto-correlations to provide accurate predictions of city-wide traffic states. However, existing methods assume a reliable and unbiased forecasting environment, which is not always available in the wild. In this work, we investigate the vulnerability of spatiotemporal traffic forecasting models and propose a practical adversarial spatiotemporal attack framework. Specifically, instead of simultaneously attacking all geo-distributed data sources, an iterative gradient guided node saliency method is proposed to identify the time-dependent set of victim nodes. Furthermore, we devise a spatiotemporal gradient descent based scheme to generate real-valued adversarial traffic states under a perturbation constraint.
Meanwhile, we theoretically demonstrate the worst performance bound of adversarial traffic forecasting attacks. Extensive experiments on two real-world datasets show that the proposed two-step framework achieves up to 67.8% performance degradation on various advanced spatiotemporal forecasting models. Remarkably, we also show that adversarial training with our proposed attacks can significantly improve the robustness of spatiotemporal traffic forecasting models.

Practical Adversarial Attacks on Spatiotemporal Traffic Forecasting Models
Fan LIU
AI Thrust&RBM. The Hong Kong University of Science and Technology (Guangzhou) fliu236econnect wwust-gz. edu. cns cns.iufaneust.fi
Hao LIU
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou) Guangzhou HKUST Fok Ying Tung Research Institute CSE, The Hong Kong University of Science and Technology liuh@ust. .kk
Wenzhao Jiang
AT Thrust, The Hong Kong University of Science and Technology (Guangzhou) #jiang431@conne wwust-uuuhug edu. en
Abstract
Machine learning based traffic forecasting models leverage sophisticated spatiotem- poral auto-correlations to provide accurate predictions of city-wide traffic states. However, existing methods assume reliable and unbiased forecasting environ- ment, which is not always available in the wild. In this work, we investigate the vulnerability of spatiotemporal traffic forecasting models and propose a practical adversarial spatiotemporal attack framework. Specifically, instead of simultane- ously attacking all geo-distributed data sources, an iterative gradient- guided node saliency method is proposed to identify the time-dependent set of victim nodes. Furthermore, we devise a spatiotemporal gradient descent based scheme to generate real-valued adversarial traffic states under perturbation constraint. Mcanwhile, we theoretically demonstrate the worst performance bound of adversarial traffic forecasting attacks. Extensive experiments proposed two-step framework achieves up to 67.8% performance degradation on various advanced spatiotemporal forecasting models. Remarkably, that adversarial training with our proposed attacks can significantly improve the robustness of spatiotemporal traffic forecasting models. Our code is available in bttpps://qiteee com usail Aakust/Adva
real-world datasets show that the
also show
Introduction
Machine learned spatiotemporal forecasting models have been widely adopted in modern Intelligent Transportation Systems (ITS) to provide accurate and timely prediction of traffic dynamics, e.g., traffic flow [1], traffic speed (2 3 and the estimated time of arrival [4, []. Despite fruitful progress improving the forecasting accuracy and utility [6) little attention has been paid to the robustness of spatiotemporal forecasting models. For example, Figure [ demonstrates that injecting slight adversarial perturbations on a few randomly selected nodes can significantly degrade the traffic
"Corresponding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

pied ctien
60
groune Fruth
35
300
150
(b)
malicious attacker
Brod
advinaar prodiction
W
300
(0)
Figure I: An illustration of adversarial attack against spatiotemporal forecasting models on the Bay Area traffic network in California, the data ranges from January 2017 to May 2017. (a) Adversarial attack of coo-distributed data. The malicious attacker may inject adversarial examples into a few randomly selected geo-distributed data sources. (e.g., roadway sensors) to mislead the prediction of the whole traffic forecasting system. (b) Accuracy drop of victim nodes. By adding less than 50% traffic speed perturbations to 10% victim nodes, we observe 60.4% accuracy drop of victim nodes in morning peak hour. (c) Accuracy drop of neighbouring nodes. Due to the information diffusion of spatiotemporal forecasting models, the adversarial attack also leads to up to about 47.23 47.23% accuracy drop for neighboring nodes.
forecasting accuracy of the whole system. Therefore, this paper investigates the vulnerability of traffic forecasting models against adversarial attacks.
In recent years, adversarial attacks have been extensively studied in various application domains, such as computer vision and natural language processing 77]. However, two major challenges prevent applying existing adversarial attack strategies to spatiotemporal traffic forecasting. First, the traffic forecasting system makes predictions by exploiting signals from geo-distributed data sources (e.g., hundreds of roadway sensors and thousands of in-vehiele GPS devices). It is expensive and impractical to manipulate all data sources to inject adversarial perturbations simultancously. Furthermore, vffteoffthh-ar traffic forecasting models propagate local traffic states through the traffic network for more accurate prediction (5). Attacking a few arbitrary data sources will result in vode-varying effects on the whole system. How to identify the subset of salient victim nodes with a limited attack budget to maximize the attack effect is the first challenge. Second, unlike most existing adversarial attack strategies that focus on time-invariant label classification [8] [9, the adversarial attack against traffic forecasting aims to disrupt the target model to make biased predictions of continuous traffic states. How to generate real-valued adversarial examples without access to the ground truth of future traffic states is another challenge.
To this end, in this paper, we propose a practical adversarial spatiotemporal attack framework that can disrupt the forecasting models to derive biased city-wide traffic predictions. Specifically, we first devise an iterative gradient- guidded method to estimate node saliency, which helps to identify a small time-dependent set of victim nodes. Moreover, spatiotemporal gradient descent scheme is proposed to guide the attack direction and generate real-valued adversarial traffic states under human imperceptible perturbation constraint. The proposed attack framework is agnostic to forecasting model architecture and generalizable to various attack settings, i.e., white-box attack, grey-box attack, and black-box attack. Meanwhile, we theoretically analyze the worst performance guarantees of adversarial traffic forecasting attacks. We prove the adversarial robustness of spatiotemporal traffic forecasting models is related to the number of victim nodes, the maximum perturbation bound, and the maximum degree of the traffic network.
Extensive experimental studies on two real-world traffic datasets demonstrate the attack effectiveness of the proposed framework on state-of-the-an spatiotemporal forecasting models. We show that attacking 10% nodes in the traffic system can break down the global forecasting Mean Average Error (MAE) from 1.975 to 6. 1329. Moreover, the adversarial attack can induce 68. 33,6%, and 56.67% performance degradation under the extended white-box and black-box attack settings, respectively. Finally, we also show that incorporating adversarial examples we generated with adversarial training can significantly improve the robustness of spatiotemporal traffic forecasting models.

---

2
Background and problem statement
In this section, we first introduce some basics of spatiotemporal traffic forecasting and adversarial attack, then formally define the problem we aim to address.
2.1 Spatiotemporal traffic forecasting
Let G: = EV.E) denote a traffic network at time step i, where V is set of nodes (e.g., regions, road segments, roadway sensors, etc.) and is a set of edges. The construction of 9e can be categorized into two types, (1) prior-based, which pre-define Gt based on metrics such as geographical proximity and similarity [I0], and (2) learning-based, which automatically learns G. in an end-to- end way (2). Note the G. can be static or time-evolving depending on the forecasting model. We as the spatiotemporal features associated to G,, where x,, E R represents the e-dimensional time-varying traffic conditions (e.g., traffic volume, traffic speed) and contextual features (e.g., weather, surrounding POIs) of node v. E V/ at t. The spatiotemporal traffic forecasting problem aims to predict traffic states for all Us V over the next time steps,
denote X, = (X1.t, "2.t.''. Xn.t)
Y Y4HEIT = (0 -+1:))
(I)
where H4-1+1:: = {(x,-++ 1---T-1---44)1.... 91-T4))..... (X,, [()} denotes the traffic states contains input features and the traffic network in previous T time steps. so(-) is the spatiotemporal traffic forecasting model parameterized by 0, and Y Y++1: = (Y (+1, Yz+2,.... Y is the estimated traffic condi- tions of interest of V from time step + to T. We denote Y Y44144 = (Y,11, Y442, ".. Y as the ground truth of H
Note the above formulation is consistent with the state-of-the-ans Graph Neural Network (GNN) based spatiotemporal traffic forecasting models [2] 10 11 12], and is also generalizable other
variants such as Convolutional Neural Network (CNN) based approaches [13
2.2 Adversarial attack
Given a machine learning model, adversarial attack aims to mislead the model derive biased predictions by generating the optimal adversarial example
arg max L(x', y; 0) s.t. || =11p S,
where z' is the adversarial example with maximum bound E under Lp norm to guarantee the perturba- tion is imperceptible human, and y is the ground truth clean example x. Various gradient-baas based methods have been proposed to generate adversarial examples, such as FGSM [14] PGD [8, MIM (}) etc. For instance, the adversarial example x' = x + (..gnVVCCCC y: 0)) in FGSM, where sign(.) is the Signum function and LCE(.) is the cross entropy loss.
Note the adversarial attack happened in the testing stage, and the attackers cannot manipulate the forecasting model or its output. On the benign testing set, the forecasting model can perform well, Based the amount of information the attacker can access in the testing stage, the adversarial attack can be categorized into three classes. White-box altack. The attacker can fully access the target model, including the model architecture, the model parameters, gradients, model outputs, the input traffic states, and the corresponding labels. Grey-box attack. The attacker can partially access the system, including the target model and the input traffic states, but without the labels. Black-box attack. The attacker can only access the input traffic states, query the outputs of the target model or leverage surrogate model to craft the adversarial examples.
2.3 Adversarial attack against spatiotemporal traffic forecasting
This work aims to apply adversarial attacks to spatiotemporal traffic forecasting models. We first define the adversarial traffic state as follow,
H (X.. G.) || S I[x] -x) Sel, se).
(3)
where S, E {0, 1 is a diagonal matrix with ith diagonal element indicating whether node is vietim node, and X is the perturbed spatiotemporal feature named adversarial spatiotemporal feature.
3

---

We restrict the adversarial traffic state by the victim node budget and the perturbation budget E. Note that following the definition of adversarial attack, we leave the topology of G. immutable we regard the adjacency relationship in an end-to-end way.
part of the model parameter that may be automatically learned
Attack goal. The attacker aims to craft adversarial traffic states to fool the spatiotemporal forecasting model to derive biased predictions. Formally, given a spatiotemporal forecasting model fo()), the adversarial attack against spatiotemporal traffic forecasting is defined as
max H 1ETics
Σ tETiest
C(fo- (H, (Hi-T-r) Y++1++))
(4a)
s.t., = =rg min
Σ ((o(HHi-T4ec) Y(41447),
(4b)
tETerato
where Trest and Terain denote the set of time steps of all testing and training samples, respectively, (()) is the loss function measuring the distance between the predicted traffic states and ground truth, and 0* is optimal parameters learned during the training stage.
Since the ground truth (i.e., future traffic states) under the spatiotemporal traffic forecasting setting is unavailable at run-time, the practical adversarial spatiotemporal attack primarily falls into the grey- box attack setting. However, investigating white-box attacks is still beneficial to help us understand how adversarial attack works and can help improve the robustness of spatiotemporal traffic forecasting models (e.g,, apply adversarial training). We discuss how to extend our proposed adversarial attack framework white-box and black-box settings in Section 3.2
3 Methodology
In this section, we introduce the practical adversarial spatiotemporal attack framework in detail. Specifically, our framework consists of two steps: (1) identify the time-dependent victim nodes, and (2) attack with the adversarial traffic state.
3.1 Identify time-dependent victim nodes
One unique characteristic that distinguishes attacking spatiotemporal forecasting from conventional classification tasks is the inaccessibility of ground truth at the test phase. Therefore, we first construct future traffic states' surrogate label to guide the attack direction,
Y YItIi (HL-T41)))) + 8441:47
(5)
where (-) is a generalized function (e.g., lanh(.), sin (-), fe)))), 64+12+7 are random variables sampled from a probability distribution (8(+1:147) increase the diversity of the attack direction. In our implementation, we derive & based on the pre- pre-trained forecasting model parameter 0*, and d++1:4+ U(-=/10,e E/ =/10.. In the real-world production [5], the forecasting models are usually updated in an online fashion (e.g., per hours). Therefore, we estimate the missing latest traffic states based on previous input data, Hi g. g. (H, where (-((-) is the estimation function parameterized by pp. For simplicity, we directly obtain ip from the pre-trained traffic forecasting model fo- (-).
With the surrogate traffic state label Y Y441144T we derive the time-dependent node saliency (TDNS) for each node as
o (F(f ( 0x.
M,
(6)
where ([ff(FHi-441) Y(+1)47) is the loss function and a is the activation function. Intuitively, M, reveals the node-wise loss impact with the same degree of perturbations. Note depending on the time step f, M, may vary. A similar idea also has been adopted to identify static pixel saliency for image classification [[S]
More in detail, the loss function L(fo (H. Y(41++7) in Equation 6 is updated by the iterative gradient-based adversarial method X
(i-1) X (-T+1 clipx -rriue 1- 1e + asign VL(fe- (i))) Ye41447))) (7)

---

where H 1-T++: is adversarial traffic states at i-th iteration, is the step size, and - (-)
is the project operation which clips the spatiotemporal feature with maximum perturbation bound e.
the time-dependent node saliency gradient Note H t-1+1 H
{ (Fle-T+ia, Ye41++)611 )" j=1
For each batch of data derived by
1 Σ
ac(fo- (Hi-T+1a), Y(+1047) } iX
g: =
(8)
where y the batch size. We use the RELU activation function to compute the non-negative saliency score for each time step,
M, [Relu(g:)l22
(9)
Finally, obtain the set of victim node St based on Mr.
{
if e E To,(Me, otherwise
(i.i),
(10)
where s denotes the i-th diagonal element of Sy. and Top(-) is a 0-1 indicator function returning if o is the top-k salient node time step t.
3.2 Attack with adversarial traffic state
Based on the time-dependent victim set, we conduct adversarial attacks to spatiotemporal traffic forecasting models. Specifically, we first generate perturbed adversarial traffic features based on gradient descent methods. Take the widely used Projected Gradient Descent (PGD) [8 for illustration, we construct Spatiotemporal Projected Gradient Descent (STPGD) as below,
(t-1) X t- ++# = clipx', Titag T++1 + asign( V [(fo (H (H (t-1)
(1))))) Y(41:47) S()), (11)
(i-1) where H' 1-7+1: is the adversarial traffic state at - 1-th iteration in the iterative gradient descent, a is the step size, and clipx, (.) is the operation to bound adversarial features in a E ball.
Note x = X, Instead of perturbing all nodes as in vanilla PGD, we only inject perturbations on selected victim nodes in S, Similarly, we can generate perturbed adversarial traffic features by extending other gradient based methods, such as MIM [9]
In the testing phase, to apply adversarial attack, where A Hi + Hi {X} X,) St +,, E H',-T+1a and AH'' {(X', X,) S, 0) || S 7. -X),, S se) E AH''--+t1: The de- tails of the adversarial spatiotemporal attack framework under the grey-box setting is in algorithm [
inject the adversarial traffic states H'' 1-14t + --
The overall adversarial spatiotemporal attack can be easily extended the white-box and black-box settings, which are detailed below.
White-box attack. Since the adversaries can fully access the data and labels under the white-box setting, we directly use the real ground truth traffic states to guide the generation of adversarial traffic states. The detailed algorithm is introduced in Appendix A.
Black-box attack. The most restrictive black-box setting assumes limited accessibility to the target model and labels. Therefore, we first employ surrogate model, which can be learned from the training data or by querying the traffic forecasting service [16. 17.. Then we generate adversarial traffic states based on the surrogate model to attack the targeted traffic forecasting model. Please refer to Appendix A.2 for more details.
We conclude this section with the theoretical upper bound analysis of the proposed adversarial attack strategy. In particular, we demonstrate the attack performance against the spatiotemporal traffic forecasting model is related to the number of chosen victim nodes, the budget of adversarial perturbations, as well as the traffic network topology.
5

---

Theorem Let Z = (o(--T+41s) and ( = fe(H'i--+ts) be the L-th layer embeddings of the forecasting model, the upper bound of the adversarial loss satisfies
-111 (A3C))'
where À denotes maximum weight bound in all layers of the forecasting model, 3 denotes parameter of the activation function in fo(.), C denotes the maximum degree of G. 1) and are the budget of number of victim nodes and perturbations, respectively.
Proof. Please refer Appendix B
Algorithm 1: Adversarial spatiotemporal attack under the grey-box setting
Input: Previous traffic data, pre-trained spatiotemporal model fa- (-), pre-trained traffic state prediction model (-), maximum perturbation budget e, victim node budget 1), and iterations K.
Result: Perturbed Adversarial traffic states H'i-T+1:-
Step 1: Tdent fy t me-edependdee vi citim nodes
Estimate current traffic state HL.- - lat by function (p(-);;
Construct future traffic state's surrogate labels Y++1++7 by Equation 5
Compute the time-dependent node saliency MM, with H and Y (4144 by Equation 6
Obtain the victim node set Sy by Equations 10
/* Step
Attack with adversarial traffic state
Initialize adversarial traffic state Ho(0)
(-T+1: H4-T+14
for i i= 1 to K do
Generate perturbed adversarial features (
1-T+1# by Equation
AH ((x (-+1 X,-T+11) St,
end
10 Return :''-++1 = Hi- 1-1114 + AH''
4 Experiments
4.1
Experimental setup
Datasets. We use two popular weal-world datasets to demonstrate the effectiveness of the proposed adversarial attack framework. (1) PEMS-BAL [18] traffic dataset is derived from the California Transportation Agencies (CalTrans) Performance Measurement System (PeMS) ranging from January 1. 2017 to May 31, 2017. 325 traffic sensors in the Bay Area collect traffic data every 5 minutes. (2) METR-LA 19 is traffic speed dataset collected from 207 Los Angeles County roadway sensors. The traffic speed is recorded every 5 minutes and ranges from March 1, 2012 to June 30, 2012. For evaluation, all datasets are chronologically ordered, we take the first 70% for training, the following 10% for validation, and the rest 20% for testing. The statistics of the two datasets are reported in Appendix C
Baselines. In the current literature, few studies can be directly applied to the real-valued traffic forecasting attack setting. To guarantee the fairness of comparison, we construct two-step baselines as below. For victim node identification, we adopt random selection and use the topology- based methods (i.e., node degree and betweenness centrality (20) select victim nodes. We also employ PageRank (PR) [21] as the baseline to decide the set of victim nodes. For adversarial traffic state generation, we adopt two widely used iterative gradient- based methods, PGD [8] and MIM (9]. to generate adversarial perturbations. In summary, we construct eight two-step baselines, PGD- Random, PGD-PR, PGD-Centrality, PGD--eereee, MIM-Random, MIM-PR, MIM-Centrality and MIM-Degree. For instance, PGD-PR indicates first identifying victim nodes with PageRank and then applying adversarial noises with PGD. Depending on the adversarial perturbation method, we compare two variants of proposed framework, namely STPGD-TDNS and STMIM-TDNS.

---

Table 1: Adversarial attack performance under the grey-box setting.
PEMS-BAY
METR-LA
Metries
G-MAE
L-MAB
G-RMSE
L-RMSB
G-MAE
L-MAE
G-RMSE
L-RMSE
Attack methods
non-uttack
1.755
4,0220
6.3504
11.8424
PGD-Random
4,9876
3.7431
8,9343
7,8006
7.8947
2.7030
13.2749
5,9501
PGD-PR
4,8599
3.5819
8.8215
7.6727
7,9003
2.7070
13,2669
5.9132
PGDD PGID-Centrai Centrality
5.1640
3,9585
9.1369
8.0333
7.8554
2.7107
13.3100
5.9422
PGD-Degree
4.9121
3.6675
8.8486
7.7263
7,9011
2.7316
13.3738
6.0661
MIM-Kandom
5.3645
4.1739
9.7082
8.6825
7.7115
2.3793
13.1724
5,6882
MIM-PR
5.2405
4.12866
9.5902
8,5600
7.7206
2.3774
13.1294
5.6548
MIM-Centrality
5.5321
4.3820
9,9312
8,9331
7.7074
2.4255
13.2233
$.7498
MIM-Degree
5.3500
4.1745
9,5808
8.5573
7.7026
2.3877
13.2570
5,8229
STPGD-TDNS
6.1329
5.1647
10.6723
9,7003
7.7191
2.6534
13,6693
6.6794
STMIM-TDNS
5.6706
4.7010
10.1336
9.1813
7.9381
2.8848
13.8592
6,9885
Target model. To evaluate the generalization ability of the proposed adversarial attack framework. we adopt the state-of-the-art spatiotemporal traffic forecasting model, Graph WaveNet (Gwnet) [2]. as the target model. Evaluation results more target models reported in Appendix
Evaluation metrics. Our evaluation focus on both the global and local effect of adversarial attacks on spatiotemporal models,
BR ((Jo(H'' (-+H+)) Y Y(4H++)), BRETI..: (-T=-+++:)), fo (Hi-T41a)),
(12a)
(12b)
where C(.) is user-defined loss function. Different from the majority target of adversarial attacks that are classification models (e.g., adversarial accuracy), traffic forecasting is defined as a regression task. Therefore, we adopt Mean Average Error (MAE) [22] and Root Mean Square Error (RMSE) [23 for evaluation. More specifically, we define Global MAE (G-MAE), Local MAE (L-MAE), Global RMSE (G-RMSE), Local RMSE (L-RMSE) to evaluate the effect of adversarial attacks on traffic forecasting. Please refer to Appendix D for detailed definitions of four metrics.
Implementation details. All experiments are implemented with Py Torch and performed on a Linux server with 4 RTX 3090 GPUs. traffic speed normalized [0, 1]. The input length T and output length T are set to 12. We select 10% nodes from the whole nodes as the victim nodes, and e is set to 0.5. The batch size y set to 64. The iteration K is set to 5, and the step size a is set 0.1.
4.2 Overall attack performance
Table the overall attack performance of our proposed approach against the original forecast- ing model and eight baselines with respect to four metrics. Note larger value indicates better attack performance and worse forecasting accuracy. Specifically, we can make the following observations. First, the adversarial attack can significantly degrade the traffic forecasting performance. For example, our approach achieves (67.79) 667.79% 62.31% and (19.88%, 14.55%) global performance degradation compared with the original forecasting results on PeMS- BAY and METR-LA dataset, respectively. Second, our approach achieves the best attack performance against all baselines. In particular, STPGD-TDNSS achieves (15.8%% 15.39% global performance improvement and (23.35%, 17.19%) local performance improvement on the PeMS-BAY dataset. Similarly, STMIM- -TMN achieves (2.44%, 2. 2000)) global performance improvement and (11 11.20%, 2.70%) local performance improve- ment on the METR-LA dataset. Moreover, we observe STPGD-TDNS and STMIM-TDNS, two variants of our framework, respectively achieve the best attack performance on PeMS-BAY and METR-LA datasets, which further validate the superiority our framework for flexibly integrate different adversarial perturbation methods. Overall, our adversarial attack framework successfully disrupts the traffic forecasting model to make biased predictions.
4.3 Ablation study
Then we conduct ablation study on our adversarial attack framework. Due to page limit, we report the result of STPGD-TDNS on the PeMS-BAY dataset. We consider two variants of our approach: (1) w/o TDNS that randomly choose victim nodes to attack, and (2) w/o STPGD that apply vanilla PGD noise to selected victim nodes. As reported in Table observe (3.91%, 6. 5.28%, 2.97%, 3.60%) and

---

Table 2: Ablation study on PeMS-BAK
G-MAE
L-MAE
G-RMSE
L-RMSE
non-atiack
1.975
4.0220
w/o TDNS
5,9024
4.8595
10.364
9.3635
w/o STPGD
4.5969
3,3876
8,4572
7.2949
STPGD-TDNS
6.1329
5.1647
10.6723
9.7003
18
10.75
10.50
STPGD-TNDS
10.25
G-AMSE
12
STRGD-TNDS
G-RMSE
STPGD-TNDS
C-RMSE
10.00
PGD-Random
PGD Random
7
PGD-Random
PGD-PR
PGD-PR
PGD-Centrality
8
PGD PGD-Centie Centrality
VGD-Centrality
9.25
PGD-Degree
PGD-Degree
5
GGD-Degree
9.00
8.75
20
30
40
20
40
60
80
100
20
40
60
80
100
120
(%)
(a) Victim node budget
(b) Perturbation budget
(c) Ratch size
Figure 2: Parameter sensitivity on PeMS-BAY.
(33.41%, 52.45%, 26.19% 32.97%) attack performance degradation four metrics by removing proposed TDNS and STPGD module, respectively. The above results demonstrate the effectiveness of the two-step framework. Moreover, observe that the STPGD module plays more important role the adversarial spatiotemporal attack.
4.4 Parameter sensitivity
We further study the parameter sensitivity of the proposed framework, including the number of victim nodes 7, the perturbation budget e, and the batch size y. Due page limit, report the result of G-RMSE on the PeMS-BAY BAY dataset. We observe similar results by using other metrics and on the METR-LA dataset. Each time we vary parameter, set other parameters to their default values.
Effect of 1)- First, we vary the number of victim nodes from 0% to 40%. As reported in Figure 2 (a), Our approach achieves the best attack performance with limited victim node budget, and the advantage decrease when the attack can be applied to more nodes.
Effect of E. Second, we vary the perturbation budget from 0% to 90%. As shown in Figure (b), the G-RMSE first increase and then slightly decrease. This perhaps because the clip function in Equation II weakens the diversity of attack noises.
Effect of Finally, we vary the batch size from to 128, illustrated in Figure (c). We observe the adversarial attack relatively stable to the batch size. However, too large batch size reduces the attack performance, which may induce over smooth of Equation
4.5 Extended analysis under different attack settings
Table reports the overall attack performance of our proposed approach against the original fore- casting model and four PGD-based baselines under the white-box and black-box attack settings. For the white-box attack, since the attacker can fully access the data and model, we re-train the forecasting model without requiring estimating the latest traffic states. For the black-box attack, we adopt STAWNET [2] the surrogate model. The experimental results are summarized in Table First, we observe adversarial attacks significantly degrade the performance of the traf- fie forecasting model under both white-box and black-box settings. For examples, our approach achieves ((68.55% 66.12%) and (56.67%, 50.78%) global performance degradation compared with the vanilla forecasting model under white-box and black-box attack. Moreover, our approach con- sistently achieves the best attack performance against baselines. To be more specific, our approach yield (4.61% .6611,.0 9. 9313% and (1.70%, 3.28%) global performance improvement under the white-bor box setting and black-box setting, respectively. In addition, observe higher attack effectiivnness under

---

Table 3: Adversarial attack performance on PeMS-BAY under white-box and black-box settings.
White-box
Black-box
Metries
Attack methods
G-MAE
L-MAB
G-RMSE
L-RMSB
G-MAE
L-MAE
G-RMSE
L-RMSE
non-uttack
2.0288
4,2476
1.9744
4.0219
PGD-Random
6.1477
5.0463
10.9217
9.5163
4.241
2.9738
7.3804
5.99
PGD-PR
6.1586
5.0713
10.7584
9.3405
4.4748
3.2605
7.9037
6,6336
PGID- Centrality
6.1723
5.0823
10.9468
9.5272
4,4859
3.3002
7.8795
6.61445
PGD-Degree
6.1507
5.0495
10.9375
9,5282
4.3577
3.1572
7.6159
6.2971
PGD-TDNS
6,4709
5,4953
12.1764
10,7262
4,5636
3,3543
8.1716
6,9388
Table 4: Performance of defense adversarial spatiotemporal attack on PeMS-BAN (Values in paren-
theses indicate std)
Attack methods
Non-attack
RGD-Random
PGD-PR
PGD Contrellttty
PGD-Degree
Defense stralegies
Non-defense
2,0288
6,1477
6.1386
6,1723
6.17
AT
2.1156
2,5456 (0.0249)
2.5539 (0.0375)
2.5660 (0.0281)
2,5394 (0.0279)
Mixup AT-TDNS
2,3090
2.7482 (0.0126)
2.7573 (0.0241)
2.7501 (0.0088)
2.7788 (0.0234)
2.0935
2.4695 (0.0036)
2.4463 (0.0075)
2.4549 (0.0023)
2.4474 (0.0069)
the white-box setting and lower attack effectivenees under the black-box setting compared to the grey-box setting. This makes sense as the white-box setting can fully access the data and label, while the black-box has more restrictive data accessibility and relies on the surrogate model to apply adversarial spatiotemporal attack.
4.6 Defense adversarial spatiotemporal attacks
Finally, we study the defense of adversarial spatiotemporal attacks. One primary goal of our study is to help improve the robustness of spatiotemporal forecasting models. Therefore, we propose to incorporate the adversarial training scheme for traffic forecasting models with our adversarial traffic states, denoted by AT-TNDS. We compare it with (I) conventional adversarial training (A7) [8] and (2) Mixup [24] with our adversarial traffic states. Note that we also tried other strategies, such as adding L2 regularization, etc., which fail to defend the adversarial spatiotemporal attack. The other state-of-the-art adversarial training methods, such as TRADE [25). cannot be directly applied in regression tasks. Please refer to Appendix E for more training details.
The results in G-MAE on the PeMS-BAY are reported in Table Overall, we observe AT' or Mixup can successfully resist the adversarial spatiotemporal attack, and AT-TDNS that combines the adversarial training scheme with our adversarial traffic states achieves best defensive performance. The above results indicate the defensibility of adversarial spatiotemporal attacks, which should be further investigated to deliver a more reliable spatiotemporal forecasting service in the future.
Related work
Spatiotemporal traffic forecasting. In recent years, the deep learning based traffic forecasting model has been extensively studied due to its superiority in jointly modeling temporal and spatial dependen- cies 100 6, 26
2 12 28]. To name few, STGCN (TO) applied graph convolution and gated causal convolution to capture the spatiotemporal information in the traffic domain, ASTGCN (II] proposed spatial-temporal attention network for capturing dynamic spatiotemporal correlations. As another example, Graph WaveNet 2 adaptively captures latent spatial dependency without requiring prior knowledge of the graph structure. The key objective of the above mentioned models is more accurate traffic forecasting. The vulnerability of spatiotemporal traffic forecasting models remains an under explored problem.
Adversarial attack. Deep neural networks have been proven vulnerable to adversarial examples X 14.. As an emerging direction, various adversarial attack strategies on graph-structured data have been proposed, including both target-attack and non-target attack (29] 30). However, existing efforts on adversarial attacks mainly focus on classification tasks with static label
p 24. Only a few
works study the vulnerability of GCN based spatiotemporal forecasting models under query-based attack [3I] and generate adversarial examples based on evolutionary algorithms [32] In this paper,
study the gradient based adversarial attack method against spatiotemporal traffic forecasting

---

models, which is model-aggostie and generalizable to various attack settings, i.e., white-box attack, grey-box attack, and black-box attack.
6
Conclusion
This paper showed the vulnerability of spatiotemporal traffic forecasting models under adversarial attacks. We proposed a practical adversarial spatiotemporal attack framework, which agnostic to forecasting model architectures and is generalizable to various attack settings. To be specific, we first constructed an iterative gradient guided node saliency method to identify a small time-dependent set of victim nodes. Then, we proposed spatiotemporal gradient descent based scheme to generate real- valued adversarial traffic states by flexibly leveraging various adversarial perturbation methods. The theoretical analysis demonstrated the upper bound of the proposed two-step framework under human imperceptible victim node selection budget and perturbation budget constraints. Finally, extensive experimental results real- world datasets verify the effectiveness of the proposed framework. The reported results will inspire further studies on the vulnerability of spatiotemporal forecasting models, as well practical defending strategies for resisting adversarial attacks that be deployed in real-world ITS systems.
Acknowledgments and Disclosure of Funding
This work is supported by the National Natural Science Foundation of China under Grant No.62102110, and Foshan HKUST Projects (FSUST2I-FYTRIOIA, FSUST2I-FYTRIO2A).
References
[1] Xu Chen, Junshan Wang, and Kunqing Xie. Trafficstream: A streaming traffic flow forecasting framework based on graph neural networks and continual learning. Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pages 3620-3626. ijcai.org, 2021.
[2] Zonghan Wu, Shirui Pan. Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial- temporal graph modeling. In Sarit Kraus, editor, Proceedings of the Twenty-Eight International Joint Conference on Artificial Intelligence, ICCAI 2019, Macao, China, August 10-16. 2019, pages 1907-1913. ijeai.org, 2019.
[3] Binbing Liao, Jingqing Zhang, Chao Wu, Douglas Mcllwraith, Tong Chen, Shengwen Yang, Yike Guo, and Fei Wu. Deep sequence learning with auxiliary information for traffic prediction. In Yike Guo and Faisal Farooq, editors, Proceedings the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, KDD 2018, London, UK, August 19-23, 2018. pages 537-546. ACM, 2018.
[4] Jing Qiu, Lei Du, Dongwen Zhang, Shen Su, and Zhihong Tian. Nei-tte: Intelligent traffic time estimation based on fine-grained time derivation of road segments for smart city. IEEE Trans. Ind. Informaties. 16(4):2659 2666, 2020.
[5] Austin Derrow-Pinion. Jennifer She, David Wong, Oliver Lange, Todd Hester, Luis Perez, Mare Nunkesser, Scongjac Lec, Xueying Guo, Brett Wiltshire, Peter W. Battaglia, Vishal Gupta, Ang Li, Zhongwen Xu, Alvaro Sanchez-Gonzalez, Yujia Li, and Petar Velickovic. ETA prediction with graph neural networks in google maps. In Gianluca Demartini, Guido Zuccon, J. Shane Culpepper, Zi Huang, and Hanghang Tong, editors, CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November
2021, pages 3767 3776. ACM, 2021.
[6] Lei Bai. Lina Yao, Can Li. Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin, editors, Advances in Neural Information Processing Systems on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
33: Annual Conference
[7] Han Xu, Yao Ma, Haochen Liu, Dehayan Deb, Hui Liu, Jiliang Tang, and Anil K. Jain. Adversarial attacks and defenses in images, graphs and text: A review. Int. J. Autom. Comput., 17(2):511-178. 2020.
[8] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Repre- sentations. ICLR 2018, Vancouver; BC, Canada, April 30- - May 3, 2018, Conference Track Proceedings OpenReview.net, 2018
10

---

[9] Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT. USA, June 18-22, 2018, pages 9185-9193, Computer Vision Foundation IEEE Computer Society, 2018.
[10] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Jérôme Lang, editor, Proceedings of the Twenty-Seventh Seventh International Joint Conference Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pages 3634-3640. ijeai.org. 2018.
[11] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial-tempora graph convolutional networks for traffic flow forecasting. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAA! 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Howaii, USA, January 27- February 1, 2019, pages 922-929. AAAI Press, 2019.
[12] Chenyu Tian and Wai Kin (Victor) Chan. Spatial-temporal attention wavenet: A deep leamnigg framework for traffic prediction considering spatial-lemporal dependencies. IET Intelligent Transport Systems. 15(4):549-561, 2021.
[13] Junho Zhang, Yu Zheng, and Dekang Qi. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Satinder Singh and Shaul Markovitch, editors, Proceedings of the Thirty-First AAAI Conference Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 1655-1661. AAAI Press, 2017.
[14] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[15] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visual- ising image classification models and saliency maps. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Bansf. AB, Canada, April 14-16. 2014, Workshop Track Proceedings, 2014.
[16] Haitao Yuan and Guoliang Li. A survey of traffic prediction: from spatio-temporal data to intelligent transportation. Data Science and Engineering, 6(1):63-8 85, 2021.
[17] Yinpeng Dong, Shuyu Cheng, Tianyu Pang. Hang Su, and Jun Zhu. Query efficient black- box adversarial attacks guided by transferbbssed prior. IEEE Transactions Pottern Analysis and Machine Intelligence, pages 1-1, 2021.
[18] Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway performance mcasurement system: mining loop detector data. Transportation Research Record, 1748(1):96- 102, 2001.
[19] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusson convolutional recurrent neural network: Data-driven traffic forecasting. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[20] Ulrik Brandes. A fasteralgorrthn for betweenness Journal of mathematical sociology, 25(2):163-
177, 2001.
[21] Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford InfoLab, 1999.
[22] Cort J Willmott and Kenji Matsuura. Advantages of the mean absolute emor (MAE) over the root mean square error (RMSE) in assessing average model performance, Climate research, 30( (11::99-822) -2, 2005.
[23] Tianfeng Chai and Roland R Draxler. Root mean square error (RMSE) or mean absolute error (MAE)?- arguments against avoiding msse in the literature. Geoscientific model development, 7(3):1247- 1250. 2014.
[24] Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.nec, 2018.
[25] Florian Tramer and Dan Boneh. Adversarial training and robustness for multiple perturbations. Advances in Neural Information Processing Systems, 32, 2019.

---

[26] Ling Zhao, Yujiao Song. Chao Zhang, Yu Liu, Pu Wang. Tao Lin. Min Deng, and Haifeng Li. T- gen: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 21(9):3848-355 3858. 2020.
[27] Hao Liu, Qiyu Wu, Fuzhen Zhuang. Xinjiang Lu, Dejing Dou, and Hui Xiong. Community-ave aware multi- task transportation demand prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 320-327, 2021.
[28] Jindong Han, Hao Liu, Hengshu Zhu, Hui Xiong, and Dejing Dou. Joint air quality and weather prediction based on multi-adversaria spatiotemporal networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4081 10899, 2021.
[29] Wei Jin, Yaxin Li, Han Yiqi Wang, Shuiwang Ji, Charu Aggarwal, and Jiliang Tang. Adversarial attacks and defenses on graphs: A review, a tool and empirical studies. arXiv preprint urXiv:2003.00653. 2020.
[30] Zijie Zhang, Zeru Zhang, Yang Zhou, Yelong Shen, Ruoming Jin, and Dejing Dou. Adversarial attacks on deep graph matching. Advances in Neural Information Processing Systems, 33:20834 20851, 2020.
[31] Lyuyi Zhu, Kairui Feng, Ziyuan Pu, and Wei Ma. Adversarial diffusion attacks on graph- based traffic prediction models, arXiv preprint arXiv:2 (04.0::000409999 2021.
[32] Fugiang Liu, Luis Miranda- Merend,,Moreno, and Lijun Sun. Spatially focused attack against spatiotemporal graph neural networks. arXiv preprint arXiv:2109.044 149004008,, 2021.
[33] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Adiyya Prakash, editors, KDD '20: The 26th ACM SIGKDD Conference Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 753-763. ACM, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes]
12

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "pCsdlz5zQG", "reviewer": "Reviewer_7aqp", "summary": "This paper presents a novel adversarial attack for spatiotemporal traffic forecasting models. Moreover, theoretically analysis are conducted to demonstrate the worst performance bound of the attack. Comprehensive experiments on real-world datasets show the effectiveness of the effectiveness of the attack, and how the robustness of the models enhances when combined with the corresponding adversarial training.", "questions": "See weakness.", "limitations": "Yes for limitations and no for potential negative societal impact.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is well-written and easy to understand.\n- Introducing adversarial robustness to spatiotemporal models is interesting and novel.\n- The theoretical analysis backs up the proposed attack well.\n- The experiments are comprehensive.\n\nWeakness:\n- I don't see clear weakness of the paper, although I wonder the generalizability of the method to other spatiotemporal-based applications beyond traffic forecasting models.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fdEpIPjCWaY", "reviewer": "Reviewer_rQBx", "summary": "This paper explores the vulnerability of spatiotemporal traffic forecasting models. They proposed a practical adversarial spatiotemporal attack framework. Experiments are conducted to verify the effectiveness.", "questions": "please see the weakness", "limitations": "please see the weakness", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nStrengths:\n1. The first attemp to attack spatiotemporal traffic forecasting models.\n2. The code is released, and thus readers can reproduce the results. \n3. The demonstration is given from empirical and theoretical view. \n\nWeakness:\n1. When the attacks are extended to black-box setting, they  employ a surrogate model, which is trained via querying the threat models. This will lead to a large number of queris, which reduces the efficiency. The authors should discuss this point.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["Privacy and Security (e.g., consent)"]}, "extra_text": {}}, {"review_id": "YctMpq3Nzgf", "reviewer": "Reviewer_EYep", "summary": "The paper tackles the problem of adversarial attacks against “traffic forecasting models”, i.e., models that must predict the conditions of “physical traffic” (e.g., cars) in real world settings. The main contribution is a generic framework of adversarial attacks against such systems, which can be declined to cover both white- and black-box adversarial settings. A comprehensive experimental evaluation on real world data validates the proposal. The paper is also enriched by an ablation study, and by strong theoretical analyses that support the overall findings", "questions": "I thank the authors for their paper, of which I particularly appreciated the research direction: instead of focusing on the (overused) attacks against image classifiers, the paper considers a completely different (but still realistic) setting. I have some comments that the authors can address in a rebuttal, which are reported below.\n\n*Unclear definitions of White/Black-box attacks.* In Section 3.2 the authors describe the potential “variations” of the proposed attack by using the well-known “white/black-box” terminology. However, it is not clear what are the actual assumptions of the considered attacker. For example, the white-box scenario is described as follows: “White-box attack. Since the adversaries can fully access the data and labels under the white-box setting, we directly use the real ground truth traffic states to guide the generation of adversarial traffic states.”. Does this mean that the “white-box” attack assumes an attacker with access to the training data and all labels? Or does this also imply that the attacker knows everything about the target model (i.e., learned parameters, weights and architecture)? Similarly, for black-box attacks the paper states that: “The most restrictive black-box setting assumes limited accessibility to the target model and labels. Therefore, we first employ a surrogate model, which can be learned from the training data or by querying the traffic forecasting service [15, 16]. Then we generate adversarial traffic states based on the surrogate model to attack the targeted traffic forecasting model.” The authors should elucidate if they assume an attacker who can “query” the target model (and whether such queries are constrained or not) or who has no access whatsoever to the targeted model, but only to a subset of the training data (in which case, it is more like a “no-box” attack [A]). For completeness, I have also checked the appendix and there is no mentioning of such details.\n\n*Realistic Feasibility.* I would be delighted if the authors could integrate (perhaps in an appendix) an analysis of the realistic feasibility of the proposed attacks. For example, consider the following statement in the introduction: “How to identify the subset of salient victim nodes with a limited attack budget to maximize the attack effect is the first challenge.” Why would a realistic attacker do that? If, as shown in Figure 1, just by injecting perturbations on “randomly selected” nodes results in a successful attack, then why going further? Would an attacker really opt for such a strategy? What is the cost and potential benefit of such a strategy?\nIndeed, I stress that real attacker operate with a cost/benefit mindset. Considered that the paper tackles a new application domain, it would be impactful to show that some attacks may require little preparation, but can lead to significant performance degradation. Of course, it is still valuable to analyze “worst-case” scenarios, but it’s important to differentiate from attacks that are more likely to occur (e.g., because they are cheaper to stage) from those who are less likely to appear (e.g., because they require a huge resource investment, which can potentially be superior than what the attacker stands to gain).\n\nSome additional issues:\n\n•\tThe following statement in the Introduction requires additional back-up: “Machine learned spatiotemporal forecasting models have been widely adopted in modern Intelligent Transportation Systems (ITS) to provide accurate and timely prediction of traffic dynamics, e.g., traffic flow [1], traffic speed [2], and the estimated time of arrival [3].” The problem is that [1,2,3] are research papers, and cannot be used to substantiate the claim that ML-related proposals are “widely adopted in modern ITS”. At best, they are well-studied in research.\n\n•\t*Training/testing time?* The paper reports that “All experiments are implemented with PyTorch and performed on a Linux server with 4 RTX 3090 GPUs.”. I am genuinely curious of how long it took to train the corresponding models on such hardware. Were the GPUs used in parallel, or did the experiments consider a single GPU “per run”? \n\nEXTERNAL REFERENCES\n\n[A]: Li, Qizhang, Yiwen Guo, and Hao Chen. \"Practical no-box adversarial attacks against dnns.\" Advances in Neural Information Processing Systems 33 (2020): 12849-12860.\n", "limitations": "I did not see any limitation mentioned in the main paper, but I **do** have one comment on this matter: the suitability of the considered datasets. I acknowledge that finding proper data is difficult in this domain, and I also acknowledge that the theoretical arguments are well-founded. However, the two considered datasets are either 5 or 10 years old. Such \"old-age\" can (slightly) impair the real world implications of the paper findings.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality:** high. I agree that there is limited work done in this specific domain, and the strong analysis and comprehensive evaluation are novel contributions.\n\n**Quality:** high. Despite lacking in some minor details, the arguments are well-supported.\n\n**Clarity:** average. The English text is good enough to allow a reader to understand the paper, but not exceptional. Presentation-wise, the paper is also appreciable.\n\n**Significance:** high. Albeit the findings are not counterintuitive (ultimately, the paper shows that “yet another application of ML can be thwarted via adversarial examples”), the considerations of a under-investigated deployment scenario of ML is commendable, and future work can greatly benefit from this paper.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ApKtVd6oTyV", "reviewer": "Reviewer_3JHx", "summary": "The paper proposes a gradient-based adversarial attack method on traffic forecasting models. They propose two-stage frameworks for practical adversarial spatiotemporal attacks, demonstrating the performance degradation.  The proposed method can selectively attack the most sensitive nodes in the graph, based on the magnitude of the gradient to generate more effective attack results.", "questions": "- Why do we only keep the negative saliency score (line 143)? What is different if we take the absolute values?\n\n- Why do we leave the topology of G_{t} immutable, when its adjacency relationship can be learned by the model parameter, which is mutable? (line 102)\n\n- I am wondering if there is any difference in performance depending on the time interval (e.g., 5 minutes, 10 minutes, 1 hour, … ), can you explain more?\n", "limitations": "- There is no significant difference between AT and AT-TDNS, which is less than 0.03 G-MAE score. Multiple runs to produce mean+-std should be examined. (Table 4)\n\n- Minor suggestion: Legends of Figure 1 b-c should be brought outside, as they interfere with the contents. \n\n- I don’t think the white-box attack is realistic, since the target model is a time-series model that predicts the upcoming states.\n\n- The authors didn’t explicitly address the limitations of the work. As said previously, the authors could include connectivity-based adversarial samples to improve the work. Also, the authors are encouraged to propose more novel ideas for acceptance for NeurIPS.\n", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\"\n- The problem is well motivated and described, and the papers are well written.  \n-  Extensive experiments are conducted to demonstrate the effectiveness of an attack. \n\n\n\nWeaknesses:\n\n- This paper lacks originality in that it proposes no novel or new concepts but just simply modify the existing adversarial attack methods little to deal with spatiotemporal forecasting models. The only novel part of the paper is the newly proposed method of TDNS, but it is still a mere extension of PGD algorithm and taking top k nodes as the victim nodes. Other than the lack of novelty, the paper was nicely written and easy to follow. I personally do not think the methods proposed in this paper are either groundbreaking or significant for future research.\n\n- I find there is no special attack designed for the time-series graph data. If we consider every time step as an observation, the attack design is very similar to PGD. And selecting the most sensitive node is similar to other data types.\n\n- The proposed method is only applicable for traffic forecasting models; And, I cannot find the explicit expression of L_{adv}.\n\n- The deviation of \\phi from the pre-trained forecasting model was not well described. Suppose \\phi is exactly the same with \\theta^{\\asterisk}. In that case, the derivative of the loss w.r.t its input is just the derivative of random noise (\\delta) at the output, making the prediction far away from its first prediction after some iterations, which is not an effective attack approach.\n\n- There is no exploration of the edge perturbation, or at least compared with the method that perturbation the edge of graph data.\n\n- Estimating the effectiveness of the attack method on more target models should be brought to the main table to show its generality.\n\n- I wonder why they kept the topology (adjacency matrix, connectivity of the road network) immutable, when perturbing the test data to generate adversarial samples. I acknowledge the point made in the paper that since a graph-based network diffuses the node features across the network, making perturbations to a subset of nodes will be sufficient for successful adversarial attack. As far as I know, graph construction is a major part of the GNN-based spatiotemporal forecasting. So the authors should include the generation of adversarial samples perturbing the original adjacency relation, at least provide an acceptable reason for dropping out that approach.\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept