Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents a novel adversarial attack for spatiotemporal traffic forecasting models. Moreover, theoretically analysis are conducted to demonstrate the worst performance bound of the attack. Comprehensive experiments on real-world datasets show the effectiveness of the effectiveness of the attack, and how the robustness of the models enhances when combined with the corresponding adversarial training.
strengths_and_weaknesses: Strengths:
- The paper is well-written and easy to understand.
- Introducing adversarial robustness to spatiotemporal models is interesting and novel.
- The theoretical analysis backs up the proposed attack well.
- The experiments are comprehensive.

Weakness:
- I don't see clear weakness of the paper, although I wonder the generalizability of the method to other spatiotemporal-based applications beyond traffic forecasting models.
questions: See weakness.
limitations: Yes for limitations and no for potential negative societal impact.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a gradient-based adversarial attack method on traffic forecasting models. They propose two-stage frameworks for practical adversarial spatiotemporal attacks, demonstrating the performance degradation.  The proposed method can selectively attack the most sensitive nodes in the graph, based on the magnitude of the gradient to generate more effective attack results.
strengths_and_weaknesses: Strength"
- The problem is well motivated and described, and the papers are well written.  
-  Extensive experiments are conducted to demonstrate the effectiveness of an attack. 



Weaknesses:

- This paper lacks originality in that it proposes no novel or new concepts but just simply modify the existing adversarial attack methods little to deal with spatiotemporal forecasting models. The only novel part of the paper is the newly proposed method of TDNS, but it is still a mere extension of PGD algorithm and taking top k nodes as the victim nodes. Other than the lack of novelty, the paper was nicely written and easy to follow. I personally do not think the methods proposed in this paper are either groundbreaking or significant for future research.

- I find there is no special attack designed for the time-series graph data. If we consider every time step as an observation, the attack design is very similar to PGD. And selecting the most sensitive node is similar to other data types.

- The proposed method is only applicable for traffic forecasting models; And, I cannot find the explicit expression of L_{adv}.

- The deviation of \phi from the pre-trained forecasting model was not well described. Suppose \phi is exactly the same with \theta^{\asterisk}. In that case, the derivative of the loss w.r.t its input is just the derivative of random noise (\delta) at the output, making the prediction far away from its first prediction after some iterations, which is not an effective attack approach.

- There is no exploration of the edge perturbation, or at least compared with the method that perturbation the edge of graph data.

- Estimating the effectiveness of the attack method on more target models should be brought to the main table to show its generality.

- I wonder why they kept the topology (adjacency matrix, connectivity of the road network) immutable, when perturbing the test data to generate adversarial samples. I acknowledge the point made in the paper that since a graph-based network diffuses the node features across the network, making perturbations to a subset of nodes will be sufficient for successful adversarial attack. As far as I know, graph construction is a major part of the GNN-based spatiotemporal forecasting. So the authors should include the generation of adversarial samples perturbing the original adjacency relation, at least provide an acceptable reason for dropping out that approach.



questions: - Why do we only keep the negative saliency score (line 143)? What is different if we take the absolute values?

- Why do we leave the topology of G_{t} immutable, when its adjacency relationship can be learned by the model parameter, which is mutable? (line 102)

- I am wondering if there is any difference in performance depending on the time interval (e.g., 5 minutes, 10 minutes, 1 hour, … ), can you explain more?

limitations: - There is no significant difference between AT and AT-TDNS, which is less than 0.03 G-MAE score. Multiple runs to produce mean+-std should be examined. (Table 4)

- Minor suggestion: Legends of Figure 1 b-c should be brought outside, as they interfere with the contents. 

- I don’t think the white-box attack is realistic, since the target model is a time-series model that predicts the upcoming states.

- The authors didn’t explicitly address the limitations of the work. As said previously, the authors could include connectivity-based adversarial samples to improve the work. Also, the authors are encouraged to propose more novel ideas for acceptance for NeurIPS.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper explores the vulnerability of spatiotemporal traffic forecasting models. They proposed a practical adversarial spatiotemporal attack framework. Experiments are conducted to verify the effectiveness.
strengths_and_weaknesses: 
Strengths:
1. The first attemp to attack spatiotemporal traffic forecasting models.
2. The code is released, and thus readers can reproduce the results. 
3. The demonstration is given from empirical and theoretical view. 

Weakness:
1. When the attacks are extended to black-box setting, they  employ a surrogate model, which is trained via querying the threat models. This will lead to a large number of queris, which reduces the efficiency. The authors should discuss this point.
questions: please see the weakness
limitations: please see the weakness
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper tackles the problem of adversarial attacks against “traffic forecasting models”, i.e., models that must predict the conditions of “physical traffic” (e.g., cars) in real world settings. The main contribution is a generic framework of adversarial attacks against such systems, which can be declined to cover both white- and black-box adversarial settings. A comprehensive experimental evaluation on real world data validates the proposal. The paper is also enriched by an ablation study, and by strong theoretical analyses that support the overall findings
strengths_and_weaknesses: **Originality:** high. I agree that there is limited work done in this specific domain, and the strong analysis and comprehensive evaluation are novel contributions.

**Quality:** high. Despite lacking in some minor details, the arguments are well-supported.

**Clarity:** average. The English text is good enough to allow a reader to understand the paper, but not exceptional. Presentation-wise, the paper is also appreciable.

**Significance:** high. Albeit the findings are not counterintuitive (ultimately, the paper shows that “yet another application of ML can be thwarted via adversarial examples”), the considerations of a under-investigated deployment scenario of ML is commendable, and future work can greatly benefit from this paper.

questions: I thank the authors for their paper, of which I particularly appreciated the research direction: instead of focusing on the (overused) attacks against image classifiers, the paper considers a completely different (but still realistic) setting. I have some comments that the authors can address in a rebuttal, which are reported below.

*Unclear definitions of White/Black-box attacks.* In Section 3.2 the authors describe the potential “variations” of the proposed attack by using the well-known “white/black-box” terminology. However, it is not clear what are the actual assumptions of the considered attacker. For example, the white-box scenario is described as follows: “White-box attack. Since the adversaries can fully access the data and labels under the white-box setting, we directly use the real ground truth traffic states to guide the generation of adversarial traffic states.”. Does this mean that the “white-box” attack assumes an attacker with access to the training data and all labels? Or does this also imply that the attacker knows everything about the target model (i.e., learned parameters, weights and architecture)? Similarly, for black-box attacks the paper states that: “The most restrictive black-box setting assumes limited accessibility to the target model and labels. Therefore, we first employ a surrogate model, which can be learned from the training data or by querying the traffic forecasting service [15, 16]. Then we generate adversarial traffic states based on the surrogate model to attack the targeted traffic forecasting model.” The authors should elucidate if they assume an attacker who can “query” the target model (and whether such queries are constrained or not) or who has no access whatsoever to the targeted model, but only to a subset of the training data (in which case, it is more like a “no-box” attack [A]). For completeness, I have also checked the appendix and there is no mentioning of such details.

*Realistic Feasibility.* I would be delighted if the authors could integrate (perhaps in an appendix) an analysis of the realistic feasibility of the proposed attacks. For example, consider the following statement in the introduction: “How to identify the subset of salient victim nodes with a limited attack budget to maximize the attack effect is the first challenge.” Why would a realistic attacker do that? If, as shown in Figure 1, just by injecting perturbations on “randomly selected” nodes results in a successful attack, then why going further? Would an attacker really opt for such a strategy? What is the cost and potential benefit of such a strategy?
Indeed, I stress that real attacker operate with a cost/benefit mindset. Considered that the paper tackles a new application domain, it would be impactful to show that some attacks may require little preparation, but can lead to significant performance degradation. Of course, it is still valuable to analyze “worst-case” scenarios, but it’s important to differentiate from attacks that are more likely to occur (e.g., because they are cheaper to stage) from those who are less likely to appear (e.g., because they require a huge resource investment, which can potentially be superior than what the attacker stands to gain).

Some additional issues:

•	The following statement in the Introduction requires additional back-up: “Machine learned spatiotemporal forecasting models have been widely adopted in modern Intelligent Transportation Systems (ITS) to provide accurate and timely prediction of traffic dynamics, e.g., traffic flow [1], traffic speed [2], and the estimated time of arrival [3].” The problem is that [1,2,3] are research papers, and cannot be used to substantiate the claim that ML-related proposals are “widely adopted in modern ITS”. At best, they are well-studied in research.

•	*Training/testing time?* The paper reports that “All experiments are implemented with PyTorch and performed on a Linux server with 4 RTX 3090 GPUs.”. I am genuinely curious of how long it took to train the corresponding models on such hardware. Were the GPUs used in parallel, or did the experiments consider a single GPU “per run”? 

EXTERNAL REFERENCES

[A]: Li, Qizhang, Yiwen Guo, and Hao Chen. "Practical no-box adversarial attacks against dnns." Advances in Neural Information Processing Systems 33 (2020): 12849-12860.

limitations: I did not see any limitation mentioned in the main paper, but I **do** have one comment on this matter: the suitability of the considered datasets. I acknowledge that finding proper data is difficult in this domain, and I also acknowledge that the theoretical arguments are well-founded. However, the two considered datasets are either 5 or 10 years old. Such "old-age" can (slightly) impair the real world implications of the paper findings.
review_title: null

Meta-review: metareview: The authors have made a significant effort to address reviewer concerns in their rebuttal. They are strongly encouraged to include these additional results and observations to either the main body of the paper or the supplement.
confidence: Less certain
Decision: Accept