Paper: IDEA: Invariant Causal Defense for Graph Adversarial Robustness (ICLR-2024)

Reviewer comment (Writing):
Minor language issues throughout the paper

Author response:
**A1:** We have thoroughly revised the paper to ensure that the language is clear, precise, and adheres to high academic standards. For ease of identification, all modifications have been highlighted in blue in the revised version of our paper.

Reviewer comment (Theory):
It is not clear why the end result of the encoder H would be a causal feature. I can see how the learned representation could be a feature set that is invariant to the features the attack but making a causal claim is not completely justified.

Author response:
**A2:** We appreciate your recognition that the learned representation is invariant to attacks. We would like to offer further clarification on our use of the term "causal feature." This terminology is specifically chosen to denote features that maintain a direct and invariant causal relationship with the labels, as supported by the principles found in the invariant causal learning literature [1,2,3,4]. For instance, the concept of Invariant Risk Minimization (IRM) posits that invariance is a defining characteristic of causal relationships [1].

In our initial paper, detailed in Section 3.1 and Section 3.2.1, we delineate our intention to identify features that hold a causal influence over the labels. By examining causal relationships and conditional independencies, we have developed the predictive and invariant goals. These goals are designed to steer the encoder $h$ towards learning these causal features, as further elaborated in Section 3.2.2.

We trust that this elaboration dispels any ambiguity surrounding our use of the term "causal".

Reviewer comment (Experiments):
It is unclear how the method generalises to multiple unseen attacks

Author response:
**A3:** As stated in Section 3.2, our IDEA aims to learn the causal features that determine labels, having strong and invariant predictability across different attack domains, rather than spurious relationships that are limited to a single domain. As stated in Section 3.2.1, the element vulnerable to the attack, denoted as $G$ (since the attack propagates from attack domain $D$ to perturbation $T$ and subsequently to ego-network $G$), exhibits certain characteristics; specifically, the conditional mutual information $I(D,Y|G)$, is substantially high, because collider. Our invariance goal is to minimize the conditional mutual information $I(D,Y|Z)$, avoiding $Z$ learning the part affected by the attack. Meanwhile, our component of domain learner aids in proving IDEA achieving adversarial robustness across all attack domains, as proved in Proposition 2. 

In addition to theoretical guarantees, experimental results further demonstrate that the IDEA is robust to attacks that do not appear in the training set, thereby confirming the effectiveness and generalization of our method against unseen attacks, and this strength is also identified by Reviewer jN54.

In conclusion, we greatly appreciate your recognition of our method and experimentation. We observe that your concerns are mainly regarding the conception, which has been stated in our initial paper. We have provided further clarification to address each concern. We hope you can reconsider your assessment and potentially improve the overall assessment based on our clarification. We believe that our work offers valuable insights and advances the state-of-the-art in adversarial graph machine learning, making a meaningful contribution to the research community. We appreciate your valuable feedback, and we look forward to any further comments or questions you may have. Thank you for your consideration.

Reviewer comment (Theory):
Why is the features learned causal and not just invariant to the attack?

Author response:
**A2:** We appreciate your recognition that the learned representation is invariant to attacks. We would like to offer further clarification on our use of the term "causal feature." This terminology is specifically chosen to denote features that maintain a direct and invariant causal relationship with the labels, as supported by the principles found in the invariant causal learning literature [1,2,3,4]. For instance, the concept of Invariant Risk Minimization (IRM) posits that invariance is a defining characteristic of causal relationships [1].

In our initial paper, detailed in Section 3.1 and Section 3.2.1, we delineate our intention to identify features that hold a causal influence over the labels. By examining causal relationships and conditional independencies, we have developed the predictive and invariant goals. These goals are designed to steer the encoder $h$ towards learning these causal features, as further elaborated in Section 3.2.2.

We trust that this elaboration dispels any ambiguity surrounding our use of the term "causal".

Reviewer comment (Experiments):
How does the method generalise to types of attacks not seen in training?

Author response:
**A3:** As stated in Section 3.2, our IDEA aims to learn the causal features that determine labels, having strong and invariant predictability across different attack domains, rather than spurious relationships that are limited to a single domain. As stated in Section 3.2.1, the element vulnerable to the attack, denoted as $G$ (since the attack propagates from attack domain $D$ to perturbation $T$ and subsequently to ego-network $G$), exhibits certain characteristics; specifically, the conditional mutual information $I(D,Y|G)$, is substantially high, because collider. Our invariance goal is to minimize the conditional mutual information $I(D,Y|Z)$, avoiding $Z$ learning the part affected by the attack. Meanwhile, our component of domain learner aids in proving IDEA achieving adversarial robustness across all attack domains, as proved in Proposition 2. 

In addition to theoretical guarantees, experimental results further demonstrate that the IDEA is robust to attacks that do not appear in the training set, thereby confirming the effectiveness and generalization of our method against unseen attacks, and this strength is also identified by Reviewer jN54.

In conclusion, we greatly appreciate your recognition of our method and experimentation. We observe that your concerns are mainly regarding the conception, which has been stated in our initial paper. We have provided further clarification to address each concern. We hope you can reconsider your assessment and potentially improve the overall assessment based on our clarification. We believe that our work offers valuable insights and advances the state-of-the-art in adversarial graph machine learning, making a meaningful contribution to the research community. We appreciate your valuable feedback, and we look forward to any further comments or questions you may have. Thank you for your consideration.

Reviewer comment (Presentation):
The drawing of the overall block diagram of the method is relatively rough.

Author response:
**Q3:** The drawing of overall block diagram of the method.

**A3:** Thank you for your constructive feedback. In response, we have further improved Figure 3, which illustrates the overall architecture, enhancing its layout and alignment. Please refer to the updated paper for a detailed view.  Furthermore, we have appended a description to Figure 3, aiming to augment its clarity and facilitate better understanding for the readers. Thank you for bringing this to our attention.

Reviewer comment (Experiments):
Does this method still have such obvious advantages in the face of the other attack methods mentioned in the article?

Author response:
**Q1:** The table only shows the excellence of this method when facing one of poisoning attacks or evasion attacks.

**A1:**  We would like to clarify that there seems to be a misunderstanding regarding the scope of the attacks tested. In fact, our IDEA has been rigorously tested against four evasion attacks. These include the graph modification attack Nettack and node injection attacks including PGD, TDGIA, and G-NIA. In addition to evasion attacks, we have also employed the widely acknowledged metattack for poisoning scenarios, testing it across different perturbation rates. This approach is a standard in defense-related research, as evidenced by its extensive use in prior studies [1,2,3,4]. The results in our initial submission broadly cover various types of attacks, including untargeted (MetaAttack) and targeted (nettack, PGD, TDGIA, and G-NIA) attacks, graph modification attacks (nettack and MetaAttack) and node injection attacks (PGD, TDGIA, and G-NIA), as well as white-box attacks (adaptive attack in Table 4) and gray-box (MetaAttack, nettack, PGD, TDGIA, and G-NIA) attacks. These have been stated in “implementation” in Section 4 in our initial submission.

Reviewer comment (Novelty):
There are also some purification methods that seem to be able to be extended to graph purification. Whether they will also encounter the limitations mentioned by the author, I hope the author can discuss or compare this. Such as the following methods:

[1] Shi C, Holtz C, Mishne G. Online adversarial purification based on self-supervision[J]. arXiv preprint arXiv:2101.09387, 2021.

[2] Liao F, Liang M, Dong Y, et al. Defense against adversarial attacks using high-level representation guided denoiser[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 1778-1787.

[3] Zhou D, Wang N, Peng C, et al. Removing adversarial noise in class activation feature space[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 7878-7887.

[4] Naseer M, Khan S, Hayat M, et al. A self-supervised approach for adversarial robustness[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 262-271.

Author response:
**Q4:** Can purification methods in CV extend to graph purification? 

**A4:** We read the computer vision purification literature you recommended [5, 6, 7, 8] and observed that they often leverage image data priors. For example, paper [5] incorporates self-supervised tasks such as image rotation that are unique to the domain of computer vision, while paper [8] depends on a pixel loss function, i.e., $\mathcal{L}_{img}$, to encourage image smoothness. These domain-specific dependencies pose significant challenges when considering the direct transposition of these methods to graph data, which inherently lacks such image-based priors.

In contrast, graph purification methods are specifically designed to exploit the unique structural and feature-related properties of graph data, making them appropriate for addressing graph-specific issues. However, graph purification defenses rely on heuristic assumptions, as detailed in our initial submission. Experimental results demonstrate that our method IDEA outperforms these existing methods when defending against attacks.

 In our revised paper, we have added a separate subsection Appendix A.4 discussing these purification methods in the computer vision field [5, 6, 7, 8]. We believe this extended discussion will provide our readers with a more holistic understanding of the challenges and potential of purification methods across various fields.

Reviewer comment (Presentation):
There are some notations and figures problems that causes the paper not easy to follow. Node j in Figure 2 should be Node k. Z in Section 3.2.1 is not defined. (.)_\cN is never defined. The overall framework only shows in Figure 3 without any introduction. Empty graph in Figure 5.

Author response:
**A3:** We apologize for any confusion stemming from inaccuracies in notations and figures previously presented. In the updated version of the paper,

- Node $j$ in Figure 2 is now corrected as node $k$.
- The notation $Z$ in Section 3.2.1 has been updated to $\Phi(G)$ for clarity.
- The symbol $\mathcal{N}$, denoting neighbors, has been clarified.

There are also some misunderstandings about our paper in the review comments:
- Framework introduction: In our initial submission, we have provided the introduction of the framework in the 3.2.4 “Overall Framework” on page 6.
- Figure 5 viewing issue: This issue may be attributed to browser-specific rendering limitations. We kindly recommend downloading the PDF version of our paper and viewing it with a PDF reader for accurate display.

We have meticulously revised the entire document to prevent any such errors in the paper. For ease of identification, all modifications have been highlighted in blue in the revised version of our paper.

Reviewer comment (Theory):
Why does causal feature would only connect with label and input data? Is it defined or derived just based on some assumption the paper made or is there anyway to automatic define the graph?

Author response:
**A4:** We respectfully contest the assertion that "causal feature would only connect with the label and input data." Contrarily, we introduce causal feature as an abstraction of underlying factors that determine both inputs and labels. This is consistent with the existing causal invariance literature [1,2,3].

Reviewer comment (Experiments):
If the attacker knew the added neural network, would the proposed method still achieve a similar improvement in the paper?

Author response:
**A5:** We have already conducted the white-box adaptive attack [9] experiments, and exported the results in Appendix D.4 “Performance under adaptive attacks”. As shown in Table 4, adaptive attacks make serious performance degradation to defense methods because they are powerful white-box attacks. IDEA outperforms all the baselines. The results offer a broader evaluation of IDEA’s performance under a hard scenario, consistently showing IDEA’s superiority.