Title: Enhancing Graph Classification Robustness with Singular Pooling
Venue: NeurIPS 2025
Accepted: yes
Keywords: Adversarial Robustness

Graph Neural Networks
Abstract: Graph Neural Networks (GNNs) have achieved strong performance across a range of graph representation learning tasks, yet their adversarial robustness in graph classification remains underexplored compared to node classification. While most existing defenses focus on the message-passing component, this work investigates the overlooked role of pooling operations in shaping robustness. We present a theoretical analysis of standard flat pooling methods (sum, average and max), deriving upper bounds on their adversarial risk and identifying their vulnerabilities under different attack scenarios and graph structures. Motivated by these insights, we propose Robust Singular Pooling (RS-Pool), a novel pooling strategy that leverages the dominant singular vector of the node embedding matrix to construct a robust graph-level representation. We theoretically investigate the robustness of RS-Pool and interpret the resulting bound leading to improved understanding of our proposed pooling operator. While our analysis centers on Graph Convolutional Networks (GCNs), RS-Pool is model-agnostic and can be implemented efficiently via power iteration. Empirical results on real-world benchmarks show that RS-Pool provides better robustness than the considered pooling methods when subject to state-of-the-art adversarial attacks while maintaining competitive clean accuracy.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the adversarial robustness of Graph Neural Networks (GNNs), specifically examining how different flat pooling methods (sum, average, and max pooling) affect their resilience in graph classification tasks. The authors first provide a theoretical analysis of these pooling operations, deriving upper bounds on their adversarial risks and identifying distinct vulnerabilities depending on the type of adversarial attack and graph structure.\n\nMotivated by this analysis, the authors propose a novel pooling strategy named Robust Singular Pooling (RS-Pool). This method constructs robust graph-level representations by leveraging the dominant singular vector of the node embedding matrix. RS-Pool is theoretically shown to enhance robustness by minimizing the sensitivity to adversarial perturbations through spectral stability properties. The proposed pooling approach is efficient, model-agnostic, and practically implemented using the power iteration method. The paper highlights the critical yet overlooked role pooling plays in the robustness of graph classification tasks and introduces a practical and effective approach to improving robustness via pooling.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\nThe paper provides a thorough theoretical treatment of adversarial robustness, establishing explicit bounds for standard pooling methods and the proposed RS-Pool. Such clear theoretical grounding significantly strengthens the paper’s contribution. The empirical results are robust and cover a diverse range of datasets (bioinformatics, molecular, social, and image-based graphs), enhancing the credibility of the claimed benefits of RS-Pool.\n\nGiven the increasing adoption of GNNs in critical applications (e.g., drug discovery, cybersecurity), enhancing robustness against adversarial attacks is highly impactful. Addressing the overlooked influence of pooling operations rather than focusing solely on message-passing layers introduces an important new dimension to the robustness discussion.\n\nThe paper is methodically organized, making complex theoretical derivations accessible. Definitions and results are clearly stated, and the flow from theoretical insights to experimental validation is logical and easy to follow.\n\n# Weaknesses (only minor, adressed during rebuttal)\n\nThe paper does not consider or discuss malicious bit-flip attacks (BFAs), a relevant and increasingly studied form of adversarial perturbation on GNNs (see references below). BFAs are not acknowledged or reviewed within the related work section, limiting the paper's comprehensiveness regarding current adversarial threats.\n\n# Conclusion\n\nI think this is a very strong paper. The proposed pooling method has a solid theoretical foundation and comes with proven guarantees regarding its adversarial risk, yet it remains technically simple, efficient, and elegant. The write-up is exceptionally clear. I enjoyed reading this paper a lot and consider it inspiring and highly relevant to my own line of work. I would be genuinely surprised if it is not accepted and would attend the talk if it was accepted as oral.\n\n# References\n\nKummer et al, \"On the Relationship Between Robustness and Expressivity of Graph Neural Networks\", AISTATS, 2025\n\nKummer et al., \"Attacking Graph Neural Networks with Bit Flips: Weisfeiler and Leman Go Indifferent\", SIGKDD, 2024\n\nWu et al., \"Securing Graph Neural Networks in MLaaS: A Comprehensive Realization of Query-based Integrity Verification\", IEEE Symposium on Security and Privacy (SP), 2024\n\nJiau et al., \"PyGFI: Analyzing and Enhancing Robustness of Graph Neural Networks Against Hardware Errors\", CoRR abs/2212.03475 2022\n\nQUESTIONS:\nThe authors should address the mentioned weakness above. Specifically, since the theoretical framework and method proposed by the authors could potentially be adapted to also address BFAs on GNNs, I suggest that they briefly mention BFAs in the related work section.\n\nLIMITATIONS:\nMostly. The authors should note in the \"Limitations\" paragraph that investigating the robustness of their method against attacks other than the discussed evasion methods (e.g., BFAs) remains an open research topic.\n\nEDIT: Adressed during rebuttal.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the adversarial robustness of Graph Neural Networks (GNNs) in graph classification tasks, focusing on the role of pooling operations. The authors present a theoretical analysis of standard flat pooling methods (sum, average, max). Motivated by these insights, they propose Robust Singular Pooling (RS-Pool), a novel pooling strategy that leverages the dominant singular vector of node embeddings to construct robust graph-level representations. Theoretical analysis shows that RS-Pool improves robustness by filtering noise-sensitive components, and empirical results on real-world benchmarks demonstrate its effectiveness against state-of-the-art adversarial attacks while maintaining competitive clean accuracy.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe paper addresses a barely explored aspect of GNN robustness by focusing on pooling operations, providing both theoretical and empirical insights. The proposed method is model-agnostic and computationally efficient (using power iteration).\n\nWeakness:\nYour attack settings are based on both features and topology, but the main results only consider feature-based attacks. Moreover, Theorems 4.2 and 4.3 appear to be simply extensions of Theorem 4 in [10] to the task of graph classification.\n\nQUESTIONS:\n1.While $d((A,X),(A’,X’))<\\epsilon$, the proof of Theorem 4.2 (line 553) gives that $ X-X’\\leq\\epsilon$. Is it because the graph topology remains unchanged? It is unclear about your settings and proofs, such as the punctuations (e.g., 632), and the representations of the matrix norm and vector norm.\n\n2.In Theorem 5.1, It seems that the values of $\\[\\tau\\]$ and $\\[\\sigma_1 - \\sigma_2\\]$ are both  small. And this bound still depends on the perturbation budget $\\epsilon$. So, from a theoretical point of view, what is the advantage of the strategy proposed in this paper compared with the three strategies mentioned in Theorem 4.2? \n\n3.Theorem 5.1 assumes that $\\sigma_1\\neq\\sigma_2$, but in practice there may be multiple principal singular values (such as symmetric graph structures). Could you elaborate on this special case?\n\nLIMITATIONS:\nSee weakness and questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe focus of the paper is The paper first defines average-case robustness as the l2 distance in the output space between the clean and perturbed graphs averaged over all perturbation within a given distance. Then they derive the effect of different pooling operators on this quantity. Next, the authors propose a new pooling method RS-Pool that projects the final node embeddings onto the first right singular vector of the embedding matrix. They also derive an upper bound for the \"adversarial\" risk for this pooling and argue that it is more favorable.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed approach is scalable, easy to implement, and well motivated. The parameter $\\tau$ allows for some control over the trade-off between clean accuracy and robustness. The focus on the pooling aspect is very interesting and under studied in the literature. The theoretical results seem correct but I did not check the proofs in detail.\n\nOne of the my main problems with the paper is the mismatch between the actual and the claimed results. This shows up in several different places:\n- As the authors already mention they analyse the average case and not the worst-case. Calling this **adversarial** robustness is highly misleading and not in line with the existing terminology in the literature. To be clear, I think the average case results are interesting and useful, but we should not be calling this adversarial. Relatedly, the statement \"(ϵ, γ)-robust in the average case sense, it is also (ϵ, γ)-robust under the worst case criterion\" is not quite accurate. The worst-case criterion would presumably have a $\\max_{\\mathcal{N}_\\epsilon}[\\cdot]$  instead of $\\mathbb{E}[\\cdot]$ in Eq. 1, just because we have a low value on average it doesn't mean we have a low value in the worst-case.\n- When they set up the threat model the authors define perturbations w.r.t. both $A$ and $X$, however, the theorems say \"Under a feature-based adversarial attack\".  Even more confusingly, in the experiments the author state \"For all attacks, we apply a perturbation budget of ϵ = 0.3, allowing up to 30% of the edges in each graph to be modified.\". However, as far as I understood, Theorem 5.1 and Theorem 4.2 only apply to feature-based permutations. This discrepancy is not ideal (see question 2). Moreover, the threat model looks at the L2 norm of the adjacency matrix which is different from the number of perturbed edges (L0 norm).\n\nThe experimental evaluation is limited. For example Table 1 shows results for a single attack budget $\\epsilon$ which is relatively large. While they consider different $\\epsilon$ in Figure 1 this should be extended to the rest of the \"main\" experiments.\n\nSome relevant works are not (properly) discussed. For example, while [1, 2]  are not exactly addressing the same problem, they are relevant enough to warrant a short discussion. Moreover, standard (non-adversarial) generalisation bounds for GNNs often have terms that depend on the product of (spectral) norms of the weights, similar to the expression derived in this paper. Showing how exactly robustness changes the bounds would be valuable. See e.g. [3] and related references.\n\nYou should label the y-axis on Figure 1 for improved clarity. Assuming that you are showing the \"adversarial\" risk $\\gamma$, how do you actually compute it? The expectation defining in Eq. 1 is untractable to compute exactly as far as I know. Section 6.1. states \"compute the distance be355 tween the pooled representations of clean and adversarially perturbed graphs\" but how are these adversarially perturbed graphs obtained?\n\nDepending on $n$ and $d$ it might be a bit cheaper to compute $H^T (H v)$ rather than precomputing $S$ since matrix-vector products are cheaper than matrix-matrix products, especially for low values of $K$. This \"trick\" is also used in the standard implementation of spectral normalisation (available in PyTorch) which also computes the right singular vector as a side-product.\n\nReferences:\n1. Cao et al., \"Adversarial Robust Generalization of Graph Neural Networks\"\n2. Sun and Lin, \"PAC-Bayesian Adversarially Robust Generalization Bounds for Graph Neural Network\" \n3. Liao et al., \":A PAC-bayesian approach to generalization bounds for graph neural networks\"\n\nQUESTIONS:\n1. Is the PGD attack adaptive? That is, do you directly attack RS-pool, differentiating through proposed pooling?\n2. Can you show empirical results against feature-based attacks that match the setup for the theory?\n3. Provide a detailed description of how the derived bounds differ from existing (not necessarily adversarial) bounds (see weakness)?\n4. Can you provide additional results for an even larger value of $\\alpha$. Can you show a similar plot for other datasets?\n\nLIMITATIONS:\nThe limitations section is there but could be expanded, e.g. about worst-case vs. average case.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the adversarial vulnerability of GNNs in graph classification by focusing on the pooling stage. After theoretically analyzing the weaknesses of standard pooling methods like sum, average, and max, the authors introduce Robust Singular Pooling (RS-Pool). This novel, model-agnostic method constructs a graph representation from the dominant singular vector of the node embeddings, which is inherently more stable against adversarial perturbations. Implemented efficiently via power iteration, RS-Pool is empirically shown to significantly enhance robustness against attacks while maintaining competitive accuracy on clean data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n- Novel Focus and Theoretical Grounding: The paper provides a novel contribution by focusing on the often-overlooked pooling stage of GNNs in the context of adversarial robustness. Its arguments are supported by a strong theoretical framework that derives robustness bounds for common pooling methods before proposing a solution.\n\n- Well-Motivated and Efficient Solution: The proposed RS-Pool method is elegantly based on established matrix perturbation theory, which shows that dominant singular vectors are stable under perturbations. It is also computationally efficient, using the power iteration method to avoid a costly full SVD, making it practical for real-world applications.\n\n- Model-Agnostic and Versatile: RS-Pool is designed to be model-agnostic, and the paper demonstrates its effectiveness with different GNN backbones (GCN and GIN) and even when combined with other existing adversarial defense methods.\n\nQUESTIONS:\n- What is the semantic meaning of the dominant singular vector? The paper uses the dominant singular vector as a robust feature summary. However, it doesn't investigate what this vector represents in terms of the graph's structure or semantics. Does it capture information about community structure, hub nodes, or specific functional motifs? Answering this could lead to more interpretable GNNs.\n\n- What information is lost by discarding other singular vectors? RS-Pool exclusively uses the top singular vector to achieve robustness. A key unaddressed question is what valuable, albeit less dominant, information is being discarded. These secondary components might be crucial for fine-grained classification tasks where subtle features matter more than overall structural stability.\n\n- How does network depth affect the spectral gap? The method's effectiveness hinges on a significant spectral gap in the node embedding matrix. The paper doesn't explore how the number of GNN layers impacts this gap. Deeper GNNs often lead to \"over-smoothing,\" where node embeddings become more similar, potentially shrinking the spectral gap and reducing RS-Pool's effectiveness.\n\nLIMITATIONS:\n- Dependence on Spectral Gap: The effectiveness of RS-Pool, both theoretically and in practice, relies on a sufficiently large spectral gap between the first and second singular values of the node embedding matrix. The paper acknowledges that a small gap, which can occur in some graph topologies, weakens the robustness guarantee and slows the convergence of the power iteration.\n\n- Computational Overhead: Despite being more efficient than a full SVD, RS-Pool incurs more computational overhead than standard pooling methods. This is especially true on very dense graphs, which could limit its use in resource-constrained applications.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}