Enhancing Graph Classification Robustness with Singular Pooling
Authors: Sofiane ENNADIR
Oleg Smirnov
Yassine ABBAHADDOU
Lele Cao
Johannes F. Lutzeyer
Venue: NeurIPS 2025 poster ()
Keywords: Adversarial Robustness
Graph Neural Networks
Primary area: deep_learning
TL;DR: We theoretically study the effect of the pooling operation on adversarial robustness, which allows us to introduce RS-Pool, a new pooling method capable of smoothing the effect of attacks.

Abstract: Graph Neural Networks (GNNs) have achieved strong performance across a range of graph representation learning tasks, yet their adversarial robustness in graph classification remains underexplored compared to node classification. While most existing defenses focus on the message-passing component, this work investigates the overlooked role of pooling operations in shaping robustness. We present a theoretical analysis of standard flat pooling methods (sum, average and max), deriving upper bounds on their adversarial risk and identifying their vulnerabilities under different attack scenarios and graph structures. Motivated by these insights, we propose Robust Singular Pooling (RS-Pool), a novel pooling strategy that leverages the dominant singular vector of the node embedding matrix to construct a robust graph-level representation. We theoretically investigate the robustness of RS-Pool and interpret the resulting bound leading to improved understanding of our proposed pooling operator. While our analysis centers on Graph Convolutional Networks (GCNs), RS-Pool is model-agnostic and can be implemented efficiently via power iteration. Empirical results on real-world benchmarks show that RS-Pool provides better robustness than the considered pooling methods when subject to state-of-the-art adversarial attacks while maintaining competitive clean accuracy.

Enhancing Graph Classification Robustness with Singular Pooling
Sofiane Ennadir"
Oleg Smirnov*
King Al Labs, Microsoft Gaming sofiane. ennadir@king. com
King Al Labs, Microsoft Gaming oleg. www.smmiiiiiiirooooooooooooooo crosoft. com
Yassine Abbahaddou
Lele Cao King Al Labs, Microsoft Gaming Leleeao@microsooft.coo com
LIX, École Polytechnique, IP Paris
yassine ybbaladoooeplllyeeccniieueee edu
Johannes F. Lutzeyer
LIX, École Polytechnique, IP Paris
j johannes... www.eeeyerrrooolyyeemii......ee.
Abstract
Graph Neural Networks (GNNs) have achieved strong performance across range of graph representation learning tasks, yet their adversarial robustness in graph classification remains underexplored compared node classification. While most existing defenses focus on the message- paassggg component, this work investigates the overlooked role of pooling operations shaping robustness. We present a theoretical analysis of standard flat pooling methods (sum, average and max), deriving upper bounds on their adversarial risk and identifying their vulnerabilities under different attack scenarios and graph structures. Motivated by these insights, we propose Robust Singular Pooling (RS-Pool), novel pooling strategy that leverages the dominant singular vector of the node embedding matrix to construct a robust graph-level representation. We theoretically investigate the robustness RS RS-Pool and interpret the resulting bound leading to improved understanding of our proposed pooling operator. While our analysis centers on Graph Convolutional Networks (GCNs), RS-Pool is model-agnostic and can be implemented efficiently via power iteration. Empirical results on weal-world benchmarks show that RS-Poo S--Po provides better robustness than the considered pooling methods when subject state-of-the-att adversarial attacks while maintaining competitive clean accuracy. Our code is publicly available at: https://gituub.oom/kin/////
Introduction
have demonstrated strong performance graph- structured data and are now a standard tool for learning node and graph representations. However, recent studies reveal that GNNs susceptible adversarial perturbations small, deliberately designed changes to the adjacency matrix or node features that can drastically affect predictions 20] This vulnerability raises concerns about their reliability and consequently, enhancing the adversarial robustness of GNNs has become a key research focus. Efforts include the design of novel attack methods 7 49 48 I7 and corresponding defense mechanisms [42 46 19]. However, prior work
Graph Neural Networks (GNNs)
24
44
38
primarily investigates node-classification tasks, with limited attention paid to ad adversarial robustness
'Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Yet, robustness at the graph level is equally critical, particularly in domains where entire graphs, rather than individual nodes, constitute the prediction target. drug discovery, for example, molecules are modeled as graphs, and predicting properties such as efficacy or toxicity is graph classification problem [35] 22]. Adversarial perturbations, editing a bond type for instance, can lead to erroneous evaluations with serious implications in precision medicine. Similarly, in cybersecurity, classifying control- graphs of binary programs aids in malware detection [34]; subtle perturbations could allow malicious code to evade the anomaly classification.
in graph-classite claphcllasiiiiatiin settings (2I]
Existing adversarial defenses for GNNs primarily focus on the message pessage-passing component, lever- aging techniques such as attention mechanisms [46,, weight regularization [I] 10), and adjacency preprocessing [42]. While these approaches are applicable tasks, graph-level prediction introduces additional challenges that go beyond message passing (18). particularly in constructing global representations from structurally diverse inputs. This step relies on pooling mechanisms [5] 91. which aggregate node features into fixed-size graph embeddings. Pooling operations are generally categorized into two types 300: hierarchical pooling, which performs multi-stage graph coarsening to produce progressively smaller subgraphs [5], and flat pooling (or readout functions), which compute global representation in a single step using aggregation functions such as sum, average, or max 19). While hierarchical pooling is a general architectural tool used throughout the network, flat pooling specifically produces the final representation used in graph-level classification. Given the importance of this final aggregation step, it is critical to investigate how flat pooling functions influence the robustness of GNNs.
both node and graph classification
This work offers a complementary perspective on GNN robustness by focusing on the pooling stage, which plays critical role in bridging node-level representations and graph-level embeddings. We center our analysis on flat pooling methods, which are widely used for their simplicity, efficiency, and strong empirical performance. Through theoretical analysis, we examine how different pooling strategies influence the expected robustness and provide insights on selecting pooling functions that balance robustness and performance under varying attack scenarios.
To pursue this goal, we begin by formalizing adversarial attacks in the context of graph classification. This framework establishes an upper bound on the model's expected adversarial risk, allowing us to quantify GNN vulnerability within defined neighborhoods of input graphs. We then theoretically analyze common pooling operations, derive their respective robustness bounds, and provide insights into their behavior under perturbations. These bounds reflect both graph structure and model-specific factors such as message-passing weights. Motivated by this analysis, we propose Robust Singular Pooling (RS-Pool), a novel pooling mechanism that constructs graph-level representations using the dominant right-singular vector of the node embedding matrix. RS-Pool improves robustness by filtering out noise-sensitive components. RS-Pool is fully differentiable, model-agnostic, and compatible with existing GNN architectures and defense methods. Since, we rely solely on the dominant singular vector in our pooling operation, do not require the expensive computation of the entire singular value dccomposition and can instead employ the far more efficient power iteration method enabling practical deployment. Empirical results across multiple adversarial settings confirm that RS-Pool achieves strong robustness with minimal impact on clean accuracy and runtime performance.
The main contributions of this work are threefold. First, we introduce a theoretical framework that quantifies the effect of flat pooling operations the resulting robustness of GNNs in graph classification. Second, we propose RS-Pool, a novel pooling mechanism that leverages the dominant singular vector of the node representation matrix to improve resilience to adversarial perturbations. Third, we develop an efficient approximation algorithm for RS-Pool and empirically validate its effectiveness in enhancing model robustness under adversarial attacks.
2 Related Work
The adversarial robustness of graph neural networks (GNNs) has attracted substantial attention 20 37 26, III, spurring a variety of attack methodologies. These attacks are typically formulated as optimization problems that seek minimal perturbations of the adjacency matrix or node features to alter model predictions. They fall into two main categories: evasion attacks, which alter the graph after training [4 49 19). and poisoning attacks, which introduce modifications during training [47 29). Although most studies target node classification, techniques such Projected

---

Gradient Descent (PGD) also extend to graph classification. Nevertheless, only few methods explicitly address the graph-level setting. For example. [39 leverage Bayesian optimization to mount attacks on graph classifiers, while 7 evaluate reinforcement learning, gradient-based optimization, and genetic algorithms that iteratively refine graphs via a fitness-driven evolutionary process.
On the defense side, numerous strategies have been proposed to strengthen GNN robustness such as input preprocessing [13). For instance, [42) remove potentially adversarial edges using Jaccard similarity. These techniques aim to suppress perturbations before message passing. More recent approaches intervene in the phessage-passing process itself. GNNGuard [46] employs attention- based edge pruning to discard vulnerable connections, and SoftMedian [7] introduces robust aggregation scheme that down-weights neighbors based on their distance to the fimension-wiss median, mitigating the impact of outliers or malicious inputs. Other architectural defenses include GCORN [. which enforces orthogonal weights to enhance stability. Beyond architectural changes, adversarial training remains highly effective. For example, [19] train GNNs on perturbed graphs, increasing robustness to future attacks. Complementing empirical defenses, certified methods are gaining traction. Techniques such randomized smoothing [4], building earlier work [50] 3
offer probabilistic guarantees that predictions remain stable under bounded perturbations.
In parallel, research on pooling strategies has also made great strides [45] 31 27 2). where different pooling methods have been presented to enhance the model's predictive ability. this work, we aim to connect these two research directions, by examining how flat pooling operations affect the expected robustness. Rather than competing with existing defenses, we offer a complementary viewpoint by analyzing the final pooling stage crucial but often overlooked component in robustness research for graph-level prediction. To the best of our knowledge, this the first systematic study of pooling in the context of adversarial defense for graph classification. While recent work by [40] offers a broad empirical benchmark of hierarchical pooling methods, assessing their robustness and generalization, their study centers graph coarsening and does not address flat pooling or its theoretical implications in adversarial settings.
3 Preliminaries
In this section, we introduce the fundamental concepts and notation that underpin our work.
Notation and Setup. Let G = (V, E) be a graph, where V and E denote its sets of nodes and edges, respectively. We use n [V] and m E to represent the number of nodes and edges. The neighborhood of a node E V is denoted by (v()) deg(u) = [.)]. A graph is typically represented by its adjacency matrix A E Ru and, when available, node feature matrix X Raxd, where d is the feature dimensionality.
{u: (v, u) E E). The degree of node is thus
Message Passing GNNs. A GNN consists of a sequence of neighborhood aggregation layers that update node representations by incorporating information from local neighborhoods. Let A denote the initial feature vector of node c. The hidden representation of node u at layer S is updated as:
AGGREGATE(() ((N(-)) N(()))): A.9 = COMBINE( (n!-))a!!))
where AGGREGATE!!! is permutation-invariant function that summarizes the features of v's neighbors, and COMBINE( integrates this summary with v's previous representation.
Pooling Operation. After L layers of message passing, a permutation-invaariat pooling function is applied to the final node representations to obtain graph-level embedding:
lgg = POOL. ((n5) v v3).
This step, often referred to as readout or flat pooling, is essential for producing a fixed-size rep- resentation of the entire graph. In this study, we aim to understand how this operation affects the expected robustness of GNNs. For tractability, we focus on flat pooling methods that aggregate node representations in a single step, specifically Sum, Average and Max pooling.
4 On the Robustness of Pooling Operations
We start by discussing the concept of the expected robustness in graph classification tasks, followed by theoretical analysis of widely used flat pooling operations.
3

---

4.1 Expected Graph Robustness
We consider a set of input graphs represented by their adjacency matrices and node features with associated labels, denoted as (AI.X; 11.31) 31)..... (Av. Xv. UN) E (A, X' ),,,, sampled from an underlying distribution D over (A, )) The goal of graph classification is to learn a function f: (A) - 3/ that predicts target label by minimizing the expected classification risk under DD
In this work, focus on the black-hox evasion setting, where the attacker has no access to the trained model f or its training data and cannot modify them. For the purpose of analysis. we extend the framework introduced in [10] to the graph classification context. Let (A, E (A, be an input graph with label E y. We assess the model's robustness by examining its behavior in the neighborhood of the input graph, measuring the expected risk of prediction changes under bounded perturbations. Given perturbation budget the expected adversarial risk of f with respect to D is defined as:
R.IFl
E (A,X)~ (A,8)EAG(AAX)
(1)
where dy = 11-112 denotes distance in the output space y. The set XA, = X{, dA,x((A, (A, X)) <}} defines the neighborhood of admissible perturbations under the budget measured by the input space distance da,x. In our work, we consider a distance that reflect both the topology and the node features while taking into account the set of permutation matrices II. which
can be written as: dA,x ((A, X), ()) minpen ||A - PAP 1l2 + +|X PX]12
From a defense perspective, the goal is to minimize the expected risk [f), thereby ensuring the model's 's predictions remain stable within the allowed perturbation budget. Bounding this risk offers valuable insights into model's ' expected robustness and its sensitivity to adversarial inputs. Based on this, we formalize the notion of GNN robustness in Definition 4.1
Definition 4.1. (Expected Robustness) A brsed function f: (A, + y is said to be (e, --robust if its expected risk satisfies [.] S 7.
We note that this definition is different from the usually used adversarial robustness in the literature, and rather adopts an average case perspective by evaluating robustness over the entire neighborhood N. (A, X). ). in contrast to worst case analysis, which considers only the most damaging perturbation. We have chosen to operate within this direction, since we consider that worst case attacks represent a subset of the neighborhood and therefore it follows that upper- bounding this quantity for the average would also give me some insights on the robustness under the worst case criterion. This distinction between average and worst case robustness has been discussed in the literature [36
4.2 On the Expected Robustness of Flat Pooling Operations
Having established formal definition of the expected robustness in the context of graph classification, we now apply this framework to assess the robustness properties of commonly used pooling operations. Specifically, examine two representative models from the message-passin
family introduced in Section 3 Graph Convolutional Networks (GCNs) (24) and Graph Isomorphism Networks
(GINs) 44 both of which are widely adopted in graph-level tasks.
Following message passing, node embeddings are aggregated through a pooling operation to produce a global graph representation suitable for downstream classification. However, this step can introduce information loss, prompting the development of various pooling strategies. In this work, we focus on three widely used flat pooling techniques, which generate graph-level representations through a single aggregation step over node embeddings: Sum, Average, and Max pooling (9]. While our theoretical analysis centers on these operations, the proposed robustness framework is general and can be extended to other mechanisms. The primary objective is to quantify the influence of each pooling strategy on the expected robustness of the model, as defined in Definition 4.1 Throughout the remainder of this paper, I| denotes the operator norm.
Theorem 4.2. Let f:= (A, - y denote a graph-based function functiiioo composed of L GCN layers, where the weight matrix of the l-th layer is denoted by W(O, Under a feature- adversarial attack with perturbation budget e, the robustmess of f in the sense of Definition 4.] Satissies the following:
f f is based on Max pooling, then it is (e, robbuttt with:

---

7 m d ( (T=))) maXNEV W.e.
If based on Sum pooling, then it (e, rr-buusst with:
= -T Enev u.c.
If is based on Average pooling, then it is (e, rrobuutt with:
7 (ITL 1) Enev Wac.
Here, th denotes the sum of normalized walks of length I.-1 originating from node and dy is the output dimensionality of the final layer.
Theorem 4.2
establishes upper bounds on the expected risk for different flat pooling operations. A common factor across all bounds is the product of the weight matrix norms, reflecting the influence of message-passing layers. The second term in each bound differentiates the pooling methods. For Sum pooling, the bound scales with the total normalized walk count, which increases with graph density. Consequently, adversarial noise propagates through multiple paths, weakening robustness and highlighting Sum pooling's vulnerability in dense graphs. In contrast, Average pooling normalizes this term by the number of nodes, diminishing the impact of increased walk counts. As graphs grow larger sparser, this normalization tightens the bound, offering improved robustness over Sum pooling. We observe the bound on Max pooling to be governed by single node in the graph for which the normalized sum of walks of length L - 1 is maximal. Consequently, the max max-pooling operator more localized than the Sum and Mean Operators for which our bounds depend on global characteristics.
The robustness of each pooling method further depends on the attack strategy. Under targeted attacks, where high-degree or influential nodes are perturbed, Max pooling can be particularly vulnerable, especially in gradient-basee settings that expose critical nodes. However, when less influential nodes are targeted, it may retain robustness. In non-targeted attacks that perturb nodes more uniformly, Max pooling may outperform Sum and Average pooling, as it aggregates over only one node. Overall, the analysis reveals that no single pooling method is optimal across all scenarios. Robustness varies with graph structure and attack type, underscoring the importance of selecting pooling strategies based on the anticipated threat model. Although our analysis focuses on GCNs, the same theoretical framework naturally extends to Graph Isomorphism Networks (GINs).
Theorem 4.3. Let f: (A, + y be composed of L GIN-layers (with its internal parameter 0) and let (())) denote the weight matrix of the E-th MLP layer. We consider the input node feature space to be bounded, i.e., ||X]|2 < B for some B R. Under a feature-based adversarial attack with perturbation budget e, the model f (e, robuttt with respect to Definition 4.1 with:
If f is based on Max pooling: I [/(0 [| (BL (max deg(u)) + c)-
If is based on Sum pooling:
1 || [[ +nE).
If f is based on Average pooling: y /n, dy (I ( (BBLI + +C)
Here, |F] is the number of edges, n. number of nodes and dr. the dimensionality of the final layer L.
As in the GCN case, the derived upper bounds for GIN highlight the influence of pooling operations on expected robustness. For Sum pooling, the expected adversarial risk grows with both the number of nodes and edges, indicating increased vulnerability in larger or denser graphs. In comparison, Average pooling normalizes by the number of nodes, which reduces the relative impact of individual perturbations and enhances robustness in large or sparse settings. Max pooling produces bound governed by the maximum node degree, suggesting heightened sensitivity in graphs with hub-like structures, particularly when high-degree nodes are targeted. These findings are consistent with GCN-based analysis and further emphasize that pooling a key factor in shaping the robustness of GNNs.
5

---

5 Robust Pooling Through Singular Decomposition
Section 4.2 demonstrated that pooling significantly influences the robustness of graph classification models. This aligns with the intuition behind message passing, where node-level perturbations propagate through local neighborhoods and ultimately affect the global representation, depending on the pooling strategy. Influential nodes with high centrality or strong connectivity tend to amplify adversarial effects, spreading their influence and creating compound vulnerabilities. Similar trends have been observed in node classification [28), where gradient- based attacks often target structurally important nodes. Mitigating the influence of such nodes can therefore improve robustness.
To address adversarial vulnerabilities, prior work has proposed various preprocessing techniques 42 13 46 [7]. However, these defenses typically operate before or during the message- message-passing stage and are primarily tailored for node classification tasks, where graphs are large and removing subset of nodes has limited impact on performance. In contrast, graph classification often involves smaller graphs for which the removal of nodes can be more critical. Removing or down-wo wwiehtigg certain nodes can disrupt information propagation and degrade clean accuracy, which is especially problematic given that it not known in advance whether graph has been attacked. As a result, achieving favorable trade-off between robustness to adversarial inputs and strong performance on clean data remains a key challenge in graph-level learning.
Given these considerations, to avoid the consequences of node removal, we propose an alternative pooling strategy that effectively balances robustness and clean performance. We introduce Robust Singular Pooling (RS-Pool), a novel method that retains informative signals in clean graphs while mitigating the influence of adversarial perturbations. RS-Pool generates the graph- representation by extracting the dominant right singular vector of the node embeddings. This design is motivated by classical results in matrix perturbation theory vectors are stable under bounded perturbations and encode the most reliable directions of variation.
which demonstrate that leading singular
In the context of graph classification, adversarial attacks distort node embeddings produced by message-passing layers. These distortions are typically localized in nature, due to the limited depth of GNNs (empirically often L E (2, 5, is observed to be tighhestperforming [4] and constrained perturbation budgets that restrict modifications to a small subset of nodes [28]. As a result, the dominant singular vector often remains stable, while less-dominant components are more susceptible to noise. By projecting onto this robust direction, RS-Pool generates a stable and expressive graph representation that maintains performance under adversarial conditions. Formally, let H E Raxd denote the matrix of node representations obtained after the pessage-passingg stage (e.g., from GCN (24). where n is the number of nodes and d the features (embedding) dimension. We consider the singular value decomposition (SVD): H = UEVT, where U E RRax., Σ E R and V E Rdxd The singular values 01 2 02 2 ... 2 minn(,.4) 2 quantify the contribution of each singular direction. Let 01 E IRd denote the dominant right singular vector, RS-Pool defines the graph-level representation as a scaled version of ty:
RS-Pool:
R nnd + Rd
H TH(
where T Ryo is scaling factor controlling the magnitude of the output embedding. The dominant singular vector captures the principal direction of variation in the embedding space, effectively summarizing the global structure encoded in H. Scaling ensures the pooled vector retains sufficient energy to serve as a meaningful input to downstream classifiers.
Crucially, RS-Pool also offers inherent robustness. Classical results from matrix perturbation the- ory [8 41 show that if the leading singular value a (H) is well-separated from the next 02 (H) (we use a. and az (H) interchangeably when unambiguous), then the top singular vector vi (H) is stable under small perturbations to H. This separation condition ensures that adversarial distortions, often low-rank and localized, have limited influence the dominant representation. Within the expected adversarial risk framework introduced in Definition 4.1 we now formalize the robustness guarantee of ERS-Pool
Theorem 5.1. Let I: (G, + y denote a graph function composed of L GCN layers and using our RS-Pool where the weight matrix of the i-th layer is denoted by W Under a feature-baser

---

adversarial attack with perturbation budget f is (E, 7)-bbust in respect to Definition 4.1 with:
TV2e 01
L.
n
[I
(1
(2)
02
8=1
with in. denoting the sum of normalized walks of length (T. 1) starting from node u, and on # 02 being the two dominant singular values.
Theorem 5.1 provides an upper bound on the expected adversarial risk of the proposed RS-Pool method. As with previous bounds, the expression includes shared terms such as the product of pressage-passing weight norms and the perturbation budget. However, a distinguishing feature of this bound is its dependence on the spectral gap of the clean representation matrix И and the scaling factor т. Notably, the bound relies solely on the spectral gap of the unperturbed node embeddings, not the perturbed input, reinforcing our earlier claim that RS-Pool mitigates the influence of adversarial noise. Furthermore, the scaling parameter T acts as a tunable control for the representation' 's sensitivity, allowing the bound be tightened and thereby improving robustness. This tunability especially beneficial in settings where the message-passingg layers exhibit high sensitivity to input perturbations.
Corollary 5.2. Let f: (g, X) + y be the graph-based function considered in Theorem 5.1 Let Y
be the upper-bound derived in Theorem 5.1 we have that f is (e, 7 rrobustt with:
min (7,nt%,27) 2T).
(3)
highlights that even when the message- mnessage-passnng passing component is highly vulnerable to adversarial perturbations, as indicated by large weight norms, RS-Pool can still ensure bounded expected risk by constraining the representation through the scaling parameter This scenario has motivated prior robustness techniques such as orthogonal regularization (6. The result demonstrates that RS- Pool serves as stabilizing mechanism, contributing to overall robustness even when earlier layers are sensitive to input perturbations. Morcover, RS-Pool retains the essential property of permutation invariance, ensuring its compatibility with standard GNN architectures.
Corollary 5.2
Lemma 5.3. RS-Pool is permutation invariant; that is. for any pair of isomorphic graphs G and Gn with its corresponding permutation matrix IL, we have fAA X) ([An,Xn):
Remark. Theorem 5. does not address cases where the largest singular value has multiplicity greater than one. Such cases are rare in practice, as GNNs' inherent smoothing tends to reduce the feature matrix rank and create a non-trivial spectral gap. Nonetheless, Wedin's 's Theorem, which underlies our proof, extends to the case where the dominant singular value has multiplicity r, yielding additional factor /rr (instead of /2) in the upper bound y, with the relevant spectral gap 0, Or+1-
Implementation and Algorithm. RS-Pool constructs the graph-level representation by extracting the dominant right-singular vector of the node embedding matrix H. As only the top singular vector is needed, computing the costly full SVD is unnecessary. Instead, we employ the truncated power iteration method to efficiently estimate the dominant vector. This iterative approach is scalable with respect to both the number of nodes and the embedding dimension. A key advantage of RS-Pool is that it remains fully differentiable, allowing seamless integration into existing GNN pipelines and enabling end-to-end training via standard backpropagation. Algorithm D outlines the forward pass of GNN model utilizing RS-Pool.
Algorithm 1 RS-Pool Forward Pass
Require: H E Raxdd T E Rx0, E Noo I: v t random unit vector in Rd 2: for t to K do
+- Hr
e t- H w
5:
w + w/|lo|l
6: end for
7: return
Computational Complexity. The main computational overhead of RS-Pool stems from approximat- ing the dominant singular vector using power iteration. Each iteration has a complexity of O(n x d), where n the number of nodes and d the embedding dimension. In practice. we find that only a small number of iterations (e.g., 2-5) is sufficient to obtain a reliable estimate. This efficiency is supported by the convergence properties of power iteration, which known to converge geometrically at a rate determined by the ratio a2 (H)/o) oz (H), where dn (H) and o2 (H) are the top two singular values of H. well-separated spectral gap ensures rapid convergence, which is commonly observed in GNN embeddings due to dominant low-frequency components. As a result, the overall complexity remains O(K x n x d) for a small K, making RS-Pool suitable for real-world graph applications.
comparison of its runtime with other pooling methods is provided in Appendix E.2
7

---

We further note that the fast empirical convergence of power iteration aligns with the theoretical robustness bound in Theorem 5.1 where the spectral gup σi (H) - ( plays a central role. A larger gap not only tightens the bound (lower 7), indicating improved robustness, but also ensures faster convergence of the power iteration method used to compute RS-Pool. This dual benefit highlights the practical effectiveness of RS-Pool, as favorable spectral properties commonly observed in GNN embeddings lead to both stable representations and efficient computation. We provide empirical validation of this observation in Appendix E.
Experimental Results
This section aims to validate our theoretical insights on real-world datasets where we aim to investigate the robustness of RS-Pool in comparison to other considered pooling methods. In line with our theoretical study, we focus on the GCN model in this section while we provide results on the GIN in Appendix E.4 We additionally provide results when considering the adversarial defenses as a backbone GNN in Appendix [..5] showcasing therefore the universality of our insights. For all our experiments, we use a 2 layers GCN (resp. GIN) classifier with identical hyper-parameters parameters across all experiments to guarantee fair comparison. Each experiment is 10 times to reduce the effect of randomness. Additional implementation details are provided in Appendix FF
Considered Benchmarks. We evaluate RS-Pool against classical flat pooling methods, including Average, Max, and Sum pooling, as well as several advanced approaches. We include Self-Attention Graph pooling (SAG) [27]. which uses node features and graph structure to compute attention scores and retain informative nodes, and TopK pooling (TopK-P) [[6), which learns to rank nodes via a projection vector and selects fixed fraction by score. We also consider Path-Integral-basee Pooling (PAN-P) 31. a TopK extension that integrates multi-hop structure into scoring. Lastly, we compare with Sort Pooling (Sort-P) [45], which orders nodes by feature magnitude and concatenates the top-k sorted nodes.
Attacks. We evaluate robustness under three adversarial attack strategies: (i) Random Attack, which randomly adds or removes edges in the input graph. The search performed over K random perturbations, and the one yielding the worst performance is selected; (ii) Genetic Attack [7]. which uses evolutionary algorithms to generate adversarial graphs via selection, crossover, and mutation; and (iii) Gradient-Based Attack (PGD) 771. which greedily modifies the graph structure by targeting edges with the highest gradient magnitude relative to the model's input. For all attacks, we apply a perturbation budget of e = 0.3, allowing up to 30% of the edges in each graph to be modified. We additionally considered the Bit-Flip Attacks (BFAs) 26 25 in Appendix E.6
Datasets. We conduct experiments on standard graph classification datasets from the TUDataset benchmark [32], spanning diverse domains. In bioinformatics graphs (PROTEINS, D&D, EN- ZYMES), small changes to residue contact links can influence biological property predictions. In molecular graphs (NCI1, ER MD), altering a bond may change the predicted molecular function. For social networks (IMDB-B, REDDIT-B), edge perturbations, such as fake user interactions, can flip the predicted graph label. In image-based graphs (MSRC_99 29), local edits to image patches can lead to misclassification. For all datasets, we use the public train/validation/ee splits when available [4]; otherwise, we adopt the same evaluation protocol and report the specific folds used.
6.1 Experimental Results
Empirical Estimation of Adversarial Risk. We begin by empirically analyzing the adversarial risk defined in Section characterized by the upper bound 7. To this end, we compute the distance be- tween the pooled representations of clean and adversarially perturbed graphs for each pooling method. the results of this analysis across the considered pooling strategies. As expected, RS-Pool consistently maintains a smaller distance between clean and perturbed representations. In addition, we see the exact behavior underlined in our theoretical results provided in Theorem 4.2 4.2 where the Sum is the one with the higher upper- bound, followed by the Max and the Average. This behavior directly correlates with lower attack success rates of our RS-Pool, as shown in subplots (b)
and (d). Complete results for all pooling methods are provided in Appendix E.3
On the Effeet of the Hyperparameter 7. As discussed in Section 5 the parameter 7 plays a critical role in RS-Pool. While it directly scales the extracted dominant singular vector, its more significant
8

---

25Poal
Sum
Mac
ca PROTEIS
[b) PROTEINS
DD
C-T1 DO
(el PROTENS
Budgeex
Attack Buaggel
Sudgel
Budieet
Accurety
Figure I: Left: Empirical estimation of adversarial risk y and corresponding attack success rate on PROTEINS ((a), (b)) and D&D ((c). (d)). Right (e): Study of the effect of the parameter which inversely controls T on the PROTEINS dataset.
Table I: Clean and Attacked classification accuracy (#: standard deviation) of the RS-Pool and other pooling strategies on different graph classification dataset when subject to adversarial attacks.
Dataset
Attack
Sum
Average
Max
SAG
TopK P
PAN-P
Sort
RS Pool
Clean
74.2 3.1
70.8 + 2.2
73.2 = 2,3
72.3
73.2
74.7 = 1.7
72.3
73.5
29
PGD
51.9
PROTEINS
Random
68.3 3.1
65.7
42.9
61.
68.2
6.1
3.2
70.4 3.1
Genetic
66.1 1.122.
65.3 3.2
39.8 5.0
4.99388
66,5
66.
63.7
68.2
PGD
6.0
12.6 2.666366
8.2
9.0
5.9 59::29
9.6
16.6 1.9
30.4 3.2
Random
61.
64.4
19.7 2.4
62.8
58.6 1.1
61.9
50,3
66.7=2.1
Clean
70.6
67.91166
68.2 1
70.4
70.9
1.74
6.
70.1 1.2
PGD
23.4
14.1
19.7 = 1.7
22.3
23.2 2.7
23.5 2.
26,3 18
NCII
Random
2..1
20.6
5.1
19.8
25.8
24.
27.2
Gemetic
2.89.1.0.8
1.77
9.2
18.7 1 1.2
23.9
19.1 2.26
25.8
26.9 0.7
Clean
3..4
2..7
27.7 1.6
26.1
27.7 4.3
2.6
29.7
328=
ENZYMES
PGD
2.1
3.
9.8
8.
1.4
5.6
11,9
Random
3.4
8.3 2.3
22
3.8 1.6
10.6
5.
9.4
11.7 2.1
Genetic
8.9
7.2
6.1
4.4
1.6
5.0
9.
9.9 2.1
Clean
5.2
62.7
56.7
63.72
3..0
2.3
56.7
62.0
IMDB-B
PGD
5.0
6..3
320 5.8
6.2
47.0
2.9
56.7
61.7 2.5
Random
52.3
62.3
45.9
62.9 1: 4,3
49.7 4.0
1..
$5.7
55.7
-
Genetic
52.0
61.0
5.0
6.8
50.9
1.7
62.
Clean
7.85
70.0
720
70.5
78.5 1.4
7.55
80.0
0.
PGD
69.0.1...
57.5
62.0 1 2.5
$5.5 1.9
55.5 1.9
57.0 1.9
56.5 1 2.5
-4.0-99 0.9
REDDIT-B
Random
76.0
69.5
67.9 = 1.2
69.0
76,5
7.
78.0
75.0=03
Genetic
7.9
69.0 155
1.4
68.5
55.5
57.0
$6.5
74.5-07
Clean
6..9
61.1 1 6.7
614 6.1
61.1 1 5.8
66.3 9: 4.2
69.7 1.8
67.1 5.4
65.9 599
ER_MD
PGD
40.4
44.5 3.8
3,3
45.9 1499225
35.2 5.3
34.1
47
Random
2.3
51.6
3.8
2.3
50.1
2.3
$2.4 3.8
58.1=59
Generic
6.12+1111
58.9 + 1.8
60.2 + 0.8
59.9 + 2.3
$1.4
52.4 4.2
$5.1 2.3
61.7 1.3
Clean
3..6113.2
88.6
88.6
90.1
87.1
89.4
1
89.7
PGD
2.8
84.1
78.0
82.6
54.6
84.1 2.3
7
85.6 85.6
MSRC_9
Random
85.6
87.8 +: 1.3
86.3 = 1.3
87.1
84.1 3.2
86.8 1.3
86.4
89.1 0.4
Gemetic
85.6 1661122
86.3 1
87.1
19
87.1 1. 2.1
84,1 1. 3.2
86.3
1.8
86.4
1. 3.2
89.4
effect lies in modulating the robustness bound in Theorem 5.1 and influencing the expected risk as detailed in Corollary 5.2 To validate this theoretically grounded effect, we examine how varying impacts both clean accuracy and robustness. In our experiments, to better isolate the role of the spectral gap, we parameterize 7 as a scaled version of the leading singular value, which is already computed during the power iteration. Specifically, we set a (X)/a, where >> 0 is tunable constant. We then study how different values of a affect performance, measuring clean accuracy and attack success rate. As shown in Figure (e), increasing (and thus decreasing r) leads to lower attack success rate, indicating improved robustness. However, this comes at the cost clean accuracy, which peaks at intermediate a before declining. These results highlight the importance of selecting an appropriate value to achieve a desirable trade-off between robustness and predictive performance.
Results Analysis. We evaluate the performance of RS-Pool and baseline pooling methods under adversarial attacks. Table presents classification accuracy on clean and perturbed data across datasets and attack types. RS-Pool consistently performs well on clean inputs and significantly outperforms baselines under adversarial conditions. The only exceptions occur in two cases involving the Random attack, where the limited perturbation budget results low attack success, allowing some baselines to retain higher accuracy due to strong clean performance. Under stronger attacks

---

Table 2: Clean and Attacked classification accuracy (+ standard deviation) of the RS-Pool and other pooling strategies on different graph classification dataset when subject to feautre-basee adversarial attacks.
Dataset
Attack
Sum
Average
Max
SAG
TopK-P
PAN-P
Sort-P
RS-Pool
Clean
3.1
70.8 =
2.3
3.3
7.92
74.7 = 1.7
72.3 1
73.5 2.9
PROTEINS
PGD
6.2 6.0
66.4
6.6
6.9
68.8 2.9
66.7 2,9
69.9 :.1
Random
5.2
6..6
68.4
6.0
6.8
69.2
70,9 70.2226
Clean
0.6
7..1
7..
0.4
74,4
74,9 7..7
0.7
D&D
PGD
3.6
66.
6.
4.89
2.7
3.15
4.9
1.1
Random
5.2
6.5
4.66
4..1
4.19
4.60
4.97
2.3
Clean 7.8
1.69
6.22
7.
7.99
70.4
70.1 6.45
NCII
PGD
3..6
36.4
5.3
36.6
3..1
3.8
39.4 366
Random
4..1
28
5..9
40.1
38.5+
40.9
38.5
41.7
Cleun 6..0
6.71
6.14
6,81
66.3
69.7
67.1
5,9
MR_MD
PGD
41,9
1.
4.6
3.5
4.8
2.1
55.1 4.5
Random
2.3
4.3
52.
53.9
52.1
512
Clean
3..6
88.6
88.6
90.1
87.1
8.14
89.7 2.
MSRC_9
PGD
2.8
85.6
8.
87.1
83.3
8,66
14
88.3
Random
88.6 3.74
87.9
89.4 1.1
87.1
8.99
89.3 3.2
such as PGD, RS-Pool shows notable gains, achieving up to a 14% uplift over the second-best method on the D&D dataset. This advantage more pronounced on datasets like D&D, which contain larger and denser graphs where the leading singular vector captures most of the informative signal. enabling more robust representation through RS-Pool. Similar observations hold for GIN and other adversarial defenses, as shown in Appendix E.4 and E.5
Feature-Based Robustness. To further demonstrate the value of our theoretical analysis, and inline with the considered setup, we evaluate adversarial attacks targeting node features. We adopt the same attack strategies used in the structural setting: Random, which adds noise to features random, and PGD, a gradient-based attack. Table 2 clean and attacked accuracies (with standard deviations) for our proposed pooling method, RS-Pool, alongside other baseline pooling strategies across multiple datasets. Consistent with the structural attack results, RS-Pool achieves higher attacked accuracy, highlighting its improved robustness against feature perturbations.
7 Conclusion
In this work, provided theoretical analysis of how pooling operations affect the adversarial robustness of GNNs in the context of graph classification. Our findings indicate that different pooling strategies exhibit varying levels of sensitivity to attack types some being more robust to targeted perturbations, while others are more vulnerable under global attacks. To address this, we introduced RS-Pool, novel pooling method that embedding matrix as the graph-level representation. We theoretically demonstrated that RS-Pool can attenuate the impact of adversarial perturbations at the pooling stage, thereby improving the model's overall robustness. Empirical evaluations on real-world datasets, using different GNN backbones and adversarial attacks, confirm that RS- achieves stronger robustness while maintaining competitive performance on clean data.
scaled version of the dominant singular vector of the node
Limitations. Our method relies on the existence of spectral gap in feature matrices, yet gap size depend on the network depth and graph topology and varies across settings, which warrants further study. Moreover, power iteration pooling scales linearly with edges but may incur nontrivial overhead on very dense graphs, limiting its use resource constrained applications.
Acknowledgements
The computation (on GPUs) was partially enabled by resources provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS) at Alvis partially funded by the Swedish Research Council through grant agreement no. "2024/22-309" J.L. would additionally like to aknowlegde support by the French National Research Agency (ANR) via the "GraspGNNs" JCJC grant ANRR22-CCF23-9880 We furthermore want to acknowledge all the reviewers for their feedback that improved our manuscript.
10

---

References
[1] Yassine Abbahaddou, Sofiane Ennadir, Johannes F Lutzeyer, Michalis Vazirgiannis, and Henrik Boström. Bounding the expected robustness of graph neural networks subject to node feature attacks. In International Conference on Learning Representations, 2024.
[2] Filippo Maria Bianchi, Daniele Grattarola, and Cesare Alippi. Spectral clustering with graph neural networks for graph pooling. In International conference on machine learning, pages 874-883, PMLR, 2020.
[3] Aleksandar Bojchevski and Stephan Günnemann. Certifiable robustness to graph perturbations, 2019.
[4] Aleksandar Bojchevski, Johannes Klicpera, and Stephan Günnemann. Efficient robustness certificates for discrete data: Sparsity-aware randomized smoothing for graphs, images and more, 2020.
[5] Chen Cai, Dingkang Wang, and Yusu Wang. Graph coarsening with neural networks. arXiv preprint arXiv:2102.01350, 2021.
[6] Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usumier. Parseval networks: Improving robustness to adversarial examples. In International conference on machine learning, pages 854 863. PMLR, 2017.
[7] Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial Attack on Graph Structured Data. In Proceedings of the 35th International Conference on Machine Learning, pages 1 1115-1124, 2018.
[8] Chandler Davis and W. M. Kahan. The rotation of cigenvectors by perturbation. ili. SIAM Journal on Numerical Analysis, 7(1):1-46, 1970.
[9] David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alan Aspuru-Guzik, and Ryan Adams. Convolutional networks on graphs for learning molecular fingerprints. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
[10] Sofiane Ennadir, Yassine Abbahaddou, Johannes F Lutzeyer, Michalis Vazirgiannis, and Henrik Boström. A simple and yet fairly effective defense for graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21063-21071, 2024.
[11] Sofiane Ennadir, Amr Alkhatib, Giannis Nikolentzos, Michalis Vazirgiannis, and Henrik Boström, Unboundattack: Generating unbounded adversarial attacks graph neural net- works. In International Conference on Complex Networks and Their Applications, pages 100-111. Springer, 2023.
[12] Sofiane Ennadir, Johannes Lutzeyer, Michalis Vazirgiannis, and El Houcine Bergou. you want to be robust, be wary of initialization. Advances in Neural Information Processing Systems, 77:23796-23822 2024.
[13) Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 1 13th international conference web search and data mining, pages 169-177, 2020.
[14] Federico Errica, Mareo Podda, Davide Bacciu, and Alessio Micheli. Fair Comparison of Graph Neural Networks for Graph Classification. In 8th International Conference on Learning Representations, 2020.
[15] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric.
In ICLR Workshop Representation Learning Graphs and Manifolds, 2019.
[16] Hongyang Gao and Shuiwang Ji. Graph u-nets. international conference on machine learning, pages 2083-2092. PMLR, 2019.

---

[17] Simon Geisler, Tobias Schmidt, Hakan Şirin. Daniel Zügner, Aleksandar Bojchevski, and Stephan Günnemann. Robustness of graph neural networks at scale. Advances in Neural Information Processing Systems, 34:7637-7649, 2021.
[18] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning. pages 2263-1272. PMLR, 2017.
[19] Lukas Gosch, Simon Geisler, Daniel Sturm, Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Adversarial training for graph neural networks: Pitfalls, solutions, and new directions. Advances in Neural Information Processing Systems, 36, 2024.
[20] Stephan Günnemann. Graph neural networks: Adversarial robustness. In Graph Neural Networks: Foundations, Frontiers, and Applications, pages 149-176. Springer, 2022.
[21] Wei Jin, Yaxing Li, Han Xu, Yiqi Wang, Shuiwang Ji, Charu Aggarwal, and Jiliang Tang. Adversarial attacks and defenses on graphs. SIGKDD Explor. Newsl., page 19-34, 2021.
[22] Steven Kearnes, Kevin McCloskey, Mare Berndl, Vijay Pande, and Patrick Riley. Molecular graph convolutions: moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 30(8):595-608, 2016.
[23] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprini urXiv:1412.6980, 2014.
[24] Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations (ICLR), 2017.
[25] Lorenz Kummer, Wilfried N. Gansterer, and Nils Morten Kriege. On the relationship between robustness and expressivity of graph neural networks. In The 28th International Conference on Artificial Intelligence and Statistics, 2025.
[26] Lorenz Kummer, Samir Moustafa, Sebastian Schrittwieser, Wilfried Gansterer, and Nils Kriege. Attacking graph neural networks with bit flips: Weisfeiler and leman go indifferent. In Proceed- ings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, page 1428-1439 New York, NY, USA, 2024. Association for Computing Machinery.
[27] Junhyun Lee, Inycop Lee, and Jaewoo Kang. Self-attention graph pooling. In International conference on machine learning, pages 3734-3743, pmlr, 2019.
[28] Kuan Li, Yang Liu, Xiang Ao, and Qing He. Revisiting graph adversarial attack and defense from data distribution perspective. In The Eleventh International Conference Learning Representations, 2023.
[29] Vijay Lingam, Mohammad Sadegh Akhondzadeh, and Aleksandar Bojchevski. Rethinking label poisoning for GNNs: Pitfalls and attacks. In The Twelfth International Conference on Learning Representations, 2024.
[30] Chuang Liu, Yibing Zhan, Jia Wu, Chang Li, Bo Du, Wenbin Hu, Tongliang Liu, and Dacheng Tao. Graph pooling for graph neural networks: progress, challenges, and opportunities. In Proceedings of the Sectt-Seccon International Joint Conference on Artificial Intelligence. IJCAI '23, 2023.
[31] Zheng Ma, Junyu Xuan, Yu Guang Wang, Ming Li, and Pietro Liò. Path integral based convolution and pooling for graph neural networks. Advances in Neural Information Processing Systems, 33:1642) 33:16421-16433 2020.
[32] Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs arXiv preprint arXiv:2007.08663. 2020.
12

---

[33] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito,o, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019.
[34] Hao Peng, Zehao Yu, Dandan Zhao, Zhiguo Ding, Jieshuai Yang, Bo Zhang. Jianming Han, Xuhong Zhang. Shouling Ji, and Ming Zhong. Evading control flow graph based GNN malware detectors via active opcode insertion method with maliciousness preserving. Scientific Reports, 15(1):9174, 2025.
[35] Aymen Qabel, Sofiane Ennadir, Giannis Nikolentzos, Johannes F Lutzeyer, Michail Chatzianas- tasis, Henrik Boström, and Michalis Vazirgiannis. Structure-aware antibiotic resistance classifi- cation using graph neural networks. In NeurIPS 2022 Al for Science: Progress and Promises, 2022.
[36] Leslie Rice, Anna Bair, Huan Zhang, and J Zico Kolter. Robustness between the worst and average case. Advances in Neural Information Processing Systems, 34:27840-27851, 2021.
[37] Lichao Sun, Yingtong Dou, Carl Yang, Kai Zhang, Ji Wang, Philip S. Yu, Lifang He, and Bo Li. Adversarial attack and defense on graph data: survey. IEFE Trunsactions on Knowledge and Data Engineering, 2022.
[38] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. In ICLR, 2018.
[39] Xingchen Wan, Henry Kenlay, Robin Ru, Amo Blaas, Michael A Osborne, and Xiaowen Dong. Adversarial attacks on graph classifiers via bayesian optimisation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 6983-6996. Curran Associates, Inc., 2021.
[40] Pengyun Wang, Junyu Luo, Yanxin Shen, Siyu Heng, and Xiao Luo. A comprehensive graph pooling benchmark: Effectiveness, robustness and generalizability. CoRR, 2024.
[41] Per-Åke Wedin. Perturbation bounds in connection with singular value decomposition. BIT Numerical Mathematics, 12:99-111. 11199911111111111111111111111,11 1972.
[42] Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu, Ad- versarial examples for graph data: Deep insights into attack and defense. In Proceedings of the Twenty-Eigg Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 4816 4823, 2019.
[43] Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective. arXiv preprint arXiv: 1ewp::1000004244 2019.
[44] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks? In 7th International Conference on Learning Representattons, 2019.
[45] Muhan Zhang. Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[46] Xiang Zhang and Marinka Zitnik. Gunguard: Defending graph neural networks against adversarial attacks. Advances in Neural Information Processing Systems, 33:9263-9275, 2020.
[47] Daniel Zügner, Oliver Borchert, Amir Akharnejad, and Stephan Günnemann. Adversarial attacks on graph neural networks: Perturbations and their patterns. ACM Transactions on Knowledge Discovery from Data (TKDD), 14(5):1-31, 2020.
[48] Daniel Zügger and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In 7th International Conference on Learning Representations, 2019.
13

---

[49] Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2018
[50] Daniel Zügner and Stephan Günnemann. Certifiable robustness and robust training for graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, jul 2019.
14

---

Supplementary Material: Enhancing Graph Classification Robustness with Singular Pooling
A Proof of Theorem 4.2
Theorem 4.2 Let f: ()) y denote graph-based function composed of L GCN layers, where the weight matrix of the E-th layer is denoted by W Under feature- adversarial attack with perturbation budget the robustness of f in the sense of Definition 4. satisfies the following:
If f is based on Max pooling, then is (6,7) robust with:
v (n, d (I221))()) maxuEv e.
If f is based Sum pooling, then is (c. robust with:
( Evev w.c.
If f is based on Average pooling, then it is (e, robust with:
[I==)))((01] Evev th.
Here, w. denotes the sum of normalized walks of length L-1 originating from node u, and dz is the output dimensionality of the final layer.
Proof. In this proof, we consider that is a graph-function that is based on layers of GCN. We recall that the GCN message- message-passing propagation is formulated for a node as
w(0n) (f-1)
(0)
E VE V(1 + d.)) +1+
(4)
where W(O E Rdd- is the learnable weight matrix with de being the embedding dimension of layer and (f) is the activation function of é-th layer. We recall that (0) =X E Ra is set to the initial node features.
Similar to [ we denote X as the original node features and denote by X' the perturbed adversarial features. We consider node u E V, we denote by h, its representation in the clean graph and K, its representation in the attacked graph. We consider that the activation functions (a (a(()) nonexpansive (1-Lipschitz continuous). From the work, we have the following result
( - ((D1 <
9 22
8=1
X, X',
Σ =EN(o)
(1 d.) (1 d,) ... (1 d
(0
E=1
with ú. being the sum of normalized walks of length (L - 1) starting from node u.
The previous results give us an idea about the behavior of each node's representation when attacked.
In the case of graph classification, additional pooling operation is added, Specifically:

---

(E)
Pool
((n lver).
(In!
graph
Hence this proof's goal analyze the following quantity
|| I
m((C) l- graph
Let's consider the Sum pooling operation, that can be written as
h (LL
Σ h
WEV
We have the following
|14
graph
11'
12(2)
graph
=
Σ
(T) - H(LL)
NEV
(by triangle inequality)
VEV
Σ
lruulle
WEV
M=1
Σ
w..
WEV
For the case of the Average pooling operation, that can be written
A(
M
NEV
We have the following analysis
11
(L) graph
(L)
|| =
(L) - n m((()
graph
V
NEV
(1 (0
S M
(by triangle inequality)
REV
[
Σ w..
NEV
In the case of Max pooling: Let's denote A H(L) HI( Since the magnitude a single entry of can never exceed the Euclidean norm of the whole row, which can be formulated as for any column our difference matrix that Auj << ||A For Max poling, we have the following
| (Z)
H papp I V dy
L [
Sevdl
max w..- wev
We additionally note that we have the following
dr.
| (L) graph
1((L))
praphl 112 Σ
max
Auj 125 Σ
Daj 2= ||
(5)
j=1

---

And we have that
[ i) (I 1/2
| E
From the two derived inequalities, we conclude that in the case of Max pooling we have
Ih gruph
1((L) graph
Il - ev/min dz}
I
[Iwoo
max i..-
uEV
By taking into account the expectancy (as shown in Definition
D. get the desired results.
B Proof of Theorem 4.3
Theorem 4.3 Let f: (A, Y be composed of L GIN-layers (with its internal parameter Ç - 0) and let W(T denote the weight matrix of the é-th MLP layer. We consider the input node feature space to be bounded, x.l||X]l2 < B for some B E R. Under feature-base adversarial attack with perturbation budget the model f is (c, -oobust with respect Definition 4.1
with:
If f is based on Max pooling: II ( (BL (max,ev deg(u)) + +).
If f is based on Sum pooling: y II 1,
( (2BL E +nc).
If f is based on Average pooling: 7 vn, dr (IZE,
2 + +o):
Here, E the number of edges, n number of nodes and dr, the dimensionality of the final layer L.
Proof. In this proof, we consider that f is graph-based function that is based on L GIN-layers (with parameter Ç = 0, usually denoted but changed avoid confusion with our attack budget). The GIN message-passing propagation process can be written for node u as:
h (E+1) = (T(+)) ((1 + C) +
(())
vEN(a)
with 7 denoting Neural Networks (usually MLP) and ( denotes the parameter of the GIN. We recall that (0) X Raad is set to the initial node features.
Similar the previous proof, base our analysis previous work (. denote X as the original node features and denote by X' the perturbed adversarial features. We consider a node V. denote h, its representation in the clean graph and hi, its representation in the attacked graph. We recall that in problem setup, consider that the activation functions (o(())sss nonexpensive (1-Lipschitz continuous).
We use the same assumptions the one considered in [ Specifically, consider that the input feature space Ho is bounded, thus each hidden space Hi of the iterative process of message passing is bounded and let B maxB; be its global maximum bound. We additionally consider that EST GIN-parameter Ç (which is very frequent in the literature). For node u E V, we have the following result:
L ((+1) (((+1)) I| (I +
From this perspective, let's consider the case of graph classification. We omit the definition of the different pooling this stated in the previous proof. We start by considering the Sum pooling

---

operation:
1/
graph
(L)
k(())
graph
=
Σ (1)) 11 (
)
uEV
Σ
(by the triangle inequality)
wEV
[BL deg(() +()
t=1
wEV
L.
II
II
BI.
Σ
deg(u) (u) ) + IV|c
E=1
wEV
In the case of undirected graph, since Σwev deg(u) = B based on the previous result, we can write:
|1 1app
n 1r grapa I|
(T |V]e).
t=1
Similar for the case of Average pooling, we have:
1/
1
h
11 (()))
NEV
(L)
h (
a
NEV
[
2BL
)
In the case of Max pooling operation, following the derivated Inequality we can directly get:
|| (2)
M(L de II (w(11)
V
[L.
max deg(u) + ueV
graph
By taking into account the expectancy (as shown in Definition]
we get the desired results.
C Proof of Theorem 5.1
Theorem 5.1 Let f: (G,X) + y denote a graph-based function composed of L GCN layers and using our RS-Pool, where the weight matrix of the i-th layer is denoted by ((()) Under a feature- based adversarial attack with perturbation budget f is (e, -robbustt in the sense of Definition 4.J with:
2e
(I
Σ (w.)²
01
02
8=1
with uh denoting the sum of normalized walks of length (L 1) starting from node u, and 01 # 02 being the two dominant singular values.
Proof. In this proof, we consider that ff is graph-function that is based on L layers of GCN. Similar to previous proof, denote by X' the perturbed node features, and W, (e) the corresponding node embedding of u at layer l. Following [I], for each node u we have:
( W1 11)
[[ Dunc,
(6)

---

Thus, to relate | - | in operator norm
[[ ² a
[ - H1 Σ |n - N4
12
Σ
(wx))
)
Consequently, we have
[ - HS [H H
(II
Σ (.))²
(7)
We consider now our proposed pooling RS-Pool, which consists of considering the dominant right singular vector of H(L), In this perspective, let:
span (H(4) and = ( and accordingly: 0 - e(S, [0, $]
with 0 being the principle angle between the two vectors. Note that ensure to always take the smaller of the two possible angles between the two dominant vectors (by controlling the signs). Building on Wedin's 's sin-O theorem 41 we can write:
H H'
sin 0 S
( (H)
From another perspective, for considered subspaces and S², when considering the principal angle, we have the following:
COs
max (x.y) = 1(1, [()1
SES. |||||||||1=1
We may assume (ui,ef) 2 O, since the sign of the dominant singular vector can be chosen by definition. From a different perspective, can express as follows:
||u o/11? (u1 2/ 01 - -vイ) 2 2(11, o1)
= 2 - 2 cos 0
4 ssn (0/2).
We additionally have the following (note that 0 E [0, fl):
0 0 sin
sin 0
sin 0
sin 0 v2
sin 0 2in
2 cos
2 cos
2 cos
From the two computed elements, we find that:
||u 2/ 11 < 2 sin(2
< 2 sin 0
V2
< / sin 0
|| Ht'l
S 2
01 o1(H) (2
22
<
III ([
(w)))
σ1 02
By also incorporating the constant
E Ryo, we obtain the final bound:
TV 2
(
Σ (wo.))
01
02
8=1
5

---

On the special case of singular value degeneracy. In the previous analysis, we consider that the dominant singular value doesn't 't have multiplicity higher than We consider that this the most dominant case in practice. Specifically, we note that RS-Pool is applied to the final node embedding matrix produced by the message-passee passing scheme. At this stage, having a dominant singular value with high multiplicity is highly unlikely, even in symmetric graphs, because the learned embeddings do not depend just on the graph structure but also on initial node features and the GNN parameters, which typically break such dominant symmetries. Moreover, GNNs are known for their inherent smoothing tendencies, which lead rank-reduction the feature matrix and result a non-trivial spectral gap in practice.
Nonetheless, extending to the case where the dominant singular value is of multiplicity r shall result in an additional scalar v(r) (rather than v(2) the derived upper- aoper-bound y and the considered spectral gap is or 0r+1-
Corollary 5.2 Let f: (G.2) be the graph-based function considered in Theorem 5.1 Then (rrouus with:
min (7, 27),
where y is the bound derived in Theorem 5.1
Proof. Let's consider the model f (9 - y that uses our proposed RS-Pool. From Theorem 5.1 we have seen that f is (e, 7) -robust with:
2c
(1w a
(w,))
01 02
Using the triangular inequality, we have the following:
|lo har
IRS-Pool(H) - RS RS-Pool(H')
[|T - -H')(H'
(H) - 21 -HH'llll
- [H])) + [H]
s 2r.
And therefore by combining the two provided bounds, we can write that f is (c, (')obbutt with:
? min moinnyy.20 (7, 2r).
D Proof of Lemma 5.3
Lemma 5.3, RS-Pool is permutation invariant; that is, for any pair of isomorphic graphs G and Gn with its corresponding permutation matrix II, we have [A, X) f(An, Xn))
Proof. Let X E IR be matrix, let's consider its SVD decomposition:
x= -
Let's X' be permuted version of the adjacency matrix X, and let P E Naxx be the corresponding permutation matrix. Since by definition p3 pTp = PPT = In. we can write the following:

---

From the previous equality, we observe that X and (PX) are identical. Consequently, they share the same largest eigenvalue and corresponding eigenspace. Thus, any unit-norm dominant eigenvector of PX must lie in the one-dimensional subspace spanned by un (X). By enforcing consistent sign convention during pooling (e.g., requiring the first non-zero entry to be positive), we obtain:
RS-Pool X') = (SS-Pool(PX) ru (PX) = (x) RS-Pool(X).
As a result, we obtain permutation invariance in our pooling operation when represents the output of the message-passingg stage.
E
Additional Results
E.1
Convergence Analysis
As outlined in Section our proposed RS-Pool method relies on estimating the dominant singular vector of input matrix using the iterative power method. This classical algorithm begins with a randomly initialized vector and performs a series of matrix-vector multiplications, progressively aligning the estimate with the dominant singular vector of the matrix. The number of iterations, denoted by K, directly influences the quality of this approximation and, by extension, the effectiveness of the pooling operation.
In this section, we empirically investigate how the choice of K impacts the accuracy of the estimated singular vector. To this end, compare the vector obtained after K iterations of the power method to the dominant singular vector computed via the more precise full SVD implemented in the gesdd routine of the LAPACK package. We use the é 22-norm to measure the distance between the estimated and true singular vectors. This allows us to quantify the convergence behavior of the power method and assess how quickly a sufficiently accurate approximation can be achieved.
Figure 2: Effect of the number of iterations on the convergence of the iterative power method. The convergence is computed through the lz distance between the estimated and true dominant singular vectors (obtained via full SVD).
Figure 2 summarizes the convergence behavior of our method across multiple datasets. We observe that in most cases, few as 2 to 5 iterations sufficient obtain reliable estimate of the dominant singular vector. This rapid convergence ensures that RS-Pool achieves stable performance in terms of both clean and adversarial accuracy, confirming its computational efficiency with small number of iterations K. Moreover, the results indicate that the spectral gap 01 (X) (x(X) is sufficiently large, which, according Theorem 5.1 guarantees robustness to perturbations within budget
E.2 Time Analysis
In this section, we analyze the computational cost of our proposed RS-Pool method, with a particular focus on its time complexity. Building on the convergence analysis presented carlier, we examine how the number of power iterations K affects training time, clean accuracy, and adversarial robustness.
Bable results on the PROTEINS dataset. We observe that increasing the number of iterations yields only marginal improvements in clean accuracy. Similarly, the attack success rate decreases by less than 2% when increasing K from to 10. In contrast, training time grows significantly due to the additional cost of repeated matrix-vector multiplications in the power iteration.
These results underscore the importance selecting an appropriate number of iterations balance model performance and computational efficiency. Taken together with the convergence properties
7

---

discussed earlier, those findings support that small number of iterations (e.g., K = 2 to 5) is sufficient achieve strong robustmess with minimal overhead.
Table Training time and Performance terms of both clean and attack success rate for different number of iterations for the PROTEINS and NCII dataset for 0.2.
DATASET
k=1
K
TRAINING TIME (IN S)
123.57
148.96
226.63
350.95
PROTEINS
CLEAN ACCURACY
73.36
73.95
73.51
73.06
ATTACK SUCCESS RATE
21.04
21.25
20.50
19.89
We now focus on comparing the training time of the proposed RS-Pool against the benchmark pooling methods that were considered both in our theoretical study and our experimental results in Section
Table 4: Mean training time (seconds) of the proposed RS-Pool in comparison to other pooling benchmarks on the different datasets.
Dataset Sum Avg
Max
SAG
TopK-P
Pan-P
Sort-P
RS-Pool
PROTEINS
56.95
56.50
60.78
81.27
71.32
68.98
71.59
148.96
D&D
1187.10
1176.93
1182.77
1206.33
1206.44
1210.92
1209.17
1293.93
NCII
131.19
132.88
138.15
203.11
180.05
170.05
186.97
249.88
ENZYMES
20.42
20.57
21.54
30.76
27.54
26.34
28.28
46,30
IMDB-B
29.05
29.42
30.88
46.50
41.43
39.04
43.32
75.61
ER MD
11.40
11.52
12.13
18.26
16.21
15.15
16.88
29.23
MSRC
6.71
7.06
9.93
8.39
9.31
18.43
E.3
Additional Results GCNN
Extending the analysis presented in Section 6 we continue our empirical evaluation of the upper bound y across the remaining pooling methods. Consistent with earlier findings, RS-Pool exhibits smaller distance between the pooled representations of clean and perturbed graphs compared to all other methods. This reduced discrepancy correlates with lower attack success rate, highlighting RS-Pool 's improved adversarial robustness.
Poot
Sort
PROTEINS
PROTBINS
DD
(Log
Attack Budg
Figure 3: Extension of Figure | Empirical estimation of adversarial risk and corresponding attack success rate on PROTEINS ((a), (b)) and D&D ((c), (d)).
E.4 Additional Results GIN
We further want evaluate our proposed RS-Pool when considering different backbone architecture,
specifically Graph Isomorphism Networks (GIN). In this setup, consider model based on 2 GIN layers the pessage-passing framework. Table repposs the mean and standard deviation of both clean and adversarial accuracy under PGD attacks with perturbation budget of = 0.3. Notably, we observe that the case of GIN, increasing the number power iteration steps used to approximate the dominant singular vector improves performance. As result, we increase the number of iterations from = (used with GCN) to 5. Overall, RS-Pool consistently outperforms the baseline pooling methods, demonstrating similar robustness improvements those observed

---

with GCN. These results support our claim that RS-Pool is model-agnostee and can be effectively integrated into different GNN architectures for graph classification.
Table 5: Clean and Attacked classification accuracy (+: standard deviation) of the RS-Pool and other pooling strategies applied on the GIN model when subject to the PGD adversarial attacks.
Sum
Average
Max
SAG
TopK-P
PAN-P
Sort-P
RS-Pool
Clean
75.6 3.1
69.3
1.5
75.8
4.1
70.5 4. 1.9
73.5 1: 4.3
73.2 4. 2.6
73.2 1 4.1
70.1 1.6
PROTEINS
Attacked
22.6
26.2 1022222. 2.9
41.3 = 1.9
32.1 + 5.9
26.5 + 5.9
21.1 + 3.6
35.8
43.1
Clean
73.5
72.9 1:11
73.5 3,4
71.8 4 0.9
+ 4.1
72.1
73.8
70.9
D&D
Attacked
12.9 3.7
28.9 5.6
5.6 2.9
24.1 + 4.9
23,9 4: 5.1
27.5 3.6
32.9 + 3.2
43.7 5.1
Cleun
33.9 4.9
27.8 1. 0.8
37.2 1. 4.3
27.8 1. 4.1
34.4 1. 0.8
30,0 J. 3.6
23.9 1. 2.8
27.8 1. 4.1
ENZYMES
Attacked
1.1 + 0.8
0.6 + 0.8
2.8 4 2.1
2,5 = 1.2
2.8 + 0.8
5.6 = 1.2
5.6 + 0.8
6.1 + 2.1
Clean
2.1
61.7 4.7
52.3 = 4.5
63.7 +: 4.1
61.0 + 5.7
60.3 + 4.7
66.0 + 3.8
63.3 4.7
IMDB-B
Atlacked
43.7 4.2
42.0 5.11
38.3 4.3
34.3 + 4.3
43.9
51.0
Cleun
68.2 0.6
72.7 2.8
71.2 1 1,9
69.3 1. 4.6
67.8 3.8
68.9 .. 2.8
61.8 1. 5.6
63.6 2.4
ER_MD
Attacked
34.1 3.2
35.6
34.1 + 1.4
36.7 0 3.2
36.7 +
35.6 1.4
40.8 +
60.1 28
Clean
93.9 2.1
89.4
9.
91.7 + 1.1
93.2 1.9
92.4 = 1.1
91
91.7
MSRC_9
Attacked
65.9 7.4
60.6 7.77
67.5 3.8
692
54.6 3.7
65.2 7.7
83.3
E.5
Additional Results Adversarial Defenses
Since RS-Pool is model-agnostic, we investigate its effectiveness when combined with existing adversarial defense strategies. We consider four representative defense categories. First, a post- processing defense via adversarial training, where adversarial examples are generated and used to retrain the model to improve robustness. We additionally consider Randomized smoothing, which consists of generating number of points within the input and then take the majority vote as a classification. We afterwards consider a set of architecture-levé defense using both R- R-GCN, which modifies the message- passiggg mechanism to improve stability under perturbations, and SoftMedian, which introduces a robust aggregation scheme that down-weights neighbors based on their distance to the dimension- wise median, mitigating the impact of outliers or malicious inputs.
Table providess the mean clean and attacked accuracy for the different considered pooling bench- marks when subject to the PDG adversarial attack. We observe similar insights in the case of the GCN and GIN, where our proposed RS-Pool out-perform all the other poolings in terms of attacked accuracy showcasing therefore its ability to work with different undelrying backbone GNNs.
E.6 Bit-Flip Attacks (BFAs)
Recently, Bit-Flip attacks have emerged as a new paradigm for attacking and consequently evaluating the robustness of deep learning models at the hardware and memory level. Unlike conventional adversarial attacks that perturb inputs or gradients, bit-flip attacks directly manipulate the binary representation of model's parameters in memory. By flipping only a few critical bits, an adversary can drastically alter the network's 's behavior, causing severe accuracy degradation classification. While such attacks have been extensively studied in the context of convolutional and feedforward architectures, their implications for Graph Neural Networks (GNNs) are only beginning to be understood. Recent work [26 25 have shown that even minimal bit-level corruption can disrupt the delicate balance of the message- message-passing propagation mechanism.
targeted mis-
In this perspective, and further demonstrate the robustness and validity of our proposed RS-Pool, we extend our evaluation to include this line of attack. Specifically, we adapt the Injectivity Bit Flip Attack (IBFA) by removing its quantization step. allowing it to operate directly on our floating-poin model. We employ the same dataset and follow an identical experimental setup, using the GIN architecture and setting the attack parameter = 5, consistent with the configuration used by the original authors (on the OGB-molHIV molecular dataset). The results of this analysis, summarized in Table indicate that indeed RS-Pool consistently outperforms all other pooling baselines, further validating its resilience to bit-level adversarial perturbations.

---

Table Clean and Attacked classification aceuracy (+ standard deviation) of the RS-Pool and other pooling strategies applied on different graph adversarial methods when subject to the PGD adversarial attacks.
Sum
Average
SAG
TopK-P
Sort.P
RS-Pool
Adv
Clean
72.4 4.0
70.2 1: 2.3
73.2 à 3.7
71.1 1. 3.7
73.7 à. 2.9
73.5 4. 2.2
73.8 2.3
72.1 4 3.1
Train.
Attacked
45.8 5.1
31.3 + 2.2
18.5 + 3.7
49.1 3.6
46.1 = 4.8
32.7 + 3.1
51.3 + 3,5
54.8 = 4.1
Clean
72.6 4.7
71.2
72.0 73.2
73.0
73.7
PROTEINS
SofiModian
Attacked
53.9
42.8 1:
41.5 5.1
36.3
52.4 + 1.2
41.7 4: 2.1
58.3 1.8
63.2 288
Clean
2.3
61.0 + 1.6
57.6 + 3.4
66.7 4:7
70.2 344
70.5 + 1.9
70.8 0.4
71.7
RGCN
Attacked
58.6 6.84
3988 2:
59.2
61.3
63.6 2: 2.3
67.3
Randomizzd
Clean
3.6
71.7 3.
72.6 + 2.2
70.8 2.3
74.1 = 3.6
75.6 3.1
74.4 +44 4.1
73.5
Smoothing
Attacked
56.2 1.5
49.7 1. 3.7
36.3 1: 3.6
57.2 4.4
56.1 1. 3.2
53.6 1. 0.7
52,9 4.3
59.3 11
Clean
3.
70.4 4.3
69.3 3.6
74.9 d:
2.5
74.7
Train.
Attacked
7.3 + 2.7
18.6 + 6.3
6.7 67-4-22
19.6 5.1
8. 1.7
7.1 + 2.6
7.3 + 1,5
31.9 4.9
Clean
74.1
68.9 +
69.0 5.1
68.4 5.1
75.7 =
77.3 + 1.6
74.9 2.9
72.9 = 3.4
SofiMedian
D&D
Attacked
51.2 1..9
8,5 3.8
46.1 5.6
6.4
10.4
Clean
74.9 1.3
63.8 1. 2.4
66.4 1. 5.1
70.6 1. 3.4
75.4 2.1
76.8 1. 1.8
73.1 2.4
71.2 1. 2.8
RGCN
Attacked
3.1
13.4 2.9
38.7 +
58.7 + 3.4
53.7
Randomized
Clean
72.1 4.39
42.5 + 2.7
69.3
74.8 1.5
76.4 1.5
76.1 1.9
72.1 +
Smoothing
Attacked
3.69
43.1 +: 3.5
22,8 + 4,5
34.1 4.1
45.1 = 4.8
41.1 + 3.5
42.2 2: 1.7
49.7 = 4.6
Clean
6..7
63.3 4.6
+: 0.9
65.5
Train.
Attacked
4.16
45.7 2.9
44.2 2.6
48.6 1.4
42.3
35.4 1. 4.2
34,9 3.3
53.9
Clean
70.0
62.9 + 6.9
62.5 + 4,5
62.5 +: 4.0
65.1 # 4.8
66.7 + 6.8
65.5 + 6.1
62.5 = 1.9
LMD
SofiMedian
Attacked
3..3
34.5
Clean
5..0
60.1 #
58A 2
5.7
63.6 =
68.5 + 6.4
66.7 3.7
59.9 3.7
RGCN
Attacked
39.2 1.2
47.5 2.1
44.5 + 3.9
49.3 5.1
28.0 + 3.8
38.6 + 1.4
46.8 1.4
51.7 0.9
Randemiazd
Clian
69.2 1.0
6..4
61.4
62.9 6..0
67.8 2.3
61.7 5.44
Smoothing
Attacked
40.4
40.4 6.4
4644
48.3 +
40.1 4.7
4.4
43.8
54.7 4.4
Table 7: Attacked classification accuracy (#: standard deviation) of the RS-Pool and other pooling strategies on different graph classification dataset when subject to Bit-Flip Attacks.
Dataset
Sum
Average
Max
SAG
TopK-P
PAN-P
Sori-P
RS-Pool
NCI1
64.7
62.2 1.9
65.9 1.3
65.2
61.2 1.7
59.2 1.7
63.5 1 1.7
66.0 i: 1.6
DD
59.8 4.1
61.8 01.8227 2.7
63.6 26
63.1
2.14
56.2 1.5
56,8 2.5
68.3 + 1.0
MD
2.5
52.3 + 1.8
51. 18
52.4
5..2 5279227
51.9
55.5 + 2.5
PROTEINS
57.0 5.8
64.2 1 1.5
64.8 2.7
63.7 0.9
61.2
60.0 + 4.9
66.7 4.7
69.6 + 1.1
E.7 On the Tightness of Upper-Bounds
In this section, we assess the tightness of the theoretical upper bound 7 established in Theorem 4.2 To this end, we compare the theoretical bound with empirical estimates computed as the norm difference between the pooled representations of clean and perturbed graphs for different values of The results are shown in Figure 4 Across all pooling methods, we observe a clear correlation between the empirical measurements and the theoretical bound, providing empirical support for the validity of our theoretical analysis.
F
Experimental Details
F1
Datasets
For our experiments, we use standard graph classification datasets from the TUDataset bench- mark selecting diverse subset to represent different domains. Specifically, we include datasets from bioinformatics and chemoinformatics (PROTEINS, ENZYMES, D&D), small molecule predic- tion (NCT1, ER MD), and image-based graphs (MSRC_9) 9). Model evaluation follows the standardized protocol of [1] using 10-fold cross-validation When public folds available, we them directly;
10

---

PROTEINS: Empirical/Thorretical Bounds
Diffennce in Pooling (L.gg Scale)
Surn Poaling
Avg Pooling
Max Pooling
18.
1.9
16"1
1.9
10
12-1
Emprical Theant
10
Grepkical Theaeer
Theendooll
20.2
0.2
6.2
03
0.4
Allack Budgel
Allack Budget
Atlack Budgel
ENZYMES: Empirrica/TThooreiicc Bounds
Diffrrncce in Pooling (Log Scalo)
Sum Pooling
Pooling
Max Pooling
1.4
1.2
30"
16"1
10
10"
12-1
Attack Budget
Attack Budget
Attack Budget
Figure 4: Analysis of the Upper-Bounds provided in Theorem 4.2 and their empirical counter-part for different values of the PROTEINS (Upper) and ENZYMES (Lower) Dataset.
otherwise, generate new folds following the same procedure. Dataset statistics and characteristics are summarized in Table
Table 8: Statistics of the graph classification datasets used in our experiments.
DATASET
#GRAPHS
#NoDES
#EDGES
#CLASSES
PROTEINS
1113
39.06
72.82
1178
284.32
715.66
NCII
4110
29.87
32.30
ENZYMES
600
32.63
62.14
IMDB-B
1000
19.77
96.53
REDDIT-B
2000
429.63
497.75
ER MD
ER_MD
446
21.33
234.85
MSRC_9
221
40.58
97.94
F.2 Experimental Setup
In all of the experiments, we use a 2-Layers GCN model. Specifically, the model uses a 2-layer convolutional architecture (consisting of two iterations of message passing and updating) stacked with a Multi-Layer Perception (MLP) as a readout. The only change the architecture was on the compare the different benchmarks in iso-architectural setting,
pooling level since the intent to ensure a fair evaluation of their robustness.
All models are trained using the Adam optimizer 23 with learning rate of 1 x 10-3 for 100 epochs. We set the hidden feature dimension to 32 and we used ReLU as our activation function for all the models. For all the experiments, of training epochs to ensure fairness following insights from previous [2] Additionally, to account for variability due random initialization, each experiment is repeated 10 times, and report the mean and standard deviation of the results. The experiments have been run on NVIDIA A40 GPU and we estimate the total number of hours of computing to be around 200 hours.
used the same initialization distribution and the same number
For our proposed RS-Pool, we set the number of iterations in the estimation algorithm to in the case of GCN while set it to K 5 in the case of GIN. In addition, the temperature parameter was tuned separately for each dataset to optimize the trade-off between clean accuracy and adversarial robustness, as detailed in Section (Figures (e))

---

F33 Implementation Details
Our implementation is provided in the supplementary materials and will be made publicly available after the review period. The code is developed using PyTorch [33], with a dense implementation of GNNs, which is required for executing the considered adversarial attacks. For all baseline pooling methods, we adapt the official implementations from PyTorch Geometric (PyG) [5] released under the MIT license.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims presented in the abstract and introduction are supported by the theoretical results in Section and further validated through empirical evidence in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the main limitations of our method in Section 7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
All theoretical claims are supported by detailed proofs in Appendices A B
and D Additionally, the problem setup and underlying assumptions are clearly defined in Section and are explicitly referenced in each theorem and proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: In addition to presenting detailed experimental setup in Appendix F we include the source code for reproducing our results the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
14

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: All experiments were conducted using publicly available datasets. When available, we used the available public train/val/test folds, and we provide all of those. To ensure reprodueibility, we provide the source code in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/
www.c//wweeeeeeee Sawelee//ooSSSbbbiicaa SwwprddeuumisssiooPeP. wwwiiisssoiiiiiiiiooi for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Appendix CeaddixFFpuu Prpovvddss details of the experimental setup, including details about the train/val/test used folds and the values of all hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the mean and the standard deviation of all results in the respective tables, based on repeated experiments using 10 different random seeds. Further details are provided in Appendix F
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We provide information about compute resources for our experiments in
Appendix [F]
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://////eerppp..cpccpptttttt/
Answer: [Yes]
Justification: Our work adheres to the NeurIPS Code of Ethics. We use public datasets and publicly available models and report on the limitations our work.
Guidelines:
The answer NA that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification: Our work has no further societal impacts apart from known impacts of graph- based models. We are not aware of any applications of our approach that would result in negative societal impacts.
16

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not release any data models that have high risk for misuse. Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
All datasets, models and repositories were cited appropriately.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets, only rely on public datasets and models.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codd/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Justification: Our work did not involve crowdsourcing nor research with human subjects. All experiments performed publicly available datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects, all experiments are performed on publicly available datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
No LLM was used this work for the core methods.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
19


Reviews:
[{"review_id": "5oXz69VOvJ", "reviewer": "Reviewer_kMFs", "summary": "This paper tackles the adversarial vulnerability of GNNs in graph classification by focusing on the pooling stage. After theoretically analyzing the weaknesses of standard pooling methods like sum, average, and max, the authors introduce Robust Singular Pooling (RS-Pool). This novel, model-agnostic method constructs a graph representation from the dominant singular vector of the node embeddings, which is inherently more stable against adversarial perturbations. Implemented efficiently via power iteration, RS-Pool is empirically shown to significantly enhance robustness against attacks while maintaining competitive accuracy on clean data.", "questions": "- What is the semantic meaning of the dominant singular vector? The paper uses the dominant singular vector as a robust feature summary. However, it doesn't investigate what this vector represents in terms of the graph's structure or semantics. Does it capture information about community structure, hub nodes, or specific functional motifs? Answering this could lead to more interpretable GNNs.\n\n- What information is lost by discarding other singular vectors? RS-Pool exclusively uses the top singular vector to achieve robustness. A key unaddressed question is what valuable, albeit less dominant, information is being discarded. These secondary components might be crucial for fine-grained classification tasks where subtle features matter more than overall structural stability.\n\n- How does network depth affect the spectral gap? The method's effectiveness hinges on a significant spectral gap in the node embedding matrix. The paper doesn't explore how the number of GNN layers impacts this gap. Deeper GNNs often lead to \"over-smoothing,\" where node embeddings become more similar, potentially shrinking the spectral gap and reducing RS-Pool's effectiveness.", "limitations": "- Dependence on Spectral Gap: The effectiveness of RS-Pool, both theoretically and in practice, relies on a sufficiently large spectral gap between the first and second singular values of the node embedding matrix. The paper acknowledges that a small gap, which can occur in some graph topologies, weakens the robustness guarantee and slows the convergence of the power iteration.\n\n- Computational Overhead: Despite being more efficient than a full SVD, RS-Pool incurs more computational overhead than standard pooling methods. This is especially true on very dense graphs, which could limit its use in resource-constrained applications.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n- Novel Focus and Theoretical Grounding: The paper provides a novel contribution by focusing on the often-overlooked pooling stage of GNNs in the context of adversarial robustness. Its arguments are supported by a strong theoretical framework that derives robustness bounds for common pooling methods before proposing a solution.\n\n- Well-Motivated and Efficient Solution: The proposed RS-Pool method is elegantly based on established matrix perturbation theory, which shows that dominant singular vectors are stable under perturbations. It is also computationally efficient, using the power iteration method to avoid a costly full SVD, making it practical for real-world applications.\n\n- Model-Agnostic and Versatile: RS-Pool is designed to be model-agnostic, and the paper demonstrates its effectiveness with different GNN backbones (GCN and GIN) and even when combined with other existing adversarial defense methods.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful comments and constructive feedback. We especially appreciate their recognition of the novelty and value of our proposed theoretical study and the proposed pooling method. Please find our responses to the raised concerns below:\n\n**Q1 - On the semantic meaning of the dominant singular vector:**\n\nWe thank the reviewer for raising this very interesting question. We first want to recall that our RS-Pool method relies on the dominant singular vector of the node embedding matrix that is produced by several iteratively applied GNN message passing layers. In general, in a GNN, we learn a function that maps from the graph space (often represented via the adjacency matrix) and node feature space (node feature matrix) to the label space as accurately as possible. Consequently, semantic properties of the considered dominant singular vector are highly dependent on the label structure that we attempt to map to and we cannot provide any general semantic interpretation for this vector without regard for the learning task and dataset. \n\nAs you rightly point out we have nice semantic interpretations for the singular vectors obtained from the node feature and graph structural space. For the node feature matrix the singular vectors can be interpreted via the long-standing theory on principal components to provide the directions that capture maximal variance of the node features. And like you mention there is a rich theory on the spectral properties of adjacency and Laplacian matrices. These singular vectors (equal to the eigenvectors for symmetric matrices) have indeed been shown to encode community structure [1] and in some instances hubs [2]. Now, our pooling strategy also works with the principal singular vector that captures the direction in which our learned embeddings vary maximally. However, what kind of semantic information is placed in this vector by the trainable GNN layers is entirely dependent on the labels that the GNN is trained to predict. This flexibility to some degree marks the strength of GNNs, as is visible by the strong empirical performance of RS-Pool that we observe in Table 1. \n\nIn general, we agree that the semantic analysis of the dominant singular vector in the context of specific learning tasks and dataset is of interest and is a promising direction for future research on the interpretability of GNNs. We will make sure to mention this direction in the revised manuscript.\n\n\n[1] A tutorial on spectral clustering. U. von Luxburg. Statistics and Computing. 2007.\n\n[2] Authoritative Sources in a Hyperlinked Environment. J. Kleinberg. Journal of the ACM (JACM). 1999\n\n---\n\n**Q2 - What kind of information is lost when considering only the dominant singular vector:** \n\nWhile it is correct that focusing on the dominant singular vector discards certain information, particularly high-frequency, node-local variations, we emphasize that in practice, this loss has limited impact on standard graph classification tasks. Empirically, as shown in Table 1 (GCN) and Table 2 (GIN), RS-Pool achieves clean accuracy comparable to that of sum pooling, which retains the full spectrum of signals. This suggests that the fine-grained information encoded in the subdominant singular vectors is rarely essential for achieving high classification performance on clean (non-attacked) graphs.\n\nOn the other hand, our motivation centers on robustness. Recent findings [3] indicate that adversarial perturbations tend to manifest in high-frequency components, precisely the directions discarded by RS-Pool. Thus, although the loss of high-frequency information is existent, it is their removal that enables RS-Pool to enhance adversarial robustness. In this sense, our method intentionally trades off a marginal amount of discriminative capacity (which is not always needed for graph classification tasks) for a significant gain in robustness. We thank the reviewer for raising this question, and we shall clarify this trade-off in the manuscript. \n\n[3] A Frequency Perspective of Adversarial Robustness. Maiya et al. arXiv. 2021.\n\n---\n\n**Q3 - On the effect of depth and over-smoothing:**\nWe appreciate the reviewer’s suggestion to investigate the role of depth in our framework. Traditionally, the adversarial robustness literature does not emphasize depth or other model’s hyperparameters, as models under attack are typically assumed to be well-trained and performant.\n\nNonetheless, we agree that in the context of GNNs, over-smoothing is an important subject to take into consideration.  However, we would like to clarify that over-smoothing is primarily a concern in node classification tasks, where discriminative power between node embeddings is critical. In contrast, in graph classification, the focus of our work, the effect of over-smoothing is less pronounced and can even be regarded as positive [4] since the collapse of the node embeddings is only problematic if the collapsed state is not label-informative.\n\nIn the regime of oversmoothing, node embeddings across the graph tend to become similar, effectively collapsing the rank of the node representation matrix $H$ to 1. This leads to the information being concentrated within the dominant singular vector and consequently $\\sigma_1$ becomes dominant and subsequent singular values tend to vanish. Therefore, instead of shrinking the spectral gap $\\sigma_1 - \\sigma_2$ will increase, resulting in better robustness guarantee of RS‑Pool as derived in Theorem 5.1, where the bound is inversely proportional to this gap. \n\nTo further illustrate our discussion, we conducted an ablation by increasing the depth of the GCN encoder and evaluated the impact on accuracy for both RS‑Pool and Sum Pooling. We pushed the number of layers until 10, which is the average graph diameter in the considered TUDataset, where at this point, most nodes have already aggregated information from the entire graph. Table 1 provides the mean and standard deviation of the accuracy, where we see that depth has not much effect on the resulting clean accuracy. We will update the manuscript accordingly.\n\n\nTable 1: Performance of a GCN on different graph classification dataset for different number of layers (depth) for RS-Pool and Sum Pooling.\n\n| Depth | Pooling | PROTEINS   | NCI1       | MSRC_9     |\n|-------|---------|------------|------------|------------|\n| 2     | Sum     | 74.2 ± 3.1 | 70.6 ± 0.8 | 88.6 ± 3.2 |\n| 2     | RS-Pool | 73.5 ± 2.9 | 70.1 ± 1.2 | 89.7 ± 1.1 |\n| 4     | Sum     | 73.2 ± 3.2 | 72.6 ± 2.0 | 89.4 ± 1.3 |\n| 4     | RS-Pool | 72.6 ± 1.7 | 72.2 ± 1.5 | 90.6 ± 0.9 |\n| 6     | Sum     | 73.3 ± 2.1 | 72.8 ± 2.2 | 91.4 ± 0.8 |\n| 6     | RS-Pool | 72.3 ± 2.6 | 72.6 ± 1.1 | 91.7 ± 1.1 |\n| 8     | Sum     | 73.8 ± 4.2 | 72.9 ± 2.6 | 87.1 ± 2.1 |\n| 8     | RS-Pool | 72.6 ± 2.2 | 72.8 ± 0.8 | 87.9 ± 2.8 |\n| 10    | Sum     | 73.4 ± 3.4 | 72.9 ± 1.7 | 89.4 ± 5.7 |\n| 10    | RS-Pool | 72.4 ± 2.6 | 72.8 ± 0.8 | 90.9 ± 3.2 |\n\n---\n[4] Understanding Virtual Nodes: Oversquashing and Node Heterogeneity. Southern et al.  ICLR. 2025."}, {"review_id": "DyOCOYSSRl", "reviewer": "Reviewer_RqyR", "summary": "The focus of the paper is The paper first defines average-case robustness as the l2 distance in the output space between the clean and perturbed graphs averaged over all perturbation within a given distance. Then they derive the effect of different pooling operators on this quantity. Next, the authors propose a new pooling method RS-Pool that projects the final node embeddings onto the first right singular vector of the embedding matrix. They also derive an upper bound for the \"adversarial\" risk for this pooling and argue that it is more favorable.", "questions": "1. Is the PGD attack adaptive? That is, do you directly attack RS-pool, differentiating through proposed pooling?\n2. Can you show empirical results against feature-based attacks that match the setup for the theory?\n3. Provide a detailed description of how the derived bounds differ from existing (not necessarily adversarial) bounds (see weakness)?\n4. Can you provide additional results for an even larger value of $\\alpha$. Can you show a similar plot for other datasets?", "limitations": "The limitations section is there but could be expanded, e.g. about worst-case vs. average case.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The proposed approach is scalable, easy to implement, and well motivated. The parameter $\\tau$ allows for some control over the trade-off between clean accuracy and robustness. The focus on the pooling aspect is very interesting and under studied in the literature. The theoretical results seem correct but I did not check the proofs in detail.\n\nOne of the my main problems with the paper is the mismatch between the actual and the claimed results. This shows up in several different places:\n- As the authors already mention they analyse the average case and not the worst-case. Calling this **adversarial** robustness is highly misleading and not in line with the existing terminology in the literature. To be clear, I think the average case results are interesting and useful, but we should not be calling this adversarial. Relatedly, the statement \"(ϵ, γ)-robust in the average case sense, it is also (ϵ, γ)-robust under the worst case criterion\" is not quite accurate. The worst-case criterion would presumably have a $\\max_{\\mathcal{N}_\\epsilon}[\\cdot]$  instead of $\\mathbb{E}[\\cdot]$ in Eq. 1, just because we have a low value on average it doesn't mean we have a low value in the worst-case.\n- When they set up the threat model the authors define perturbations w.r.t. both $A$ and $X$, however, the theorems say \"Under a feature-based adversarial attack\".  Even more confusingly, in the experiments the author state \"For all attacks, we apply a perturbation budget of ϵ = 0.3, allowing up to 30% of the edges in each graph to be modified.\". However, as far as I understood, Theorem 5.1 and Theorem 4.2 only apply to feature-based permutations. This discrepancy is not ideal (see question 2). Moreover, the threat model looks at the L2 norm of the adjacency matrix which is different from the number of perturbed edges (L0 norm).\n\nThe experimental evaluation is limited. For example Table 1 shows results for a single attack budget $\\epsilon$ which is relatively large. While they consider different $\\epsilon$ in Figure 1 this should be extended to the rest of the \"main\" experiments.\n\nSome relevant works are not (properly) discussed. For example, while [1, 2]  are not exactly addressing the same problem, they are relevant enough to warrant a short discussion. Moreover, standard (non-adversarial) generalisation bounds for GNNs often have terms that depend on the product of (spectral) norms of the weights, similar to the expression derived in this paper. Showing how exactly robustness changes the bounds would be valuable. See e.g. [3] and related references.\n\nYou should label the y-axis on Figure 1 for improved clarity. Assuming that you are showing the \"adversarial\" risk $\\gamma$, how do you actually compute it? The expectation defining in Eq. 1 is untractable to compute exactly as far as I know. Section 6.1. states \"compute the distance be355 tween the pooled representations of clean and adversarially perturbed graphs\" but how are these adversarially perturbed graphs obtained?\n\nDepending on $n$ and $d$ it might be a bit cheaper to compute $H^T (H v)$ rather than precomputing $S$ since matrix-vector products are cheaper than matrix-matrix products, especially for low values of $K$. This \"trick\" is also used in the standard implementation of spectral normalisation (available in PyTorch) which also computes the right singular vector as a side-product.\n\nReferences:\n1. Cao et al., \"Adversarial Robust Generalization of Graph Neural Networks\"\n2. Sun and Lin, \"PAC-Bayesian Adversarially Robust Generalization Bounds for Graph Neural Network\" \n3. Liao et al., \":A PAC-bayesian approach to generalization bounds for graph neural networks\"", "was_revised": true, "final_justification": "The authors addressed all of my concerns, hence I'm increasing my score.", "rebuttal": "We thank the reviewer for their constructive feedback. \n\n**W1 - On the “average” versus “worst-case” robustness:** Indeed, as pointed out by the reviewer, we rather approach the subject from what we define as the “expected adversarial risk”, which differs from the more commonly used adversarial robustness definition. In this perspective, rather than considering only the worst-case adversarial examples, we consider the whole input’s neighborhood.  We believe that our quantity is of complementary value to the typically considered “worst-case” robustness, since in certain settings it is preferable to be robust to “average” attacks rather than to the “worst-case” scenarios [1]. To clarify this ambiguity we will replace the term “adversarial robustness” by “expected adversarial robustness” throughout our revised manuscript.  We further note that in our proof, we place no assumption on the sampling distribution of the perturbed graphs within the considered neighborhood defined by the budget $\\epsilon$. Hence, the upper-bound $\\gamma$ applies to any point within the considered neighborhood $\\mathcal{N}\\_{\\epsilon}$ (refer to the equation following Line 553 in Appendix A). Therefore, with a suitable choice of the sampling distribution of the perturbed graphs, the derived bounds also applies to the worst case (as defined by different attack schemes) since by definition the set of worst-case perturbations are also part of the neighborhood ($\\mathcal{S}\\_{\\epsilon}(A, X) \\subseteq \\mathcal{N}_{\\epsilon}(A, X)$). Hence, while we have focused on the expected adversarial case, our proofs and the derived bounds can be easily adapted to take into account the worst-case adversarial robustness by adapting the definition of the expected adversarial risk. We will make sure to revise the manuscript accordingly to make clear such ambiguity and clarify that the provided bound can also be adapted to the worst-case (by adapting the definition in Eq. 1). We thank the reviewer for pointing this out. \n\n\n[1] Robustness between the worst and average case. Rice, Bair, Zhang & Kolter. - NeurIPS 2021\n\n**Q2 & W2 - Regarding feature-based attacks:**\n\nOur empirical results, focused on structural perturbations, as structural attacks and defenses are well-implemented, documented and more commonly studied. Nonetheless, we agree with the reviewer that extending our experiments to feature based adversarial attacks allows us to show a direct connection between our theoretical and practical results, and we thank the reviewer for this suggestion.\n\nWe ran the requested experiments by considering two main attacks: The first is random attack (which consists of adding random sampled noise in the node features), and the second is the PGD attack (which uses the gradients). Table 1 provides the mean accuracy (and the standard deviation) of the different considered pooling methods. These new results show that our proposed RS-Pool also outperforms all the other pooling strategies for feature based attacks. We will revise the manuscript accordingly and add these results. \n\nTable 1: Results on feature-based attacks.\n\n|  | Attack |     Sum    |   Average  |     Max    |     SAG    |   TopK-P   |    PAN-P   |   Sort-P   |     RS-Pool    |\n|----------|--------|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:--------------:|\n| PROTEINS | PGD    | 68.4 ± 4.0 | 64.6 ± 2.2 | 66.4 ± 4.1 | 64.3 ± 3.6 | 64.9 ± 6.9 | 68.8 ± 2.9 | 66.7 ± 2.9 | **69.9 ± 4.1** |\n| PROTEINS | Random | 69.5 ± 5.2 | 67.6 ± 2.9 | 68.4 ± 3.2 | 69.0 ± 4.0 | 68.8 ± 6.5 | 69.2 ± 3.2 | 70.2 ± 2.6 | **70.9 ± 2.8** |\n| D&D      | PGD    | 36.9 ± 3.6 | 36.9 ± 6.8 | 46.4 ± 6.4 | 49.9 ± 1.8 | 28.7 ± 3.7 | 37.5 ± 4.1 | 47.9 ± 4.8 | **51.8 ± 1.1** |\n| D&D      | Random | 49.0 ± 5.2 | 44.0 ± 6.5 | 45.7 ± 4.6 | 54.1 ± 3.0 | 40.9 ± 5.1 | 51.0 ± 4.6 | 49.7 ± 4.9 | **54.9 ± 2.3** |\n| NCI1     | PGD    | 38.6 ± 2.8 | 36.4 ± 3.5 | 37.9 ± 5.3 | 36.6 ± 2.2 | 35.1 ± 2.2 | 38.4 ± 1.8 | 36.2 ± 3.6 | **39.4 ± 1.9** |\n| NCI1     | Random | 41.1 ± 2.0 | 38.1 ± 2.9 | 40.9 ± 5.5 | 40.1 ± 2.9 | 38.5 ± 1.0 | 40.9 ± 2.4 | 38.5 ± 4.8 | **41.7 ± 1.3** |\n| ER_MD    | PGD    | 41.9 ± 3.2 | 45.3 ± 1.4 | 45.3 ± 4.6 | 45.7 ± 3.5 | 45.3 ± 3.8 | 45.3 ± 2.1 | 43.1 ± 3.5 | **55.1 ± 4.9** |\n| ER_MD    | Random | 54.3 ± 2.3 | 50.2 ± 4.3 | 52.1 ± 6.4 | 53.9 ± 5.5 | 52.1 ± 4.1 | 51.3 ± 5.2 | 53.6 ± 7.7 | **56.2 ± 5.1** |\n| MSRC_9   | PGD    | 84.8 ± 2.8 | 85.6 ± 4.0 | 85.6 ± 2.8 | 87.1 ± 1.1 | 83.3 ± 1.1 | 85.6 ± 3.9 | 84.1 ± 1.9 | **88.3 ± 1.1** |\n| MSRC_9   | Random | 86.4 ± 3.7 | 88.6 ± 1.9 | 87.9 ± 2.8 | 89.4 ± 1.1 | 87.1 ± 2.1 | 87.9 ± 3.9 | 88.6 ± 3.2 | **90.2 ± 1.1** |\n\n**Q1 - Is the PGD attack adaptive ?**\n\nAs pointed out by the reviewer, for each pooling method (including RS-Pool), we differentiate through the pooling, making the attack adaptive. We will add an algorithm describing the attack framework in the appendix of our revised manuscript (note that our implementation is available in the supplementary materials). \n\n**Q3 & W4-Linking to the existing bounds:**\n\nWe thank the reviewer for the different provided references, we agree that citing them and discussing them in detail is important and we will edit the manuscript accordingly. We note that the (adversarial) generalization bounds provide understanding and quantify the difference in terms of the loss from the training to unseen (either perturbed or non-perturbed) test graphs. In our work, we rather consider the specific perturbation effect of the pooling layer choices at inference time. We hence consider a set of flat pooling methods to theoretically study their effect and accordingly we propose a new pooling RS-Pool, which we show theoretically and empirically to have some benefits in enhancing the overall robustness of the model. We believe that studying the generalization bounds of the pooling layer (also our RS-Pool) can be a great way to combine both directions in future work. \n\n\n**W3 - On the considered attack budgets:**\n\nWe agree with the reviewer that extending to other budgets is indeed very useful to further showcase the value of our proposed method. We therefore have extended the results to 10% and 20% attack budgets in the case of the PGD attack (the most performant attack in Table 1 of our paper). We will additionally extend this study to the other considered attacks and models and update the appendix accordingly. Table 2 reports the mean and standard deviation of the resulting attacked classification accuracy. We see that our RS-Pool is still out-performing the other pooling methods in smaller budgets. We furthermore want to point out that for the binary adjacency matrices we consider, the $L_0$ and $L_2$ norms are equivalent and hence, there is no discrepancy between the norm considered by the threat model and the number of perturbed edges.\n\nTable 2: Results for different attack budgets.\n\n|  | Budget |     Sum    |   Average  |     Max    |     SAG    |   TopK-P   |    PAN-P   |   Sort-P   |   RS-Pool  |\n|----------|:------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|:----------:|\n| PROTEINS |   0.1  | 63.6 ± 3.0 | 53.6 ± 1.9 | 48.2 ± 2.9 | 60.1 ± 1.5 | 60.7 ± 2.7 | 62.2 ± 2.3 | 62.5 ± 1.5 | 65.1 ± 2.3 |\n| PROTEINS |   0.2  | 55.1 ± 2.6 | 40.2 ± 1.9 | 27.1 ± 0.8 | 55.7 ± 1.8 | 50.3 ± 2.8 | 45.2 ± 1.1 | 56.2 ± 2.9 | 56.9 ± 3.6 |\n| D&D      |   0.1  | 21.7 ± 1.0 | 45.4 ± 4.0 |  6.8 ± 3.0 | 44.2 ± 5.4 | 22.2 ± 4.2 | 23.9 ± 0.9 | 28.2 ± 4.1 | 51.3 ± 3.1 |\n| D&D      |   0.2  | 11.5 ± 3.2 | 32.3 ± 3.4 |  6.8 ± 1.8 | 26.2 ± 5.6 | 11.3 ± 2.7 | 10.1 ± 4.3 | 15.2 ± 5.0 | 38.6 ± 3.3 |\n| ER_MD    |   0.1  | 49.1 ± 4.3 | 53.9 ± 2.3 | 44.2 ± 2.6 | 53.2 ± 2.9 | 48.7 ± 2.5 | 47.9 ± 3.2 | 42.3 ± 2.6 | 58.4 ± 2.7 |\n| ER_MD    |   0.2  | 40.4 ± 4.1 | 47.2 ± 2.2 | 43.4 ± 2.9 | 49.1 ± 1.1 | 43.1 ± 2.8 | 36.0 ± 2.5 | 36.7 ± 2.3 | 54.3 ± 3.9 |\n| MSRC_9   |   0.1  | 84.1 ± 1.9 | 87.9 ± 2.8 | 84.8 ± 1.1 | 86.4 ± 1.9 | 84.1 ± 1.9 | 85.6 ± 2.8 | 85.6 ± 2.8 | 88.6 ± 1.9 |\n| MSRC_9   |   0.2  | 82.6 ± 2.8 | 84.8 ± 3.4 | 83.3 ± 4.3 | 85.6 ± 2.8 | 79.5 ± 2.1 | 85.6 ± 2.4 | 75.8 ± 3.5 | 86.4 ± 3.2 |\n| ENZYMES  |   0.1  | 10.6 ± 2.8 | 10.0 ± 1.4 |  9.4 ± 2.8 | 11.1 ± 3.4 |  6.1 ± 0.8 |  7.8 ± 1.6 | 12.2 ± 1.6 | 14.4 ± 2.8 |\n| ENZYMES  |   0.2  |  5.0 ± 2.7 |  3.9 ± 3.4 |  5.0 ± 2.7 |  8.9 ± 3.9 |  1.7 ± 1.4 |  3.9 ± 2.1 |  7.2 ± 1.6 | 12.8 ± 3.5 |\n\n\n**Q4 - On the effect of $\\alpha$ and extending the study to other datasets:**\n\nWe note that $\\alpha$ is a hyper-parameter controlling the trade-off between clean and attacked accuracy (based on Corollary 5.2, recall that $\\tau = \\sigma_1(X)/\\alpha$). Typically, depending on the use-case and the user’s need to sacrifice clean accuracy for better robustness, this parameter should be tuned. While we have focused on the PROTEINS dataset in Figure 1 (e), the insights translate to other datasets. In response to your question, we considered 3 additional datasets and higher values of $\\alpha$. We will add the required figures in the appendix (which will be based on the table below).\n\n| | $\\alpha$ |   1  |  50  |  100 |  200 |  500 | 1000 | 2000 |\n|:-------:|:------------------:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n| D&D     | Clean Accuracy     | 74.6 | 75.1 | 74.8 | 74.7 | 74.7 | 74.3 | 73.6 |\n| D&D     | Success Rate       | 66.1 | 51.3 | 50.8 | 50.4 | 45.6 | 40.5 | 28.4 |\n| NCI1    | Clean Accuracy     | 70.1 | 67.9 | 65.4 | 65.1 | 64.6 | 62.1 | 61.8 |\n| NCI1    | Success Rate       | 44.9 | 43.4 | 41.2 | 40.5 | 38.5 | 37.5 | 37.6 |\n| ER_MD   | Clean Accuracy     | 65.9 | 66.2 | 64.0 | 63.8 | 63.4 | 58.8 | 56.9 |\n| ER_MD   | Success Rate       | 16.4 | 16.3 | 15.1 | 14.8 | 14.6 | 5.9  | 4.1  |\n\n\n**On the trick:** We want to warmly thank the reviewer for the proposed “trick”, indeed such an adaptation can reduce the training time of our method, we will edit the code accordingly and provide more details on this implementation in the revised manuscript."}, {"review_id": "JvveVHkNO1", "reviewer": "Reviewer_FeJc", "summary": "This paper investigates the adversarial robustness of Graph Neural Networks (GNNs) in graph classification tasks, focusing on the role of pooling operations. The authors present a theoretical analysis of standard flat pooling methods (sum, average, max). Motivated by these insights, they propose Robust Singular Pooling (RS-Pool), a novel pooling strategy that leverages the dominant singular vector of node embeddings to construct robust graph-level representations. Theoretical analysis shows that RS-Pool improves robustness by filtering noise-sensitive components, and empirical results on real-world benchmarks demonstrate its effectiveness against state-of-the-art adversarial attacks while maintaining competitive clean accuracy.", "questions": "1.While $d((A,X),(A’,X’))<\\epsilon$, the proof of Theorem 4.2 (line 553) gives that $ X-X’\\leq\\epsilon$. Is it because the graph topology remains unchanged? It is unclear about your settings and proofs, such as the punctuations (e.g., 632), and the representations of the matrix norm and vector norm.\n\n2.In Theorem 5.1, It seems that the values of $\\[\\tau\\]$ and $\\[\\sigma_1 - \\sigma_2\\]$ are both  small. And this bound still depends on the perturbation budget $\\epsilon$. So, from a theoretical point of view, what is the advantage of the strategy proposed in this paper compared with the three strategies mentioned in Theorem 4.2? \n\n3.Theorem 5.1 assumes that $\\sigma_1\\neq\\sigma_2$, but in practice there may be multiple principal singular values (such as symmetric graph structures). Could you elaborate on this special case?", "limitations": "See weakness and questions.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\nThe paper addresses a barely explored aspect of GNN robustness by focusing on pooling operations, providing both theoretical and empirical insights. The proposed method is model-agnostic and computationally efficient (using power iteration).\n\nWeakness:\nYour attack settings are based on both features and topology, but the main results only consider feature-based attacks. Moreover, Theorems 4.2 and 4.3 appear to be simply extensions of Theorem 4 in [10] to the task of graph classification.", "was_revised": true, "final_justification": "The rebuttal resolves my concerns about the overall attack framework and the meaning of Theorem 5.1.  As the authors promise to include the related discussions and additional experiments in the manuscript, I think the paper can be received.", "rebuttal": "We appreciate the reviewer’s time and effort in reviewing our paper and for recognizing the significance of studying the effect of pooling in terms of adversarial robustness. In what follows, we try to address the raised questions and weakness:\n\n**Q1 - On the distance in the proof of Theorem 4.2 and the punctuation:**\n\nWe apologize for the raised confusion in this part. Indeed, in Theorem 4.2, we consider feature-based adversarial attacks, which perturb only the node features, and therefore the topology remains unaltered. Hence, the distance (in terms of the norm) between $X$ and $X’$ is smaller than $\\epsilon$. We will work on the writing clarity, fix the punctuation and more clearly distinguish matrix and vector norms accordingly. \n\n---\n\n**Q2 - On the derived bound in Theorem 5.1:** \n\nAs spotted by the reviewer, the bounds derived in Theorem 5.1 and Theorem 4.2 share several terms that  depend on the weight norms, the message-passing framework and the effect of the structure (expressed in terms of the walks within the graph). The distinctive contribution and the main advantage of RS-Pool becomes more apparent in Corollary 5.2, where the bound is further refined to expose the influence of the hyperparameter $\\tau$. This parameter, unique to RS-Pool, offers user control over the robustness-accuracy trade-off, allowing practitioners to directly control the upper bound on adversarial risk, and consequently the model’s underlying robustness. This is not possible with the classical flat pooling approaches considered in Theorem 4.2.\n\nAs highlighted in lines 286–287, a key insight is that RS-Pool functions as a robustness stabilizer, even when the preceding message-passing layers are not inherently robust. Unlike traditional pooling methods, which aggregate over potentially perturbed final node embeddings, our proposed pooling rather focuses on the dominant singular vector keeping therefore the dominant, low-frequency signal and attenuates high-frequency components, where adversarial perturbations typically arises [1]. This filtering contributes directly to the tightening of the bound and the improved empirical robustness demonstrated in our experiments. We will clarify this point further in the revised version of our manuscript. \n\n[1] A Frequency Perspective of Adversarial Robustness. Maiya et al. arXiv. 2021.\n\n---\n\n**Q3 - On the special case of singular value degeneracy:**\nWe thank the reviewer for pointing out this special case.  We note that RS-Pool is applied to the final node embedding matrix produced by the message-passing scheme. At this stage, having a dominant singular value with high multiplicity is highly unlikely, even in symmetric graphs, because the learned embeddings do not depend just on the graph structure but also on initial node features and the GNN parameters, which typically break such dominant symmetries. Moreover, GNNs (and actually also Transformers) are known for their inherent smoothing tendencies, which lead to rank-reduction in the feature matrix and result in a non-trivial spectral gap in practice [1, 2, 3, 4, 5].\n\nNonetheless, as pointed out by the reviewer, we agree that the provided upper-bound doesn’t take into account this specific case, but an extension of Theorem 5.1 to the case where $\\sigma_1 = \\sigma_2$ is immediate. Specifically, as previously discussed in Wedin’s Theorem, which is the main basis of our proof, extending to the case where the dominant singular value is of multiplicity $r$ shall result in an additional scalar $\\sqrt{r}$ (rather than $\\sqrt{2}$) in the derived upper-bound $\\gamma$ and the considered spectral gap is $\\sigma_r - \\sigma_{r+1}$. While such a special case is unlikely to arise in practice, we will provide the details of this derivation in the revised manuscript for completeness.  \n\n[1] Anisotropy Is Inherent to Self-Attention in Transformers.- Godey et al. - EACL 2024.\n\n[2] How do vision transformers work? - Park N. & Kim S. - ICLR 2022.\n\n[3] Anti-oversmoothing in deep vision transformers via the fourier domain analysis: From theory to practice. - Wang et al. - ICLR 2022.\n\n[4] Attention is not all you need: Pure attention loses rank doubly exponentially with depth. - Dong et al. - ICML 2021.\n\n[5] How expressive are transformers in spectral domain for graphs? - Bastos et al. - TMLR 2022."}, {"review_id": "qGVR63HeVn", "reviewer": "Reviewer_EjgL", "summary": "The paper investigates the adversarial robustness of Graph Neural Networks (GNNs), specifically examining how different flat pooling methods (sum, average, and max pooling) affect their resilience in graph classification tasks. The authors first provide a theoretical analysis of these pooling operations, deriving upper bounds on their adversarial risks and identifying distinct vulnerabilities depending on the type of adversarial attack and graph structure.\n\nMotivated by this analysis, the authors propose a novel pooling strategy named Robust Singular Pooling (RS-Pool). This method constructs robust graph-level representations by leveraging the dominant singular vector of the node embedding matrix. RS-Pool is theoretically shown to enhance robustness by minimizing the sensitivity to adversarial perturbations through spectral stability properties. The proposed pooling approach is efficient, model-agnostic, and practically implemented using the power iteration method. The paper highlights the critical yet overlooked role pooling plays in the robustness of graph classification tasks and introduces a practical and effective approach to improving robustness via pooling.", "questions": "The authors should address the mentioned weakness above. Specifically, since the theoretical framework and method proposed by the authors could potentially be adapted to also address BFAs on GNNs, I suggest that they briefly mention BFAs in the related work section.", "limitations": "Mostly. The authors should note in the \"Limitations\" paragraph that investigating the robustness of their method against attacks other than the discussed evasion methods (e.g., BFAs) remains an open research topic.\n\nEDIT: Adressed during rebuttal.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\nThe paper provides a thorough theoretical treatment of adversarial robustness, establishing explicit bounds for standard pooling methods and the proposed RS-Pool. Such clear theoretical grounding significantly strengthens the paper’s contribution. The empirical results are robust and cover a diverse range of datasets (bioinformatics, molecular, social, and image-based graphs), enhancing the credibility of the claimed benefits of RS-Pool.\n\nGiven the increasing adoption of GNNs in critical applications (e.g., drug discovery, cybersecurity), enhancing robustness against adversarial attacks is highly impactful. Addressing the overlooked influence of pooling operations rather than focusing solely on message-passing layers introduces an important new dimension to the robustness discussion.\n\nThe paper is methodically organized, making complex theoretical derivations accessible. Definitions and results are clearly stated, and the flow from theoretical insights to experimental validation is logical and easy to follow.\n\n# Weaknesses (only minor, adressed during rebuttal)\n\nThe paper does not consider or discuss malicious bit-flip attacks (BFAs), a relevant and increasingly studied form of adversarial perturbation on GNNs (see references below). BFAs are not acknowledged or reviewed within the related work section, limiting the paper's comprehensiveness regarding current adversarial threats.\n\n# Conclusion\n\nI think this is a very strong paper. The proposed pooling method has a solid theoretical foundation and comes with proven guarantees regarding its adversarial risk, yet it remains technically simple, efficient, and elegant. The write-up is exceptionally clear. I enjoyed reading this paper a lot and consider it inspiring and highly relevant to my own line of work. I would be genuinely surprised if it is not accepted and would attend the talk if it was accepted as oral.\n\n# References\n\nKummer et al, \"On the Relationship Between Robustness and Expressivity of Graph Neural Networks\", AISTATS, 2025\n\nKummer et al., \"Attacking Graph Neural Networks with Bit Flips: Weisfeiler and Leman Go Indifferent\", SIGKDD, 2024\n\nWu et al., \"Securing Graph Neural Networks in MLaaS: A Comprehensive Realization of Query-based Integrity Verification\", IEEE Symposium on Security and Privacy (SP), 2024\n\nJiau et al., \"PyGFI: Analyzing and Enhancing Robustness of Graph Neural Networks Against Hardware Errors\", CoRR abs/2212.03475 2022", "was_revised": true, "final_justification": "My initial review of the paper was highly positive and it remained throughout the rebuttal phase. I have read the other reviewers remarks as well as the authors replies and while I do agree with some minor technical issues the other reviewers point out, I find the paper as a whole -- the fundamentally novel theoretical perspective and thorough analysis, the proposed elegant practical solution as well exceptional write up --  still sufficient to strongly recommend acceptance. I genuinely believe this work will have high impact across several sub domains of graph learning with GNNs and make a significant contributions towards rendering GNNs safer and thus more applicable in real world settings. This expectation of a broader impact has been reinforced by the authors preliminary experiments during rebuttal, which showed that RS-Pool also leads to a favorable robustness against BFAs, which are an attack concept orthogonal to what was originally considered by the authors.", "rebuttal": "We thank the reviewer for their review and for acknowledging the novelty of our theoretical analysis and the effectiveness of our proposed RS-Pool. We are indeed very grateful for the kind comments about our manuscript, and we hope that it can help increase the robustness of GNNs especially in critical applications as identified by the reviewer. \n\nWe additionally thank the reviewer for pointing out the Bit-flip attacks, which we were not aware of. It is indeed a nice direction of research for adversarial attacks, which is very relevant in the case of GNNs, and especially in the case of graph classification given the limited number of attacks focusing on this specific task. We will add a thorough discussion of this line of research in our related work section. We have additionally found an implementation of this specific attack in the context of GNNs, which takes into account the TU Dataset family, on which we base our experimental setup. We will accordingly try these experiments and provide them in the updated version of our manuscript."}]

Author rebuttal:

Meta-review: This paper proposes a singular pooling mechanism to improve the robustness of graph classification models under adversarial perturbations. The method is well-motivated, theoretically grounded, and shows consistent improvements over baselines across multiple benchmarks. Reviewers appreciate the clarity of presentation, the solid empirical and theoretical evidence, and the potential impact on robust graph learning. Several concerns are raised about the scope of experiments and broader generality, but these are well-addressed in the rebuttal. Overall, the paper's contribution is novel, technically sound, and it establishes a useful and well-substantiated direction for developing more robust graph classification methods.
Decision: Accept (poster)