Aggregation Buffer: Revisiting DropEdge with a New Parameter Block
Authors: Dooho Lee
Myeong Kong
Sagad Hamid
Cheonwoo Lee
Jaemin Yoo
Venue: ICML 2025 poster (2025)
Keywords: Graph neural networks
DropEdge
data augmentation
edge robustness
node classification
degree bias
structural disparity
Primary area: deep_learning->graph_neural_networks
TL;DR: We propose Aggregation Buffer, a plug-in parameter block designed to address the fundamental limitations of DropEdge and significantly improve the robustness of various GNN architectures.

Abstract: We revisit DropEdge, a data augmentation technique for GNNs which randomly removes edges to expose diverse graph structures during training. While being a promising approach to effectively reduce overfitting on specific connections in the graph, we observe that its potential performance gain in supervised learning tasks is significantly limited. To understand why, we provide a theoretical analysis showing that the limited performance of DropEdge comes from the fundamental limitation that exists in many GNN architectures.
Based on this analysis, we propose **Aggregation Buffer**, a parameter block specifically designed to improve the robustness of GNNs by addressing the limitation of DropEdge. Our method is compatible with any GNN model, and shows consistent performance improvements on multiple datasets. Moreover, our method effectively addresses well-known problems such as degree bias or structural disparity as a unifying solution. Code and datasets are available at https://github.com/dooho00/agg-buffer.

A Aggregation Buffer: Revisiting DropEdge with a New Parameter Block
Dooho Lee Myeong Kong'
Sagad Hamid"
12
Cheonwoo Lee' Jaemin Yoo
Abstract
GNNs are able to account for interaction patterns and struc- tural dependencies, source of knowledge that enables im- proving the performance in sumissuper learning tasks, even with limited observations (Ying et al., 2018; 8; Brody et al., 2022; Song et al., 2022).
We revisit DropEdge, a data augmentation tech- nique for GNNs which randomly removes edges to expose diverse graph structures during train- ing. While being a promising approach to ef- fectively reduce overfitting on specific connec- tions in the graph, we observe that its potential performance gain in supervised learning tasks is significantly limited. To understand why, we provide a theoretical analysis showing that the limited performance of DropEdge comes from the fundamental limitation that exists in many GNN architectures. Based on this analysis, we propose Aggregation Buffer, a parameter block specifically designed to improve the robustness of GNNs by addressing the limitation of DropEdge. Our method is compatible with any GNN model, and shows consistent performance improvements multiple datasets. Morcover, method ef- fectively addresses well-known problems such as degree bias or structural disparity as unifying so- lution. Code and datasets are available at ht tts:ss: / //github.. com/ doohoC0 0/ / laggaagguuuaa buffer.
While leveraging edge structure has proven highly effective, it often makes a GNN overfit to certain structural properties of nodes mainly observed in the training data. As result, the model's performance suffers considerably in the pres- ence of structural inconsistencies. For example, it is widely known that GNNs perform worse on low-degree nodes than on high-degree nodes even when their features are highly informative, since high-degree nodes are the main source of information for their training (Tang et al., 2020; Liu et al., 2023; Subramonian et al., 2024). Moreover, GNNs exhibit poor accuracy on nodes whose neighbors have conflicting structural properties, such as heterophilous neighbors in ho- mophilous graphs, or vice versa (Wang et al., 2024; Mao et al., 2024). These problems clearly highlight the two faces of GNNs their reliance on edge structure is the key to their success, while also making them more vulnerable.
Common approaches to enhance robustness against input data variations in supervised learning are random dropping techniques such as DropOut (Srivastava et al., 2014). For GNNs, DropEdge (Rong et al., 2020) has been introduced as a means to increase the robustness against edge perturba- tions. DropEdge removes a random subset of edges at each iteration, exposing a GNN to diverse structural information. However, the performance gain by DropEdge is limited in practice, and DropEdge is typically excluded from the stan- dard hyperparameter search space of GNNs in benchmark studies (Dwivedi et al., 2023; Luo et al., 2024).
1. Introduction
Graph-structured data are pervasive across various research fields and real-world applications, as graphs naturally cap- ture essential relationships among entities in complex sys- tems. Graph neural networks (GNNs) have emerged as a powerful framework to effectively incorporate these rela- tionships for graph-related tasks. In contrast to traditional multi-layer perceptrons (MLPs), which solely consider node features, GNNs additionally take advantage of edge informa- tion to incorporate crucial interrelations between node fea- tures (Kipf & Welling, 2017; Gasteiger et al., 2019; Hamil- ton et al., 2017; Veličković et al., 2018). As a consequence,
In this work, we provide a theoretical analysis on the reason why DropEdge fails, We study the objective shift caused by DropEdge and highlight the implicit bias toaustobsss trade- off in its objective function. Then, we prove that the failure of DropEdge is not because of its algorithm, but the induc- tive bias existing in most GNN architectures, based on the concept of discrepancy bound in comparison to MLPs.
'School of Electrical Engineering, KAIST, Daejeon, Re- public of Korea *Computer Science Department, University of Münster, Münster, Germany. Correspondence to: Jaemin Yoo <jaemin kc.
Building on these insights, we propose Aggregation Buffer (AGG a new parameter block which can be integrated to to any trained GNN as a post-processing procedure. AGG effectively addresses the architectural limitation of GNNs,
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada, PMLR 267, 2025. Copyright 20255 by the author(s).

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
allowing DropEdge to significantly enhance the robustness of GNNs compared its original working mechanism. We demonstrate the effectiveness of AGG in improving the robustness and overall accuracy of GNNs across 12 node classification benchmarks. In addition, we show that AGG works as a unifying solution to structural inconsistencies such as degree bias and structural disparity, both of which arise from structural variations in graph datasets.
DropEdge
DropEdge
Figure I. DropEdge generates various reduced rooted subgraphs for center nodes (*) by randomly removing edges.
2. Preliminaries
In node-level tasks, a GNN can be considered as taking the k- k-hop subgraph of each node as its input. For each node i, the edge removal operation in DropEdge can be interpreted as transforming the rooted subgraph Gi, centered node i, into reduced rooted subgraph, denoted as Gi. Figure I illustrates how DropEdge modifies a rooted subgraph.
Notation. Let G = (V, E') be an undirected graph, where V is the set of nodes and E is the set of edges. We denote the adjacency matrix by A E (0. 13IV where at = if there an edge between nodes i and j. and aij = otherwise. The node feature matrix is denoted by X E RIV xdo, where dy is the dimensionality of features.
Definition 3.1 (Rooted Subgraph). A rooted subgraph Gj
Graph Neural Network. A graph neural network (GNN) consists of multiple layers, each performing two key oper- ations: aggregate (AGG) and update (UPDATE) (Gilmer et al., 2017; Hu et al., 2020b). For each node, AGG gathers information from its neighboring nodes in the graph struc- ture, while UPDATE combines the aggregated information with the node's 's previous representation. With ( X, we formally define the I-th layer (t) E R/I[xd as
(V, F2,) is k-hop subgraph centered on node i, where Vi is the set of nodes within the k-hop neighborhood of node i and E, denotes the set of edges between nodes in Vi2
Definition 3.2 (Reduced Rooted Subgraph), Given a rooted subgraph G. as an input, reduced rooted subgraph G, (Vi, E3) is a subgraph of Gi created by DropEdge, where V; C V2 and É, C E, is the edge set induced by V.
3.2. Hlas-Robusttess Trade-off
H W D (H (11),, ()),,,,, H UPDATE (1) (H ((H")))))
In a typical classification task, the objective is to minimize the Kullback-Leibler (KL.) divergence between the true pos- terior ((y)(C)) and the modeled one Q(y: as
where di is the dimensionality of embeddings from the I-th layer, and a learnable weight matrix w E Rd 1x4 is typically used to transform representations between layers. We also denote H (s:t) E [glV|x(d. (d..+...))d))) as the concatena- tion of node embeddings from layer s to along the feature dimension, as (s:t) [| ... (+), where denotes the concatenation operator and t.
((0)=
(P 9. (P Q(
(1)
where 0 is the set of parameters used for modeling Q. When DropEdge is used during the training of GNN, it perturbs the given rooted subgraph and creates a reduced subgraph G. which leads to the following shifted objective function:
(())= DKG. (P
3. Revisiting DropEdge
The shifted objective È can be decomposed as follows:
We give an overview of DropEdge and formalize its objec- tive shift in node-level tasks. We then analyze the implicit bias-robustness trade-off in its objective and exhibit an un- expected failure of DropEdge on improving robustness.
[( DKL (Pr.
Epllog Q9y) log (())) (3)
The first term corresponds to the standard objective func- tion [(0) in Equation 1, which ensures that optimizing [(0) remains aligned with its intended purpose. particularly measures how well the model approximates the true poste- rior and can be referred to as bias, as it relies on observed labels collected to represent the true distribution.
3.1. Overview of DropEdge
DropEdge (Rong et al., 2020) is a data augmentation algo- rithm for improving the generalizability of GNNs by intro- ducing stochasticity during its training. More specifically, it modifies the graph's 's adjacency matrix A using a binary mask M& E {0, 1)1VIxIV, by creating an adjacency matrix A = M& @ A, where @ is the element-wise multiplication between matrices. The matrix M is generated randomly to drop a subset of edges by setting their values to zero.
The second term measures the expected difference between log Q (9.) and log Qu G.). This can be understood as measuring the robustness of a GNN against edge perturba- tions, as minimized s.t. the GNN produces consistent
2

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Test Accuracy
E
ing that other drop ratios would lead to degradation.
0.5
This behavior is consistently observed across all datasets in our experiments, raising the question of whether DropE- dge can truly improve the performance. While trade-off between bias and robustness is expected, this outcome is unusual compared to data augmentation methods in other domains (Srivastava al., 2014; DeViiss, 2017: Hou et al., 2022). In most cases, small perturbations of data do not 150 200 250
0.48
Accuracy
0.86
0.46
0.44
0.85
0.42
0.84
0.4
100
250
0
Epochs
Epochs
Bias term in C
Robustness term in Eq
significantly interfere with the primary learning objective, allowing robustness optimization to improve generalization. However, in GNNs trained with DropEdge, optimizing ro- bustness immediately increases the bias term on test data, preventing sufficient robustness to be achieved.
0.44
0.08
0.42
0.07
Loss
0.4
0.06
0.38
0.05
This phenomenon highlights fundamental challenge: the minimization of Êq. in terms of both bias and robustness, is inherently difficult achieve within the standard training framework of GNNs, limiting the effectiveness of DropEdge and similar techniques in improving edge-robustness.
0.36
0.04
50
100
150
200
250
0
50
150 Epochs
100
200
250
Epochs
GCN DropEdge GCN,,Ours)
Figure 2. Accuracy and loss terms test data during the training of a GCN at PubMed. We illustrate the average of 10 independent runs, with shaded regions representing the minimum and maximum values. While DropEdge decreases the robustness term compared to standard GNNs, it leads to increasing the bias term, eventually resulting in similar test accuracy to standard GNNs.
3.4. Reason of the Failure: Core Limitations of GNNs
The robustness term in LQ can be optimized only when a GNN is able to produce similar outputs for different adja- cency matrices, namely A and A. To study the poor efficacy of DropEdge, we analyze how well a GNN can bound the difference between its outputs given different inputs, which we refer as the discrepancy bound. Our key observation is that the failure of DropEdge is not rooted in its algorithm but rather in the inductive bias of GNNs, suggesting that it cannot be addressed optimally with existing GNN layers.
predictions for Gj and G. However, as the expectation in- volves the true distribution P, it can only be computed if P is known. By assuming that Q is sufficiently close to P, we can rewrite L as follows:
Definition 3.3 (Discrepancy bound). Let H²() and H. be the outputs of the I-th layer of a network given different inputs H H((-1)
[Q( = DKL (Pl ())
(4) We discuss the validity of this approximation in Appendix E. The interplay between the terms in La naturally introduces a bias-robustnen trade-off. The first term, which is equal to f. enables learning the true posterior accurately. The second term works as a regularizer, promoting consistency across different reduced rooted subgraphs. Finding an optimal balance between bias and robustness key to maximize the performance of GNNs on unseen test graphs.
DKL (Q(
(f-1) and ! (1-1 The discrepancy bound of
the 1-th layer is a constant C, such that
H H 00 112 S CII H. H 1
where C' is independent of the specific inputs.
As a comparison, we first study the discrepancy bound of MLPs in Lemma 3.5 and move to GNNs. Proofs for all theoretical results in this section are in Appendix C.
Lemma 3.4. Commonly used activation fanctions-Rele Sigmoid, and GELU-ard parameterized linear transforma- tion satisfy Lipschitz continuity.
3.3. Unexpected Failure of DropEdge
DropEdge is designed to enhance the robustness of GNNs against structural perturbations. To evaluate its effectiveness, we train GNN with and without DropEdge and measure LQ on the test set. As shown in Figure 2. DropEdge success- fully regularizes the robustness term compared to standard GNNs. However, this comes at the cost of increasing the bias term, leading to a similar total La and no overall per- formance improvement. It is notable that we have carefully tuned the drop ratio of DropEdge for this example, suggest-
Lemma 3.5. Given an MLP with activation function o, the discrepancy bound at the 1-th layer is C = Lo ( (|2 where Lg is the Lipschitz constant of a.
Theorem 3.6. In an I--layer MLP with activation function o, the discrepancy bound at the L-th layer can be derived for every intermediate layer I < L as
H .H2 (111 S C H()) 1110122 1|2,
3

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
where C
1.0 11) I
12.
Step 1. Integrate AGGg into each layer of the trained GNN
Step 2. Train AGGg with DropEdge
GNN
Theorem 3.6 implies that reducing discrepancies in inter- mediate representations can minimize discrepancies in the final output, allowing parameters in each layer to effectively contribute to the model's 's robustness. On the other hand, the linear discrepancy bound does not hold for GNNs. We formalize this observation in Theorem 3.8.
Q(v (G:)
AGGB
GNN
Lbius
Frozen Weight
Drop Edge
Q R (G.)
AGG
Ceobuse Q2(3i19i)
Lemma 3.7. Commonly used aggregation functions in reguarr random walk normalized, and symmetric normalized satisfy Lipschitz contimuity.
AGG
UPDATE
C
GNN
Figure 3. Illustration of AGGg and its training scheme. After the integration into pre-trained GNN, AGG is trained using Lxc with DropEdge, while the pre-trained parameters remain frozen.
Theorem 3.8. Given a graph convolutional network (GCN) with any non-linear activation function a and different adja- cency matrices A and Ag, the discrepancy bound cannot be established as a constant C independent of the input. Theorem 3.9. Under the same conditions as Theorem 3.8, the discrepancy of GCN at layer I is bounded as
2. Effectiveness. Pre-training without DropEdge avoids the suboptimal minimization of the bias term observed in Section 3.3. As a result, AGG g can focus entirely on optimizing the robustness term.
[H where C La 11, C2 = C A2 and Å is the normalized adjacency matrices of A.
(1) H 11 & G - C2.
(1)
No Knowledge Loss. Freezing the pre-trained parame- ters prevents any unintended loss of knowledge during the training of AGG The integration of AGG can even be detached to get the original model back.
The key difference between GNNs and MLPs arises from the AGG operation in GNN layers. While the inclusion of the AGG operation enables a GNN to utilize the graph structure, becomes problematic when aiming for robust- ness under different adjacency matrices. As demonstrated in Theorem 3.9, discrepancies can arise purely due to dif- ferences in the adjacency matrices, as a form of C2, even if the pre-aggregation representations are identical. This issue ultimately hinders the optimization of the robustness term in Lq. as observed in Section 3.3.
The main idea of our approach is to assign distinct roles to different sets of parameters: the pre-trained parameters fo- cus on solving the primary classification task, while B is dedicated to mitigate representation changes caused by in- consistent graph structures. Given AGG while its details will be discussed later, we modify a GNN layer as
H H = AG (H("), .A + AGG!! (1) (H (01-1) A),
where AGGB can leverage all available resources until the current layer including the adjacency matrix A and the preceding representations H (0:1-1), We henceforth refer to the GNN model augmented with AGGB as GNNB.
4. Achieving Edge-Robustness
Our analysis in Section 3 shows the difficulty of optimizing La due the nature of GNNs. As solution, we propose Aggregation Buffer (AGGa). ) a new parameter block which can be integrated into a GNN's backbone as illustrated in Figure 3. AGG is specifically designed to refine the output of the AGG operation, mitigating discrepancies caused by variations in the graph structure introduced by DropEdge.
4.2. Essential Conditions for AGG
The important part of our approach is to decide the actual function of AGG Existing methods for enhancing GNN layers, such as residual connections (He et al., 2016; Chen et al., 2020) and JKNet (Xu et al., 2018), are not considered as AGG since they fail to satisfy the essential conditions that AGG must meet to achieve purpose. To derive our own approach that is better than existing methods, we first introduce the two essential conditions for AGGg-
4.1. Aggregation Buffer: A New Parameter Block
Unlike the standard training strategy, where an augmenta- tion function is used during training, we propose a two-step approach; given a GNN trained without DropEdge, we in- tegrate AGGB into each GNN layer and train AGGg with DropEdge while freezing the pre-trained parameters. This two-step procedure provides several advantages:
C1: Edge-Awareness.. When the adjacency matrix A is perturbed to Á, AGG should produce distinct outputs to compensate for structural changes:
AGG (A) + AGG ( (A).
1. Practical Usability. Our approach can be applied to any trained GNN. Separate training of AGGg enables modular application even to already deployed models.
This condition ensures that AGGg adapts to structural vari- ations by modifying its output accordingly. Existing layers
4

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
that depend only on node representations, such residual connections and JKNet, fail to meet this condition they produce identical outputs regardless of structural perturba- tions when the input representations remain the same.
Unlike the bias term, the robustness term does require access the true distribution. This independence enables its application to all nodes, including unlabeled nodes, pro- moting comprehensive edge-robustness for the graph. On the other hand, it may not be effective to apply the bias term to all nodes as well, since it relies on an assumption that the pre-trained model distribution Q approximates Palso the unlabeled nodes, which is hardly true in practice.
C2: Stability. For any perturbed adjacency matrix A C A created by random edge dropping, A should produce outputs with a smaller deviation from the original output when given A, compared to when given A:
| B (A)lIF ||F < ||AGG] (A)lle-
5. Related Works
This condition ensures the knowledge learned by the origi- nal GNN to be preserved, contained in the frozen pre-trained parameters, by minimizing unnecessary changes under the original graph structure. At the same time, it provides suffi- cient flexibility to adapt and correct for structural perturba- tions, thereby optimizing edge-robustness without compro- mising the integrity of the original representations.
Random Dropping Methods for GNNs, Several random- dropping techniques were proposed for GNNs to improve their robustness, complementing the widely-used DropOut (Srivastava et al., 2014) method used in classical machine learning (You et al., 2020; Zhang et al., 2021; Li et al., 2023; Fang al., 2023). DropEdge (Rong et al., 2020) removes random subset of edges, while DropNode (Feng et al., 2020) removes nodes along with their connected edges. Ex- isting graph sampling methods can also be seen as variants of these approaches. DropMessage (Fang et al., 2023) in- tegrates DropNode, DropEdge, and DropOut by dropping propagated messages during the message-passinn phase, of- fering higher information diversity. While these methods aim to reduce overfitting on edges in supervised learning, their performance improvements have been modest.
Our Solution. We propose simple structure-awar form of AGGB which satisfies both conditions above:
JB H (0--1) A) = (D + H (at- W
where D is the degree matrix of adjacency matrix A. Since it is degree-normalizer linear transformation, its computa- tion is faster than the regular AGG operation. When com- puted in parallel, integrating AGG does not increase infer- ence time, ensuring efficient execution.
Sub-optimalities of GNNs. Incorporating edge informa- tion for its prediction is the core idea of GNNs. However, it also makes GNNs vulnerable to structural inconsistencies in the graph, making it suffer from well-known problems like degree bias and structural disparity. Degree bias refers to the tendency of performing significantly better on high- degree (head) nodes than on low-degree (tail) nodes (Tang et al., 2020; Liu et al., 2023). Tail-GNN (Liu et al., 2021) transfers representation-traasllee while Coldbrew (Zheng et al., 2021) uses existing nodes as virtual neighbors for tail nodes. While both approaches improve the performance on tail nodes, they degrade the per- formance on head nodes and rely on manual degree thresh- olds. TUNEUP (Hu et al., 2023) fine-tunes GNNs with pseudo-labels and DropEdge, differing from our method by not freezing pre- trained parameters, lacking AGGg, and us- ing different loss function. GraphPateher (Ju et al., 2024) attaches virtual nodes to enhance the representations of tail nodes. Structural disparity arises when neighboring nodes have conflicting properties, such as heterophilous nodes in homophilous graphs. Recent studies (Wang et al., 2024; Zhu et al., 2020; Mao et al., 2024) show that MLPs out- perform GNNs in such scenarios, implying that avoiding edge-reliance is often more beneficial. Our work addresses both issues holistically, improving GNN generalization by enhancing edge-tobustness through the idea of AGG
Theorem 4.1. yn satisfies the conditions Cl and C2.
Proof. The proof is in Appendix D.
4.3. Objective Function for Training AGGB
We train the A to minimize an objective function, LRC, referred as the robustness controllle loss, which has a few adjustments from fq. First, we introduce hyperparameter A to explicitly balance the strength between the bias term Lhins and the robustness term Lrobust
from head to tail nodes,
(Rc Loiss(0)) + Crobusi (f
(5)
where 0g refers to the set of parameters in AGGB-
Then, we reformulate the bias term by replacing P with Q. Since our method involves two-stage training. we can safely assume that the modeled distribution Q of the pre-trained GNN is a good approximation of the true distribution P at least within the training data. As a result, the bias term can simulate knowledge distillation in training data:
Lrins (0g) [inl Eiev,, DKL (2 (.]l
Crodust (0 = T DRL (g) (.)),
where Vi refers to the set of (labeled) training nodes, and QB represents the modeled distribution of the GNN en- hanced with AGGB, which we refer to as GNN
5

---

Aggregation Buffer: Revisiting DropEdge with a New Parameter Block
Table 1. Accuracy (%) of all models for test nodes grouped by degree. Head nodes refer to the top 33% of nodes by degree, while tail nodes refer to the bottom 33%. Bold values indicate the best performance, and underlined values indicate the second-best performance. Standard deviations are shown as subseripts. Our GCNg g achieves at least the second-ber best in 31 out 36 settings.
Method
Core
Cf Cleeer
Publieed
Miki-cs
A.Photo
Corrpute
CB
Prysics
Arxiv
Actor
Squirrel
Cheme lean
OVERALL PERFORMANCE
MLF
04.80ia1 a
60 70.50
84 02.2.
80.97
81181ga..0
15.08ann12
56 56.41.0.34
34
32.50a1 a1
32 10
GCN
83.44 as
72.45
68.47
a0
22.21
88, 88.24
21.86
15.18
71.80
30. 30.16
41,67a2 4n
40 19 30
Drupfdge
63.27 ss
72:29
86 21
60.22
02. 02.14 c9
66 08
91.01
05. 05.13am 10
71 71.78
20 86
38.60g2 or
40 10.51
DropNode
63.05 an
72
86 16.5
91.80
88.17
91.93_a
95.11g0 16
71.72g0.10
28.07_a.4
38.01
39.74gm
DhapMessage
K3.4ba1.0
72.44
a6, 86.56
80.30lao.r
12.13
88.02200.4
92.08
86.14an
71 71.565a..30
29.0 29.62.1.5
28.76aa
401 401/8 or
TUNEUP
61.59g1
75
86, a3
80.50100
92
88.14,000
90.89.o
71.81
28, 1..
41.49ga
40 00.24
GrighPatcher
a3.57
72.22
86 2laa.
50.64
92.80 a0.57
88. 10.m
94.74.
05, a5.25
72.06 12.06
28. 17.77oas uout
41 41.89 dn
60.3 60.35
(Oun)
84. 34.8....
73.32.00ss
87.56 87.56.0.37
80. 76.2..0
92.64
88, as.7
93. 08.54.0.37
95
72.43.0.34
30 30:06.0..4
42. 42.39.211
40 40.96:2493
ArcurAty ON HEAD Nonts (HiGH--nrerea))
MLP
65.86g1.0 as
70.90 20.99
84. 70
80.06
88. 88.5821.32
86.09.a.s
94 34.08.u.24
07. 07.50
63 23:90
$4.27
25. 00.800a
20. 74
OCN
84.70a1 on
79. 79.10
87. a..36
65.19
04.8512m
00.72,0.m
98.15.0.a0
07.6410
40.81,0.10
27. 1..33
25.1
38. 00.51aai
Frge
84.7422m
87. 77.
24.00
16.500g1.75
00. 10
98. 14.14
07.61am
20.67
27.51 27.51
33. 25.64424m ss
37. 77.55.. 54
DeupNode
84.822a 47
19.01,1.44
87. 20.3
65.02
01.89
00.53.n..e
20.19.0
07, 27,5
50. 50.73,03:
32.33 28.02.1.15
35. 26.92,,
DropMessage
84.86a1 on
39335
84.84...4
84.966004
91.l1.amm
1d 9d.28....2
50.77002
27.60.1.00
201-22aa
85
TUNEUP
84.58 as
79.43
S7. 78.
85, 35 30
94.73g in
90. 911022 io
92. 22.12.....
07, 07.26 10
50.74
26.50
34.85aa at
35. 88.8825
86.21g1 GrophiPtccher
79 79.00
87. 6
96, 28
91.51g0
10.20
UT 4610
80.89
26, 26.86
an 72:4
36. B8,40gg
GCNs(Ous))
822. ai
79.41an00
88 88.14go.a on
B6.liga os
91.84
90.70gg so
90.87 as
97.70
50.5...
27.65_1.4
25.38 ma
37.08 on
ACCURACY ON TAIL N NoDES (Low DECREE)
MLP
63.20l11 so
80 80.27 $1.42
$1.30s5.43
$1,91
75.51
92. 92.96.o.2 as
22.76
49.71g0.30
34. 17.334
35.50
28. 28.94a5.o
OCN
70.70 75
65.77 40
85. 14
77.8 77.83gg as
27.88
83, 83.35
90. 50.04 st
02.74
62. 7..76
45.8 45.85 on
S7. 77.177. o1
Page
g0.61
63.54
85 21 20.34
88. 18 27.00
83.65
90. 00.09.o.12
02,662n50
62 02.65:0033
31.94-1.01
43.20
34.91
DropNode
80.19g as
65.50g1.20 2s
65, Ma
27
87.60 87.6021. in
83,28g no
90. 10.1220.4
92.67
22.
30.77
422 42.76 on
34. 20.33,g ae
DmppMsssage
79.71g1
63.75g1
St.d
77.90l
88 88.07
90.35_0
92.7220
03 20
30.78_
40.04,6
34, 14.160.0
TUNEUP
80
06.3543 us
85, 12
78.13 20
87.87
83: 83.46 so
88 98_0 sa
91.64
31. 31.09
45.51 ga os
37. 37.50
Grughiuncher
81.18g on
60.34 17
si 38
TH.M
89.24 28 os
80, 48
53. 63.56...11
29: 22
46.24
3M 22
GCNs(Ous)
82 12.06..
67.17_1.7
86 36.86
79 26.26..0
88.53,1
84.61go.
54.07
64.40 50
32 32.26_.1.9
47.06..413
37.35
6. Experiments
ing eight datasets, use independently randomized 10%//0%%/80% split for training, validation, and test, respec- tively. Our experiments are conducted using a two-layer GCN (Kipf & Welling, 2017), with hyperparameters se- lected via grid search based on validation accuracy across five runs, following prior works (Luo et al., 2024). For all baselines, we choose the hyperparameters as reported in the respective works or perform grid searches if no reference is available. Detailed hyperparameter settings and search spaces can be found in Appendix B.
Datasets. We evaluate the accuracy of node classification for 12 widely-used benchmark graphs, including Cora, Cite- seer, Pubmed, Computers, Photo, CS, Physics, Ogbn-arxiv, Actor, Squirrel and Chameleon (Shchur et al., 2018; Hu et al., 2020a; Pei et al., 2020). For Squirrel and Chameleon, we use the filtered versions following prior work (Platonov et al., 2023). These datasets are frequently used in prior works (Ju et al., 2024) and cover graphs from diverse do- mains with varying characteristics. Detailed descriptions of these datasets are provided in Appendix A.
Evaluation. All reported performances, including the ab- lation studies, are averaged over ten independent runs with different random seeds and splits, where we provide both the means and standard deviations. To assess edge-robustness, we evaluate the performance w.r.t. degree bias and structural disparity. For degree bias, we provide the performance on head nodes (the top 33% of nodes by degree) and tail nodes (bottom 33%), as defined in prior works (Ju et al., 2024). For structural disparity, nodes are grouped similarly based on their homophily ratio. Homophilous nodes are the top 33% with the highest homophily ratios, while heterophilous nodes comprise the bottom 33% with the lowest ratios.
Baselines. We compare GCN, (GCN with AGG with its pre-trained model GCN (Kipf& & Welling, 2017) as well as closely related graph learning methods, grouped into two categories. The first category includes random-dropping data augmentation algorithms for GNNs such as DropE- dge (Rong et al., 2020), DropNode (Feng et al., 2020), and DropMessage (Fang et al., 2023). The second category in- cludes state-of-the-an the-art methods for addressing the degree bias problem such as TUNEUP (Hu et al., 2023) and Graph- Patcher (Ju et al., 2024), which are relevant since degree- robustness can be seen as a special case of edge-robustness. Lastly, we include standard MLPs, providing a baseline for full edge-robustness as no rdge-information is utilized.
6.1. Overall Performance
We compare GCN B with several basclines and present the results in Table 1. In terms of overall performance, GCN B achieves the highest accuracy in 9 and the second-best in 3
Setup. We adopt the public dataset splits for Ogbn- arxiv (Hu et al., 2020a), Actor, Squirrel and Chameleon (Pei et al., 2020: Platonov et al., 2023). For the remain-
6

---

Aggregation Buffer: Revisiting DropEdge with a New Parameter Block
Table 2. Performance (%) of all models grouped by node homophily ratio. Homophilous nodes represent the top 33% of nodes with the highest homophily ratio, while heterophilous nodes represent the bottom 33%. Bold values indicate the best performance, and underlined values indicate the second- best performance. Standard deviations are shown as subscripts.
Metbod
cora
Citeseez
Fublec
Wiki-Cs
Photo
Computer
C5
Physics
Arxiv
Actoz
Squizzel
Chamelcon
ACCURACY ON HOMOPHILOUS NODES
ML.P
21.68
76. 2.
50.00
86.30
63.6322.41 41
20.0
98 20
24.60la0.14
36.68-1.52
25.84
43.50
OCN
22.09
87. g.
05.9 25
91.05
96.45
94.47 10.50
g9.25
99 1.15
05. 05.13go.s
39. a9.07
48.71
15.15
Droplidge
02.38a1.s
88.00 88.0610.0
96.1229.5
94.40at.42
96330554
94.72:0.41
99.28
99. .0132:011
85.08
38.30:1.0
41.25
42.30:4.22
DropNode
02.81
87 S4
96.17an.32
93.99 c..0
a6.48
06.29
99.31 2017
99. 1011
05.02
40.73
42 42.67at.00
Drophlesage
92.51a1.7
88 06.0611.0
16.18:022
94.49-c.34
96.30
94.6220.47
99 39.37:0.0
99. 22.32:013
90.79.0.06
1..0
20.22
11.25:2435
TUNEUP
83. 13
88 31
06.012000
91.05gras
96.30as no
20.57
99. Ma
99, 26
05.6 05.07
37 94g2er
5..88am ro
47 17.9 :7.89
Gmphhetcher
83 03.29
87 38.1.0
26.04
94.08
98. 18
04.05 20.5
38.30
99.44 99.44.0.07
05.72
34 7622611
48.71 18.77
44.8 44.83a1.22
OCN (CNnsOOus)
94. 93.13: av
88.78 88.78 sa
05,83
91.65
96.544m mi
95.30gu.av am
08.64
99 13
05, 05.00ga.1
38, 18.26
49.52:: au
47 47.01
ACCURACY ON IieTEROPHILOUS NODES
MLP
50.93
41.88 11.02
ST.
,0.90
g1.7 24
68.84.0.m
06.966...0
08,08 p..77
.4.53,0.1a
31 16.66, g..7
g2.56
29.53
OCN
64.18ga.
41 06.g sa
07 34.0 at
61.89g1
81.74g0 15
71.42gin5 ns
g6.8
86 8000
32.51 1g)
19. 13 16.
42. 12.19
33. 13.774g74
Diop
24
41.784127
67.1240
60.97
k1.50quu
71.06g109
86.47
21.700
19.29
41.50ggous
37.01.1..2
DropNode
41.59g1 os
67.24 67.2449
SU.07 on
71.38gi
76.50g04
86 4620
21.91
57
18.50 on
41.54g4 on
36.78.1
DropMesaage
41 41.84,os1
07:2330mm
61.48.2.a
11.87,0as
77.27
86.47 co
32.29
19.49.1.s
40.79
37.90-r.
TU TUNEUP
a
42.74gi or
67.0920 en
62.50
81 ot
74.16
St $1.672005
31.68
18.24g00
42.13g5 os
20.00g
GruphPaccher
64.17=2.:
41 1.
66.41 08.41
58.09-ras
81,461100
71 71.502109
78.67a0s
86 57.77
33.38
18. 18.49:138
42.41aa
34. 15.7.0
OCN (Ours)
65.55 54
03.24g100
70.774mm
52.44g
62.29 os
72 72.02:1.. 28
62.75a044
88 43,0
34.02
19.96.1.0
42.42=40r or
35 15.00
out of 12 cases. Random-dropping methods such as DropE- dge, DropNode, and DropMessage fail to consistently out- perform GCN. This suggests that it is hard to enhance the cdge-robustness of GNNs solely by performing data aug- mentation function, due to the inductive bias of GNNs as we have claimed in Section 3. Although GCNB is also based on DropEdge, consistently improves the performance of GCN since it effectively addresses its vige-vuunnrrability.
dropping approaches f'ail to consistently improve the tail performance. The models specifically designed for degree bias, TUNEUP and GraphPatcher, improve the tail perfor- mance but the improvement is relatively marginal.
Especially in heterophilous graphs (Actor and Squirrel), tail nodes generally outperform head nodes, contradicting typical degree bias trends. Degree-bias methods, which rely on the principle of transferring information from head tail nodes, show limited effectiveness in these cases. However, GCNB still improves the performance by enhancing edge- robustness without being restricted to specific information flow, showing that edge-robustness is a broader focus.
One notable observation is that MLPs surpass all models in the CS and Actor datasets. This indicates that edges can contribute negatively to node classification depending on the structural property. On these datasets, our GCN n achieves the highest performance among all GNNs, demonstrating its effectiveness for enhancing edge-robustness.
6.3. Addressing Structural Disparity
TUNEUP and GraphPatcher generally improve the perfor- mance of GCN, demonstrating that addressing degree bias enhances the overall accuracy. However, their effectiveness compared to the base GCN is more limited than previously reported. Unlike previous works (Hu et al., 2023; Ju et al., 2024), which used basic hyperparameter settings, our ex- periments involve an extensive grid search to find optimal GCN configurations, making improvements more challeng- ing. Despite this, GCN B significantly outperforms the base GCN, highlighting that edge-robustness is a critical factor for performance improvements in general.
We report the accuracy of the methods on homophilous and heterophilous nodes in Table 2 to evaluate structural dispar- ity. Consistently with recent findings (Mao et al., 2024), our experiments show that MLPs generally outperform GNNs on heterophilous nodes, while GNNs perform better on homophilous nodes. Methods for addressing degree bias, particularly GraphPatcher, show some improvements on het- erophilous nodes but inconsistently, indicating a correlation between degree bias and structural disparity, although the two problems seem distinct. GCN achieves the highest GNN performance in heterophilous nodes on 9 datasets, demonstrating that enhancing edge-robustness can effec- tively mitigate structural disparity.
6.2. A Addressigg Degree Bias
We assess the performance on head and tail nodes to eval- uate the impact of our method on degree bias, as shown in Table 1. GCNB successfully mitigates degree bias, achiev- ing at least the second-best -bst accuracy on tail nodes in 10 and on head nodes in 9 datasets, demonstrating that enhancing edge-robustness effectively reduces degree bias. Random-
6.4. Generalization to Other GNN Architectures
An important advantage of AGG B is its broad applicability to most GNN architectures due to its modular design. To evaluate its versatility, we conduct extensive experiments
7

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Table Accuracy of different GNN models before and after the integration with AGG AGG e achieves consistent and significant performance improvements across various architectures.
Table 5. Accuracy with alternative loss functions to train AGGB-
Pubmed
CS
Arxiv
Chame
Pseudo-lahel
86.6220.37
92.4820.15
71.8440.18
40. 24..01
Self-distillation
86.1 15
02.02.0.25
72.18+0.19
40.22:4:22
Pubmed
CS
Acxiv
Chameleon
Cross-cntropy
86.67 20.16
23.29
71.942o.1
10.76:4.19
SAGE SAGEg
#0.07
92.44.0.0
70.92+0.1
37.34
Lac (train only)
86 36.8910.5
92.67
72. 12.2610.15
40.31
28.0
93. 03.364
71.16=0.14
37.85+3.50
Lac (ours)
87.56.0.37
93.54.0..7
72.43.0016
40.96
GAT
20.64
90.50
71.86 14
28.54
G
87.47 10
93.09
72. 12.26 22.14
39.0812.84
SGC
84,01
90,80
69.15aa.0 os
38. 18.24
Tahle 6. Accuracy with various architectural hyperparametens,
SGCB
84.77 #1.02 oz
91. 20.0
69. 69.55-20.4
38.91 +3.08
AGGg significantly enhances performance all configurations.
GIN
85. +0.42
87.88 $0.51
63.94
39.84
GIN
87.18 27.18
88. ; 1.00
65. 20.775
41.72:2.41
Pubned
Arxiv
GCN
GCNA
GCN
GCN#
Table 4. Accuracy with different layer architectures used as AGGB, with none of the altemaaties outperforming our proposed design.
NUMBER OF LAYERS
2
86.48
87.56
71.80=0.10
72.43.0.10
4
84.82+0.34
87 $7.36=0.39
71.53=0.20
72. 72.42+0.20
Pubmed
CB
Arxiv
Chameleon
6
83.4610:24
86.64 60.4
70.77
71.79.0.21
8
82. 82.6840.19
06.18
70.17
71.36
JKNet
87 27.45go:s
93.36
72.19
4..29
Residual
87 46gn
92.05g0.2
72.29
39.77
HIDDEN DIMENSION SIZE
AGG
86. 20.55
91.63
72.27
40.69
64
86.54
87.18=0.32
70.
70.43::0.10
AGG (ours)
87. 37.56
93. 03.54
72. 12.1
10.96_445
128
86.5640.25
87.36
70.92.
71.244.0.18
256
86.54
87.54g0.2 23
71.392a.0s
71.8710.12
512
86, 46.4810.17
87.56 g0.27
71.80
72.43.0.16
on four well-known architectures: SAGE (Hamilton et al., 2017), GAT (Veličković al., 2018), SGC (Wu et al., 2019), and GIN (Xu et al., 2019). In Table 3, we compare the accu- racy of each model before and after the integration of AGGg- These results show that AGGB consistently delivers signif- icant performance improvements across all architectures, demonstrating its wide applicability and effectivvness.
ACTIVATION FUNCTION
ReLU ELU Sigmoid Tanh
46.48
07.56=0.27
=1.80=0.10
:2.43:0010
86.51 10.1
86.95 26
71.50.0.2z
71.97
86.66
86.62
71.54:0.14
00.20
85.28
86.12go.17
71.7220.15
72.25
improves performance over the base GCN. Additionally, we test other loss functions, including the cross entropy using labels, knowledge distillation from the pre-trained model (Hinton, 2015; Zhang et al., 2022), and pseudo-labeling (Lee et al., 2013; Hu et al., 2023). Although these alterna- tives lead to performance improvements when combined with AGGB, their effectiveness and consistency are limited compared to our objective function LRC.
7. Ablation Studies
Different Layer Architectures. In line with Section 4.2, we evaluate alternative layer architectures for AGG in- cluding residual connections, JKNet, and the AGG layer of GCN, while not changing other components of our method. Table 4 shows that our proposed design consistently out- performs these alternatives. Especially, the standard AGG shows no improvement on Pubmed and even degrades per- formance CS from the base GCN. This suggests that using an additional AGG operation to resolve structural inconsistencies is ineffective as it introduces another in- consistency, as described in Theorem 3.9. These results highlight the importance of the conditions we propose in Section 4.2 for designing effective AGG Comprehensive results across all datasets, accompanied by an in-depth dis- cussion of this ablation study, are presented in Appendix G.
Architectural Hyperparameters. For broader applicabil- ity, we expect AGGB to enhance robustness across diverse architectural hyperparameters not only in 2-layer GNNs. In Table 6, we present experimental results with varying num- bers of layers, hidden dimension sizes, and activation func- tions in GCN. AGG consistently improves performance in all configurations, even when the base model performs poorly due to overly deep layers or small hidden sizes. No- tably in deep networks, where the performance typically de- grades due oversmoothing, AGG significantly mitigates this decline, suggesting that the lack of robuustebsss is potential reason of oversmoothing. These results demon- strate that AGGB is broadly applicable to GNNs regardless of their architectural hyperparameters. Further experimental results involving deeper architectures and additional datasets are presented in Appendix H.
Following Section 4.3, we ex- plore alternative loss functions for training AGGg. First, we evaluate a variant of LRC by restricting the robustness term to the training set. While this approach is less effective than the proposed loss on all four datasets, it still consistently
Different Loss Functions.
8

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Impact Statement
Table Effect of each key component in the proposed method.
In this paper, we aim to advance the field of graph neural networks. Our work is believed to improve the reliability and effectiveness of GNNs on various applications. We do not foresee any direct negative societal impacts.
Pubmed
CB
Arxiv
Chame ShameLeoo Leon
GCNB
87 07.
93. 0..37
72.43
40. 40.96:4.8s
(-) Freezing
87.50
93.50
10.91
40.77 s6
(-) g
86.8220.60
91.6720.41
72.21
40.35
(-) Pre-truin
87.22.00.14
92.7840.17
71.1540.2
39.3923.06
References
Effect of Each Component. We study the effect of each key component of our approach in Table 7. First, the accu- racy usually degrades without freezing the parameters of the pre-trained GNN, indicating that freezing these parameters prevents unintended loss of the original knowledge. Next, we test the performance without AGGg, which is equivalent to fine-tuning the pre-trained GNN using LRc- This leads to significant performance degradation, even performing worse than the pre-trained GNN on the CS dataset. This supports our theoretical findings that the original GNN ar- chitecture is inherently limited in optimizing the robustness term in LQ. Finally, training GCNp B in an end-to-end man- ner without pre- pre-training also leads to performance degra- dation, demonstrating that the two-step training approach effectively optimizes both the bias and robustness terms.
Brody, S., Alon, U., and Yahav, E. How attentive graph attention networks? In International Conference on Learning Representations, 2022.
Chen, M., Wei, Z., Huang, Z., Ding, B., and Li, Y. Simple and deep graph convolutional networks. International conference on machine learning, pp. 1735.1 PMLR, 2020.
Deieies, T. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 700,04452, 2017.
Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Ben- gio, Y., and Bresson, X. Benchmarking graph neural networks. Journal of Machine Learning Research, 24 (43):1 2023.
8. Conclusion
Fang. T., Xiao, Z., Wang, C., Xu, J., Yang, X., and Yang, Y. Dropmessage: Unifying random dropping for graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 4267-4275, 2023.
In this work, revisited DropEdge, identifying critical limitation--DrpEEagee fails to fully optimize its robustness objective during training. Our theoretical analysis revealed that this limitation arises from the inherent properties of the AGG operation in GNNs, which struggles to maintain con- sistent representations under structural perturbations. To ad- dress this issue, we proposed Aggregation Buffer (AGGB), parameter block designed to improve the AGG opera- tions of GNNs. By refining the aggregation process, AGG effectively optimizes the robustness objective, making the model significantly stronger to structural variations. Exper- iments on 12 node classification benchmarks and various GNN architectures demonstrated significant performance gains driven by AGGB. especially for the problems related to structural inconsistencies, such as degree bias and struc- tural disparity. Despite its effectiveness, our approach has limitations as a two-step approach; its performance relies on pre-trained knowledge, as AGG focuses primarily on improving robustness. A potential direction for future work to design a framework that enables AGG,, to be trained end-to-end, allowing simultancous optimization of both bias and robustness without dependency pre-training.
Feng, W., Zhang, J., Dong, Y., Han, Y., Luan, H., Xu, Q.. Yang. Q., Kharlamov, E., and Tang. J. Graph random neural networks for semi-supervised learning on graphs. Advances in neural information processing systems, 33: 22092-22109 2020.
Gasteiger, J., Bojchevski, A., and Günnemann, S. Com- bining neural networks with personalized pagerank for classification graphs. In International Conference on Learning Representations, 2019.
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chem- istry. In International conference on machine learning. pp. 1263-1272. PMLR, 2017.
Hamilton, W., Ying, Z., and Leskovec, Inductive repre- sentation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Acknowledgements
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
This work was supported by the National Research Founda- tion Korea (NRF) grant funded by the Korea government (MSIT) (RS-201 R8S-2024-00344 425 and RS-2024-000 00406985..

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Hinton, G. Distilling the knowledge in a neural network. arXiv preprint arXiv: 1XXi::55330023//// 2015.
J., Wu, R., Sun, W., Chen, L., Tian, S., Zhu, L.., Meng. C., Zheng, Z.., and Wang, W. What's behind the mask: Under- standing masked graph modeling for graph autoencoders. In Proceedings of the 29th ACM SICKDD Conference on Knowledge Discovery and Data Mining, pp. 1268-1279, 2023.
Hou, L., Pang, R. Y., Zhou, T., Wu, Y., Song. X., Song, X., and Zhou, D. Token dropping for efficient BERT pretraining. In Muresan, S., Nakov, P., and Villaviceneio, A. (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties ( (Volume I: Long Papers), pp. 3774-3784, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. http:////w/22222..looooo....
Liu, Z., Nguyen, T.-K., and Fang, Y. Tail-gnn: Tail-node graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1109-11199 2021.
Liu, Z., Nguyen, T.-K., and Fang, Y. On generalized degree fairness in graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 4525-4533 2023.
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118- 18-22333, 2020a.
Luo, Y., Shi, L., and Wu, X.-M. Classic GNNs strong baselines: Reassessing GNNs for node classification. In The Thirty-eight Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks Truck, 2024.
Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., and Leskovec, J. Strategies for pre-training graph neu- ral networks. In International Conference on Learning Representations, 2020b.
Mao, H., Chen, Z., Jin, W., Han, H., Ma, Y., Zhao, T., Shah, N., and Tang, J. Demystifying structural disparity in graph neural networks: Can one size fit all? Advances in neural information processing systems, 36, 2024.
Hu, W., Cao, K. Huang, K., Huang, E. W., Subbian, K., and Leskovec, J. Tuneup: A training strategy for improving generalization of graph neural networks, 2023. URL https:://pppeeeeeeeeeee net net/forum? forum? id= id-8xuFDlyCoh.
Pei, H., Wei, B., Chang, K. C.-C., Lei, Y., and Yang, B. Geom-gen: Geometric graph convolutional networks. arXiv preprint arXiv:2002.0522 2020.
Ju, M., Zhao, T., Yu, W., Shah, N., and Ye, Y. Graph- patcher: mitigating degree bias for graph neural networks via test-time augmentation. Advances in Neural Informa- tion Processing Systems, 36, 2024.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In The Eleventh International Conference on Learning Represemationss 2023,
Kingma, D. P. and Ba, Adam: A method for stochastic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Confer- ence Track Proceedings, 2015.
Rong, Y., Huang, W., Xu, T., and Huang, J. Dropedge: Towards deep graph convolutional networks node clas- sification. In International Conference on Learning Rep- resentations, 2020.
Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings. OpenReview.net, 2017.
Shehur, O., Mummme, M., Bojehevski. A., and Günnemann, S. Pitfalls of graph neural network evaluation. arXiv preprint 1811.0588, 2018.
Song, Z., Yang, X., Xu, Z.,, and King, Graph- semi- supervised learning: A comprehensive review. IEEE Transactions on Neural Networks and Learning Systems, 34(11):8174-8194 2022.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1229-1958 2014.
Lee, D.-H. et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural net- works. Workshop on challenges in representation learning, ICML,, volume 3, pp. 896. Atlanta, 2013.
Subramonian, A., Kang, J., and Sun, Y. Theoretical and empirical insights into the origins of degree bias in graph
10

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
neural networks. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
Zhang, H., Wu, Q., Yan, J., Wipf, D., and Yu, S. From canonical correlation analysis to self-supervised graph neural networks. Advances in Neural Information Pro- cessing Systems, 34:76-89, 89, 2021.
Tang, X., Yao, H., Sun, Y., Wang. Y., Tang, J., Aggarwal, C., Mitra, P., and Wang, S. Investigating and mitigating degree-related biases in graph convoltuional networks. In Proceedings of the 29th ACM Interaational Conference on Information & Knowledge Management, pp. 1435- 1444, 2020.
Zhang, S., Liu, Y., Sun, Y., and Shah, N. Graph-less neural networks: Teaching old mips tricks via distillation. In International Conference Learning Representations, 2022.
Veličković, P. Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. Graph attention networks. In International Conference on Learning Representations, 2018.
Zheng, W. Huang, E. W., Rao, N., Katariya, S., Wang, Z., and Subbian, K. Cold brew: Distilling graph node rep- resentations with incomplete missing neighborhoods. arXiv preprint a 1.04840, 2021.
Wang, J., Guo, Y., Yang, L., and Wang, Y. Understanding heterophily for graph neural networks. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 50489-50529. PMLR, 21-27 Jul 2024.
Zhu, J., Yan, Y., Zhao, L., Heimann, M., Akoglu, L., and Koutra, D. Beyond homophily in graph neural networks: Current limitations and effective designs. Advances in neural information processing systems, 33:7793-7804, 2020.
Wang, M., Zheng, D., Ye, Z., Gan, Q., Li, M., Song, X., Zhou, J., Ma, C., Yu, L.. Gai, Y., et al. Deep graph library: A graph-centrie, highly-performant package for graph neural networks. arXiv preprint 151.:1199,.0115,, 2019.
Wu, F., Souza, A., Zhang. T., Fifty, C., Yu, T., and Wein- berger, K. Simplifying graph convolutional networks. In International conference machine learning, pp. 6861- 6871. PMLR, 2019.
Xu, K., Li, C., Tian, Y.. Sonobe, T., Kawarabayashi, K.-i., and Jegelka, S. Representation learning on graphs with jumping knowledge networks. In International confer- ence on machine learning, pp. 5453-5462. PMLR, 2018.
Xu, K., Hu, W., Leskovee, J., and Jegelka, How powerful are graph neural networks? In International Conference Learning Representations, 2019.
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. I.., and Leskovec, J. Graph convolutional neural net- works for web-scale recommender systems. In Proceed- ings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 974-983, 2018.
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., and Shen, Y. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33:5812-5823, 2020.
Zeng. H., Zhou, H., Srivastava, A., Kannan, R., and Prasanna, V. GraphSAINT: Graph sampling based in- ductive learning method. In International Conference on Learning Representations, 2020.

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
A. Dataset Overview and Training Configuration for Base GCN
We selected the 12 datasets based on prior work (Ju et al., 2024). For Squirrel and Chameleon, we use the filtered versions provided by (Platonov al., 2023) via their public repository: https: / //gittiii //gittu....//.. com/yandexx readxxxeeeeacc.. heterophi lous- wowus-grappaa graphs, The remaining 10 datasets are sourced from the Deep Graph Library (DGL) (Wang et al., 2019). All graphs are treated undirected. Detailed statistics are provided in Table 8.
Table 8. Statistics of datasets and hyperparameters used for training the base 2-layer GCN.
Cora
Citesser
PubMed
MIKI-CS
Photo
Computer
C5
Phyeica
Arxiv
Actirr
Squirrel
Chame Teon
# nodes
2, 708
3.327
19, 717
11,70
7,650
13, 752
18, 333
34, 403
160. 343
7,600
2,334
800
edges
556
651
431,726
238,
491, 722
163, 788
495, 924
166, 243
391
93, 996
18,
e features
133
3. 703
500
300
745
767
6, 805
8,415
128
932
2,089
2,325
classes
10
Homophily Ratio
0.8100
0.7355
0.8024
0.6543
0.8272
0.7772
0,8081
0.9314
0.6542
0.2167
0.2072
0.2361
Hidden Dim
512
512
512
512
512
512
256
64
512
64
256
256
Learning Rate
1e-2
Weight Decay
Se-4
Se-4
Se-4
Se-4
Se-4
5e-4
Se-4
5er5
5ei5
Se-4
5e-d
5e-4
Dropout
0.5
0.7
0.7
0.3
0.5
0.3
0.2
0.5
0.5
0.7
0.2
0.2
AGG Scheme
Sym
Sym
Sym
Sym
Sym
Sym
Sym
Sym
Sym
Sym
To train the base GCN, we conduct grid search across five independent runs for each dataset, selecting the best hyperpa- rameter configuration based the highest validation accuracy, following the search space outlined in (Luo et al., 2024). The search space included hidden dimensions [64, 256, 512], dropout ratios [0.2, 0.3, 0.5, 0.7], weight decay values [0, 5e-4, 5e-5], and learning rates [le-2, 1e-3, 5e-3]. We use the Adam optimizer (Kingma & Ba, 2015) for training with early stopping based on validation accuracy, using patience of 100 epochs across all datasets.
We also consider two GCN aggregation schemes following prior work (Ju et al., 2024): (i) symmetric normalization, typically used in transductive settings, formulated
AGG H A) D D AD H
and (ii) random-walk normalization, commonly used inductive settings, given by
AGG (H (1-1) A)=
= DD -H
We select the aggregation scheme that achieves higher validation accuracy.
For the Squirrel and Chameleon datasets, we observe significant performance degradation when using standard GCN architectures. Therefore, guided by recommendations from (Platonov et al., 2023), which highlights data leakage issues and proposes filtered versions of these datasets, we incorporate residual connections and layer normalization into GCN. For reproducibility, detailed hyperparameter settings used for training the base GCN on each dataset are provided in Table 8.
B. Experiment Configurations for Baselines and Proposed Method
All experiments conducted NVIDIA RTX A6000 GPU with 48 GB of memory. We sincerely thank all the authors of baseline methods for providing open-source implementations, which greatly facilitated reproducibility and comparison.
MLP. For MLPs, we perform grid search using the exact same hyperparameter search space as the base GCN. This extensive search, which is often overlooked for MLPs, leads to a unique observation: well-tuned MLPs can outperform GNNs on certain datasets, such as Actor and CS.
Random Dropping Methods For DropEdge (Rong et al., 2020), DropNode (Feng et al., 2020), and DropMessage (Fang al., 2023), we the official repository of DropMessage: https: / liigtthuu thub. com /twpt//ttoooeeaaaaaee which offers unified framework for empirical comparison of the random dropping techniques. We conduct a grid search over drop ratios from 0.1 to 1.0 in increments of 0.1 for each method.
GraphPatcher. For GraphPatcher (Ju et al., 2024), we use the official repository: https: /ttp///gtttaaa/a www.tttwb.coooo..tt jumxglhf/ GraphPatcher. We adopt the provided hyperparameter settings for overlapping datasets. For the remaining datasets, we perform hyperparameter search over five independent runs, following the search space suggested in the original paper.
12

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Table Hyperparameters used train AGG² integrated with pre-trained 2-layer GCN
Cora
Citeseer
PubWed
Wiki-CS
Photo
Computer
CS
Physics
Arxiv
Actor
Sqairrel
Chameleon
1.01
0.5
0.1
0.1
DropOut
0.7
0.0
0.5
0.2
0.7
0.2
DropEdge
0.2
0.2
0.2
0.5
0.2
0.2
0.7
TUNEUP. For TUNEUP (Hu et al., 2023), as the official implementation is not publicly available, we implemented the method ourselves. For the second training stage of TUNEUP, we conduct a grid search over DropEdge ratios from 0.1 to 1.0, and use the same search space for learning rate, dropout ratio, and weight decay as in the base GCN. Although TUNEUP was also manually implemented by the GraphPatcher authors, our extensively tuned implementation consistently yields higher performance across the most of datasets.
Aggregation Buffer. A is trained after being integrated into a pre-trained GNN. For training AGGB, we use the Adam optimizer with fixed learning rate of le-2 and weight decay of 0.0 across all datasets. It is noteworthy that further performance gains may be achievable by tuning these hyperparameters for each dataset individually. Since the hidden dimension and number of layers are determined by the pre-trained model, they are not tunable hyperparameters for AGGB. Training of AGGB early stopped based on validation accuracy, with patience of 100 epochs across all datasets.
A requires tuning on three key hyperparameters: the dropout ratio, DropEdge ratio, and the coefficient A, which balances the bias and robustness terms in LRC, described in Equation (5). The search space used for these hyperparameters in our experiments is as follows:
values: [1, 0.5, 0.1],
DropEdge ratio: [0.2, 0.5, 0.7, 1.0).
Dropout ratio: [0, 0.2, 0.5, 0.7].
For hyperparameter tuning, we follow the same process used for training the base GCN, conducting a search across five independent runs and selecting the configuration with the highest validation accuracy. To ensure reproducibility, we provide the detailed hyperparameters for training AGG across datasets in Table 9, and release our implementation as open-source https:://////c. /ttp//gttttt//tt www homm/doooo0////ooooooooooo -aw.//gggg........ooaooooo
C. Proofs in Section 3
C.1. Proof of Lemma 3.4
Activation functions play pivotal role in Graph Neural Networks (GNNs) by introducing non-linearity, which enables the network to model complex relationships within graph-structured data. Ensuring that these activation functions are Lipschitz continuous is essential for guaranteeing that similarly aggregated representation can result simliar output after applying the activation function. In this section, we formally derive the Lipschitz continuity of three widely used activation functions: Rectified Linear Unit (ReLU), Sigmoid, and Gaussian Error Linear Unit(GELU).
C. 1.1.1. DEFINITION OF LIPSCHITZ CONTINUITY
A function f R R is said to be Lipschitz continuous if there exists constant L 2 0 such that for all y E IR
[f(x) f(v) & Tl vl
C. 1 2. RECTIFIED LINEAR UNIT (RELU)
The Rectified Linear Unit (ReLU) activation function is defined as: ReI.U(x) = max(0, x))
To prove that ReLU is I-Lipschitz continuous, we need to show that:
[eeL.((x) ReLUU() s | vl Va, E
13

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Case 1: and y
In this case,
(z)U(x) = and ReLU(yy = y. ReI.U()) - ReLUU()) = = - yl S |x y/-
Thus,
Case 2: < O and 0
Here,
ReLU(z) = and ReLU(y) =
Therefore,
(e)UU(x) ReLU())) 00 - 0 |x y/-
Case 3: 2 and < (without loss of generality)
In this scenario,
ReLU(x) = and ReLU(y) 0.
Thus,
ReLLU(x) ReLU(y)] |ar 0 |x| |o y/- This inequality holds because 2 and y implying |x| S vl.
In cases, [Re..U(x) ReLU(()) J yl. Therefore, ReLU is I-Lipschitz continuous.
C.1.3. SIGMOID FUNCTION The Sigmoid activation function is defined as:
o 1
The derivative of the Sigmoid function is:
d'(x) alz)(1 d(x)).
Using the fact that Vo IR, (x) > 0, (.1)o(()) > we apply AM-GM inequality:
(x) + (1-of (l-o(z))
vo(x)(1-
2
Squaring both sides,
(
2
20(z)(1-o())
2
Thus,
0 0 dz)(x)= = -1 - 4
By the Mean Value Theorem, for any x, y E I, there exists some c between r and y such that:
|a(x) -v)) [0)]l) vl-
Using that [c)(c) S 1. we have for all x, y E H
(a(x) - o N 1
4 - y/-
Therfore, Sigmoid is -Lipschitz continuous.
14

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
C.1.4. GAUSSIAN ERROR LINEAR UNIT(GELU)
The GELU activation function expressed as:
GELU(x) xx()),
where (xx) is the cumulative distribution function (CDF) of the standard normal distribution:
((x)) 1 + erf
1
(
I
2
V22
First, we compute the derivative of Gz)U
GELU(x)= = ((x) + x
dx
where ((x) is the probability density function (PDF) the standard normal distribution:
<(x)
1//2
2nT
In order to show the boundedness of the derivative, we examine the second derivative:
GELU = x2²6xz)
dx
Setting the second derivative equal to zero to find critical points:
d²
GELU(x) = = (2x)) =0. 0.
dx
Since ((x) > 0 for all x E IR, the extrema of GELU(z) occurs at x = +
d.
d
Hence, it is enough to examine the value of GELU(x) = ((x) + zz0(x) at :0,, -
dx
lim d GELU(x)= lim (xx) lim =
2-000 dx
2000
20000
d GELU(x)= lim (xx) + lim =
lim x- dr
x--00
1 (1 + erf(1)) + V2 -e- 2 1.129
GELU(V2) 4(V2) V2 2(V2)
dx
V2T
1
dx GELU(-V2) Φ v2. - 2 = (1 erf(-1)) - V2
e -..22
2t
using that
lim ((x) 1, lim 4(x)) = 0, lim rδ(x) lim
1
=²/2
1100
/2m
15

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Thus,
d
GELU(1) $ 1.13, Vo E R. dx
Therefore, GELU is 1.13-Lisschitz continuous.
C.2. Proof of Lemma 3.5
The spectral norm of a matrix satisfies the following sub-uuliip property:
|BBl2 <
Using this property, we establish the discrepancy bound for 1-layer propagation in standard neural networks. For two intermediate representations ( and ( we have:
H 1 H 12 l1o H (1 (11 W() = b(0) (H W + 1l1
Lo (H (1) w( + b() - (H + 1l
$ Lo H - H 1 ((
where Lg is the Lipschitz constant for activation function
C.3. Proof of Theorem 3.6
The discrepancy in the final output representation can be bounded recursively follows: (
H L))
1| H1[41
L, W (1)
||
H1|
H 1 1|2
L ( 122
(((-1) ( (1- -11 1 11-22)
L I1
||
[ H11112
1=1+1
C|
, 12.
where C = Lo (1-
[ (1() represents the cascade constant.
C.4. Proof of Lemma 3.7
In this proof, we aim to show how minimizing the discrepancy each aggregation step effectively bounds the final representation discrepancy. To ensure consistent analysis, assume that the representation matrix H, is normalized, satisfying |, < V By quantifying propagation of discrepancy across linear transformations and various aggregation operations, we demonstrate that controlling intermediate discrepancies reduce the discrepancy in the final output.
C.4.1. REGULAR AGGREGATION
For regular aggregation, the representation discrepancy satisfies:
|AG (H) 1) A1) - " (H) (1-1) A2)ll2
(A, w) A,H,
l|
[[(A, (((11)))) A, + A H
[1-1]
(1-1)
A₂ (1-1) wr00)12
s (IAA (111 A, [111)
1|
A
(1-1) A2H, (11) (2)) 11
([A41)22|H [1-1) - (111)
+ A A2 (1-1) [|2)1 w 12
s |4/l (1-1) H 1/2 + [V]] A1 - Azll2
16

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
This shows that if A Ag, the representation discrepancy is linearly bounded by the input difference.
C.4.2. ROW-NORMALIZEED AGGREGATION
For row-normalized aggregation, the representation discrepancy satisfies:
||GG " (H| (1-1) Ai) - AG (H2 (1-1) A2)l12
= "|] [ (1-1) ( (1) 2 AAHH2 (1-1) w (0)1 |/2
|I(DT' 1, (1-1) D | [1-1]
+ Di A₁ 2 (1-1) D2' A₂H2 [1-1) w
S (I] A, (1-1) DJ A A [1
||2 |D A H (1-1)
2 D22 A (1 ||)) w [
$ (I A1l22 H [111) - (111))
(
12 + | 1 A -D2 1 A (1-11) 11 12)]
< W 12 H (111 H (1-1)
12 + [ W (0) |2| Dj 1 AA D A2l2
noting that ||
ADT'A,ll A,112 = because low-normalized matrices have their largest eigenvalue equal to 1. This demonstrates that the discrepancy is linearly bounded if A A2.
C.4.3. SYMMETRIC-NORMALIZER AGGREGATION
For symmetric-nnormalized agg.eaan the representation discrepancy satisfies:
| (H) (1-1) A.) - O ( (H2 (1-1) AA2)ll2
=I D : H (1-1) W( D22 A2
H (1-1)
H
W (1) 1|2
[I A H|| D
Di D 1 [111]
Dj D H (1-1) D2 A2 1 H (1-1) w 12
< Pi A, " (((( Dj D -0-2H, 2
[1-1)
1|2 + D A D H (--1)
D2 1 A2D2
H f- (2) [|2) w
< DI $ AAD 12 (111) H
(1-1)
12 | D D2 A. #1 H 1111) 1
< [ (1) |2| 1
112 -1
(1-1) H
H (1-1)
22 D
A -D2
where |D] D 112 = -1 due to its normalization. This follows from the fact that 1 - DI 3 A, is positive
T x = x DJ D, &æ where y is eigenvector for D; - A with corresponding eigenvalue 0. This implies that if A₁ = A2, the representation discrepancy is linearly bounded by the input difference, similar to the case of regular aggregation.
semi-definite, and there exists x = D such that x
C.5. Proof of Theorem 3.8
Before we proceed with the proof, let us first establish the following lemma.
Lemma C.1. Let A, B E RT be two distinct matrices (A + B), and let a m + R mm be a non-constant, continuous, element-wise function. Then there exists some E R.mxn such that
(AAZ) + (BZ)
Equivalently, no such a can satisfy o(AZ) = (BZ) for all Z if A
Proof. Since A + B, there exists at least one index (i,j) such that dij * bij- Denote dsy = and bij (a > 0, 0). We will construct particular Z that reveals the difference under a. Define Z so that its j-th row is scalar variable E R and all other entries are zero. Then, each (i,t)-entry of AZ is az, while the corresponding entry of BZ is bz for I 1=1 1, Because is applied element-wise, ar (AZ)) a (BZ) implies o(az) obz)). We analyze two cases:
Case 1: a = or = 0: Without loss of generality, let b = 0. Then (az) = (()) must hold for all z. This forces or to be the constant, contradicting the given assumption.
17

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Case 2: a + 0 and b + 0: Without loss of generality, let b. Then, o(az) obb)) for all z implies
(
b ((
o(z) - -
for any n Since b/a < 1, by the continuity of a, (=) 1o a constant on that range, again contradicting the non-constant assumption.
= =o()). Hence a would be
Thus, in either case, we find that the hypothesis (AAZ) oBB for all Z forces to be constant, which is contradiction. Therefore, there must exist some Z for which (A42 * (BZ).
Now, we use the above lemma to show that, if A 7 A2, then constant can bound the difference of GCN outputs in terms the difference of inputs. Suppose, for contradiction, that there exists C' such that for every pair H H (1-1) exists, the following holds:
H ()) - 12 1o (1 w()) - o (A2H, ( w (1) S H (111) H 1
In particular, consider the case where H (1-1) = H (1-1) Then || (1-1) - H2 1 0, so the right-hand side of above
equation is zero. Thus, we must have a (1 w()) o (AgH, ( w()). However, by Lemma C.I, there exists a suitable choice of H so that HW) corresponds to of the lemma, leading to o (Á, & (A2)HW()) H (1-1) H. 11-1) H, have
|ooA ()( H
() 0 C| H||
This contradiction shows that no such input-independent constant C can exist.
C.6. Proof of Theorem 3.9
In GNNs with row-normalized or symmetric-normalized aggregation, the propagation of representation discrepancy is bounded as:
H 112 < L= (H (1-1) (H( A.) A
(H (1-1) A2)ll
S La L
H (1-1) H 11 + L W (1 ||2 A - A22l2
where A represents the normalized adjacency matrix and La be the Lipschitz constant of the activation function. This result shows that adjacency matrix perturbations introduce an additional error term, which grows through layers.
D. Proofs in Section 4
D.1. Proof of Theorem 4.1
Note that once condition C2 satisfied, then C1 automatically holds. Thus, suffices to show
|| D < DD -1 Mlpp where D, = DIII, M H 10: w()).
Since À A by edge removal, each diagonal entry d, of D. satisfies 0 < d, s d, Thus 1 d,
+ for all i.
Write D M and D. M row by row. If M., denotes the i-th row of M, then
(D M); = & M (D. M)x.
+ M
de
The Frobenius norm
D M | = Σ d MM
D.
M 2 = Σ 2/2 11.
18

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Pubmed
A.Photo
CS
0.08
0.14
GCN Training
0.06
0.12
0.04
0.02
0.08
0.1
Amw www.waaaau
Loss
0.02
www.
Loss
0.06
0.04
-0.02
0002
0.02
150
Epochs
Epochs
Epochs
0.08
0.05
0.14
AGG Training
0.06
0.04
0.03
0.1
Loss
0.04
0.02
0.08
0.01
0.04
0.02
50
100
150 Epochs
200
250
50
100
200
250
60
80
100
120
Epochs
Epochs
(log Q(c();) - log Q(c(.))
Figure 4. Changes in two different approximations of the robustness loss. Er|log Q(v -og Q(yr during training of the base GCN (top row) and AGG (bottom row). Each curve represents average over 10 independent runs, with shaded areas indicating the minimum and maximum values. Blue represents the robustness term in our proposed bobusmess-controlle loss, where / is approximated by Q. Orange represents the label-based approximation, where P approximated using ground-truth labels. Both approximations exhibit similar trends: robustness loss gradually emerges during GCN training and is further optimized during AGGg training.
Because 1 2 1 whenever d, < dr, each term in the sum for D 1 'M is greater or equal. Therefore
D D M
| S DD. MM.M.
Note that the equality holds if and only if 1|, for every j's such that dj < dj. Such a configuration occupies a measure-zero subset of whole space, and thus arises with probability zero in typical real-world scenarios.
Substituting back completes the proof:
|AGG (9 (A)]IE- 1 < D.D M |AGGE (Á)]le-
E. Validity of the Approximation on Robustness Loss
Between equation 3 and equation 4, approximate the robustness term in the shifted objective under DropEdge. Specifically, the expectation with respect to the true distribution P is approximated using the model's predictive distribution Q as follows:
Epllog Q(vi log Q(v:(.)) = DKL (KL G.)
This approximation is based on the assumption Q P. Since the true distribution P is inaccessible during training, this assumption allows the term to be computed in practice.
Although the assumption Q P may not strictly hold particularly in the early stages of training it becomes increasingly valid as training progresses. Since the model is trained using cross-entropy loss, it explicitly minimizes the KI. divergence (P(y)G.) (alillQQ(1999 (()))))), gradually aligning Q with P on the training distribution. Moreover, our framework employs two-step training procedure, in which this approximation utilized only after the base GCN has been trained. This staged design ensures that the approximation is applied under more reliable conditions, promoting stable and effective optimization of the proposed bobustness-controlle loss, LRC-
19

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Table 10. Test accuracy under varying levels of random edge removal (%) across 12 datasets. A value of 100% indicates that edges are removed, whereas 0% indicates complete edge removal. Bold entries denote the highest performance for each setting. AGG significantly enhances robustness, outperforming the baselines (GCN, DropEdge) in 56 out of 60 cases.
Copa
Citeseer
Pubned
wiki-ca
GCN
DropEdgee
GCNp
GCN
Droplidge
GCN₂
GCN
DrapEdge
GCNz
GCN
Droplidge
GCNr
100%
g1.44
1..57
8..04
12.29
7..32.0.a
86
1.17
a0.4
.7.56
p0.26
80.22
60.75.0.42
75%
81.85=1.2s
81.78-1.35
84.53:1.3
71.54:0.4
73.16-1.01
85.99 6..2
88.01an.12
87.52:0.35
79.28
79.34
60.17:0.47
50%
79.63-1.42
79.09
84.31
70
40
72.90_1.a
50.00
87.51_000
77.85ga9
48
79. 20
25%
76.28_11.
1.000
84.4421.8
60.35
22.66.1.m
24
24.90g024
46.51 a5
77.8330.7
098
72.63g1
72.33
84.1 84.17g1.5
68.122005
67.5 67.54go0
1.
83.80gn
84. 84.18go.00
86.81 g0:
60.00ga.0
72.54g0a9
72. 86
A.Photo
Computerr DropEdge
Physica DropEdge
GCN
DropBidge
GCNg
GCN
GCNg
GCN
DropEdge
GCNz
GCN
GCN2
100%
92.21
92. 14.1.42
02.444.1.4
08
88 88.08
88. n...
91.01g0.1
93.54.0.37
95.18
13
95.79 75.097
75%
92.08
02.02-1.44
92.38_1.8
88.01g0
87
21.41
91.432003
94, 94.88ga11
14.8720
95.77.001
50%
91.47
91.77=1.39
92.29
87.41
87.38
88. 58.5
20
90.71an22
83.56-0.42
94.58
91.31s020
95.78
25%
90.79 1:77
90.92.1.51
91.90 01.9001.51
86.20
86.28
88.02
89.91
89.95
93.53
a3.99
95. 95.12:0.17
0%
84.8
85. 15.99-1..c
86.11
76.77=1.4
78.17
12.50-119
93.10=a.3:
=7.17=
93.18:0.74
94.33
94.63=044
95. 95.44:0.20
Arxiv
Actor
Squirrel
Chameleon
DropEddge
GCNr
GCN
DropEidge
GCNz
GCN
DrapEidge
GCNp
GCN
DrapEdge
GCNp
100%
71 80
71.78g
72.43-011
30.16:u0.7
29. 22
30.5 30.5820.4
11 11.07:2.42
11.66g211
42.39
10.1 10.19
40.23g4::
40.38 40.96.4.x5
75%
70.41a0.17
70.42aa2:
71.57
30.72.0.se
90.33-6.9
30.70.a.aa
40.64:2.59
40.71-27:
41.45-2.54
39. 39.52.4.a7
39.51e421
40.16-4..s
50%
67. 27.
13
69.95-uu14
31 09
30.19=0a0
31.26:1.0s
30.88-42.9
39.63
40.86-2.21
40.13:5.14
39,85 39.631+4.c8
25%
22.8120022
66.28_u114
31. 31.22:00.
31.20 az
31.00
37 37.47a230
37.332220
40.06=2.m 0s
38.41
28.4622.0
38.57=4.39
0%
44.09.0.4
44.55enas
53.57
33.08-1.35
32.80=1.0z
30.70.s.77
82.18.2.51
32.40.1..r
35.40
33.2449.35
31.22::.10
36.29:4.64
To empirically evaluate the validity of this approximation, we estimate the expectation under P using ground-truth labels as proxy. Specifically, we computed the following quantity:
ΣΣ Q(dg)))) - log Q(rl.)),
where y: (c) denotes the one-hot encoded ground- -ruth label for node i. While this proxy samples only one class per node and may be affected by label noise, still offers practical estimate for validating the approximation. As shown in Figure 4, the trend of this label-based quantity closely mirrors that of the approximated KL divergence, indicating that our approximation effectively captures the underlying behavior. Furthermore, AGG exhibits robust optimization behavior even under this label-based approximation, demonstrating its effectiveness in terms of robustness optimization.
F. Assessing Edge-Robustneg via Random Edge Removal at Test Time
While we previously demonstrated the edge-robustness benefits of AGG through improvements in degree bias and structural disparity, we now provide a more direct evaluation by measuring model performance under random edge removal during inference. Specifically, we assess how test accuracy degrades as edges are randomly removed from the input graph. We compare three models: (1) a standard GCN trained normally, (2) a GCN trained with DropEdge, and (3) GCNs, which incorporates AGG,, into pre-trained GCN. The results are presented in Table 10.
GCNg significantly outperforms both DropEdge and standard GCN in 56 out of 60 cases, indicating that AGG enables GCNs to generate more consistent representations under structural perturbations, thereby exhibiting superior edge-robustness.
Interestingly, in of the cases where GCN n does not outperform the baselines, the performance of the standard GCN improves as edges are removed- sppcifiilyyy on the Actor dataset. This aligns with our observation in Table I that an MLP outperforms GCN on this dataset, suggesting that leveraging edge information may not be beneficial. These findings imply that the edges in Actor are likely too noisy uninformative. Nevertheless, even on Actor, GCN n maintains stable accuracy under edge removal, highlighting that AGG, still contributes to enhanced edge-robustness.
In contrast, models trained with DropEdge often show marginal improvements or even performance degradation compared to standard GCNs. This supports our claim that DropEdge alone is insufficient for achieving edge-robustness, due to the inherent inductive bias of GNNs.
20

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Table 11. Accuracy across different layer architectures used AGG2. Blue indicates improvement over the base GCN, and bold text highlights the best pest-eerfon architecture per dataset. The rightmost column reports the average rank. Our proposed design consistently improves performance and achieves the best overall ranking.
Mahhod
Cor
di Ciceseer
Pubsted
Wiki CS
A. P.oooo
A. Conputter er
Physlice
Ansiv
Action
Siguierel mel
Chaneleon
Rank
GCN
44
T2:45 10
50,9810117
80.20 3032
92.21a3.35
88,24 28.24
91,85_a
38.18
71,90us..
00.1010m7
41.07.5.42
20.19)
4.75
AGG
64.01
72: ss
56 87
70. es
91.74 24
58.00.2
91 63
85 02
72.27
29.30 29.00g1::
40.18
40 20
4.92
Resilul
63 63.29
72.7
87.0644.25
80.80.1.0.
92
46.2740.15
72.20
40.57
41.92.200
39-77.418
IKNet-atlle
72. 70.
57.4544.25
80.78
67.77 ua
99 16.36.. ce
95. 25.86..a a..2
72.39aa.
30.70.3.21
40.70.2.30
20.2
AGGy (kmple)
84.28.1.57
72. 12.79
87. 37.58.0.3s
80.70
92.00
a8.50
92.01_a37
50 20
72.09
30013
41.9112.55
40.542415
3,33
A joun))
64 88
73. 35
87. 56
80 70.7
92 64g.
68 16
93. 03.54gg0
05 70g0
72 72.43a0 14
30.50g0
42 30g2
40.90 40.966.
1.58
G. Extensive Ablation Study of Alternative Layer Architectures
In this section, we extend the results presented in Table 4 to all 12 datasets used in our experiments. As shown in Table 11, although several alternative layer architectures provide performance gains in specific cases under our training scheme and loss, only our original A design consistently and significantly improves performance across all datasets.
We also evaluate simplified, single-layer variant of AGG that restricts the usable representation to only the immediate previous layer, ((-11), and is formulated as:
D + H
This variant satisfies the same theoretical conditions- CC (edge-awareness) and C2 like our proposed architecture. While it improves performance on 10 out of 12 datasets matasets-making it the most competitive alternative- the improvements are relatively marginal compared to those achieved by AGG
in Section 4.2, just
The motivation for integrating representations from all preceding layers, rather than relying on a single layer, is to mitigate the risk of accumulating structural discrepancies. Ideally, AGGB could fully correct such inconsistencies at each layer. In practice, however, residual discrepancies may persist in intermediate layers and propagate through the network. ultimately affecting the final output. Relying solely on (1-1) risks amplifying these unresolved issues, whereas aggregating H (0:-1) enables AGGB to leverage earlier, potentially less corrupted representations, leading to more robust corrections.
Importantly, the performance gains from A are not solely attributed to multi-layer integration. We also compare it with a JKNet-style block, which similarly aggregates outputs from all previous layers and is formulated as H (0:-1) AGGB outperforms this JKNet-style design on out of 12 datasets, while the JKNet-style variant even degrades performance on 4 datasets. This result suggests that the inclusion of degree normalization, (D I)- a key component of AGG that ensures satisfaction of the conditions outlined in Section 4.2 (i.e., (1) edge-awareness and (2) stability) is crucial for achieving consistent performance improvements across diverse datasets.
Although AGG performs robustly in our experiments and ablations, we acknowledge that concatenating all preceding representations can introduce information redundancy and noise, particularly as GNN depth increases. However, this is not currently critical issue, as GNNs typically achieve optimal performance relatively shallow depths (e.g., two layers) due to smoothing. That said, as deeper GNNs become more effective in future research, developing more streamlined integration mechanisms that reduce redundancy and noise presents a promising direction for extending this work.
H. Extensive Ablation Study on Deeper GCNs
In this section, we evaluate the effectiveness of AGG when applied to deeper GCN, using 4, 8, 12, 16, and 20-layer models across 6 datasets. In addition to the standard GCN and BC we include two more variants: (1) GCN trained with DropEdge, and (2) DropEdgeg. where A is integrated into a GCN trained with DropEdge.
These variants are included to assess whether AGGB can provide further improvements beyond what DropEdge achieves, particularly in deep architectures. This is motivated by the original DropEdge paper (Rong et al., 2020), which highlights its effectiveness in alleviating oversmoothing and demonstrates more substantial performance gains in deeper GNNs. The results are presented in Table 12.
AGGB improves performance in 28 out of 30 configurations, demonstrating that its effectiveness is robust to architectural depth. Notably, performance gains tend to increase with depth, suggesting that deeper GNNs are more susceptible
21

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Table 12. Accuracy with varying GCN depths. DropEdge denotes a GCN trained with DropEdge followed by integration of AGGg- All experiments use hidden dimension of 256 and learning rate of 0,001. For fair comparison. the edge dropping ratio is fixed at 0.5 for both DropEdge and AGG,, and the hyperparameter is fixed 1.0. Integrating AGG improves performance 56 out 60 configurations.
Cora
Pubree
A. Computer
GCN
GCNs
DropEsige
DrupEdge
GCN
GCNN
DropEilge
DropEdges
GCN
GCNi,
DropEdge
DropEdlge
4
81.98
$2.50.1.06
82.51
8..96.1.80
43.73
86, 50
g4.13
a7
86.86
86
86.23
86 1..34
27.54
29.989_1.m
24.64
10.86ig1.44
83.07a0 16
90 48
83. 23.18a0.0 sn
85.49.0
80.00g2.
79.78
80.21:1192
12
73. 4222as
77.70=1.v
78. 78.41
79.65
82 Masa.
85. 4850
82.58 32.58am.3
84.64 84.44.0
78.082122
76. 76.45- :6.05
76.17
77. 77:16:
69.78
69.78.a.45
75.67 a..3
77.92
79.39
82.08:0.45
85.
81.76.0.47
83.78:0.a
73.50
74.83
74. 74.89
76. 76.20:1.41
61.15es..s
72.23:3.44
73.89-2.0
76.31=3.41
81.75
85.23
81.17a0.38
83.20
67.99
68.63::444
72.14
73.69:3.05
Squirrel
Chaneleon
GCN
GCN₂
DropEdge
Droplidge,
GCN
GCNs
DropEdge
DrogEdge,
GCN
GCNz
Droplidge
DropEdge,
4
89 40
91. as
89.70ga.
91 51
33.96
31.37
33 43.76
53 33.82.1.54
39 17
40.00-a.0
37.95 37.95g4m7
58 46
91:33: 34
68 38.56
91 24:0..
31.06
34.41=1.99
31.1441.47
34.41:1.35
37.35
37.38+2.56
99:3..
37.06+4.31
86.8423445
90.3 20.57:0.2s
27
91 08
34.78 78
34.70:1.9
31.91a3.
35, 35.05:1.s
37. 37.18
37.32 37.32..611
36. 46:11:4444
35.25
16
85 55.0440.51
89 89.98
87.41
90.76 sa
34.49
34.70.1.0
34.77.1.30
34.90:12s
36.35.5.7
36.
34.92at.12
35 35.888
20
82.34
88.59:1.47
85,5620.30
88.77=1.00
31,23:1.24
34.80:1.s
34,83e1.31
$4.92-1.58
35.03.43*
37.30=4.52
31.37
34.06-3.92
Table 13. Accuracy from independent runs of a 2-layer GCN (hidden dimension: 256). and after integration of AGG (GCN B).
evaluated on the public split of larger datasets: Flickr, Ogbn-arxiv, and Reddit. The GCN is trained using fixed hyperparameters (learning rate: 0.001, dropout: 0.5), while AGGg is trained with fixed parameters (A = 1.0, DropEdge rate: 0.5). Integrating AGGg consistently improves overall performance, with the largest gains observed in low-degree, heterophilic nodes.
Flickr
Arxiv
Redd it
GCN
GCN₃
GCN
GCNs
GCN
GCNg
Overall Accuracy
12.50go.15
52.84.0.08
71.06g0.10
71.37
94.61=0.01
94.89_0.01
High-degree Nodes
49.66+0.28
49.87
80.06:0.11
79.95
98.81
98. 0.00
Low-degree Nodes
$3.93
54.58
62.32=0.05
63. 63.16:0.04
88.06=0.01
88.84_0.08
Homophilous Nodes
80.58 40.49
80.44
94.87.0.02
94.60_0.aa
99. 09.7
99.66
Hcterophilous Nodes
18.00+0.07
18.18
32.30=a.00
03.08
84.10=0.01
85.01-o.00
to structural inconsistencies as representations undergo repeated aegregation-thar creating greater opportunities for improvement via enhanced edge-robustness.
As expected, DropEdge yields more substantial improvements in deeper architectures, while its effects remain marginal in shallow ones. Importantly, integrating AGGB into DropEdge-trained models significantly boosts performance in 28 out of 30 settings. This demonstrates that AGGB provides a distinct benefit speefft-sseiiiillllyy... enhanced ddge-robustness. These results reinforce claim that DropEdge alone is insufficient for addressing edge-robustness, regardless of model depth, and that AGGg offers principled approach to mitigating structural inconsistencies in deep GNNs.
I. Additional Experiments Larger Datasets
To further demonstrate the broad applicability of AGG we include results on three larger datasets: Arxiv (Hu et al., 2020a), Reddit (Hamilton et al., 2017), and Flickr (Zeng et al., 2020), all of which are loaded from the Deep Graph Library (DGL). As shown in Table 13, AGG consistently improves performance across all three datasets, in line with earlier findings. In all cases, the performance gains primarily stem from improvements low-degree and heterophilous nodes, highlighting that the observed benefits are indeed driven by enhanced edge-robustness. It is also worth noting that these results are obtained without any hyperparameter tuning. This suggests that further improvements are possible with tuning- as the larger performance gain observed on Arxiv in Table 1.
Additionally, we conduct the edge removal experiments described in Appendix F. The performance degradation from random edge removal significantly reduced when using AGGB, further validating its effectiveness on larger-scale datasets.
22

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
Table 14, Accuracy under random edge removal (%) in test, using the same experimental settings Table 13. AGGg consistently improves performance across all edge removal ratios and datasets, with greater gains at higher removal ratios.
Flickr
Arxiv
Reddit
GCN
GCNa
GCN
GCNa
GCN
GCNB
100%
52.50+a.15
52.84
71.06=a.10
=1.37
94.61+0.01
94.89
75%
50.95
51 61
69.58.0.08
70.59.0.07
94.472001
94.87=0.01
50%
47.49
49. 66
67 17
69.00.0..0
94.18
94.82..0.01
25%
41.01
45. 45.32
65.50_.0.6
93.52
94.38_
0%
31.7311.33
39. 19.57
15.36
54.56.0.04
38. 08.91.2.01
45.48 27
J. Generalizing Theorem 3.9 to Other GNN Architectures
In this section, we extend our discrepancy analysis-Theeore 3.9 beyond GCN broader class of GNN architectures. We provide proofs for three representative models: GraphSAGE. GIN, and GAT, which are also used in our experiments to assess the generalizability of AGGB, as presented in Section 6.4. These results theoretically demonstrate that the issue of non-optimizable edge-robustness is not specific to GCN, but is fundamental limitation shared across various GNN architectures one that AGG, is designed to address. We omit SGC from this analysis, as it can be regarded as a linearized variant of GCN and is therefore already covered by the proof in Appendix C.
J.1. GraphSAGE
The GraphSAGE layer is formulated as:
H (AGG (HLLL), A) H w, o (A + H H (
where A = D-1 A denotes the normalized adjacency matrix. Then, the discrepancy at layer / satisfies:
H (1)
H (1
la S L. |AGG (H("") A1) AGG
H (H (1-1) A2) H w H. !1 W ( l|2
L. |A (i-) A.H (1-1) W 2 + La I || ([[1)) H, 1112
2
L. (1W l1a I (1) [12) H (1-1) H (1-1) 2|2 + L. V |
(1) [2 -All2
s HH (1-1) H (1 + C2
where C1 = Lo (I) W
2 + I|w [la,, C2 = Lo w| 12 - Azll2-
J.2. GIN
The GIN layer is formulated as:
H (AGG ())) +1+(11 (1) H()H))))) = (AH + + (1+6(0) ) H()))))
where e is learnable scalar layer l. Then, the discrepancy layer satisfies:
H (1)
H 11 ~ CIIAGG (1") .A)) (H. AA)) + (1 (1+ H (-1) (1 + ) H 1111
112
s H (-
A.H! (- +| ||
(1 1111 -HH --
$ C(|4) 2 ++1+ H (11) (--)) H 11 + A2llz
< C [[ (1-1)
-1, - ( +C2
where C' the discrepancy bound of MI.P(() [ = C(I A 1|2 + + - - -A2l12-
23

---

Aggregation Buffer: Revisiting DropEdge with New Parameter Block
J.3. GAT
The GAT layer defined
h (1)
Σ JENG
oi w (P
exp(LeakyReLU(a [W', w'', ('h "
Oij ΣLEN, exp(LeakyReLU(a [ ( w'n
MoyLeakyReLUppT"hh w'th +q W', +qTWh
ΣKEN; exp(LeakyRel.U(p Wh. "-1) q ('
where a E R2F is the original attention weight vector, and p, q E F are its components such that a H|- = p-)++a(()
The induced attention matrix can be interpreted as:
A Row.Vom((axpLLallykeellddddi W' ) )A + Adiag(q ('H('-1) ))))
Since A row-stochastic, we have |2 = 1. The discrepancy is thus bounded by:
H (1)
H 11 S L. (rIIAG (1-) AA) - OGG (H2 (111) A2)]ll2
s T. I (1-1) w H 10
s W (1) | 1
(1-1)
(1-1) 2 + AH (t- (1-1) - AAH (1-1) 112
$ (. H [1-1] H (111)
112 + L. W 1
(1-1) 12l1 - -Abll
|1
(1-) -H +
where C₁ = Lo
12,, C2 = C1IVIII
- - -A112.
24


Reviews:
[{"review_id": "8KnBsMLPEH", "reviewer": "Reviewer_G3VD", "summary": "This paper analyzes DropEdge, which is widely used in GNNs. it shows that DropEdge has limited effectiveness for GNNs. Through theoretical analysis, the authors show the limitation comes from fundamental constraints in GNN architectures. They propose \"Aggregation Buffer\" (AGGB), a parameter block that can be added to any pre-trained GNN and trained with DropEdge. AGGB addresses structural inconsistencies in graphs, improving performance on 11 benchmark datasets while effectively mitigating common GNN problems like degree bias and structural disparity.\n\n## After rebuttal\n\nI am satisfied wth the authors' response.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your detailed review and insightful questions. We hope the responses below address your concerns.\n\n**Revising the sentence explaining the way of data split**\n\nThank you for your thoughtful suggestion. We acknowledge that for datasets such as Cora, Citeseer, and Pubmed, the settings you mentioned are more commonly used. \nIn the final version, we will revise it to:\n``Experiments are repeated 10 times, each with a independently randomized 10\\%/10\\%/80\\% split for training, validation, and test, respectively.\"\nWe hope this clarification addresses your concern.\n\n**Discussions related to other data augmentation techniques in graph, non-dropping(e.g G-Mixup) and adaptive dropping(e.g NeuralSparse and xAI-Drop) methods.**\n\nG-Mixup is inspired by Mixup in computer vision and generates synthetic graphs by interpolating between estimated graphons of different groups. While effective for graph classification, it is not directly applicable to node-level tasks—our focus—since node classification typically involves a single large graph rather than many samples.\nNeuralSparse trains a dedicated sparsification network to remove task-irrelevant (noisy) edges deterministically. In this sense, it functions more as an architectural variant preceding the GNN than as a stochastic augmentation method. \n\nxAI-Drop drops nodes with the low fidelity scores in high probability and can be viewed as a parametric, biased variant of DropNode. This method also produces reduced subgraphs (Definition 3.2), and such adaptive strategies could replace DropEdge in our framework to yield richer structural signals and potentially boost AGG$_B$’s performance. \nWe view this integration as a promising direction for future work and will include this discussion in the final version.\n\n**W1. Could the Aggregation Buffer approach be effectively applied to other common GNN tasks such as graph classification and link prediction?**\n\nFor graph classification, the readout function aggregates node representations into a graph-level output.\nWe can view the input as a set of rooted subgraphs, $S\\_i ={\\\\{G_j\\\\}}^{m}\\_{j=1}$, where $m$ is the number of nodes in graph $i$. \nThe classification objective can then be expressed as $D_{KL}(P(y_i|S_i)||Q(y_i|S_i))$.\nSince DropEdge produces a set of reduced subgraphs $\\tilde{S}_i$, our bias–robustness framework can be applied similarly. \n\nFor link prediction, framed as a binary classification task for edge existence, the model input can be considered as a pair of rooted subgraphs, $\\pi_{(u, v)} =\\{G_u, G_v\\}$. \nThe objective becomes $D_{KL}(P(y_{(u,v)}|\\pi_{(u,v)})||Q(y_{(u,v)}|\\pi_{(u,v)}))$, with data augmentation defined analogously on, $\\tilde{\\pi}_{(u,v)}$.\n\nAlthough these tasks share the bias–robustness trade-off perspective, each requires task-specific adaptations. In graph classification, the set of node representations at each intermediate layer varies with node dropping, making discrepancy between different-sized sets hard to define. In link prediction, operations such as dot product, absolute difference, Hadamard product, or concatenation followed by an MLP require separate theoretical treatment. Thus, while the bias–robustness trade-off view can be extended, AGG$_B$’s layer-wise correction mechanism demands new theoretical insights and task-specific methods—a promising direction for future work.\n\n**W2. To what extent can your findings regarding discrepancy bounds be generalized to other GNN architectures?**\n\nThank you for this question. We extended our discrepancy analysis beyond GCNs by considering the more generalized layer-wise update:\n$$\n\\mathbf{H}^{(l)} = \\sigma(\\mathbf{A}^* \\mathbf{H}^{(l-1)} \\mathbf{W}_1^{(l)} + c \\mathbf{H}^{(l-1)} \\mathbf{W}_2^{(l)}),\n$$\nwhere $A^*$ is a (possibly normalized) adjacency matrix and $c$ is a constant. Under this formulation, Theorem 3.9 still holds, with constants $C_1$ and $C_2$ depending on model parameters and adjacency differences, can be shown by extending the arguments in our proof at Appendix D.\nThis abstraction encompasses a broader GNN architectures. For instance, inductive GCN corresponds to $A^* = (D+I)^{-1}A$ and $c=0$; GraphSAGE uses $A^* = D^{-1}A$ and $c=1$; GIN sets $A^* = A$ and $c = 1 + \\epsilon^{(l)}$; and for GAT, with $c=0$ and we can bound the norm of $A^*$ due to its row-stochasticity.\nThus, our theoretical findings regarding discrepancy bounds apply to a wide range of GNN architectures. A formal proof for these cases will be included in the final version.\n\nThank you again for your insightful review and for taking the time to consider our responses.", "claims_and_evidence": "I found the following claims in the paper. \n1.  DropEdge has limited performance gains in supervised learning tasks due to fundamental limitations in GNN architectures. This has been well verified with both theoretical analysis and empirical evidence\n2.  The limited effectiveness of DropEdge stems from the AGG operation in GNNs and its inability to maintain consistent representations under structural perturbations. The authors develop a theoretical framework using discrepancy bounds (Theorems 3.8 and 3.9) to show that unlike MLPs, GCNs cannot establish a constant discrepancy bound independent of input when adjacency matrices differ.\n3. Aggregation Buffer (AGGB) effectively addresses GNN limitations. This is also well supported with both theory and emprical results. \n4. AGGB consistently improves performance across different GNN architectures and datasets.  Comprehensive results in experiment study support this claim.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria in this paper are well-justified and appropriate for addressing GNN robustness issues. The Aggregation Buffer design directly targets the identified theoretical limitations in GNN architectures with its degree-normalized linear transformation satisfying both edge-awareness and stability conditions. The two-step training approach is new and sound. \n\nFor evaluation, the paper employs a comprehensive approach using 11 diverse benchmark datasets, comparing against relevant baselines, The thorough ablation studies examine different AGGB architectures, loss functions, hyperparameter variations, and component contributions.", "theoretical_claims": "Yes, I briefly check the theorical claims, but not in detail. I don't find errors.", "experimental_designs_or_analyses": "Yes.   I am convinced with the experimental design. Almost all benchmark datasets with different sizes are used.   DropEdge and other drop based methods like Drop Message are included as baselines.  I am convinced by the experiments.  The ablation studies and in-depth analysis are also comprehensive. \n\nI have a minor concern/suggestion. \nThe authors mention that \"use a 10%/10%/80% split for training, validation, and test, which is common for semi-supervised learning.\"  I suggest the authors revise this sentence.  Since in the literature, they usually don't use this setting. For example, the cora, they often have two settings, one is with 20 samples for training per class; the other setting is with 1000 for testing, 500 for validation, and the others for training.", "relation_to_broader_scientific_literature": "This paper's work on improving GNN robustness through Aggregation Buffer connects to several research threads in graph learning. It extends theoretical understanding of DropEdge by providing a formal analysis of its limitations, building on discrepancy bound concepts commonly used in domain adaptation. The paper's findings on GNN vulnerability to structural perturbations align with literature on GNN over-smoothing. Its two-stage training approach with parameter freezing shares conceptual similarities with knowledge distillation techniques. By framing various structural inconsistency problems (degree bias, heterophily) as manifestations of edge-robustness limitations, the paper offers a unifying perspective that connects previously separate research directions in GNN architecture design.", "essential_references_not_discussed": "I think the related work part is already good. There are no other essential references that should be discussed. \n\nI think it would be better if the author also included some discussions on general data augmentation techniques in graphs, not limited to random drop-based.  For example, the ones based on mixup.\n\n[1] G-Mixup: Graph Data Augmentation for Graph Classification (ICML 22)\n\nBesides, for the drop-based ones, I also suggest adding some references about adative dropping, like \n\n[1] Robust Graph Representation Learning via Neural Sparsification [ICML 20]\n[2] Learning to Drop: Robust Graph Neural Network via Topological Denoising (WSDM 21)\n[3] xAI-Drop: Don't use what you cannot explain (LOG 24)", "other_strengths_and_weaknesses": "Strengths\n1.Strong theoretical foundation that connects the empirical limitations of DropEdge to fundamental properties of GNN architectures\n2. Novel characterization of the bias-robustness trade-off in GNN training that explains previously observed phenomena\n3. The proposed AGGB is simple yet effective, requiring minimal computational overhead\n4. Extensive empirical validation across diverse datasets demonstrates practical utility\n\nWeaknesses\n\n1. The paper focuses exclusively on node classification tasks; application to other GNN tasks\n2. The theoretical analysis primarily focuses on GCNs; stronger theoretical connections to other GNN architectures would strengthen the paper", "other_comments_or_suggestions": "I don't have other comments.", "questions_for_authors": "1.  the current paper focuses on node classification tasks, but could the Aggregation Buffer approach be effectively applied to other common GNN tasks such as graph classification and link prediction? Would the theoretical insights about discrepancy bounds and the bias-robustness trade-off transfer directly to these tasks, or would they require significant adaptation?\n2. the theoretical analysis primarily focuses on GCNs,  to what extent can your findings regarding discrepancy bounds be generalized to other GNN architectures like GAT, GraphSAGE, or GIN?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iXKH4mbZpi", "reviewer": "Reviewer_jKJV", "summary": "This paper revisits DropEdge, a data augmentation technique for GNNs that randomly removes edges to enhance robustness. While DropEdge helps mitigate overfitting, its performance gains in supervised learning are limited due to an inherent inductive bias in GNN architectures. To address this, the authors propose Aggregation Buffer ($AGG_B$), a parameter block that improves GNN robustness and enhances DropEdge’s effectiveness. $AGG_B$ can be integrated as a post-processing step in any GNN model. Empirical results on 11 node classification benchmarks show that $AGG_B$ significantly improves accuracy and mitigates degree bias and structural disparity. The paper provides a theoretical analysis of DropEdge’s limitations and demonstrates that $AGG_B$ serves as a unifying solution to structural inconsistencies in graph data.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful review. Our response is organized around three points: (1) novelty, (2) ablation study on the AGG$_B$ design, and (3) additional experiments.\n\n**Q1. The novelty of the method is limited. For example, two-stage training with DropEdge is proposed in TUNEUP.**\n\nThere were previous works that considered degree bias and structural disparity as separate issues.\nOur main contribution is to offer a new perspective, reframing these issues as instances of a broader problem: **structural inconsistency**.\nWe propose AGG$_B$, which directly addresses this general problem.\nTo the best of our knowledge, no previous work has solved these two problems at once or even considered these problems together, but our AGG$_B$ consistently and significantly outperforms the approaches designed specifically for certain problem.\n\nTUNEUP performs two-stage training to utilize the pseudo-labels from a classifier for semi-supervised node classification.\nThe reason why they used DropEdge is to reduce the effect of overfitting caused by the imperfect knowledge of pseudo labels; our approach is totally different.\nOur two-stage training is motivated by our theoretical results showing that existing GNNs are inherently impossible to solve the structural inconsistency.\nTo bypass the inherent limitation of GNNs, we separate robustness optimization into a second stage and introduce a carefully designed parameter block, AGG$_B$, specifically tailored to optimize robustness effectively.\n\n**Q2. Is the performance improvement simply due to the use of $H^{(0:l-1)}$?**\n\nNo. In response to your suggestion, we strengthened our ablation study ([link - Table A, B](https://shorturl.at/2cblM)) by including a single-layer  variant, $(D+I)^{-1}H^{l-1}W^{l}$ and extending across all datasets (beyond the 4 originally used).\nThis variant satisfies conditions C1 and C2 while limiting the usable information to the immediate previous layer. \nThe results show that while alternative layer architectures provide gains under our training scheme and loss, our original AGG$_B$ consistently works the best.\nIt is also noteworthy that this experiment has a comparison with JKNet-style block, which also uses the information from all previous layers, and our AGG$_B$ consistently outperforms it.\nFollowing your concerns, we also replaced Chameleon with its filtered version and added filtered Squirrel. The overall performance trends remain consistent.\n\nIn fact, the reason why we chose to use all previous layers is not because of performance.\nIf AGG$_B$ fails to fully resolve the structural discrepancies at intermediate layers, these inconsistencies may propagate into deeper layers, making the effect of AGG$_B$ partially reflected.\nBy referencing earlier representations, AGG$_B$ can access less corrupted information.\nWe will add more in-depth discussion on the choice of our parameter block in the final version.\n\n**Q3. Marginal performance improvement and more additional experiments is requires**\n\nWe believe that a difference in experimental setups should be considered when evaluating the significance of improvements.\nUnlike many studies that train GNNs from scratch, our method is applied to trained GNNs. \nFew approaches—such as curriculum learning (e.g., TUNEUP) or graph augmentation (e.g., GraphPatcher)—operate in this setting.\nWhile prior works often fix hyperparameters for base GNNs, we performed extensive grid searches, demonstrating that our method yields gains beyond what hyperparameter tuning can achieve—by addressing fundamental architectural limitations.\nAlthough this setup may boost base accuracy and lead to smaller apparent gains, we believe it more accurately reflects the true effectiveness and robustness of our method.\n\nFollowing your advice, we conducted three additional experiments:\n\n(1) **Performance under edge removal** ([link - Table C](https://shorturl.at/GjlH9)) :\nWe directly evaluated robustness driven by AGG$_B$ under random edge removal. AGG$_B$ significantly outperformed standard GCNs—even those trained with DropEdge—demonstrating its edge robustness.\n\n(2) **Experiments with deeper GCNs** ([link - Table D](https://shorturl.at/bhlls)) :\nAGG$_B$ improved performance in 28 out of 30 configurations, with larger gains at greater depths, where increased aggregations makes models more vulnerable to structural inconsistency. Even on GCNs trained with DropEdge, AGG$_B$ boosted performance in 28 out of 30 cases, highlighting its distinct mechanism.\n\n(3) **Experiments on larger datasets** ([link - Table E, F](https://shorturl.at/DvhF3)) :\nOn larger datasets, AGG$_B$’s performance remained consistent with earlier observations, underscoring its broad applicability.\n\nWe appreciate your valuable insights and important concerns of our work. We will incorporate these additional findings in the final version. Thank you again for your detailed feedback.", "claims_and_evidence": "The proposed \\( AGG_B \\) takes \\( H^{(0:l-1)} \\) as input, rather than the standard 1-hop neighborhood representation \\( H^{l-1} \\). It is well known that incorporating \\( H^{(0:l-1)} \\) can enhance performance, as demonstrated in works like JKNet. However, this raises an important question: **Is the performance improvement due to the aggregation function and loss function introduced by the authors, or simply due to the use of \\( H^{(0:l-1)} \\)?**  \n\nTo clarify this, I strongly recommend an **ablation study** where everything remains unchanged except that \\( H^{l-1} \\) is used instead of \\( H^{(0:l-1)} \\). This would help isolate the true contribution of \\( AGG_B \\). Without this analysis, the authors' claim remains inconclusive.", "methods_and_evaluation_criteria": "1. Novelty of the method is limited. For example, two stage training with dropedge is proposed in Tuneup [1]. \n2. the Chameleon dataset used in the paper is known to be problematic[2]. Please use the filtered dataset instead.\n\n\n[1] Hu, Weihua, et al. \"TuneUp: A Simple Improved Training Strategy for Graph Neural Networks.\" arXiv preprint arXiv:2210.14843 (2022).\n[2] Platonov, Oleg, et al. \"A critical look at the evaluation of GNNs under heterophily: Are we really making progress?.\" arXiv preprint arXiv:2302.11640 (2023).", "theoretical_claims": "No, i did not check the proofs in the appendix.", "experimental_designs_or_analyses": "1. The performance improvement introduced by \\( AGG_B \\) appears to be marginal, with gains of less than 1% on many datasets. Considering the standard deviation, it remains inconclusive whether \\( AGG_B \\) is genuinely effective.  \n   - I suggest the authors evaluate \\( AGG_B \\) on deep GNNs, as DropEdge tends to perform better with increased depth. This would provide a clearer understanding of its impact.  \n\n2. The authors make claims regarding heterophilous graphs, yet they only test on two heterophilous datasets, and one of which is problematic.  \n   - A broader evaluation on more diverse heterophilous datasets is necessary to support these claims.  \n\n3. Given the marginal performance improvements and large standard deviations, it would be beneficial to include experiments on larger-scale datasets to assess \\( AGG_B \\)’s scalability and effectiveness.  \n\nOverall, the results presented are not convincing, and additional experiments—especially on deeper GNNs, more heterophilous datasets, and larger-scale benchmarks—are necessary to substantiate the claims.", "relation_to_broader_scientific_literature": "Two stage training of GNN with the use of Dropedge is proposed in Tuneup. The training framework is similar.", "essential_references_not_discussed": "N.A.", "other_strengths_and_weaknesses": "The writing and presentation are clear and well-structured.", "other_comments_or_suggestions": "N.A.", "questions_for_authors": "please see weaknesses above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HiVJUfjgs3", "reviewer": "Reviewer_Yzw2", "summary": "This paper analyzes the robustness of GNN under dropping edges and proposes Aggregation Buffer (AGGB) as a solution, which enhances the robustness of GNN through a two-step training strategy while maintaining the knowledge of the original model. AGGB optimizes the shortcomings of DropEdge and improves the performance of GNN under unstable graph structures.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your detailed review and valuable questions. We have reordered your questions since we believe Q1 and Q3 are closely related. We hope the following responses adequately address your concerns.\n\n**Q2. Does the current design of AGG$_B$ which relies on all preceding representations lead to information redundancy or introduce unnecessary noise? Is there a more streamlined way to integrate this information?**\n\nThank you for raising this point. \nThere can be information redundancy in our design, but our rationale for integrating representations from all previous layers is to minimize the propagation of structural discrepancies.\nIdeally, AGG$_B$ would fully correct inconsistencies at each layer; however, in practice, some discrepancies may remain unresolved in intermediate layers and can accumulate in deeper ones.\nRelying solely on the immediately preceding layer $H^{(l-1)}$ risks carrying these issues forward, whereas leveraging all prior representations $H^{(0;l-1)}$ allows AGG$_B$ to access earlier, less corrupted information for more robust corrections.\n\nFor detailed analysis, we have strengthened our ablation study ([link - Table A, B](https://shorturl.at/2cblM)) (a) by including a single-layer variant $(D+I)^{-1}H^{l-1}W^{l}$ of AGG$_B$ and (b) running it for all datasets (beyond the 4 originally used).\nAlthough this variant improves GNN performance in most cases, our original design consistently yields stronger results across all datasets and achieves the highest overall ranking. \nDeveloping a more streamlined integration that minimizes information redundancy and noise is a promising future direction of this work.\n\n**Q1. If there is still a large deviation between Q and the true distribution P, is it reasonable to assume that Q is sufficiently close to P?**\n\nThank you for this insightful question.\nThe assumption $Q \\approx P$ is used in our approximation between Eq. (3) and Eq. (4):\n$$\nE_{P}[ \\log Q(y_i |G_i) - \\log Q(y_i|\\tilde{G_i})] \\approx D_{KL} (Q(y_i |G_i) \\Vert Q(y_i|\\tilde{G_i}))\n$$\nThis approximation is adopted because the true distribution P is inaccessible.\nWe agree that assuming $Q \\approx P$ is generally not valid when Q significantly deviates from P. \nHowever, as the training proceeds with the bias term $D_{KL}(P(y_i|G_i) \\Vert Q(y_i|G_i))$ optimized, it brings $Q$ closer to $P$, at least on the training distribution. \nFurthermore, our two-step training scheme leverages this assumption in loss function only after base GCN is trained, which makes the assumption more reliable in practice.\n\nTo support the reliability of our framework, we conducted an experiment to indirectly assess the true distribution $P$ using the labels by the below formulation.\n$$\n\\frac{1}{N} \\sum_{i=1}^N \\sum_{c=1}^{C} y_i(c) (\\log Q(c|G_i) - \\log Q(c|\\tilde{G}_i))\n$$\nAlthough this expression uses one-hot ground truth labels (thus only sampling one class per node) and the labels include noise, it still offers a useful proxy to assess the validity of the approximation.\n\nOur results ([link - Figure A](https://shorturl.at/MeZ73)) show that its shape closely mirrors that of the approximated KL divergence, indicating that our approximation captures trends well. \nFurthermore, AGG$_B$ shows robust optimization behavior even under this label-based approximation, demonstrating its effectiveness in terms of robustness optimization.\nWe will include this discussion in the final version.\n\n**Q3. Currently, the method trains GNN first and then $AGG_B$. Is it possible to introduce a mechanism for training $AGG_B$ when pre-training GNN?**\n\nThis is an insightful question that relates closely to the assumption discussed above. \nThe assumption $P \\approx Q$ is used once more in the bias term of Eq. (5) due to inaccessibility to true distribution.\n$$\nD_{KL} (P(y_i |G_i) \\Vert Q_B(y_i|G_i)) \\approx D_{KL} (Q(y_i |G_i) \\Vert Q_B(y_i|G_i))\n$$\nIn our two-step scheme, the pre-trained $Q$ is optimized to be close to $P$, making this approximation more valid, especially when restricted to training samples (Sec. 4.3)\nHowever, if AGG$_B$ were trained jointly with the GNN from scratch, $Q$ would initially deviate significantly from $P$, and the robustness-controlled loss could interfere with the optimization of the standard loss, leading to suboptimal guidance.\n\nNonetheless, we agree that incorporating AGG$_B$ into the end-to-end training of GNNs would enhance the applicability and elegance of our approach. \nAs noted in our conclusion, this is a promising direction for future work, and we are actively exploring strategies to integrate it into joint training.\n\nThank you again for your insightful reviews and for taking the time to read our response.", "claims_and_evidence": "The main claim of the paper is that GNN cannot effectively cope with the adjacency matrix perturbations caused by DropEdge due to its aggregation operation , resulting in performance degradation. To this end, the authors proposed AGGB, a module that can enhance the robustness of GNN, aiming to solve this problem. In the experimental section, the authors verified the effectiveness of AGGB through a series of comparative experiments. The experimental results provide evidence to support the claims in the paper.", "methods_and_evaluation_criteria": "The proposed method mainly includes Aggregation Buffer (AGGB), which is used to enhance the robustness of GNN under structural perturbations. The proposed evaluation criteria is appropriate.", "theoretical_claims": "In the paper, several key theoretical claims are proposed. I checked the claims. In general, the theoretical claims are effectively supported by mathematical derivation and theoretical proof.", "experimental_designs_or_analyses": "The experimental datasets are common datasets for semi-supervised learning. The evaluation method of degree bias and structural disparity is one of the key innovations of the experimental design in the paper. By evaluating the performance of the head node and the tail node respectively, the robustness of the model under different degree distributions can be better measured. For structural differences, nodes are divided into homogeneous nodes and heterogeneous nodes based on their homogeneity ratio. This is also an innovation.", "relation_to_broader_scientific_literature": "GNN is widely used in various graph data tasks. DropEdge (Rong et al., 2019) proposed to enhance the robustness of the model by randomly deleting edges in the graph. However, this method only trains the model by perturbing the graph structure, and does not fundamentally solve the sensitivity of the GNN structure to perturbations. Based on this, this paper further proposes to introduce the AGGB module to correct the performance of GNN under perturbations, so that the model can adapt to different graph structure changes and improve robustness.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\n1. The proposed method is a novel approach for improving the robustness of GNNs under graph perturbations, as it addresses an important issue of structural changes in graph data.\n\n2. The paper does a good job of clearly explaining the theories behind the designation and shows why existing methods fail to address these issues. \n3. The experimental design is solid, with a wide range of datasets and a novel evaluation framework, including tests for robustness under GNNs. \n\nWeaknesses:\n\n​There are no major weakness in this paper.", "other_comments_or_suggestions": "See the questions.", "questions_for_authors": "1. If there is still a large deviation between Q and the true distribution P, is it reasonable to assume that Q is sufficiently close to P?\n2. In the current design, AGGB relies on the representations of all the first 𝑙 layers and the adjacency matrix. Does this design lead to information redundancy or introduce unnecessary noise? Is there a more streamlined way to integrate this information?\n3. Currently, the method trains GNN first and then AGGB. Is it possible to introduce some AGGB mechanisms when pre-training GNN?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zvRwYsWSoT", "reviewer": "Reviewer_Zx9t", "summary": "This paper revisit dropedge, it claims the robustness of GNNs will grow bad during training, which yields poor performance. This paper propose aggregation buffer, a block designed to address this problem.\n\n## Update after rebuttal\n\nI recommend accept as a poster.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your detailed review and valuable questions. We hope the following responses address your concerns.\n\n**Why the two conditions are enough for the bias-robustness trade-off is unclear.**\n\nThank you for raising this insightful point. \nOur two conditions and the layer-wise correction mechanism of AGG$_B$ are motivated by our discrepancy-bound analysis (Sec. 3.4). \nCondition C1 encourages AGG$_B$ to adapt to structural variations, while C2 ensures stability by minimizing unnecessary changes when the graph structure is stable. \nTogether, these conditions mitigate representation inconsistencies caused by structural variations in a *layer-wise manner*.\n\nEstablishing a rigorous theoretical link between layer-wise corrections and robustness—defined on the *final output of a GNN*—is indeed challenging.\nHowever, it is intuitive that consistent corrections at each layer yield a more robust final representation—a notion supported by our empirical results. \nMoreover, in the case of a single-layer GNN, the layer-wise and final-output views coincide, offering a clear example where this connection holds.\n\n**W1. The improvements seem marginal.**\n\nWe believe that a difference in experimental setups should be considered when evaluating the significance of improvements.\nOur method aims to enhance trained GNNs—similar to curriculum learning (e.g., TUNEUP) or graph augmentation (e.g., GraphPatcher)—rather than training GNNs from scratch.\nWhile prior works often use fixed hyperparameters for base GNNs, we performed a grid search even for the base models, boosting the *base accuracy* to improve upon.\n\nAlthough this setup may yield smaller apparent gains, it more accurately reflects the true robustness and effectiveness of methods.\nNotably, our approach is the only method that consistently improves performance across all datasets, suggesting that these gains stem from architectural changes--the integration of AGG$_B$--rather than hyperparameter tuning.\nThese results support our view that addressing structural inconsistency offers further opportunities to enhance GNN performance.\n\n**W2. The paper aims to improve the robustness of GNNs, but the experiments are conducted on the original graph.**\n\nThank you for your helpful suggestion.\nWe focused on original graphs because our target is robustness among nodes with different structural properties within the same graph, rather than across graphs. \nTo evaluate this, we measured performance across groups based on node degrees (low vs. high) and structural roles (heterophilic vs. homophilic), which are closely linked to structural inconsistency. \nOur method showed significant gains for low-degree and heterophilic nodes, supporting the claim that AGG$_B$ improves edge robustness. \n\nNevertheless, we agree that testing under graph perturbation is informative. \nIn response to your feedback, we conducted additional experiments using random edge removal in test graphs ([link - Table C](https://shorturl.at/GjlH9)).\nAGG$_B$ significantly improves the standard GCN—even those trained with DropEdge—demonstrating its robustness benefits.\nFurthermore, GNNs trained with DropEdge did not retain performance any better than those without it, reinforcing our claim that DropEdge alone is insufficient for robustness due to inherent inductive bias.\n\n**Q1: There is a recent paper [1] that report high performance of classic GNNs, (e.g. 85.10 on Cora). Is there difference between your setup and theirs?**\n\nWe carefully reviewed [1] and identified several key differences in experimental setups.\nIn our work, we use 10 random splits per dataset, running one trial per split. \nWe select hyperparameters based on validation results from 5 of these splits to avoid overfitting to a specific partition.\n\nIn contrast, [1] employs fixed public splits—using a single public split for Cora—with 5 runs using random weight initializations.\nThis approach may yield higher accuracy and lower variance by tuning specifically for that split, but it might not generalize well to other partitions.\nThis approach may yield higher accuracy and lower variance due to tuning specifically for that split, but it might not generalize well to other data partitions.\n\nAdditionally, their hyperparameter search space is broader, including batch/layer normalization, residual connections, number of layers, additional linear transformations before and after GNN layers, and maximum epochs.\nOur search includes dropout, hidden dimension, learning rate, and weight decay, following standard practices in previous GNN literature [2]. These differences likely explain the performance gap.\n\nThank you again for your insightful review and for considering our response.\n\n[1] Classic gnns are strong baselines: Reassessing gnns for node classification.\n\n[2] Pitfalls of Graph Neural Network Evaluation", "claims_and_evidence": "Yes.\n\nThe main claim is that Dropedge is helpful for the robustness but harmful for the bias. The evidence of this claim is in Figure 2. And this claim is also intuitive and sounds correct to me (I actually know the fact Dropedge is helpful for the robustness from a paper called Dropmessage).", "methods_and_evaluation_criteria": "Yes.\n\nThe method is to add a new aggregation block that satisfied two conditions (actually one condition, since C2 implies C1). From Figure 2 we see this method works. But in theory, why these two conditions is enough for the bias-robustness trade-off is a bit unclear.", "theoretical_claims": "Yes. No obvious issue.\n\nTheorem 4.1 basically says the proposed block satisfied the two condition, I have checked the correctness.", "experimental_designs_or_analyses": "Yes. No obvious issue.\n\nTable 1 uses strong baselines, Table 2 uses some different encoders. The authors also include results of high-degree and low-degree nodes, I do not know why this is necessary, but it is harmless. The overall experiments are decent, at least in my batch.", "relation_to_broader_scientific_literature": "The paper is related to DropEdge, JKNet.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "S1: The paper is clear and the analysis seems reasonable.\n\nS2: The experiments are comprehensive.\n\nW1: The improvements seem marginal.\n\nW2: The paper aims to improve the robustness of GNNs, but the experiments are conducted on the original graph.", "other_comments_or_suggestions": "No.", "questions_for_authors": "Q1: There is a recent paper [1] that report high performance of classic GNNs, e.g., the raw GCN achieves 85.10 on Cora. Is there difference  between your setup and theirs?\n \n[1]Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes Aggregation Buffer (AGGB), a simple yet effective parameter block designed to enhance GNN robustness by addressing the limitations of DropEdge. The approach is backed by solid theoretical analysis using discrepancy bounds and is compatible with various GNN architectures.
The authors provide comprehensive experiments across multiple datasets and architectures, including deeper and larger models, and address reviewer concerns thoroughly in the rebuttal. While some reviewers noted that performance gains are modest, the conceptual novelty and practical utility of AGGB make the contribution meaningful.
Reviewer jKJV has questioned the novelty of the proposed AGGB relative to TuneUp, yet, the authors have convincingly highlighted the differences in the approach and particularly the purpose. 

Final recommendation: Accept (poster). The paper offers a well-motivated, general solution to structural inconsistencies in GNNs and is of value to the ICML community.
Decision: Accept (poster)