Beyond Random Masking: When Dropout meets Graph Convolutional Networks
Authors: Yuankai Luo
Xiao-Ming Wu
Hao Zhu
Venue: ICLR 2025 Poster (2025)
Keywords: Graph neural networks
Dropout
Primary area: learning on graphs and other geometries & topologies

Abstract: Graph Convolutional Networks (GCNs) have emerged as powerful tools for learning on graph-structured data, yet the behavior of dropout in these models remains poorly understood. This paper presents a comprehensive theoretical analysis of dropout in GCNs, revealing that its primary role differs fundamentally from standard neural networks - preventing oversmoothing rather than co-adaptation. We demonstrate that dropout in GCNs creates dimension-specific stochastic sub-graphs, leading to a form of structural regularization not present in standard neural networks. Our analysis shows that dropout effects are inherently degree-dependent, resulting in adaptive regularization that considers the topological importance of nodes. We provide new insights into dropout's role in mitigating oversmoothing and derive novel generalization bounds that account for graph-specific dropout effects. Furthermore, we analyze the synergistic interaction between dropout and batch normalization in GCNs, uncovering a mechanism that enhances overall regularization. Our theoretical findings are validated through extensive experiments on both node-level and graph-level tasks across 14 datasets. Notably, GCN with dropout and batch normalization outperforms state-of-the-art methods on several benchmarks, demonstrating the practical impact of our theoretical insights.

Published as a conference paper at ICLR 2025
BEYOND RANDOM MASKING: WHEN DROPOUT MEETS GRAPH CONVOLUTIONAL NETWORKS
Yuankai Luo!.² & Xiao-Ming Wu² & Hao Zhu
'Beihang University, Beijing, China "The Hong Kong Polytechnic University, Hong Kong $Data6I Sydney, Australia
ABSTRACT
Graph Convolutional Networks (GCNs) have emerged as powerful tools for learn- ing on graph-structured data, yet the behavior of dropout in these models re- mains poorly understood. This paper presents a comprehensive theoretical analy- sis of dropout in GCNs, revealing that its primary role differs fundamentally from standard neural networks preventing oversmoothing rather than co-adaptation. We demonstrate that dropout in GCNs creates dimension-specifie stochastic sub- graphs, leading to a form of structural regularization not present in standard neural networks. Our analysis shows that dropout effects are inherently degree- dependent, resulting in adaptive regularization that considers the topological im- portance of nodes. We provide new insights into dropout's role in mitigating oversmoothing and derive novel generalization bounds that account for graph- specific dropout effects. Furthermore, analyze the synergistic interaction be- tween dropout and batch normalization in GCNs, uncovering a mechanism that enhances overall regularization. Our theoretical findings are validated through ex- tensive experiments on both node-level and graph-level tasks across 14 datasets. Notably, GCN with dropout and batch normalization outperforms stae-of--the-att methods several benchmarks, demonstrating the practical impact of our theo- retical insights.
INTRODUCTION
The remarkable success of deep neural networks across various domains has been accompanied by the persistent challenge of overfitting, where models perform well on training data but fail to generalize to unseen examples. This issue has spurred the development of numerous regularization techniques, among which dropout has emerged as a particularly effective and widely adopted ap- proach (LeCun et al., 2015). Introduced by Srriastava et al. (2014), dropout addresses overfitting by randomly dropping out" a proportion of neurons during training. effectively creating an ensem- ble of subnetworks. This technique has proven highly successful in improving generalization and has become a standard tool in the deep learning toolkit. The effectiveness of dropout has prompted extensive theoretical analysis, with various perspectives offered to explain its regularization effects.
Some researchers have interpreted dropout as a form of model averaging (Baldi & Sadowski 2013),
while others have analyzed through the lens of information theory (Achille & Soatto (018)). Wager et al. 2013) provided insights into dropout's 's adaptive regularization properties, and Gal & Ghahra-
mani (2016) established connections between dropout and Bayesian inference. These diverse the- oretical frameworks have significantly enhanced our understanding of dropout's 's role in mitigating overfitting in traditional neural networks. However, as the field of deep learning has expanded to encompass more complex data structures, particularly graphs, new questions have arisen regarding the applicability and behavior of established techniques. Graph Neural Networks (GNNs), espe- cially Graph Convolutional Networks (GCNs), have demonstrated remarkable performance tasks involving graph-structured data (Kipf & Welling 2017). Naturally, researchers and practitioners have applied dropout to GNNs, often observing beneficial effects on generalization (Hamilton al. 2017).
'Hao Zhu the corresponding author and led the writing of the paper.

---

Published as a conference paper at ICLR 2025
While dropout was originally designed to prevent co-adaptation of features in standard neural net- works, our analysis reveals that its primary mechanism in GCNs fundamentally different. We demonstrate that dropout's 's main contribution in GCNs is mitigating oversmoothing by maintaining feature diversity across nodes, rather than preventing co-adaptation as in standard neural networks. This finding represents significant shift in our understanding of how regularization operates in graph neural networks. Specifically, we demonstrate that:
Dropout in GCNs creates dimension-specific stochastic sub-graphs, leading to a unique form of structural regularization not present standard neural networks.
The effects of dropout are inherently dereeede,,,,,nt, with differential impacts on nodes based on their connectivity, resulting in adaptive regularization that considers the topolog- ical importance of nodes in the graph.
Dropout plays a crucial role in mitigating the oversmoothing problem rather than co- adaption in GCNs, though its effects are more nuanced than previously thought.
The generalization bounds for GCNs with dropout exhibit a complex dependence on graph properties, diverging from traditional dropout theory.
There exists a significant interplay between dropout and batch normalization in GCNs, revealing synergistic effects that enhance the overall regularization.
Our theoretical framework not only provides deeper insights into the mechanics of dropout in graph- structured data but also yields practical implications for the design and training of GCNs. We vali- date theoretical findings through extensive experiments demonstrating the practical relevance of our analysis. This work bridges a critical gap in the theoret- ical understanding of regularization in GCNs and paves the way for more principled approaches to leveraging dropout in graph representation learning. Furthermore, we validate our theoretical find- ings through extensive experiments, demonstrating that GCNs incorporating our insights on dropout and batch normalization outperform several state-of-the-ae methods on benchmark datasets. This practical success underscores the importance of our theoretical contributions and their potential to advance the field of graph representation learning.
both node-level and graph-level tasks,
RELATED WORK
Dropout in Neural Networks. Overfitting can be reduced by using dropout Hinton et al. (2012)
prevent complex co-adaptations on the training data. Since its inception, several variants have been proposed to enhance its effectiveness. DropConnect (Wan et al.) 2013) generalizes dropout by randomly dropping connections rather than nodes. Gaussian dropout Srivastava et al. (2014) replaces the Bernoulli distribution with a Gaussian one for smoother regularization. Curricul lum dropout (Morerio et al. (017) adaptively adjusts the dropout rate during training. Theoretical inter- pretations of dropout have provided insights into its success. The model averaging perspective (Baldi & Sadowski 2013) views dropout as an efficient way of approximately combining exponentially many different neural networks. The adaptive regularization interpretation (Wager et al. 2013) shows how dropout adjusts the regularization strength for each feature based on its importance. The Bayesian approximation view (Gal & Ghahramani, 2016) connects dropout to variational inference in Bayesian neural networks, providing a probabilistic framework for understanding its effects.
Regularization in Graph Neural Networks. are prone to overfitting and smoothngg (Li (Lietal al. 2018). Various regularization techniques (Yang et al. 2021 Rong et al. 2020; Fang et al. 2023 Feng et al. 2020) have been proposed to address these issues. Drophide (Rong et al. (020) randomly removes edges from the input graph dur- ing training, reducing over-smoothing and improving generalization. Graph diffusion-based meth- ods (Gasteiger et al. 2019) incorporate higher-order neighborhood information to enhance model robustness. Spectral-based approaches (Wu et al. (201)) leverage the graph spectrum to design effec- tive regularization strategies. Empirical studies have shown that traditional dropout can be effective in GNNs (Hamilton et al. 2017), but its interaction with graph structure remains poorly understood. Some works have proposed adaptive dropout strategies for GNNs (Gao & Jil 2019), but these are primarily heuristic approaches without comprehensive theoretical grounding.
Graph Neural Networks (GNNs), while powerful,
2

---

Published as a conference paper at ICLR 2025
Theoretical Frameworks for GNNs. Despite the empirical success of Graph Neural Networks
(GNNs), establishing theories to explain their behaviors is still an evolving field. Recent works have
made significant progress in understanding over-smoothing (Li et al. 2018 Zhao & Akoglu 2019
Oono & Suzuki, 2019; |ong et al. 2020), interpretability (Ying et al. 2019 Luo et al. 2020; Vu
& Thail 2020, Yuan al. 2020; 2021D, expressiveness (Xu al. 2018 Chen al. 2019] Maron
et al.
et al. 2019 Feng et al. 2022), and generalization (Scarselli et al. [0188] Du
al. 2019 Verma & Zhang 2019; Garg et 2020 Zhang et al. 2020) Oono & Suzuki 2019
; Liao et | 2020] Esser et al., 2021, Cong et al. (021). Our work aims to complement
these existing theoretical frameworks by focusing on the practical aspects of dropout in GNNs, widely used regularization technique that has not been thoroughly examined from a theoretical perspective. Previous works have provided valuable insights using classical techniques such as Vapnik-Chervonenkis dimension (Scarselli et al.] 2018). Rademacher complexity (Lv) 2021; Garg ct al. 2020). and algorithm stability (Verma & Zhang 2019). Recent efforts (Oono & Suzuki, Esser ct al. 202I have also made strides in incorporating the transductive learning schema of
2019
GNNs into theoretical analyses. We bridge the gap between theoretical understanding and practical implementation of GNNs, offering insights into how dropout affects generalization and performance in graph-structured learning tasks.
3 THEORETICAL FRAMEWORK
In this section, we develop a rigorous mathematical framework to analyze the behavior of dropout in Graph Convolutional Networks (GCNs). We begin by establishing notations and definitions, then formalize the GCN model with dropout, and finally introduce key concepts that will be central to our analysis.
3.1 NOTATIONS AND DEFINITIONS
Notations.
Let (V,E, X) be undirected graph with I'Vl nodes and |&] edges, where X nxdd represents the node feature matrix with da input features per node. We denote by A R the adjacency matrix, D dlag(deg......dee the degree matrix where deg, Σ, Aij-
and Ā D AD- the normalized adjacency matrix.
Graph Convolutional Networks (GCNs), An L-layer GCN performs the following layer-wise transformation:
H = O(AH"
where H E Bandd is the feature matrix, W Rd-ixd is the weight matrix, o()) is non-linear activation, and O X. The feature energy measures representation smoothness:
E(H
Σ (i,DE6
11
1
2181
Dropout in GCNs. For layer I, dropout applies a random mask MD e R where each element M is drawn independently from Bernoulli(1 - p). The forward pass with dropout is defined as: ij
1 M O'
HD
(3)
where @ denotes element-wise multiplication and p is the dropout probability.
Batch Normalization. When incorporating batch normalization, the layer transformation becomes:
T (-
where BN applies feature-wist normalization BN(X) o BBB and batch statistics pa. of
X-Ha +B with learnable parameters Voite

---

Published as a conference paper at ICLR 2025
Original
After Fextare
1,5olgraphh
fubegrah
Stopedo
B-UAI
ISGA
Res
Figure I: Illustration of how dropout creates dimension-specific sub-graphs. From left to right: the original graph with complete feature vectors, the graph after applying dropout (where x indicates dropped features), and the resulting sub-graphs for each feature dimension. Different colors indicate different feature dimensions, and grayed-out nodes show where features are dropped, preventing message passing along those paths in the next convolution.
3.2 DIMENSION-SPECIFE STUCHASTIC SUB-GRAPHS
We demonstrate how dropout creates dimension-specific sub-graphs in Figure ! At each iteration dropout induces dimension-specifie stochastic sub-graphs G.! (V.8,))) with:
8, ( ((,, v) E & M/ 0 and Md) 0).
(5)
The coupling between feature dropout and graph topology is captured by the feature-topology cou- pling matrix:
C = oo
0}.
(6)
which measures how dropout simultaneously affects connected nodes' features. This interaction manifests in each node's ' effective degree:
ell = IIj E NO): ak, M O and M0 Off = Σ degit
representing the actual count of node i's neighbors that maintain feature connections after dropout. We consider path p (Vp..... .. V&)) active for feature j when all nodes along the path retain this fea- ture, i.e., 1 wj wij
M M # O. To elucidate the specific impact of dropout embedding features,
introduce these concepts:
Theorem (Sub-graph Diversity). The expected number of distinct sub sub-graphs per iteration is:
BIE -j= d/I d,(1 (1 -11pp -
where d, the number of features at layer p the dropout probability, and [8] is the number of edges in the original graph (The complete proof is in the Appendix. A.7).
This theorem reveals that dropout in GCNs leads to rich set of sub- sub-graphs. providing a form of structural data augmentation unique to graph-based models. The diversity these sub-graphs increases with both the dropout probability and the number of features d, This suggests that higher-dimensional GCNs with moderate dropout rates can benefit from wider range of struc- tural variations during training, potentially leading to more robust and generalizable representations. Moreover, this mechanism allows the GCN to implicitly explore different graph structures without explicitly modifying the input graph. This could be particularly beneficial for tasks where the opti- mal graph structure is uncertain or where multiple relevant sub-structures exist within the data.
Theorem (Expected Active Features per Path). For path P of length k, the expected number of features for which is active is:
BlHactive features for P] d,(1-))**
-
This theorem demonstrates that while individual long paths are unlikely to be active for any given feature, the dineni-aimessional nature of GCNs allows for effective long-range information flow through the ensemble effect across features. This theoretical insight further supported by our empirical analysis in Appendix A.6
4

---

Published as a conference paper at ICLR 2025
3.3 DEGREE-DEPENDER DEPENDENT NATURE DROPOUT EFFECTS
The interaction between dropout and the graph structure leads to form of degree-dependent regu- larization in GCNs. This means that the effect of dropout varies based the connectivity of each node, creating an adaptive regularization scheme that considers the topological importance of nodes in the graph.
Theorem (Degree-Deppendent Dropout Effect). The expected effective degree and its variance given by:
-1 Bideg = (1 deg: and Varideg,, degi(1 (1 (1- -1-))),
(8)
where deg; is the original degree of node and p is the dropout probability.
This theorem highlights that dropout affects nodes differentially depending on their degree. High- degree nodes, typically more influential within the graph, exhibit less variation in their effective degree due to dropout, potentially resulting in more stable representations for these important nodes. This observation is empirically confirmed the analysis of a 2-layer GCN presented in Appendix A.6] Consequently, the degree-dependent nature of dropout in GCNs results in adaptive regularization, where the regularization effect naturally adjusts to the local graph structure.
Corollary (Relative Stability of High-Degree Nodes). The coefficient of variation of the effective degree, defined as CV[deg,, = Vor[deg,) 1/B|deg olf decreases with increasing node degree:
CVideg VT -1 p)²
Vdeg/(1 - p)
This corollary further confirms that high-degree nodes experience relatively less variation in their effective degree due to dropout. Figure [iglutttOillusrattes that the CV decreases as node degree increases. This degree-dependent effect distinguishes dropout in GCNs from its application in standard neural networks and suggests that the optimal dropout strategy for GCNs may need to consider the graph structure explicitly.
3.4 ROLE oF DROPOUT IN OVERSMOOTHING
Oversmoothing is a well-known issue in GCNs, where node representations become indistinguish- able as the number of layers increases. Our analysis reveals that dropout plays crucial role this context, though its effects more nuanced than previously thought.
Theorem 5 (Dropout and Feature Energy). For GCN with dropout probability the expected feature energy at layer I is bounded by:
degmax {
BIECH
)
where E(X) is the energy of the input features and W are the weight matrices (The complete proof is in the Appendix(A.2).
The derived bound demonstrates how dropout affects feature energy through the interplay of network depth ()), graph structure (through degmus and A), and weight properties (IW().) Note that this analysis only provides upper bound; the absence of lower bound in this derivation is due to limitations in bounding certain terms. We will later show that when considering batch normalization, can establish the existence of a lower bound, providing more complete characterization.
3.5
GENERALIZATION BOUNDS wITH GRAPU-SPECIIC DROPOUT ErrECTS
The unique properties of dropout in GCNs, such as the creation of stochastic sub-graphs and degree- dependent effects, influence how these models generalize to unseen data. Our analysis provides novel generalization bounds that explicitly account for these graph-specific dropout effects. offer- ing insights into how dropout interacts with graph structure to influence the model's generalization capabilities.
5

---

Published as a conference paper at ICLR 2025
Decot--a
1.3
Ohhen De v01-44
Chapr Da
100
Deatun 22400
0.4
40C
700
200
100
000
100
000
40
-BC
Figure 2: Feature energy vs dropout rates.
Figure 3: BN feature energy vs dropout rates.
Theorem 6 (Generalization Bound for L-Layer GCN with Dropout). For an L-layer GCN F with dropout probability pt at layer and Lip-ichciitzz activation function or, with probability at least 1 5 over the training examples, the following generalization bound holds:
BoIL(F(X)) -B -BSL(((())) of 1og(1/0) 1=1
Σ
Lioss Li
(l)))))()) 10 (-))
where ED is the expectation over the data distribution, Ey is the expectation the training (10)
ples, is the loss function with Lipschitz constant Loss, L = IT= constant from layer I to output, 12 is the spectral norm of the weight matrix at layer i, || is
[||)))) is the Lipschitz
the spectral norm of the normalized adjacency matrix, and XG) is the fractional chromatic number of the dependency graph G induced by the message passing structure.
This generalization bound reveals how the network's s stability depends on the loss function's Lip- schitz constant, layer-wise Lipschitz constants capturing weight effects, graph structure through Xr) feature activations, and dropout rates. This leads several key insights: First, network depth affects stability through the layer-wise Lipschitz constants 4. The multiplicative accumula- tion of weight and graph effects (TT III suggests deeper GCNs require careful regu- larization as perturbations can amplify through layers. Second, the graph structure fundamentally for GCNs due to message passing (versus X)( for MLPs), GCNs gain natural regularization from their graph structure. This effect strengthens with graph connectivity since larger x leads to better stability. Combined with the fact that the normalized adjacency matrix has bounded spectral norm (ll I 1), this provides built-in stabilizing mechanism unique to GNNs. Third, examining layer-specific terms reveals the interplay between weights ||WO),, feature magnitudes ||o A H and dropout rates pi- The contribution of each layer to the overall bound suggests that adaptive layer-wise dropout rates might be more effective than uniform dropout, particularly when certain layers process more criti- cal features. Finally, the bound mathematically explains the dropout rate trade-off through the term (pt/((1 (G)). Higher dropout provides stronger regularization but increases noise, while the graph structure (through X/) moderates this effect. This helps explain why moderate dropout rates often work best in practice, with the optimal rate depending the graph's 's connectivity pat- terns. This thcoretical insight aligns with empirical observations that GNNs often benefit more from dropout than MLPs, as the graph structure provides additional stability through (G) while allowing effective information flow via message passing.
influences stability through (G). Since X)(
3.6 INTERACTION OF DROPOUT AND BATCH NORMALIZATION IN GCNs
While dropout provides a powerful regularization mechanism for GCNs, its degree-dependent nature lead to uneven regularization across nodes. Batch Normalization (BN) offers complementary approach that can potentially address this issue and enhance the benefits of dropout. Our analysis reveals how the combination of dropout and BN creates synergistic regularization effect that is sensitive to both graph structure and feature distributions.
Theorem 7 (Layer-wiss, wise Energy Lower Bound for GCN with Dropout and BN). For an L-layer Bydi Graph Convolutional Network with dropout rate p, batch normalization parameters (8 Ya del at

---

Published as a conference paper at ICLR 2025
each layer with probability at least (1 - -8),, , the expected feature energy at each layer I satisfies:
) p p.degmin
Σ 218((1 p) d=1
1012117 (8
where 1.2..... indicates the layer, degmin is the minimum degree in the graph, (8] is the total number of edges, ① is the standard normal CDF and 5 .Y are the BN parameters for dimension d at layer 1 (The complete proof is in the Appendix A.4).
Our theoretical analysis reveals crucial interplay between dropout and batch normalization in GCNs. The lower bound feature energy combines three essential components: (I) A graph struc- tural term degao that captures the network connectivity, (2) A dropout-ing induued scaling factor _ that 2
amplifies preserved features, and (3) A BN-controlled feature activation term Σ D(8" /Y >/""))))'))))))))))))))) that establishes a non- zero energy floor. This interaction operates through several key mechanisms: (1) The BN shift parameters / directly contribute to feature energy through their squared magni- tude, while the ratio /8) /y_ determines the proportion of features preserved through ReLU activa- tion via the standard normal CDF &b Higher positive values of this ratio increase feature preserva- tion. (2) Dropout's s T-p factor enhances this feature preservation effect, creating a controlled amplifi- cation that prevents feature collapse. This amplification naturally weighted by graph connectivity, with minimum degree degmin ensuring baseline protection even for sparsely connected nodes. (3) The entire bound scales with the graph's minimum degree, illustrating how the mechanism adapts to the underlying graph structure, providing stronger guarantees for more densely connected graphs. This theoretical framework explains our empirical observations Figures 2 & 3 where batch nor- malization effectively moderates the energy dynamics in GCNs. By establishing non-zero lower bound on feature energy, BN prevents complete feature collapse regardless of weight updates, while dropout enhances feature discrimination. Their joint application creates a specialized regularization mechanism for graph-structured data, where BN's parameter-controlled feature preservation inter- acts with dropout-induced sparsity to maintain robust node representations across graph topologies.
3.7 COMPARISON WITH OTHER DROPOUT VARIANTS
Various dropout mechanisms have been proposed for GNNs, each applying masks at different stages of message passing. We formally characterize these variants through their masking operations and their effects in Table The key distinction of standard dropout lies in its masking, which creates unique bub-graph structures for each feature dimension. This leads quadratic effect on the effective degree, providing stronger regularization than other variants, While DropNode and DropEdge apply coarse-grained masks uniformly across features, and DropMessage operates at the message level, dropout's 's feature-specific approach provides finer-grained control over information flow.
Table I: Comparison of different dropout variants GNNs. Each method characterized by its masking operation M,, the resulting sub-graph formation Gt, and expected effective degree Bldeg where p is the dropout probability.
Method
Masking Operation
Sub-graph Formation
Expected Effective Degree
Droe
M, A(iMaot H W))) G, CVV VVVooropeees \ 16. flli V Varappell)
deg. IT [esoo( p)
DropEdge
M, (Messe Ax "W")
G. CV.E) Extopped)
(1 pideg,
DropMessage
M2 A(Manu @ (HO-1 D G CV. D &lMari # O1)
pldeg,
Dropout
M, MYn @ (-00) G = CV. J) E|M # 0. M. OI) (1 p) deg.
4 EXPERIMENTS
To validate our theoretical analysis, we conducted extensive experiments on a variety of datasets, considering both node-level and graph-level tasks. We implemented dropout technique several popular GNN architectures: GCN (Kipf & Welling 2017). GraphSAGE (Hamilton et al. 2017). GAT (Veličkovié et al., 2018), and GatedGCN (Bresson & Laurent 2017). For each model, compared the performance with and without dropout. Our code is available at hhttps://gitiee com/L1ooyk11999/roppouuu theory
7

---

Published as a conference paper at ICLR 2025
Table 2: Node classification results (%). The baseline results are taken from Deng et al. (2024) Wu et al. 2023). The top 1st 2nd and 3rd results are highlighted. "dp" denotes dropout.
Cora
CiteSeer
PubMed
Computer
Photo
CS
Physics
WikiCS
oghn-
products
modes
2.708
3,327
19,717
13,752
7,650
18,333
34,493
11,701
169,343
2,449,029
* odges
5,278
4,732
44,324
245,861
119,081
81,894
247,962
216,123
1,166.243
61,859,140
Metric
Accurayy
Accuracy?
Accuracyt
Accuracyt
Accuracy
Accuracy
Accuricy
Accuracy
Accuracyt
Accuracyt
GCNI
85.19 soas
73.20 200
80.32 2014
91.04 cm
94,30 um
92.22 2:1
95 97.97-2 st:
78.68 c.3
72.74 sun
79.42 2029
GPRGNN
83,17 .03s
71.86
79.75
89.32
94.49 2314
95.13
96.85
78.12
71.10
79.76
APPNP
83.32 2029
71.78
Lose
80.14
1012
90.18 can
94.32
2314
94.49
.m
96.54 cm
78.87 .m
72.34 2024
78.84 L00
IGNN
82.97 sore
71.74
80.67
83,40
89.92 +m7
92.85 e.
96.24
71.49
72.88 cus
81.79 -054
GraphGPS
-1
72.73
1m
79.94 .its
91.19 vas
95.06
93.93 vart
97.12-1
78.66
70,97 .0a
00M
NAGphormer
82.12
71.47 -33
79.73 and
91.22 .a. 95.49 +311 95.75 6:30
97.34 cm
77.16.
70.13
eas
73.55 .021
Exphormer
.135
71.63
95.35 1m 94.93 xba
78.54
00M
GOAT
83.18 133
71.99 -13
79.13 .nis
90.96 +0.
92.96
94.21 +.
96.24-1x
77.00
72.41 va.
82.00 -00
NodeFurmer
82.20
72.50
79,90
86,98
93.46
a3 95.64
96.45
74.73
59.90
73.96
SGFoemer
84.50 .033
72.60
Jazs
80.30 JoMe
92.42 1o3e
95.58 1336
95.71 1834
96.75 .:31
80.05
.04
72.63 2012
81,54 2043
Polynormer
83.25 2053
72.31
79,24
93.68
96,46
2:26
95.53
2016
97.27
80,10
73,46 site
83.82
GCN
85.22 2008
73.24 2000
81.08 1.1
93.15 .am
95.03 a
94.41 ri15
97.07 2:::
80.14 e
73.13 sum
81.87 2041
Dirichlet energy
74.671
9.934
4,452
8.020
3,765
20.241
8,966
6.109
8.021
7.771
GCN wyo dp
83.18 133 70.48 -nus
79.40 100 90.60 .as 94.10 ai
94.30 a03 96.92 1as
77.61 13 72.05 .02
77.50
Dirichlet energy
2.951
0.170
0.247
0,592
1.793
3,980
0.318
1.592
1,231
1.745
GCN BN
84.97 aoc3
72.97
.oas 80.94
92.39
94.38
93.46 a:34
96.76
..4
71,93 a01
79.37 .012
SAGE
84.14 Lo33
71.62 .023
77,86 aa
92.65 4021
95.71 1:20
95.90 6:30
97.20 2.30
80.29 2057
72.72 a01
82.69 .023
SAGE dp
83.06
69.68
.uz3
76.40
90.17 1000
94.90 .a
95.80
97.06 .tm
78.84
71.37 1021
79,82
SAGE w/o BN
83.89 sos>
71.39 s0rs
77,26 ue
92.54 va.
95.51
94.87 .015 1m
97.03 -:e
79.50 1053
71.52 san
80.91 2015
GAT
83.92
72.00
80.48
93.47
95.53
94.49
96.73
80.21
72.83
80.05
GAT w/j dp
82.58 10
71.08
79.28 .nd
92.94 .a.
93.88
94.30 .014
96.42 .m
78.67 .en
71.52 .a.
77.87 -025
GAT w/o BN
71.82
80.43
92,16 ca.
95.05 +3
93.33 em. 96.57
79.49
71.68 sn.
78.21
4.1 DATASETS AND SETUP
Datasets. For node-level tasks, we used 10 datasets: Cora, CiteSeer, PubMed (Sen et al. 2008 ogbn-arxiv, ogbn-products (Hu et al., 2020). Amazon-Computer, Amazon-Phote Coauthor-CSS Soauthor-Physics (Shchur et al. (0118), and WikiCS (Mernyci & Cangea 2020). Cora, CiteSeer, and PubMed are citation networks, evaluated using the remi-supervised setting and data splits from Kipf & Welling (2017). Computer and Photo (Shchur et al.] (0188) are co-p pprchasa networks. CS and Physics (Shchur et al., 2018) are co-authorship networks. We used the standard 60%/20%/20% training/validationneest splits and accuracy as the evaluation metric (Chen et al. 2022 Shirzad et al.) Deng al. 2024). For WikiCS, we adopted the official splits and metrics (Mernyei & Cangea,
2023
2020 For large-scall graphs. included agbn-arxiv and -ronduroducts with 0.16M to 2.4M
nodes, using OGB's 's standard evaluation settings (Hu et al. 2020).
For graph-level tasks, we used MNIST, CIFARIO (Dwivedi et al. 2023), and two Peptides datasets (functional and structural) (Dwivedi et al. 2022). MNIST and CIFARIO are graph versions of their image classification counterparts, constructed using 8-nearest neighbor graphs of SLIC superpixels. We follow all evaluation protocols suggested by Dwivedi et al. (2023). Peptides-fune involves clas- sifying graphs into 10 functional classes, while Peptides-struce regresses 11 structural properties.
All evaluations followed the protocols in (Dwivedi et al. 2022).
Baselines. Our main focus lies on the following prevalent GNNs and transformer models from Polynormer (Deng et al.) 2024): GCN (Kipf & Welling 2017). SAGE (Hamilton et al. 2017). GAT
Veličković et al. (2018). GCNII (Chen 20.20. (Veličković et al.[ APPNP Gasteiger
et al. 2018). GPRGNN (Chien et al.] (020). SGFormer (Wu et al. 2023), Polynormer (Deng et al. 2024.. GOAT (Kong et | 2023). NodeFormer (Wu et al., 2022), NAGphormer (Chen al. 2022); GT Dwivedi & Bresson (020)) Kreuzer et al. (2021) I). MGT et al. (2023). Gutteride
let al. (2023), PMixrr He et al. (022). GRIT Ma et al. (2023) GraphGPS (Rampásek)
eet al. 2022). Exphormer (Shirzad et al. 2023). CKGCN (Ma et al.) 2024), GRED (Ding et al.)
2024). Graph Mamba Behrouz & Hashemi (2024). We report the performance results of basclines
primarily from (Deng et al. | with the remaining obtained from their respective original papers or official leaderboards whenever possible, as those results are obtained by well-tuned models.
Experimental Setup. We implemented all models using the PyTorch Geometric library (Fey & Lenssen 2019). The experiments are conducted on a single workstation with RTX 3090 GPUs For nove- tasks, we adhered to the training protocols specified in (Deng al.
2024 Luo al.
8

---

Published as a conference paper at ICLR 2025
F-Norm
Foature Average Distance
Dirchkt
No Roppapa
So Desoet
85
Drspon
Inppet
10
500
600
50
350
Ckm
Ciobbrr
Polbled
Dum
Clieber
PobSfed
Chre
Cieeber
PalMdel
Figure 4: Effect of dropout on feature F-norm, average pair distance, and Dirichlet energy.
2024b a), employing BN and adjusting the dropout rate between 0.1 and 0.7. In graph-level tasks, we adopted the settings from (Tönshoff et al. 2023 Luo et al. 2025), utilizing BN with a consistent dropout rate of 0.2. All experiments were run with different random seeds, and we report the mean accuracy and standard deviation. To ensure generalizability, we used Dirichlet energy (Cal & Wang 2020 as an oversmoothing metric, which proportional our feature energy.
4.2
NODE-LEVEL CLASSIFICATION RESULTS
The node-level classification results in Table 2 not only align with theoretical predictions but also showcase the remarkable effectiveness of dropout. Notably, GCN with dropout and batch nor- malization outperforms state-of-the-art methods several benchmarks, including Cora, CiteSeer, and PubMed. This superior performance underscores the practical significance of our theoretical insights. Consistently across all datasets, models employing dropout outperform their counterparts without it, validating our analysis that dropout provides beneficial regularization in GNNs, distinct from its effects standard neural networks. The varying levels of improvement observed across different datasets support our theory of degree-depende structure. Furthermore, the consistent increase in Dirichlet energy when using dropout provides em- pirical evidence for our theoretical insight into dropout's crucial role in mitigating oversmoothing in GCNs, particularly evident in larger graphs. The complementary roles dropout and batch normal- ization are demonstrated by the performance drop when either is removed, supporting our analysis of their synergistic interaction in GCNs.
dropout effects that adapt to the graph
4.3 GRAPH-LEVEL CLASSIFICATION RESULTS
Our graph-level classification results, presented in Tables andd further validate the broad applica- bility of our theoretical framework. First. compared to recent SOTA models, observe that simply tuning dropout enables GNNs to achieve SOTA performance on three datasets and is competitive with the best single-model results the remaining dataset. Second, the significant accuracy im- provements on graph-level tasks such as Peptides-fun and CIFARIO highlight that our insights ex- tend beyond node classification. The varying degrees of improvement across different graph datasets are consistent with our theory that dropout provides adaptive regularization tailored to graph proper- ties. Third, the consistent increase in Dirichlet energy when using dropout supports our theoretical analysis of dropout's ' role in preserving feature diversity.
These results robustly validate our theory, showing that dropout in GCNs produces dimension- specific stochastic sub-graphs, has degree-dependent effects, mitigates oversmoothing, and offers topology-aware regularization. Combined with batch normalization, dropout enhances GCN per- formance on graph-level tasks, affirming the relevance and utility of our framework and suggesting directions for improving GNN architectures.
4.4 MITIGATING OVERSMOOTHING RATHER THAN CO-ADAPTATION
In traditional neural networks, dropout primarily prevents co-adaptation of neurons. However, our theoretical framework suggests that dropout in GCNs serves fundamentally different purpose: mit- igating oversmoothing rather than preventing co-adaptation. To validate this hypothesis, we exam- ined how dropout affects weight matrices in a 2-layer GCN, focusing specifically spectral norm changes (see Appendix (A.3). We further analyzed three key metrics to quantify dropout's influence feature representations, shown in Figure The left panel of Figure demonstrates that the

---

Published as a conference paper at ICLR 2025
Table 3: Graph classification results on two pep-Table 4: Graph classification results on im-
tide datasets from LRGB (Dwivedi et al. 2022). age datasets from (Dwivedi et al. 2023
Model
Peptides-fumg
Peptides-struct
Model
MNIST
CIFARIO
graphs
15,535
15,535
# graphs
70,000
60,000
Avg. # modes
150.9
150.9
Avg. nodes
70.6
117.6
Avg. edges
307.3
Avg. edges
564.5
941.1
Metric
AP
MAE
Metric
Accuricy
Accuracy 1
GT
0.63226 60n7s
0,2529 -03006
GT
90.831 ccas
59.753 -0.23
SANE
0.6439 00007
0.2545
2000012
SANS
GraphGPS
0.6535
0.2500
GraphGPS
98.051
72.298
MGT+ WavePE
0.6817 DuRe4
0,2453 a00025
MGT+ MGT+WavePE
DRew
0.7150 00044
0,2536 _..0015
DRew
Exphormer
0.6527
0.248 -.,007
Exphommer
98.550 cos
74.696
STaph-MLPMixer
0.6970 e0000
0.2475 20,9015
Staph-MLPMixx
97.422 10.110
73.961 00.300
GRIT
0.6988 nire?
0010
GRIT
98,108 mam
76.468 -0881
CKGCN
0,6952 Cotoa
0.2477
CKGCN
98.423
72.785
GRED
0.7085 DXECT
0,2503
-0.00
GRED
98.383 10002
76.853 _0.15
Graph Mamba
0.6972 tunee
0.2477
0.0009
Graph Mamba
98.392 +8.183
74.563
GCN
0.7015
0.2437
GatedGCN
98.684 10:13
76,931 0.36)
Dirichlet energy
9.649
6,121
Dirichlet energy
1.119
1.541
GCN w/o dp Dirichlet energy
0.6484 no0s
0.2541 1016
GatedGCN w/o dp
98.235 c.12
71.384 _0335
6,488
3.725
Dirichlet energy
0.987
0.845
Frohenius norm of features remains relatively stable regardless of dropout application, indicating that dropout does not uniformly scale all features. The middle panel reveals that dropout consis- tently doubles the average pairwise distance between nodes, helping maintain distinct node repre- sentations. Most significantly, the right panel shows that dropout substantially increases Dirichlet energy. This dramatic rise Dirichlet energy, compared to the modest changes in Frobenius norm and pairwise distances, provides compelling evidence that dropout enhances discriminative power between connected nodes, explaining its effectiveness preventing oversmoothing rather than sim- ply reducing co-adaptation.
4.5
COMPARISON WITH DROPOUT VARIANTS
To further explore the practical impact of these different regularization techniques, we conducted hyperparameter tuning for DropEdge, DropNode, and DropMessage on the Cora, Citeseer, and Pubmed datasets. The results, summarized in Table 5 demonstrate that while these methods yield comparable performance, traditional dropout generally performs best.
Table 5: Experimental results of different regularization methods on Cora, Citeseer, and PubMed.
ClicSeer (CCN)
Pubbied (GCN)
Cora (SAGE)
CiteSeer (SACE)
PubMed
Cora
CiteSeer
GNN
55.18 1.22
20:45 x 0.45
70-401 1.05
$3.06 0.5
59:06 = 015
70 00 =1 1.40
$2.55 = 1.47
71.08 1.42
75.25 0.58
85.2 2
4.63
$1.8 = 1.16
84.14 9.63
T1 11:02:0 0.39
77.00 1177
8342 + 1.29
72.00
M0.15 9.39
GNN Dreplize
0.38
=
77.58
$4322
71.30
84.75 78-78-000
7 1.19
5U52 0.s.
8 062
-1322-1..4
18.30 0.90
81.46
71
78.36 1:33
5 CONCLUSIONS
Our comprehensive theoretical analysis of dropout in GCNs has unveiled complex interactions be- tween regularization, graph structure, and model performance that challenge traditional understand- ing. These insights not only deepen our understanding of how dropout functions in graph- graph-structured data but also open new avenues for research and development in graph representation lcarning. Our findings suggest the need to reimagine regularization techniques for graph-based models, explore adaptive and structure-aware dropout strategies, and carefully balance local and global information in GCN architectures. Furthermore, the observed synergies between dropout and batch normaliza- tion point towards more holistic approaches to regularization in GNNs. As we move forward, this work lays a foundation for developing more robust and effective graph learning algorithms, with potential applications in dynamic graphs, large-scale graph sampling, and adversarial robustness. Ultimately, this research contributes to bridging the gap between the empirical success of GNNs and their theoretical foundations, paving the way for designing graph learning models.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Hao Zhu was supported by the Science Digital Program in Commonwealth Scientific and Indus- trial Research Organization (CSIRO). Yuankai Luo received support from National Key R&D Pro- gram of China YFB350005)))00) NSFC Grant 62172026, National Social Science Fund of China 22&ZD153, the Fundamental Research Funds for the Central Universities, State Key Laboratory of Complex & Critical Software Environment (SKLCCSE), and the HK PolyU Grant P0051029.
REFERENCES
Alessandro Achille and Stefano Soatto. Information dropout: Learning optimal representations through noisy computation. IEEE transactions on pattern analysis and machine intelligence, 40 (12):2897-2905, 2018.
Pierre Baldi and Peter J Sadowski. Understanding dropout. Advances in neural information pro- cessing systems, 26, 2013.
Ali Behrouz and Farnoosh Hashemi. Graph mamba: Towards learning on graphs with state space models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 119-130, 2024.
Xavier Bresson and Thomas Laurent. arXiv:1711.07553, 2017.
arXiv preprint Residual gated graph convnets.
Chen Cai and Yusu Wang. A note over-smoothing for graph neural networks. arXiv preprint arXiv:2006. 13318, 2020.
Jinsong Chen, Kaiyuan Gao, Gaichao Li, and Kun He. Nagphormer: A tokenized graph transformer for node classification in large graphs. In The Eleventh International Conference on Learning Representationss 2022.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. International conference on machine learning, pp. 1725-1735. PMLR, 2020.
Zhengdao Chen, Soledad Villar, Lei Chen, and Joan Bruna. On the equivalence between graph isomorphism testing and function approximation with gnns. Advances in neural information processing systems, 32, 2019.
Eli Chien, Jianhao Peng, Pan Li. and Olgica Milenkovie. Adaptive universal generalized pagerank graph neural network. International Conference on Learning Representations, 2020.
Weilin Cong, Morteza Ramezani, and Mehrdad Mahdavi. On provable benefits of depth in training graph convolutional networks. Advances in Neural Information Processing Systems, 34:9936- 9949,20211
Nima Dehmamy, Albert-LászIó Barabási, and Rose Yu. Understanding the representation power of graph neural networks in learning graph topology. Advances in Neural Information Processing Systems, 32, 2019.
Chenhui Deng, Zichao Yue, and Zhiru Zhang. Polynormer: Polynomial-eppessin graph trans- former linear time. arXiv preprint axXiv:2403.01232. 2024.
Yuhui Ding, Antonio Orvieto, Bobby He, and Thomas Hofmann. Recurrent distance filtering for graph representation learning. In Forty-first International Conference on Machine Learning, 2024.
Simon Du, Kangeheng Hou, Russ R Salakhutdinov, Barnabas Poczos, Ruosong Wang, and Keyulu Xu. Graph neural tangent kernel: Fusing graph neural networks with graph kemels. Advances in neural information processing systems, 32, 2019.
Vijay Prakash Dwivedi and Xavier Bresson. generalization of transformer networks to graphs.
arXiv preprint arXiv:20/2.09680 2020.

---

Published as a conference paper at ICLR 2025
Vijay Prakash Dwivedi, Ladislav Rampášek, Mikhail Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long range graph benchmark. arXiv preprint arXiv:2206.08 2022.
Vijay Prakash Dwivedi, Chaitanya Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. Journal of Machine Learning Research, 24(43):1-48, 2023.
Pascal Esser, Leena Chennuru Vankadara, and Debarghya Ghoshdastidar. Learning theory can (sometimes) explain generalisation in graph neural networks. Advances in Neural Information Processing Systems, 34:27043-27056, 2021.
Taoran Fang, Zhiqing Xiao, Chunping Wang, Jiarong Xu, Xuan Yang, and Yang Yang. Dropmes- sage: Unifying random dropping for graph neural networks. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 37, pp. 4267-4275, 2023.
Jiarui Feng, Yixin Chen, Fuhai Li, Anindya Sarkar, and Muhan Zhang. How powerful are k-hop message passing graph neural networks. Advances in Neural Information Processing Systems, 35: 4776-4790, 2022.
Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for semi-supervised learning on graphs. Advances in neural information processing systems, 33:22092 -2103,, 2020.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint 1XXxv:1933.02288 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
Hongyang Gao and Shuiwang Ji. Graph u-nets. In international conference on machine learning, pp. 2083-2092. PMLR, 2019.
Vikas Garg. Stefanie Jegelka, and Tommi Jaakkola. Generalization and representational limits of graph neural networks. In International Conference on Machine Learning, pp. 3419-3430. PMLR, 2020.
Johannes Gasteiger, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.0599) 2018
Johannes Gasteiger, Stefan Weißenberger, and Stephan Günnemann. Diffusion improves graph
learning, Advances in neural information processing systems, 32, 2019.
Benjamin Gutteridge, Xiaowen Dong, Michael M Bronstein, and Francesco Di Giovanni. Drew: Dy- namically rewired message passing with delay. In International Conference on Machine Learning. pp. 12522-1266, PMLR, 2023.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs Advances in neural information processing systems, 30, 2017.
Xiaoxin He, Bryan Hooi, Thomas Laurent, Adam Perold, Yann LeCun, and Xavier Bresson. A generalization of vit/mlp-mixer to graphs. In International Conference on Machine Learning, pp. 12745. PMLR, 2023.
Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arxiv 2012. arXiv preprint
05800 2012.
Weihua Hu, Mathias Fey, Marinka Zitik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:221 18 18-22133, 2020.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017. URL https:
//openreview. hwwparrvvieeenee/oumu/iiiiiieeie www.eet//ooummii-sJJuayyyyy//
12

---

Published as a conference paper at ICLR 2025
Kezhi Kong, Jiuhai Chen, John Kirchenbauer, Renkun Ni, C. Bayan Bruss, and Tom Gold- stein. GOAT: A global transformer on large-scale graphs. In Andreas Krause, Emma Brun- skill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Pro- ceedings of the 40th International Conference on Machine Learning, volume 202 of Proceed- ings of Machine Learning Research, pp. 17390. PMLR, 23-29 Jul 2023. URL https: //proceedings. mlr httpaannnsmm..prpseeeeeee 2TTess/v202/kong23ee html
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Re- thinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34:21618-21622 2021.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553))36-4444 2015.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Renjie Liao, Raquel Urtasun, and Richard Zemel. A pac-bayesian approach to generalization bounds for graph neural networks. arXiv preprint arXiv:2012.076690, 2020.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. Advances in neural information pro- cessing systems, 33:1 12:19920-1990 2020.
Yuankai Luo, Qijiong Liu, Lei Shi, and Xiao-Ming Wu, Structure-aware semantic node identifiers for learning on graphs. arXiv preprint arXiv:2405.16435, 2024a.
Yuankai Luo, Lei Shi, and Xiao-Ming Wu. Classic GNNs are strong baselines: Reassessing GNNs for node classification. In The eiiht Conference on Neural Information Processing Sys- tems Datasets and Benchmarks Track, 2024b. URL http:://ppenrevvee.nnt/foou xkl 4KdGeE.
Yuankai Luo, Lei Shi, and Xiao-Ming Wu. Unlocking the potential of classic gnns for graph-level tasks: Simple architectures meet excellence. arXiv preprint arXiv:2502.00923, 2025.
Shaogao Lv. Generalization bounds for graph convolutional neural networks via rademacher com-
plexity, arXiv preprint arXiv:2102.1023: 2021.
Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriant Puncet K Dokania, Mark Coates, Philip Torr, and Ser-Nam Lim. Graph inductive biases in transformers without message passing. arXiv preprint 1.Xiv:2305.17558 2023.
Liheng Ma. Soumyasundar Pal, Yitian Zhang, Jiaming Zhou, Yingxue Zhang, and Mark Coates. Ckgconv: General graph convolution with continuous kernels. arXiv preprint arXiv:2404. 13604, 2024.
Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman. Invariant and equivariant graph networks. arXiv preprint arXiv: 18/.099// 2018.
Péter Mernyei and Cátällia Cangea. Wiki-es: A wikipedia-based benchmark for graph neural net- works. arXiv preprint 40Xiv:2007.0290112200 2020.
Pietro Morerio, Jacopo Cavazza, Riccardo Volpi, René Vidal, and Vittorio Murino. Curriculum dropout. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3544- 3552, 2017.
Nhat Khang Ngo. Truong Son Hy, and Risi Kondor. Multiresolution graph transformers and wavelet positional encoding for learning long-range and hierarchical structures. The Journal of Chemical Physics, 159(3), 2023.
Kenta Oono and Taiji Suzuki. Graph neural networks exponentially lose expressive power for node classification. arXiv preprint arXiv: 1905. 10947, 2019.
13

---

Published as a conference paper at ICLR 2025
Ladislav Rampášek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. arXiv preprint arXiv:2205.22454, 2022.
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification. In International Conference on Learning Repre- sentations, 2020. URL http:tss//opeereeiiettt wwtp:iiwweeeeeeeeeeeeeeeeeeeeeee ftt/m//rmmm/// wiw.mmididdiiixxiidk...
Franco Scarselli, Ah Chung Tsoi, and Markus Hagenbuchner. The vapnik chervonenkis dimension of graph and recursive neural networks. Neural Networks, 108:248 259,, 2018.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad Collective classification in network data. Al magazine, 29()):33933 2008.
Oleksandr Shehur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv: /8/ 1.8./..0. 1.05868, 2018.
Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica J Sutherland, and Ali Kemal Sinop. Exphormer: Sparse transformers for graphs. arXiv preprint arXiv:2303.06147. 2023.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting The journal of machine learning research, 15(1):1929-1958 2014.
Jan Tönshoff, Martin Ritzert, Eran Rosenbluth, and Martin Grohe. Where did the gap go? reassess- ing the long-range graph benchmark. arXiv preprint arXiv:2309,03367, 2023.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference Learning Representations, 2018.
Saurabh Verma and Zhi-Li Zhang. Stability and generalization of graph convolutional neural net- works. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Dis- covery & Data Mining, pp. 3539-1548, 2019.
Minh Vu and My T Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. Advances in neural information processing systems, 33:12225-12233 2020.
Stefan Wager, Sida Wang, and Percy S Liang. Dropout training as adaptive regularization. Advances in neural information processing systems, 26, 2013.
Li Wan, Matthew Zeiler, Sixin Zhang, Yann Le Cun, and Rob Fergus. Regularization of neural networks using dropconnect. In International conference on machine learning, pp. 1058-1066. PMLR, 2013.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Sim- plifying graph convolutional networks. In International conference machine learning, pp. 6861-6877 PMLR, 2019.
Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf. and Junchi Yan. Nodeformer: A scalable graph structure learning transformer for node classification, Advances in Neural Information Processing Systems, 35:27377-24401, 2022.
Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian, and Junchi Yan. Simplifying and empowering transformers for large-graph representations. In seirth-eeventh Conference on Neural Information Processing Systems, 2023. URL https://
Seenreview.net/oore wwt.tt//ooomm...:dpxwwwww.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference Learning Representations, 2018.
Han Yang, Kaili Ma, and James Cheng. Rethinking graph regularization for graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1573, 2021.
14

---

Rui-Ray Zhang and Massih-Rezz Amini. Generalization bounds for learning under graph- dependence: a survey, Mach. Learn., 113(7):3929-399) April 2024. ISSN 0885-6125, doi: 0111077/1109944-2--06666 Published as a conference paper at ICLR 2025
Zhitao Ying. Dylan Bourgeois. Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. Xgnn: Towards model-level explanations of graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 430-438, 2020.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International conference on machine learning, pp. 12241- 12252. PMLR, 2021.
http::/::://do http:::///ooooor///////// 10. 1007 /000//1099444 024 06536-3
Shuai Zhang, Meng Wang, Sijia Liu, Pu Chen, and Jinjun Xiong. Fast learning of graph neural networks with guaranteed generalizability: one-hidden-layer case. In International Conference on Machine Learning, pp. 1126--11277. PMLR, 2020.
Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. arXiv preprint arXiv: 12223, 2019.
A APPENDIX
A.1 PROOF OF THeoReM
1
Proof. Let's approach this proof:
Step 1: For a single feature j. the probability that an edge is present in the sub-graph E. E.h is (1 - pp? both endpoints need to retain this feature.
Step 2: The probability that an edge is not present in (((,0) () is (1-p)² ppp p)2
Step 3: For a sub-graph to be identical to the original graph, all edges must be present. The proba- bility of this is: ((1 p) (1 2)
Step 4: Therefore, the probability that shhi different from the original graph (i.e., unique) is 2)261
Step 5: Define an indicator random variable X₁ for each feature j:
X, = {
if & is unique otherwise
Step 6: We have:
P(X; 1)= 1- -1- 0) (1-p)
Step 7:
The expected value of Xj is:
B[X]] P(X, 1) P 0) 22
Step 8: The total number of unique sub-graphs is Σdh Xy. By the linearity of expectation:
di
BiE 1j=
dll
X11
BXJ d(1 - (1 - 20))
This completes the proof.
15

---

Published as a conference paper at ICLR 2025
A.2 PROOF THEOREM 5
Proof. We start with the definition of feature energy:
)(H
Σ
1 10
2 1J58
Step 1: Taking the expectation:
BIE(H
B 19131.
2/
1
Step 2: Since Ea.eelllh,ll /1/ 111111111]11 2 2 deg.llb.l1?:
Σ
Bllk
n2/1111
1
Σ
BlIl
M
1- M oz
1
2|
2181
1,E8
Σ
B 1
M о 16]
2181(1 p)?
1,88
Σ
[(1 - [1(l=> (01 || (12]113] 2(1 - TT
21 p)2
LEE6
1
Σ degill 10
Tr AZ)
18
[8
where z; o(Ei Auh!-"Ww
Step 3: Since deg: S degmax for all
degmax
degmax,
degi
1 Pp.
Step 4: By ReLU non-negative homogeneity and submultiplicative property:
||
s
Step 5: By dropout scaling with probability pt
11
1 p
Step 6: By applying steps 4-5 recursively:
1 (
y-1 1 A
11
|
10 13XX
1=1
Step Combining all inequalities:
degmax
BIE(H )
W
[8
1 p
1=]
A.3 PROOF THBOREM 6
Proof. The proof proceeds in several steps:
Step 1: Dependency Graph. Let G (V, be the dependency graph where vertices V represent nodes in the graph, and an edge (i, f) e & exists nodes and j are connected through message passing via A. The graph & is fixed across all layers as it is determined by the structure of A.
16

---

Published as a conference paper at ICLR 2025
Step 2: Dropout Effect Perturbation. At layer I with dropout probability Pt. let 8 be the perturbation matrix:
80
M o -(
(1I)
P-Pi
where M has elements drawn from Bernoulli(1 pt).
Step 3: Perturbation Propagation. Let F((X) denote the network output with dropout applied up to layer I. With Li-Liischitz activation:
L
7 201
(L (L.l -l
(12)
By operator norm properties:
[IF(X) Fi-1(X)li s 1/
(13)
Step 4: Bounding Matrix Perturbation. Let of denote the (i, jitth entry of 80h By Janson's
inequality for dependent variables over G (Zhang & Amini 2024):
Σ
Σ 0 $
B
S
(14)
Taking the square root and using the definition of Frobenius norm:
1r]
(15)
Pt
10 HH
(16)
pilx
where use B[( B[(M B[M Pt-
Step 5: Loss Stability. By the Lipschitz property the loss function:
- Loss (x) F1-1(x)]1=
(17)
Looss Li La
Pj
-
(18)
(1 ()
Step 6: Final Concentration Bound. Using McDiarmid's inequality and noting the impact of message passing through (G),, with probability at least 1 6:
- -
o log(1/0)
Σ
Lioss 4. (1- 1--PPix
Pt
lo (HH
l
(19)
The bound shows that GNNs (x)()) > due to message passing) achieve better stability than MLPs (x)(G) = 1, no message passing), with the benefit increasing with graph connectivity.
A.4
PROOF OF THEOREM
Proof. Step 1: Start with feature energy and node representation:
E(H") =
Σ 1 ())
11
h
2/
(E)E
h
M. o (
P
where 2/º 2 and 2 (BNN Auhi (W))))))
17

---

Published as a conference paper at ICLR 2025
Step 2: For the BN output before ReLU at layer I, for each feature dimension d (I.....d)) d/):
) 10)
) = BNAAH
(0)
+B
vd
(o +
Step 3: For ReLU activation z max(0, y) at layer I, for each dimension d:
B((e)))] 2 ((6 /> 1
where ① is the standard normal CDF. Step Using the BN-induced bound:
1 11=
10
d Σ
D /1 -B8 >
d=1
Step 5: For feature energy with merged terms:
E(H = 2/
(12g/1 + 11x] 2(2,")² 2
G
2
(11201
+ 11 11
(1 11
2/
(URR
1
10
1012,1112 1|=/
2/8
(11z (12 11250133
1- p 2|E
degill || 10
p 2/8
p degmin
1 112
P
218)
Then with BN bound:
di
E(H
p degmin
vi (n (
1-
2
A.5 EFFECT OF DROPOUT ON MAX SINGULAR VALUES OF THE WEIGHT MATRICES
We analyze why dropout leads to larger weight matrices in terms of spectral norm [W Consider the gradient update for weights ( between layers:
OL
(AH())
)
(A(H")) @ " - p)) aHe
aL.
aL
(20)
aW
where p is the dropout rate and M the dropout mask. This leads to weight updates:
aL
aL 2H
M (AH )
2H (((H M /1 P))
(21)
The 1/(1 p) scaling factor in dropout has two key effects: 1) For surviving features (where
M 1), the gradient amplified by 1/(1 p). This leads to larger updates for these weights
18

---

Published as a conference paper at ICLR 2025
during training. 2) During each iteration, different subsets of features survive, but their gradients consistently scaled up. Over many iterations, this accumulates to larger weight values despite the unbiased expectation maintained by dropout. Specifically, with dropout rate when p 0.5, sur- viving gradients are doubled. This amplification effect compounds over training iterations. While dropout maintains unbiased expected values during forward propagation, the consistent gradient scaling during backward propagation leads to systematically larger weight magnitudes. Empirically, observe that higher dropout rates correlate with larger spectral norms [[I] (as shown in Fig- ure 5), supporting this theoretical analysis. The increased weight magnitudes directly contribute to higher feature energy ((H during inference, as:
2
Σ 1n (5,)E
h 1
222)
2
where larger weights produce more distinctive features between connected nodes, helping mitigate oversmoothing.
Effect of Dropout Max Singular Values of W
No Dropout
Dropout=0.7
4-
2-
Layer (Cora)
Layer (Cora)
Layer (Pubmed)
Layer 2 (Pubmed)
Figure 5: Effect of dropout on max singular values of the weight matrices.
A.6 EMPIRICAL VALIDATION or THEORETICAL PROPERTIES
In this section, we provide empirical evidence supporting the theoretical properties derived Sec-
tion 3.
Comensiinn-Speifiee Stochastic Sub-graphs. Figure how varying dropout rates impact number of edges &, in stochastic sub -rbbgr of a 2-layer GCN, defined by Equation [3] across the Cora and Citeseer datasets. We observe that higher dropout rates correlate with fewer edges in these sub-graphs. This variation demonstrates dropout's role in GCNs form of structural regulariza- tion, where dimension-speecffcc stochastic sub-graphs are generated. Each feature dimension samples different sub-graph from the original graph each iteration. This mechanism provides rich set of structural variations during training, potentially enhancing the model's ability to capture diverse graph patterns. Figures 7 & illustrate the behavior of active features along paths of length and within 2-layer GCN equipped with 16 hidden dimensions, across varying dropout rates. Notably, dropout rate of 0.6, the average number of active features approaches zero. This characteristic also underscores the importance of multidimensional feature spaces in ensuring robust information transmission under feature dropout.
Degree-Dependers Nature of Dropout Effects. Figure 9 demonstrates that dropout affects the effective degree of nodes. Figure 10 illustrates that the CV decreases as node degree increases. This degree-dependent effect distinguishes dropout in GCNs from its application in standard neu- ral networks and suggests that the optimal dropout strategy for GCNs may need to consider the graph structure explicitly. Figure II presents empirical evidence supporting Theorem 3 (Degree- Dependent Dropout Effect), which predicts that high-degree nodes experience relatively less varia- tion in their effective degree due to dropout. The figure shows classification accuracy the Cora
19

---

Published as a conference paper at ICLR 2025
1000
Das Rige Oerts
Lmggh 1 Poloo (Bdgm) langle Poalo
Lmgth Poto (84gm) Loggh Paalo
Chose - Eag Conta
2000
0990
Drrpee Bone
Drepred Bat
Blain
Figure 6: Sub-graph size.
Figure 7: Active path on Cora. Figure 8: Active path on Citeseer.
Doca Bippett
Despot
Omppot
00
080
83
544
Dtgno
Figure 10: Effective CV de- Figure 11:
Figure 9: Effective degree.
Accuracy Cora.
gree.
dataset broken down by node degree, demonstrating that nodes with higher degrees consistently achieve better performance. This aligns with theoretical finding that high-degree nodes maintain more stable representations under dropout, as their effective degree has lower coefficient of variation. The observed pattern confirms that dropout naturally provides adaptive regularization that adjusts the local graph structure, with stronger stabilizing effects for topologically important nodes.
20


Reviews:
[{"review_id": "Pvt0OjNSp2", "reviewer": "Reviewer_UjFe", "summary": "This paper investigates the role of dropout in GCNs, addressing a gap in understanding how dropout interacts with graph structure in these models. The authors provide a theoretical analysis that dropout in GCNs generates dimension-specific stochastic subgraphs, which introduces a unique form of structural regularization that doesn’t appear in traditional neural networks. The study highlights that dropout’s effects vary based on node degree, leading to adaptive regularization that leverages topological node importance. The paper also discuss dropout’s capacity to reduce oversmoothing and presents generalization bounds tailored to graph-specific dropout effects. Additionally, it explores the combined effect of dropout and batch normalization in GCNs, identifying a mechanism that enhances overall regularization.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper focuses on the role of dropout in GCNs, specifically analyzing its unique interactions with graph structure. This originality is meaningful to the community.\n- The work presents a well-developed theoretical framework, introducing concepts like dimension-specific stochastic subgraphs, adaptive regularization based on node degree, and graph-specific generalization bounds.\n- Including comprehensive experiments across 16 datasets for both node-level and graph-level tasks is encouraging.", "weaknesses": "- The authors provide generalization bounds for graph neural networks with dropout. However, further clarification is needed on how this finding offers insights into understanding and designing graph neural networks, or any specific guidance on selecting dropout rates. With this theory, is it possible to get the best dropout rate with a specific graph structure and GNN? This would help demonstrate the practical relevance of the theory. Additionally, can the experiments provide corresponding analyses regarding this theory? For example, whether the change in performance at different dropout rates is consistent with the change in generalization bounds can be analyzed from the theory.\n- The use of dropout or similar strategies designed specifically for graphs is also widely applied in GNNs, like DropNode, DropEdge, DropMeassge, etc [1, 2, 3]. The authors may need to discuss its relevance to this study, including whether the proposed theory can analyze these methods and the essential difference and connection between dropout and these methods. Compared to traditional dropout, does dropout on the graph structure more directly enhance the performance of graph neural networks?\n\n[1] Dropedge: Towards deep graph convolutional networks on node classification\n\n[2] Dropmessage: Unifying random dropping for graph neural networks\n\n[3] Graph random neural networks for semi-supervised learning on graphs", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yKh1fQYnUZ", "reviewer": "Reviewer_iGzb", "summary": "This paper focuses on the theoretical analysis of dropout in Graph Convolutional Networks (GCNs) and its impact on regularization and model performance.\n\nThis paper establishes a mathematical framework to analyze dropout's behavior in GCNs. \nIt shows the dropout in GCN is similar to adaptive regularization that considers the topological importance of nodes, and is effective in mitigating over-smoothing in GCNs. And the dropout has synergy with batch normalization in GCNs for enhanced regularization.", "questions": "- Can the theoretical analysis of dropout in GCNs provide insights on how to select the dropout hyperparameter?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a mathematical framework that deepens the understanding of dropout in Graph Convolutional Networks (GCNs), addressing its relation to adaptive regularization and batch normalization.\n2. The empirical analysis is extensive, including empirical observation of theorems and evaluation results on various datasets.\n3. The idea of using active path subgraphs to understand graph feature dropout is interesting.", "weaknesses": "1. Dropout is a general and well-known technique, to achieve performance gain via dropout, the question can be how to tune the parameter. Can the theoretical analysis of dropout in GCNs provide insights on how to select the dropout hyperparameter?\n2.  The paper primarily focuses on dropout in GCNs, but it may not sufficiently compare the method with other graph learning regularization techniques, e.g. [1], [2]\n\n[1] Tackling Over-Smoothing for General Graph Convolutional Networks. Wenbing Huang, Yu Rong, Tingyang Xu, Fuchun Sun, Junzhou Huang.  (extension of DropEdge) Arxiv 2020\n\n[2] Rethinking Graph Regularization for Graph Neural Networks. Han Yang, Kaili Ma, James Cheng. AAAI 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "138cOdBpgA", "reviewer": "Reviewer_tgNZ", "summary": "This paper develops a comprehensive theoretical framework analyzing how dropout uniquely interacts with Graph Convolutional Networks (GCNs), revealing that it creates dimension-specific stochastic sub-graphs and provides degree-dependent adaptive regularization. The research provides new theoretical insights into dropout's role in mitigating oversmoothing and its synergistic interaction with batch normalization, deriving novel generalization bounds specific to graph structures. These theoretical findings are validated through extensive experiments across 16 datasets, demonstrating improved performance on benchmark datasets like Cora, CiteSeer, and PubMed.", "questions": "How does the computational overhead compare to traditional dropout implementations?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper demonstrates rigorous theoretical analysis with a comprehensive mathematical framework for understanding dropout in GCNs, introducing well-defined concepts like dimension-specific sub-graphs and feature-topology coupling matrices.\n\nThe research reveals novel insights about unique interactions between dropout and graph structure, particularly showing how dropout creates dimension-specific stochastic sub-graphs and exhibits degree-dependent effects leading to adaptive regularization.\n\nThe analysis is thorough and multi-faceted, examining structural regularization, oversmoothing mitigation, and interaction with batch normalization, supported by extensive experiments across 16 datasets for both node-level and graph-level tasks.\n\nThe work successfully bridges theory and practice, providing actionable insights for GCN design and training while demonstrating improved performance on benchmark datasets like Cora, CiteSeer, and PubMed.", "weaknesses": "The experimental validation lacks detailed information about the 16 datasets used, and the comparative analysis with state-of-the-art methods could be more comprehensive. Some experimental results mentioned in figures are truncated in the provided content.\n\nThe theoretical framework makes limiting assumptions about undirected graphs, and doesn't adequately address the extension to directed graphs. The interaction between dropout and different activation functions, as well as the impact of graph density on dropout effectiveness, need more exploration.\n\nThe paper lacks clear guidelines for selecting optimal dropout rates based on graph properties, analysis of scalability to very large graphs, and discussion of computational overhead for implementing the theoretical framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wumckDPIQ3", "reviewer": "Reviewer_qw1j", "summary": "The paper performs a comprehensive theoretical analysis of dropout in the case of Graph Convolution Networks (GCN) from multiple perspectives: dimension-specific graph structure modification during training, degree-dependent effect on nodes, impact on over-smoothing, and it’s combined effect with batch normalization.", "questions": "- Section 3.2 introduces the concept of dimension-specific subgraphs. How does this impact the aggregated node representation (including all dimensions)? Since the performance of a GCN ultimately depends on the aggregated representation, it would be insightful to explore this relationship.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "- The theoretical analyses are detailed and sound. \n- Mathematics and the overall logic of the paper is easy to follow.\n- The paper attempts to better understand the internal workings of dropout in GCNs.", "weaknesses": "While I like the theoretical analyses presented in the paper, I think the experiments do not quite align to support the theoretical claims. Below are my concerns with the paper.\n\n- The authors referring to the dropout and batch normalization as “our approach” in lines 409-410 and 466 is misleading since the techniques have been well-established in deep learning for improving performance. The contribution of the authors lies in the detailed theoretical analysis of these techniques within the context of a GCN. While it is a valuable contribution, the techniques should not be claimed as their approaches.\n- The major concern is with the conclusions drawn from the experiments. It is already established in deep learning that dropout and batch normalization enhance performance through regularization. Therefore, only comparing the performance in Tables 1, 2, and 3 does not provide sufficient evidence that the observed improvements are specifically due to the additional effects of dropout in graph neural networks, as analyzed in the theorems. The authors need to design experiments that can directly validate their theoretical analysis.\n- Section 3.4 describes an interesting connection between dropout, the number of GCN layers, and over-smoothing. However, the authors fail to provide experimental evidence to support this relationship. Demonstrating how dropout affects over-smoothing in GCN with varying layer depths would strengthen the paper.\n- Line 472 (regarding Table 1) and line 483 (regarding Table 2) draw contrasting conclusions about the effect of dropout on Dirichlet Energy. What is the reason behind this difference in the behavior of dropout?\n- Minor: Repeated use of variable 'd' for denoting degree (line 133) and node feature dimensionality (line 141).\n\nWith a sound experimental design that can directly validate the theoretical claims, I believe the paper would be a good contribution to the GCN community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)