Title: A CLT for Polynomial GNNs on Community-Based Graphs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph Neural Networks

Neighbor Aggregation

Convergence of Measures

Central Limit Theorem

GNN Oversmoothing

Stochastic Block Model
Abstract: We consider the empirical distribution of the embeddings of a $k$-layer polynomial GNN on a semi-supervised node classification task and prove a central limit theorem for them. Assuming a community based model for the underlying graph, with growing average degree $\nu_n\to\infty$, we show that the empirical distribution of the centered features, when scaled by $\nu_{n}^{k-1/2}$  converge in 1-Wasserstein distance to a centered stable mixture of multivariate normal distributions. In addition, the joint empirical distribution of uncentered features and labels when normalized by $\nu_n^k$ approach that of mixture of multivariate normal distributions, with stable means and covariance matrices vanishing as $\nu_n^{-1}$. We explicitly identify the asymptotic means and covariances, showing that the mixture collapses towards a 1-D version as $k$ is increased. Our results provides a precise and nuanced lens on how oversmoothing presents itself in the large graph limit, in the sparse regime. In particular, we show that training with cross-entropy on these embeddings is asymptotically equivalent to training on these nearly collapsed Gaussian mixtures.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the asymptotic properties of the product $A^k X$ where $A$ is an adjacency matrix and $X$ is a matrix of node features under a sparse community graph model, and in doing so are able to characterize the asymptotic distribution of the embedding vectors produced through this matrix product. In particular, they are able to give guarantees for a Wasserstein-bound CLT for these features (both after being re-scaled by a suitable power of the average degree of the network, and after the typical centering/scaling one expects for CLT results). This is then used to give some discussion and insights into the behavior of using these features for classification, and the well known phenomenon of over-smoothing in GNNs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths of the paper:\nThe results in the paper are quite technical in their nature (for instance, requiring some detailed analysis into random walks of graphs) and are generally presented well and are of high quality. The mathematical details in the appendix are correct, and are covered at an appropriate level of detail. From a probabilistic perspective, the results of the paper are interesting. To my knowledge, the mathematical results in of themselves are new.\n\nWeaknesses of the paper:\nIt is generally unclear how this paper fits into the larger literature on GNNs and the phenomenon which the paper seeks to explain, and the extent to which it is novel is unclear. The GNN model being studied is also relatively simple - even ignoring the removal of the non-linear activation function between layers, the features being studied are simply node features propagated by a power of the adjacency matrix. With the exception of Section 4.1 (where there is only a discussion of a linear model applied to the features), there is no reference to trainable features for this model. Moreover, there are already a large number of papers which study the GNN oversmoothing phenomenon, some of which already give explanations under more realistic assumptions than the current paper. The overall novel contribution in terms of understanding these models from a ML perspective appears to be minimal.\n\nQUESTIONS:\n1. I am surprised that the results do not extend to the dense regime - for instance, by rewriting Assumption 3 as $( \\nu_n / n)^{1/2} \\cdot \\sqrt{n} ( \\pi_l - | \\mathcal{C}_l | / n ) = o(1)$, it appears that if $\\nu_n = n$, then a standard assumption of assuming that the community assignments are drawn from a Multinomial distribution would break this assumption. Can you please give some intuition as to what breaks down in this regime?\n\n2. I am a bit confused by the presentation of Assumption 1, where the average degree parameter $\\nu_n$ is said to need to grow to infinity as the number of nodes does (which is a standard assumption). There is then also the assumption that $\\nu_n B_{ll'} / n \\leq C/n$ for some constant $C$, which when rearranged gives that $\\nu_n B_{ll'} \\leq C$, which contradicts $\\nu_n \\to \\infty$. I'm assuming this is a typo and that it just suffices to assume that $B_{ll'}$ is bounded, which under the assumptions that $L$ is fixed with $n$ is going to be the case anyways. \n\n3. Can you please provide some intuition for why the covariance matrices in Theorem 1 and Theorem 2 have no dependence on the covariance structure of the node features $X_i$? Just thinking out loud about the variance of the node features, we know that by the law of total variance, it will break down into two terms - one corresponding to how the covariance of the node features is smoothed by the random walks on the graph of length $k$, and the other corresponding to the uncertainty in the graph mixed by the mean signal of the $X_i$. This second term is what appears in the variance bound - why does this end up being the dominating term? \n\n4. Can you please give a summary of where your results fit into the existing literature on theoretical guarantees for GNNs? For example, there is a substantial literature already which covers the oversmoothing phenomenon in GNNs - to what extent is the discussion in this paper novel? For example, to cite some well known papers which discuss this phenomenon: \n\n- In \"Graph Neural Networks Exponentially Lose Expressive Power for Node Classification\" by Oono and Suzuki, they are able to give a characterization of this phenomenon for Erdos-Renyi graphs within GCNs with ReLU activations, and are able to show subspace collapse in the learned embeddings which is achieved exponentially in the number of layers.\n- In \"A Note on Over-Smoothing for Graph Neural Networks\" by Cai and Wang, this is extended to handle other non-linear activation functions.\n- \"Not too little, not too much: a theoretical analysis of graph (over)smoothing\" by Keriven studies the same type of model as in the current paper, albeit with the Laplacian rather than the adjacency matrix (which is the more frequently used model). \n- \"Demystifying Oversmoothing in Attention-Based Graph Neural Networks\" by Wu et al studies GNNs with attention mechanisms, and also handles the non-linear regime.\n\nCan you explain the novelty in your contribution compared to these papers? To be clear, this last question is the one of main importance to me in terms of understanding the overall contribution of the work. An understanding of whether the results in this paper develop substantial **new** understanding for the behavior of GNNs will be the main factor in whether my overall evaluation would increase.\n\nLIMITATIONS:\nThe authors have adequately addressed the societal impact of their work. As it comes to the limitations of the theoretical analysis, there is not really much of a discussion at all except a sentence at the very end of the conclusion referring to non-linear GNNs and different graph models. I would appreciate a more involved discussion about a) why their results cannot be extended easily to non-linear GNNs, GNNs with learnable weight matrices within layers, GNNs with Laplacian matrices, and so on; and b) a discussion about what aspects of the theory break down when moved to the either the dense regime where $\\nu_n = \\Omega(n)$ or the regime where $\\nu_n = O(1)$. This would improve the overall contribution of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors show two CLT-type convergence results for linear GNNs on CSBM random graphs when the number of nodes go to infinity, and the graph is sparse. The first result pertains to rescaled node features, which they show converge to a mixture of Gaussians with vanishing covariances. The second result concerns a centered and again-rescaled version of these features, which they show converge to a sum of centered Gaussians with specific covariances. They show two applications, one concerning the training of classifiers on these features for which they give a uniform deviation result, and one concerning some remarks on oversmoothing, when the number of layers grows. The proofs rely mostly on moment computations.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n- the paper is well-written and clear, despite heavily technical content\n- one notable strength of the results is that, as CLTs, the limit does not depend on the distribution of the input node features (only on its mean), which is quite original in this literature. Is this due to the sparsity? (looking at the proof it allows to discard some terms)\n- beyond the convergence results themselves, the applications on training and oversmoothing are interesting\n\nWeaknesses\n- the references are quite scarce, given this large literature, especially on the oversmoothing aspect. Several papers performed similar theoretical analyses before (with shrinking Gaussian communities, collapsing covariances, and so on), and while that of the authors is new (as far as I know), it must commented with regard to these previous results. It would help better outline the novelty and strength of the results: while the current formulation gives an impression of novel \"idea\" that is relatively incorrect, it is rather a new take on a classical idea. See eg [1,2,3] (non-exhaustive list).\n\nIn the same vein, linear GNNs are very often called \"Simplified Graph Convolution\" (SGC), as per the seminal paper [4]. To my knowledge, polynomial GNNs are more often used for polynomial filters, where every power of the graph matrix (eg Chebnet).\n\n[1] Baranwal et al. \"Effects of Graph Convolutions in Multi-layer Networks\"\n[2] Keriven \"Not too little, not too much: a theoretical analysis of graph (over)smoothing\"\n[3] Wu et al \"A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks\"\n[4] Wu et al. \"Simplifying Graph Convolutional Networks\"\n\n- could the result on training linear GNNs be pushed further? It feels a bit \"unfinished\" at the moment. Surely the limit problem is a known classification problem, with guarantees?\n- while Theorem 2 indeed feels like a CLT, this is not quite the case of Theorem 1, which uses vanishing covariances. Unless I am missing something, a Gaussian with vanishing covariance converges to a Dirac in Wasserstein norm, does that means that the distribution really converges to a sum of Diracs? What would be the implication of this for the other results? (training, oversmoothing). This may be discussed\n- In theorem 2 the centering of the features uses their unknown mean. Could it be replaced by some kind of empirical average? Otherwise it must be clearly written that this is a theoretical result that uses oracle quantities.\n- the authors mention the importance of sparsity, but that \"some\" results may carry to the dense setting. Is it possible to provide a bit more details what would not carry, and why?\n- minor comment: it is just a personal appreciation, but I feel like some of the language used by the authors oversells a bit the results; eg I'm not sure a result pertaining to linear GNNs on sparse CSBM can be said to have \"profound implications for the training of GNNs\" in general. Paradoxically, this is a bit detrimental to the just appreciation of the results, which I find to be valuable and carry interesting intuitions by themselves.\n\nQUESTIONS:\nSee above for several questions.\n\nLIMITATIONS:\nLimitations are appropriate.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper establishes CLTs for the features of a particular class of GNNs when the input data follow a form of stochastic block model. The limits distributions in the CLTs are mixtures of Gaussians with parameters expressed in terms of the ingredients of the block model. Insights into the performance of linear classification and the over smoothing phenomenon are derived from the CLTs and the theory is illustrated with numerical experiments.\n\nSTRENGTHS AND WEAKNESSES:\nFrom a rigorous mathematical point of view, GNNs have remained a somewhat murky subject. In the ML literature, even the motivation for GNN’s is often not well explained beyond simple heuristics, and their has been an absence of careful statistical formulation of GNN’s operation, nevermind statistical theory of the sort set out in this paper. As such I believe the work set out in this paper could make an important step forward and stimulate more work in this area. \n\nThe mathematical clarity of the paper is very good. I have experience in statistical theory of networks and graphs, but am not an expert in GNNs; for me the presentation of theory made it easy to digest.\n\nThe paper focuses on one particular class of GNN’s: poly-GNN’s, which have a relatively simple linear structure. \nMy questions below include a request to clarify exactly where this paper sits in the literature on GNN’s, its originality,  and for comments to be added on the prospects of the theory being extended beyond GNN’s. Subject to that being clarified, I think it would be wrong to regard the focus on poly-GNN’s as a “weakness” of the paper, so much as a natural limitation in its scope. \n\nI think there are some opportunities for other small improvements — see questions below. \n\nIf the authors can further clarify the originality of this work and its position in the literature about the theory of GNNs I am willing to up my scores about significance and originality, and hence boost my overall score for the paper. Clarification of originality and place in the literature is of course a very important matter, so without this addressed I have currently rated as \"borderline reject\".\n\nQUESTIONS:\n1) The work in this paper has the potential to stimulate similar analysis of GNN's other than Poly-GNN's, can the authors very briefly comment on what the main opportunities and obstacles might be? If they need to save some space to accommodate this, I would recommend condensing section 1.1, I think it could easily be half the length and still do the job of summarising the existing contributions.\n\n2) Regarding the place of this work within the literature, I think more should be said about the current state of theory for GNN's and poly GNN's in particular. Are there any existing CLT's, or other theory related to behaviour of GNN's under block models etc? If, in any sense, the results in this paper are the first of their kind, I think that must be highlighted. More broadly to frame originality, what was the inspiration for this work? Is it based on any existing analysis, or something in a completely new direction? I think these questions should be answered in the paper, not just in the response to the review.\n\n3) A technical question, related to reqs (10), (11) and elsewhere: inside the expectation their is a sup over a (presumably) uncountable family of random variables (associated with each of the Lipschitz functions). Consider this uncountable sup, how do you know the sup is measurable, and hence well-defined as a random variable?\n\n4) Line 230 \"...significant practical implications...\" On first reading this, I was expecting that the theory would be use derive some guidance or new techniques for how GNNs should be use in practice. I'm not sure I got that, but rather some (detailed) insights into phenomena that might be observed in practice. The authors may consider re-wording slightly here to manage the readers' expectations.\n\n5) Line 238 \"...as these are the quantities typically used by the classifier” Please clarify - the degree normalised features depend on nu_n; even if one accepts the assumption that the data are generated by the block model, it seems too much of a stretch to assume that the true value of nu_n is known.\n\n6) Line 272 \"...is a well-documented empirical observation” Please add references to the literature.\n\nA couple of typos:\n\nAbstract \"Our results provides a...\"\n\nFigure 1 caption “Ten gradients steps…”\n\nLIMITATIONS:\nThe authors have answered “No” to the question of whether limitations are discussed. This feels like a missed opportunity!! Surely there must be something you can say here?!\n\nThey have also answered “No” to the question about open access to data and code. Whilst I agree the numerical experiments are not a focal point of the paper, it seems an unnecessary restriction to not make it easy for the community to reproduce or further experiment with the numerical examples. They have also answered “No” to the question about compute resources. Again this feels unnecessary: just add it as a brief comment in the supplementary material. \n\nThe authors have answered “No” to the question about societal impacts. This is not inappropriate given the theoretical nature of the contribution.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a rigorous theoretical analysis of the asymptotic behavior of k-layer Polynomial Graph Neural Networks (Poly-GNNs) on sparse community-based graphs, such as those modeled by the Contextual Stochastic Block Model (CSBM). The authors derive two central limit theorems (CLTs) characterizing the empirical distributions of Poly-GNN embeddings:\nTheorem 1 proves that the joint distribution of degree-normalized node embeddings and class labels converges to a mixture of Gaussians with vanishing covariances.\nTheorem 2 shows that the centered and scaled embeddings converge to a stable mixture of multivariate Gaussian distributions.\nThe paper also analytically characterizes mean and covariance collapse under repeated aggregation, offering a precise mathematical explanation for the oversmoothing phenomenon observed in deep GNNs. A key contribution is the demonstration that training a linear classifier with cross-entropy loss on Poly-GNN features is asymptotically equivalent to training on the limiting Gaussian mixture, offering theoretical insight into GNN training dynamics.\n\nSTRENGTHS AND WEAKNESSES:\nQuality\nStrengths:\nThe mathematical exposition is rigorous and detailed.\nThe CLTs are proven with clear assumptions, and proof sketches are well-motivated.\nExplicit formulas for limiting means and covariances are derived, which are novel and impactful.\nWeaknesses:\nNo experimental validation beyond illustrative figures; this limits the practical understanding of how well the theoretical predictions manifest in real datasets.\nAssumptions (e.g., community-based graph structure and νₙ → ∞) may be restrictive in real-world applications.\n\nClarity\nStrengths:\nThe paper is very well written and logically structured.\nEach section flows naturally into the next, and technical results are motivated with clear intuition.\nThe discussion on oversmoothing via spectral analysis is particularly enlightening.\nWeaknesses:\nSome parts of the proof outline (e.g., sub-Gaussian arguments, moment bounds) assume familiarity with advanced probability without intuitive explanation.\nNotation can be heavy for non-theory audiences, especially in the oversmoothing section.\n\nSignificance\nStrengths:\nOffers an original, quantitative understanding of GNN oversmoothing—previously treated heuristically in much literature.\nBridges the gap between feature propagation theory and classifier training behavior.\nWeaknesses:\nApplicability is currently limited to polynomial GNNs on stochastic block models, which may reduce perceived generality.\nImpact on GNN design may be indirect unless extended to more expressive models.\n\nOriginality\nStrengths:\nThe use of CLTs to characterize GNN embeddings under label-aware graph models is novel.\nProvides, to my knowledge, the first exact analytical treatment of Poly-GNN oversmoothing via spectral power iteration arguments.\nWeaknesses:\nThe Poly-GNN class considered is a simplification of practical GNNs (e.g., lacks nonlinearity or learnable weights), which might limit originality for practitioners.\n\nQUESTIONS:\nApplicability Beyond Poly-GNNs: Can the authors comment on how the analysis might extend to nonlinear or weighted GNNs (e.g., GCN, GAT, GPR-GNN)? What technical barriers exist?\nSensitivity to νₙ Assumptions: Many results hinge on νₙ → ∞ and νₙ = o(n). Are the convergence rates or limiting distributions significantly affected if νₙ remains constant or grows slowly?\nPractical Implications for Model Design: Given that the class means collapse to a 1D subspace, can these results motivate architectural changes (e.g., early stopping, skip connections, regularization)?\nEmpirical Validation: Would the authors consider adding numerical simulations (beyond Figures 1–3) to illustrate how closely the empirical distributions of embeddings follow the theoretical Gaussian mixtures, especially for moderate n?\nBeyond CSBM: The authors mention the challenge of generalizing beyond community-based graphs. Can they speculate on potential extensions or possible failures of the current CLTs in graphs with heavy-tailed degree distributions or assortative mixing?\n\nLIMITATIONS:\nNo — The paper does not adequately discuss limitations, although the authors indirectly allude to model constraints. I recommend:\nExplicitly stating the limitations of the Poly-GNN assumption (e.g., no learnable weights or non-linearities).\nDiscussing how sensitivity to νₙ or non-community-based graphs may affect the validity of the results.\nConsidering the gap between asymptotic and finite-n behavior, especially since real graphs are not infinitely large or cleanly community-structured.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}