Title: Deeper with Riemannian Geometry: Overcoming Oversmoothing and Oversquashing for Graph Foundation Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: MPNN

Oversmoothing

Oversquashing

Riemannian geometry

Graph Foundation Models
Abstract: Message Passing Neural Networks (MPNNs) are the building block of graph foundation models, but fundamentally suffer from oversmoothing and oversquashing. There has recently been a surge of interest in fixing both issues. Existing efforts primarily adopt global approaches, which may be beneficial in some regions but detrimental in others, ultimately leading to the suboptimal expressiveness. In this paper, we begin by revisiting oversquashing through a global measure -- spectral gap $\lambda$ -- and prove that the increase of $\lambda$ leads to gradient vanishing with respect to the input features, thereby undermining the effectiveness of message passing. Motivated by such theoretical insights, we propose a local approach that adaptively adjusts message passing based on local structures. To achieve this, we connect local Riemannian geometry with MPNNs, and establish a novel nonhomogeneous boundary condition to address both oversquashing and oversmoothing. Building on the Robin condition, we design a GBN network with local bottleneck adjustment, coupled with theoretical guarantees. Extensive experiments on homophilic and heterophilic graphs show the expressiveness of GBN. Furthermore, GBN does not exhibit performance degradation even when the network depth exceeds $256$ layers.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this submission, the authors propose to mitigate both oversmoothing and oversquashing issues of MPNNs. From the lens of Riemannian geometry, this paper begins with showing the increase of lambda leading to gradient vanishing, and thus present a message-passing paradigm of local adjustment, leveraging the proposed nonhomogeneous boundary condition. Consequently, a provable GBN is designed with the Robin condition, and is evaluated on both homophilic and heterophilic graphs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nS1. The studied problem is significant given oversmoothing and oversquashing are longstanding issues in MPNNs. The authors theoretically discover the possible negative impact of globally increasing the spectral gap, and it is interesting to study the local adjustment MPNN alternatively.\n\nS2. The established theory is solid. The authors unify the MPNN formalism from Riemannian dynamics (manifold heat kernel) and, in order to jointly address the both issues, construct a nonhomogeneous boundary condition on the manifold with rigorous elaboration. No error is detected in the proof.\n\nS3. The proposed GBN is novel and its complexity is the same as vanilla GCN. The authors design a new message passing paradigm based on nonhomogeneous Robin condition with theoretical guarantees. \n\nS4. The experiments on homophilic and heterophilic graphs are convincing. The authors evaluate the ability of GBN against oversmoothing and oversquashing, and show model performance in very deep network. Codes are available.\n\nWeaknesses\n\nW1. The authors give the time complexity comparison with some baselines, while GraphNormv2, G2-GCN and SWAN are not listed.\n\nW2. Further details should be provided in the paper, e.g., is Lemma 5.3 only gives an existence about the solution of over-smoothing? See Q3. The theory in this paper is not friendly to the readers not familiar with Riemannian dynamics and PDEs.\n\nQUESTIONS:\nQ1. Link prediction performance comparison with GCN, GAT, GIN and SAGE on typical benchmark datasets such as Cora, Citeseer and CS.\n\nQ2. Does GBN need to explicitly identify the boundary nodes? Does there exist graphs having no boundary node, or to what extent? Under that case, GBN degenerates to GCN exactly? \n\nQ3. Please specify the computational overhead again the vanilla GCN.\n\nQ4. Implementation details, i.e., the initial values of boundary condition coefficients, and the neural network architecture for the indicator I.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper identifies that existing global remedies for oversquashing in MPNNs, specifically rewiring approaches that enlarge the graph's spectral gap, actually accelerate gradient vanishing and oversmoothing. For this, they draw on the continuous analogue of studying heat flow on Riemannian manifolds. To overcome these issues, they introduce *Graph Boundary-conditioned MPNNs* which adapt the message passing to include a learnable nonhomogenous Robin boundary condition, which can adaptively regulate information flow and mitigate both issues. They demonstrate the theoretical benefits of such a boundary condition for the continuous analogue, e.g., by analyzing Dirichlet energy decay. Empirically, the authors show that GBN outperforms a large set of baselines on standard small to medium-scale homophilic/heterophilic benchmarks, scales beyond 256 layers with no performance drop on Cora, and effectively mitigates oversquashing measured by performance on the standard graph transfer task.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- To the best of my knowledge, the idea of considering such adaptive boundary conditions is novel and, in my view, a quite elegant way compared to the \"static\" graph rewiring approaches that have been proposed.\n- The method seems to introduce no significant computational overhead (i.e., scales similarly to GCNs).\n- Strong empirical performance for node classification, and the claims of mitigating oversmoothing and oversquashing are effectively verified as there is no performance degradation even with 256 layers and Dirichlet energy stays roughly constant.\n\n**Weaknesses:**\n- The main body leans heavily on Riemannian geometry (§ 5) to motivate GBN, but most of the benchmark graphs lack any clear manifold interpretation which makes the link seem somewhat loose. The resulting core GNN update in § 6 is barely discussed in the main body. Also, the fact that the boundary set is implemented dynamically via a learned, soft gate is only vaguely mentioned in the main body at lines 247-248 and only looking at the appendix/code helped clear things up for me. I understand that this might be a natural thing to do in PDE circles, but I would suspect that a GNN-focused audience might benefit from a more self-contained presentation.\n- For claiming scalability, the experiments are quite small-scale (and for the oversquashing experiment only on synthetic data). To validate (a) scalability, it could make sense to evaluate on `ogbn-{arxiv, products}`. For (b) long range dependencies, it might make sense to also evaluate on some tasks of the long range graph benchmark (such as `peptides-{func, struct}`) [1].\n\n[1] *Long Range Graph Benchmark.* Vijay Prakash Dwivedi, Ladislav Rampášek, Mikhail Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, Dominique Beaini. NeurIPS 2022.\n\nQUESTIONS:\n- Overall presentation and clarity of the paper should be improved, mainly self-containedness of § 6 (see sec. on Weaknesses), but also the general writing (just for example, the term \"GBN\" is introduced without explanation in the abstract, or the title of § 6 is \"GBN: A Provable Neural Architecture\").\n- Evaluation on the benchmarks from the Weaknesses section would make a stronger case for GBN.\n- The paper could dedicate more space to providing some intuition for GBN. After all, as far as I understand Eq (18) also implements some shift operator wrt. a weighted graph, whose filter response could be analyzed. Also, it would be valuable for intuition to, e.g., plot the boundary assignment $I_i = \\zeta(x_i, \\eta)$ for some specific graphs.\n- The related work section could mention some more recent approaches to achieve similar goals (such as, e.g., [2]).\n\n[2] *Unitary convolutions for learning on graphs and groups.* Bobak T. Kiani, Lukas Fesser, Melanie Weber. NeurIPS 2024.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses two fundamental, interconnected problems in MPNNs: oversmoothing and oversquashing.\nThe authors challenge the prevailing \"global\" approaches (e.g., graph rewriting) that aim to fix these issues by modifying the entire graph structure, often by increasing the spectral gap. Through a theoretical analysis grounded in Riemannian geometry. As an alternative, the paper proposes a \"local\" approach that refines the message-passing mechanism itself. By modeling MPNNs as solvers for a heat equation on a Riemannian manifold. This theoretical foundation leads to their proposed model, the Graph Boundary conditioned message passing Neural network (GBN).\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. The paper tackles oversmoothing and oversquashing, which are critical and well-recognized challenges that limit the depth and expressive power of modern GNNs.\n2. The authors provide theoretical analysis, connecting MPNNs to heat flow on Riemannian manifolds. \n3. GBN offers a promising direction for mitigating oversquashing without modifying the graph's topology. This is a significant advantage over conventional graph-rewiring methods, which can be computationally expensive and risk altering essential structural information.\n\n### Weaknesses\n1. The authors claim to \"put forward a local treatment and introduce the local Riemannian geometry for the first time.\" This claim is questionable, as several well-studied methods already leverage local geometric concepts. For instance, methods based on Ollivier-Ricci Curvature [1] and Forman-Ricci Curvature [2] are intrinsically local, as they operate on small neighborhoods. This undermines a central claim of the paper's novelty.\n\n2. The paper fails to compare against some highly relevant baselines. For example, PANDA [3] is another recent attempt to tackle oversquashing without altering the graph topology, making it a critical point of comparison that is missing from the evaluation.\n3. The paper's use of \"external inputs\" to combat oversmoothing appears conceptually similar to well-established techniques like initial residual connections. \n4. The paper argues that edge deletion, used by some methods to combat oversmoothing, \"hinders the information flow.\" This may be a mischaracterization of prior work [2, 4]. Methods like Spectral Graph Pruning or those using Ricci curvature typically only prune edges in overly dense regions, which is not necessarily a hindrance to overall information flow and may even be beneficial.\n5. The introduction of multiple MLPs for the boundary conditions adds a new set of hyperparameters. The paper does not explore the model's sensitivity to these choices.\n\n> References\n\n[1] Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature\n\n[2] Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvature\n\n[3] PANDA: Expanded width-aware message passing beyond rewiring.\n\n[4] Spectral Graph Pruning Against Over-Squashing and Over-Smoothing.\n\nQUESTIONS:\nSee weaknesses above.\n\nLIMITATIONS:\nNo additional limitations are identified.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel local approach to tackle both oversmoothing and oversquashing in message passing neural networks (MPNNs). By integrating local Riemannian geometry, the authors design a Graph Boundary conditioned Neural network (GBN) that adaptively adjusts message passing based on local graph structures. The approach is supported by theoretical guarantees and experiments show that GBN achieves superior performance on various graph datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper demonstrates notable strengths in its theoretical rigor and experimental thoroughness, with theorems (e.g., 4.2, 4.3) formally guaranteeing the approach's efficacy against oversmoothing and oversquashing, and comprehensive ablation studies validating component contributions. The experimental setup is standard and reproducible, providing clear details for experts. \n\nHowever, its clarity suffers from dense theoretical notation (e.g., integral symbols and Riemannian geometry terms) that lack self-contained definitions, potentially alienating non-specialists. In significance, the work makes substantial contributions to MPNN research by addressing long-standing challenges, though its impact on broader fields remains limited. Regarding originality, while the integration of local Riemannian geometry is innovative, the paper could enhance distinctiveness by explicitly contrasting with related work (e.g., prior studies on Riemannian geometry and MPNNs cited in line 83), clarifying how the proposed GBN uniquely advances the paradigm. Strengthening these comparative analyses would solidify its claim to originality.\n\nQUESTIONS:\n1. In the “Message of the Section” on line 162, it is stated that the method of increasing the spectral gap to reduce oversquashing impairs the effectiveness of message passing. However, in the “A more intuitive explanation” on line 158, it is mentioned that reducing oversquashing can be achieved by adding edges to allow more messages to flow. These two statements seem contradictory. I would like the authors to clarify whether the message passing becomes more unobstructed or more hindered when reducing oversquashing. A clear and consistent explanation would increase my evaluation of the paper's logical coherence.\n2. Are the graph rewriting operations (adding or deleting edges) and the operations for adjusting the spectral gap independent of each other? It is not clear from the paper how these two aspects interact. If the authors can elaborate on their relationship, such as whether one operation affects the other or if they can be applied simultaneously in a coordinated manner, it will enhance my understanding of the proposed approach and potentially increase the paper's score in terms of comprehensiveness.\n3. In the ablation study, when the learnable and adjustable $\\gamma$ and $\\beta$ are fixed to $\\gamma_0$ and $\\beta_0$, the accuracy does not drop significantly. I am curious about how these fixed values $\\gamma_0$ and $\\beta_0$ were selected. If there was a non-learning heuristic method used to choose them, please describe it. Additionally, if such a method was used, does it lead to a reduction in computational complexity? A detailed response on this would improve my assessment of the practicality and methodological soundness of the ablation study in the paper.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}