Title: Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spectral Graph Neural Networks
Abstract: ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness in capturing local graph structure. Despite their success, MPNNs are limited in their ability to capture long-range dependencies between nodes. This has led researchers to adapt MPNNs through *rewiring* or make use of *Graph Transformers*, which compromise the computational efficiency that characterized early spatial message passing architectures, and typically disregard the graph structure. Almost a decade after its original introduction, we revisit ChebNet to shed light on its ability to model distant node interactions. We find that out-of-box, ChebNet already shows competitive advantages relative to classical MPNNs and GTs on long-range benchmarks, while maintaining good scalability properties for high-order polynomials. However, we uncover that this polynomial expansion leads ChebNet to an unstable regime during training. To address this limitation, we cast ChebNet as a stable and non-dissipative dynamical system, which we coin Stable-ChebNet. Our Stable-ChebNet model allows for stable information propagation, and has controllable dynamics which do not require the use of eigendecompositions, positional encodings, or graph rewiring. Across several benchmarks, Stable-ChebNet achieves near  state-of-the-art performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper revisits ChebNet, a classical spectral Graph Neural Network (GNN), and proposes a stabilized variant, Stable-ChebNet, for long-range graph tasks. The authors analyze the instability of high-order Chebyshev polynomial filters using a continuous-time ODE interpretation and propose antisymmetric weight parameterizations with dissipative terms to mitigate gradient explosion. Empirical evaluations on long-range benchmarks such as LRGB and OGB show that Stable-ChebNet achieves strong performance, competitive with or exceeding attention-based graph Transformers and rewiring techniques. While the approach is grounded in classical spectral theory, it is presented with modern theoretical and empirical rigor.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Rigorous theoretical motivation, interpreting ChebNet as a dynamical system and identifying stability issues.\n\n- Efficient implementation that avoids heavy architectural modifications or positional encodings.\n\n- Solid benchmark results on long-range tasks.\n\n- Reintroduces a classical method with modern improvements, offering a computationally lighter alternative to Transformers.\n\nWeaknesses:\n- The work does not sufficiently engage with recent advances in long-range modeling using adaptive attention and multiscale architectures such as hierarchical self-attention [1], multiresolution message passing [2], multiresolution graph transformers [3], and virtual node [4].\n\n- The analysis assumes linear activations and simplified training dynamics, which may not reflect real-world performance under nonlinearity and optimization noise.\n\n- There is no in-depth ablation to isolate the benefits of antisymmetric weights and the dissipative term.\n\nReferences:\n\n[1] Thuan Nguyen Anh Trang, Khang Nhat Ngo, Hugo Sonnery, Thieu Vo, Siamak Ravanbakhsh, Truong-Son Hy, Scalable Hierarchical Self-Attention with Learnable Hierarchy for Long-Range Interactions, Transactions on Machine Learning Research (TMLR). URL: https://openreview.net/pdf?id=qH4YFMyhce\n\n[2] Truong-Son Hy and Risi Kondor, Multiresolution Equivariant Graph Variational Autoencoder, Machine Learning: Science and Technology, Volume 4, Number 1, DOI 10.1088/2632-2153/acc0d8. URL: https://iopscience.iop.org/article/10.1088/2632-2153/acc0d8\n\n[3] Nhat Khang Ngo, Truong-Son Hy, and Risi Kondor, Multiresolution Graph Transformers and Wavelet Positional Encoding for Learning Long-Range and Hierarchical Structures, Journal of Chemical Physics, Volume 159, Issue 3, DOI 10.1063/5.0152833. URL: https://pubs.aip.org/aip/jcp/article/159/3/034109/2903066/Multiresolution-graph-transformers-and-wavelet\n\n[4] Chen Cai, Truong-Son Hy, Rose Yu, and Yusu Wang, On the Connection Between MPNN and Graph Transformer, ICML 2023, Proceedings of Machine Learning Research 202:3408-3430. URL: https://proceedings.mlr.press/v202/cai23b/cai23b.pdf\n\nQUESTIONS:\n- How does Stable-ChebNet compare in performance and scalability to recent models using hierarchical self-attention [1], multiresolution message passing [2, 3] or virtual node [4]?\n\n- Would integrating learnable hierarchies or dynamic clustering (as in adaptive attention or pooling-based GNNs) [2] further enhance ChebNet’s capacity for long-range dependency modeling?\n\n- Could the antisymmetric formulation be extended to more expressive spectral bases such as graph wavelets or multiresolution spectral domains [5, 6]?\n\nReferences:\n\n[1] Thuan Nguyen Anh Trang, Khang Nhat Ngo, Hugo Sonnery, Thieu Vo, Siamak Ravanbakhsh, Truong-Son Hy, Scalable Hierarchical Self-Attention with Learnable Hierarchy for Long-Range Interactions, Transactions on Machine Learning Research (TMLR). URL: https://openreview.net/pdf?id=qH4YFMyhce\n\n[2] Truong-Son Hy and Risi Kondor, Multiresolution Equivariant Graph Variational Autoencoder, Machine Learning: Science and Technology, Volume 4, Number 1, DOI 10.1088/2632-2153/acc0d8. URL: https://iopscience.iop.org/article/10.1088/2632-2153/acc0d8\n\n[3] Nhat Khang Ngo, Truong-Son Hy, and Risi Kondor, Multiresolution Graph Transformers and Wavelet Positional Encoding for Learning Long-Range and Hierarchical Structures, Journal of Chemical Physics, Volume 159, Issue 3, DOI 10.1063/5.0152833. URL: https://pubs.aip.org/aip/jcp/article/159/3/034109/2903066/Multiresolution-graph-transformers-and-wavelet\n\n[4] Chen Cai, Truong-Son Hy, Rose Yu, and Yusu Wang, On the Connection Between MPNN and Graph Transformer, ICML 2023, Proceedings of Machine Learning Research 202:3408-3430. URL: https://proceedings.mlr.press/v202/cai23b/cai23b.pdf\n\n[5] Truong-Son Hy and Risi Kondor, Multiresolution Matrix Factorization and Wavelet Networks on Graphs, ICML 2022 (Workshop on Topology, Algebra, and Geometry in Machine Learning), Proceedings of Machine Learning Research 196:172-182. URL: https://proceedings.mlr.press/v196/hy22a/hy22a.pdf\n\n[6] Truong-Son Hy, Viet Bach Nguyen, Long Tran-Thanh and Risi Kondor, Temporal Multiresolution Graph Neural Networks For Epidemic Prediction, ICML 2022 (Workshop on Healthcare AI and COVID-19), Proceedings of Machine Learning Research 184:21-32. URL: https://proceedings.mlr.press/v184/hy22a/hy22a.pdf\n\nLIMITATIONS:\n- The proposed approach focuses narrowly on stabilizing ChebNet rather than expanding its expressiveness. By contrast, several recent works have introduced architectural innovations for long-range modeling (please see the references above).\n\n- Stable-ChebNet assumes a fixed graph structure and does not address dynamic graphs or graphs with evolving topologies.\n\n- The long-range benchmarks used still rely on synthetic or curated datasets; performance in realistic, noisy, or application-specific domains (e.g., biological networks or scientific simulations) is untested.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors note that the early ChebNet spectral GNN has surprisingly good capabilities for modeling long-range interactions on graphs. They revisit ChebNet and uncover that it suffers from an unstable behaviour during training. The authors are therefore proposing stable variants of ChebNet, that obtain competitive and state-of-the-art results on their empirical evaluation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\n1. The paper is well-written and easy to follow.\n2. The paper continues a recent line of research where spectral models are reassessed for their effectiveness in modeling long-range interactions, with a notable related example being the Spatio-Spectral GNN [[1]]. I believe that this line of work (and the broader analysis of the behaviour of signals and weights of GNNs) is an important and promising direction going forward.\n3. To my knowledge, this is the first paper that re-evaluates the performance of an overlooked model like ChebNet and demonstrates its competitiveness on tasks that are difficult for message-passing GNNs.\n4. The design choices behind Stable-ChebNet are principled and well-justified. The authors also connect their approach to neural networks inspired by differential equations and provide theoretical support.\n5. Stable-ChebNet performs well compared to MPNNs, Graph Transformers, and rewiring-based methods, particularly given that it does not rely on positional encodings.\n\n**Weaknesses**:\n\n1. The greatest weakness is that the empirical evaluation section is somewhat lacking. The paper would have been stronger if datasets that are not considered \"long-range\" were included, or even other datasets from the LRGB (such as PCQM-contact). Moreover, some datasets that are common when discussing long-range capabilities and over-squashing are also omitted (such as NeighborsMatch from [[2]]). The node-level datasets are also lacking, with only ogbn-proteins and ogbn-arxiv being used in the empirical evaluation. Since Stable-ChebNet improves signal propagation, I am wondering if it could also obtain better performance on heterophilic datasets, such as the ones introduced in [[3]].\n2. Figure 1 is confusing and not very informative, especially in the context of the introduction. I suggest the authors clarify the figure's purpose and consider moving it to a later section.\n3. The related work discussion is not fully fleshed out. Although Section 2 (\"Background\") mentions prior work, the coverage of spectral methods, rewiring strategies, and Graph Transformers is limited. The paper also shares conceptual connections with recent sequential and state-space models, such as [[4]], which explore similar ideas from a signal processing perspective. I recommend expanding the related work section in the appendix and, if space allows, in the main text.\n\n**Conclusion:**\n\nOverall, I believe that this is a good paper and should be accepted. However, my initial score is 4 (borderline accept). I will update my score to a 5 (accept) if the authors expand their empirical evaluation and address some of the other weaknesses I raised. Please note that I will not lower my score if the additional experiments turn out to be weak, since negative results would also be very informative, and the paper currently contains a good amount of strong results. I believe that the paper, the core idea, and the direction of research are strong, even if some results do not reach state-of-the-art performance. That said, the current empirical results are not sufficient for a practitioner or researcher to form a clear understanding of the model's overall performance.\n\nI may further update my score depending on the comments and opinions of the other reviewers, particularly if they raise points that I strongly agree with or that significantly influence my assessment.\n\n\n[1]: https://arxiv.org/abs/2405.19121\n[2]:https://arxiv.org/abs/2006.05205\n[3]:https://arxiv.org/abs/2302.11640\n[4]:https://arxiv.org/pdf/2303.06349\n\nQUESTIONS:\n- Please see the \"Weaknesses\" sections above.\n\n- Did the authors also consider performing experiments with positional encodings (such as LapPE or RWSE)? The computation overhead is minimal, and I wonder if they could obtain further performance gains with this. This could also be an interesting and \"low-hanging\" addition to Table 5, since peptides-func normally benefits from positional encodings.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper shows the limitations of the ChebNet framework in propagating information in long range graphs. Specifically, the authors show that as the order of the chebyshev polynomial grows the response function becomes unstable (eigenvalues of the Jacobian exponentially increases). In order to circumvent this issue, the authors propose to restrict the weight matrices to be antisymmetric so as to have imaginary eigenvalues for the Jacobian. This bounds the norm of the Jacobian. In the modified method (Stable-Chebnet) the authors cast the Chebnet as a continuous ordinary differential equation and the layerwise update is obtained using the forward euler method. The method performs comparably to some SoTA methods on synthetic and long range benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1)\tThe paper provides a novel analysis on ChebNet’s limitations and proposes a solution\n2)\tThe method obtains competitive results over chebnet and current SoTA methods\n3)\tThe paper is well written\n\n**Weaknesses**\n\n1)\tWhile the overall method achieves good results, some studies should be provided on the benefit of the individual components. For example, the authors should provide ablations of their method using only antisymmetric weight, only the forward Euler method etc.\n2)\tThe paper should also provide a study showing ability of the modified chebnet to approximate an arbitrary filter function. It would be interesting to see if it is able to better approximate the filters overcoming limitations of Chebnet here or whether some new limitations are introduced.\n\nQUESTIONS:\n1)\tHow would the method perform on omethods using different basis functions ([1,2]). Would the proposed limitations still exist?\n2)\tSince the paper argues that the limitations of Chebnet arises due to exploding response function with filter order: What if the norms would be constrained after every layer In Chebnet using layernorm, instancenorm etc. How would the method behave in this case? A study might help understand the importance of current method.\n3) Please see weakness section above\n\n[1] Xiyuan Wang, & Muhan Zhang. (2022). How Powerful are Spectral Graph Neural Networks.\n\n[2] Mingguo He, Zhewei Wei, Zengfeng Huang, & Hongteng Xu. (2022). BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper reevaluates the spectral GNN ChebNet on long-range graph tasks showing that it still performs competitively compared to baseline methods. It also introduces Stable-ChebNet, a slight modification of ChebNet to enforce layer-wise stability that achieves near state-of-the-art performance across a variety of experiments.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- Paper is well-written, well-structured and most visualizations are helpful and clear.\n- There are both theoretical and empirical validation for the claims made.\n- The authors have successfully identified and filled a niche in the GNN literature/benchmarking. ChebNet is indeed often omitted as a baseline in the literature.\n- It is clearly communicated why ChebNet should be included as a baseline in the future, especially on longe-range interaction graphs.\n\n**Weaknesses**\n1. Experiments are somewhat limited. The full results on LRGB and OGB could be provided.\n2. Generally, the choice of methods to compare against is inconsistent. For example, in Table 3 and 4, completely different baselines are considered. Additionally, on long-range interaction tasks, one should also compare against methods like S2GCN or BuNN [2] that were created exactly for this task. Also, more recent experimental settings on standard GNNs like GAT from [1] are omitted, even though these would perform best on ogbn-proteins in the table provided.\n3. The discussion on how to proceed based on these findings could be more pronounced. Yes, SpectralGNNs are strong baselines, but still if you look at the leaderboards of the benchmarks, e.g. on the OGB datasets, you don’t find any pure GNN approach anymore. How could this trend be changed?\n4. Figure 5 and Table 2 seem to show the exact same information. Why were there no other baselines included that were constructed to work well for long-range interactions?\n\nQUESTIONS:\n1. Could you please elaborate why Figure 1 illustrates how classical ChebNet filters can exhibit unbounded dynamics? What do you mean by temperature here? Why does a temperature close to zero for most of the nodes at step T illustrate unbounded dynamics? \n2. Overall Stable-ChebNet indeed achieves superior results on the synthetic long-range interaction datasets, but how can these findings translate to actual real-world datasets? The results from the long-range benchmark peptides dataset show that it can match the performance there, but the advantage over e.g. simple GCN is small. Do these results not suggest that we should just use more powerful methods like S2GCN, combining the two paradigms and be done with it?\n3. Why did you evaluate ChebNet only on these three real-world datasets and did not include comparisons on standard homophilic node classification, the heterophilic datasets from Platanov et al. [3] and for the ones from OGB also on ogbn-products and some graph property prediction tasks? This would significantly simplify the inclusion of ChebNet as a standard baseline in future work.\n4. The Barbell graph describes an extremely artificial scenario, which limits the transferability of these findings to real-world graphs. Could you elaborate further on how such a graph could arise in real-world scenarios?\n5. Did you consider the findings and discussions from [1]? There has recently been a lot of discussion about fair and comparable evaluation of GNNs. They show that in many papers the baselines are not properly tuned. Do you think there is a similar issue with ChebNet, and did you use a similar approach? Or if not, could you use the ideas from [1] to improve the results of ChebNet even further (e.g. adding layer/batch norm)?\n\n[1] Luo et al., Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification, NeurIPS 2024 Track on Datasets and Benchmarks\n\n[2] Bamberger et al., Bundle Neural Network for message diffusion on graphs, ICLR 2025\n\n[3] Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?, ICLR 2023\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}