Title: Less but More: Linear Adaptive Graph Learning Empowering Spatiotemporal Forecasting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spatiotemporal learning; Graph Representation Learning; Time Series; Deep Learning
Abstract: The effectiveness of Spatiotemporal Graph Neural Networks (STGNNs) critically hinges on the quality of the underlying graph topology. While end-to-end adaptive graph learning methods have demonstrated promising results in capturing latent spatiotemporal dependencies, they often suffer from high computational complexity and limited expressive capacity. In this paper, we propose MAGE for efficient spatiotemporal forecasting. We first conduct a theoretical analysis demonstrating that the ReLU activation function employed in existing methods amplifies edge-level noise during graph topology learning, thereby compromising the fidelity of the learned graph structures. To enhance model expressiveness, we introduce a sparse yet balanced mixture-of-experts strategy, where each expert perceives the unique underlying graph through kernel-based functions and operates with linear complexity relative to the number of nodes. The sparsity mechanism ensures that each node interacts exclusively with compatible experts, while the balancing mechanism promotes uniform activation across all experts, enabling diverse and adaptive graph representations. Furthermore, we theoretically establish that a single graph convolution using the learned graph in MAGE is mathematically equivalent to multiple convolutional steps under conventional graphs. We evaluate MAGE against advanced baselines on multiple real-world spatiotemporal datasets. MAGE achieves competitive performance while maintaining strong computational efficiency.


Peer reviews:
comments: SUMMARY:
This paper studies the problem of GNN-based spatiotemporal forecasting. Motivated by the deficiency of RELU in modeling dynamical graph topology used behind GNNs, this paper removes RELU and proposes a sparse yet balanced MOE strategy for efficiently approximating the full-rank Linear Adaptive Graphs. Extensive experiments on a large number of benchmark datasets show the superior performance of the proposed method against a sizeable collection of baselines.

STRENGTHS AND WEAKNESSES:
Strengths:
- This paper is well motivated by illustrating the deficiency of RELU in modeling dynamical graph topology used behind GNNs.
- The proposed method is technically sound.
- The experiments compare the proposed method against a large collection of baselines on a number of benchmark datasets.

Weaknesses:
- Some design choices are not well justified, particularly the necessity of the kernel-inspired approximation. What if you directly apply MOE on the full-rank linear adaptive graphs, which completely avoid the low-rank bottleneck? Theoretically, this will be more expressive than your proposed method at the cost of higher computational cost. This trade-off should be rigorously discussed and studied in the experiments. What is the Pareto frontier of this trade-off?
- The proposed MAGE can be regarded as a base building block, which could be integrated with other spatiotemporal GNN models. Its advantages should be better justified by integrating with several other SOTA models. It would be interesting to see whether the performance improvement persists in these cases.

QUESTIONS:
Please see the weaknesses.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-16

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the issues of edge noise and high complexity in traditional adaptive graph convolution methods by introducing a sparse mixture-of-experts linear graph learning approach. Theoretical analysis is provided to demonstrate the method’s effectiveness. Extensive experiments and comparisons with baseline models validate the efficacy and efficiency of the proposed model.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\nS1: This paper presents an novel integration of the mixture-of-experts mechanism with spatiotemporal graph learning, and designs some novel techniques with solid theoretical contributions.\n\nS2: The experiments conducted on 15 datasets are informative and comprehensive. And a 20-fold increase in efficiency is impressive.\n\nS3: The authors have open-sourced the code to facilitate reproducibility. Focusing on efficient model for large-scale data is a cutting-edge research direction.\n\n**Weakness:**\n\nW1: The numerical formatting in Table 2 is inconsistent. Some values use separators while others do not. There are also spelling errors. For example, in line 289, there is an extra space inside the parentheses: (60%). Figure 2(a) is suggested to be presented in tabular form to improve readability.\n\nW2:  The paper argues that the RULE function may introduce noise, but further experimental validation is needed to support this claim. \n\nW3: Some important spatiotemporal forecasting benchmarks should be included for further comparison, such as LLGformer [1].\n\nW4: The existing mixture-of-experts models in the spatiotemporal data domain should be discussed [2][3]. \n\n[1] Jin, D., Huo, C., Shi, J., He, D., Wei, J., & Yu, P. S. (2025, April). Llgformer: Learnable long-range graph transformer for traffic flow prediction. In Proceedings of the ACM on Web Conference 2025 (pp. 2860-2871)\n\n[2] Jiang W, Han J, Liu H, et al. Interpretable cascading mixture-of-experts for urban traffic congestion prediction. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024: 5206–5217.\n\n[3] Li S, Cui Y, Zhao Y, et al. ST-MoE: Spatio-temporal mixture-of-experts for debiasing in traffic prediction. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2023: 1208–1217.\n\nQUESTIONS:\nQ1: Is there a way to avoid multiple experts learning redundant knowledge?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce an efficient sparse yet balanced mixture-of-experts strategy to tackle the complexity associated with current adaptive graph learning methods. They underpin the validity of their approach with comprehensive theoretical analyses. Compared with 14 spatiotemporal models across 17 datasets, their model demonstrates remarkable performance, while achieving significant improvements in memory efficiency, with a 10× reduction, and training efficiency, with a 20× increase.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nS1. The proposed model is reasonably motivated and logically well-designed, effectively addressing various challenges step-by-step.\n \nS2. The idea of sparsing and balancing among the mixture of experts is admirable.\n \nS3. This paper presents new theoretical insights.\n\nS4. Extensive experiments including 17 datasets and 14 baselines prove the competitive performance and efficiency of their model. \n\n\nWeakness:\n\nW1. The authors have not provided an analysis or explanation for why PatchSTG fails to run on certain datasets. \n\nW2. Spatiotemporal forecasting is a subset of time series, necessitating further discussion around additional time series forecasting models, such as PatchTST and TimeMixer. \n\nW3. Is it possible to achieve performance gains by expanding the number of experts?\n\nRef:\n\n[1] Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730.\n\n[2] Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L., ... & Zhou, J. (2024). Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405.14616.\n\nQUESTIONS:\nWhat is the model's potential for scalability on extremely large-scale spatiotemporal graphs?\n\nLIMITATIONS:\nYes, the authors have discussed limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Mixture-of-Adaptive Graph Experts (MAGE), a framework for adaptive graph learning in spatiotemporal forecasting. The method uses a mixture-of-experts approach, where multiple learned graphs are combined using node-specific affinity weights. MAGE aims to reduce computational complexity and improve flexibility compared to traditional adaptive graph methods. The paper provides comprehensive theoretical analysis and experimental results on several real-world datasets to support its approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n(1) The proposed MAGE framework addresses the computational bottleneck of traditional adaptive graph learning by introducing a kernel-based approximation, which reduces the graph construction complexity from quadratic to linear with respect to the number of nodes. This design is particularly practical for large-scale spatiotemporal datasets and is supported by both theoretical analysis and empirical benchmarks.\n\n(2) By using a mixture-of-experts mechanism, MAGE allows for node-specific combinations of multiple learned graphs. This provides additional flexibility in capturing diverse and dynamic spatiotemporal dependencies, which could be beneficial in heterogeneous or complex forecasting scenarios.\n\n(3) The paper conducts extensive experiments across 17 real-world datasets from multiple domains, and includes ablation studies and efficiency comparisons. These experiments demonstrate that the method can achieve competitive, and sometimes state-of-the-art, results in both accuracy and efficiency when compared to a wide range of existing baselines.\n\n\n**Weaknesses:**\n\n**W1:** A major weakness of the paper is its oversimplified and outdated characterization of existing adaptive graph learning methods. The authors claim that dynamic graphs are typically constructed via pairwise embedding inner products, resulting in $O(N^2)$ computational complexity. However, this only describes a subset of early adaptive graph models such as AGCRN and GWNet. In fact, many recent adaptive graph construction approaches Implicit learning-based methods, avoiding explicit dense adjacency computation altogether. Furthremore, a large number of modern methods already employed top-K selection, sparse routing, or on-the-fly edge generation, achieved much lower complexity in practice [1] [2]. Therefore, presenting the $O(N^2)$ embedding inner product approach as representative of the entire field is inaccurate and ignores the significant methodological diversity and existing research progress that have been made in adaptive graph learning.\n\n\n\n**References:**\n\n\n[1] Huang, Siyuan, et al. \"Tailoring self-attention for graph via rooted subtrees.\" *Advances in Neural Information Processing Systems* 36 (2023): 73559-73581.\n\n[2] Qian, Chendi, et al. \"Probabilistic Graph Rewiring via Virtual Nodes.\" *The Thirty-eighth Annual Conference on Neural Information Processing Systems*.\n\n\n\n\n\n**W2: Limited Global Structural Consistency, Expressiveness, and Interpretability.** The design of MAGE enables each node to construct its own adaptive adjacency structure by selectively combining expert-generated graphs using node-specific affinity weights. While this approach allows for high local flexibility, it fundamentally breaks the consistency of a unified global graph structure. Unlike traditional GNNs, where all nodes share the same adjacency matrix. As a result, important global properties such as overall connectivity, cycles, community structure, or long-range dependencies may not be explicitly modeled or preserved. This lack of structural coherence can be particularly problematic in tasks where global graph topology or prior knowledge plays a crucial role.\n Additionally, the \"personalized\" and fragmented nature of the learned adjacency makes it extremely difficult to interpret or visualize the resulting graph, limiting its practical utility for scientific analysis, knowledge discovery, or downstream applications that require graph explanations. The method also lacks theoretical justification regarding its ability to model or recover key global attributes (such as symmetry, normalization, or spectral properties) that are well-studied in the GNN literature. Without systematic empirical or theoretical analysis on these aspects, it is unclear whether the proposed approach can generalize or perform robustly on problems where global graph characteristics are critical.\n\n\n\n**W3: Lack of Direct Graph-Level Validation and Analysis.** The experimental validation in this paper is largely indirect. The majority of the experimental section focuses on demonstrating that using MAGE leads to improved spatiotemporal forecasting performance on certain datasets, as measured by standard prediction error metrics. However, such improvements only provide indirect evidence of the effectiveness of the proposed graph learning method. The paper lacks more direct graph-level validation from the perspective of graph structure analysis. For example, there is no comparison of the sparsity or connectivity patterns of the graphs constructed by MAGE versus other methods, to show whether MAGE successfully preserves the most important edges. Additionally, the paper does not include any visualization or alignment of the learned graphs with the true underlying system structure at time $t$, to demonstrate that the edges constructed by MAGE meaningfully correspond to real, strong connections in the data. This omission makes it difficult to assess whether the proposed method actually learns more accurate or interpretable graph structures.\n\n\n\n**W4:** **Insufficient Empirical Evidence for the Claimed Drawbacks of ReLU Activation.** The authors claim that the use of ReLU activation in existing adaptive graph learning methods amplifies edge-level noise, resulting in the creation of spurious or unreliable connections in the learned adjacency matrix. However, this assertion is only supported by a simple theoretical analysis, without any dedicated empirical validation. The paper does not provide ablation studies, visualizations, or quantitative experiments to directly demonstrate the negative impact of ReLU-induced noise or to substantiate the practical advantages of their proposed alternative. As a result, the significance of this issue remains unconvincing.\n\nQUESTIONS:\nFor questions, please see the above raised Weakness 1 - Weakness 4. \n\nAside from W1-W4, I also have another question about the storyline of this paper. \nThis paper **has unclear motivation**. **The storyline in the Introduction is not convincing and lacks logical coherent, particularly in the introduction section.** The authors first claim that the commonly used ReLU activation function in adaptive graph learning disproportionately amplifies negative edge weights and reinforces noisy edges. To address this, they propose replacing ReLU with a kernel-based approximation scheme. However, they then argue that this approximation introduces a low-rank bottleneck in the learned adjacency matrix, which subsequently motivates the introduction of their Mixture-of-Experts (MAGE) framework. Thus, my question is: Was the motivation for introducing MAGE simply to counteract the side effects introduced by using a kernel-based approximation scheme as a replacement for the ReLU activation function?\nThis sequence of arguments is problematic: if the main issue with ReLU is noise amplification, it is unclear why the only possible alternative must be a kernel-based function that suffers from low-rank limitations. The authors do not sufficiently justify why other activation or similarity functions that avoid both noise amplification and low-rank issues could not be used instead. As a result, the motivation for MAGE appears ad hoc and unconvincing, undermining the overall contribution and logical flow of the paper.\n\nLIMITATIONS:\nPlease see the above raised Weakness 1 - Weakness 4", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}