Title: SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater
Venue: NeurIPS 2025
Accepted: yes
Keywords: Graph neural networks

dynamic graphs

temporal information
Abstract: Dynamic graph learning is crucial for accurately modeling complex systems by integrating topological structure and temporal information within graphs. While memory-based methods are commonly used and excel at capturing short-range temporal correlations, they struggle with modeling long-range dependencies, harmonizing long-range and short-range correlations, and integrating structural information effectively. To address these challenges, we present SALoM: Structure Aware Temporal Graph Networks with Long-Short Memory Updater. SALoM features a memory module that addresses gradient vanishing and information forgetting, enabling the capture of long-term dependencies across various time scales. Additionally, SALoM utilizes a long-short memory updater (LSMU) to dynamically balance long-range and short-range temporal correlations, preventing over-generalization. By integrating co-occurrence encoding and LSMU through information bottleneck-based fusion, SALoM effectively captures both the structural and temporal information within graphs. Experimental results across various graph datasets demonstrate SALoM's superior performance, achieving state-of-the-art results in dynamic graph link prediction. Our code is openly accessible at https://github.com/wave5418/SALoM.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on temporal graph learning. It proposes a new method, SALoM, to capture long-range neighborhood temporal correlations. SALoM adopts ODE to update memory and introduces a long-short memory updater to balance long-range and short-range temporal dependencies. And to incorporate both structural and temporal information, it adopts information bottleneck-based fusion, where the structural information is obtained by co-occurrence encoding. SALoM outperforms baselines, especially on USLegis, UNtrade, and UNvote.\n\nSTRENGTHS AND WEAKNESSES:\nS1. This paper presents a novel approach to capturing long-range neighborhood temporal correlations and balancing the learning of temporal and structural information. The components of SALoM are technically sound. \n\nS2. SALoM demonstrates outstanding performance across several datasets, highlighting its effectiveness in addressing certain predictive scenarios.\n\nW1. A major concern is that the datasets used in the experiments are outdated. Several studies have shown that these datasets are no longer sufficient for evaluating the full capacity of DGNNs. I recommend evaluating SALoM on the more recent TGB[R1] and TGB-Seq[R2] benchmarks to ensure a comprehensive assessment.\n\nW2. There may be a potential flaw in the implementation of SALoM, which is shared by other memory-based methods such as TGN, DyRep, and JODIE. Specifically, when considering two edges, (a,b,t1) and (a,c,t1), these edges are placed in different batches during testing because there are too many edges occurring at t1. However, when testing (a,c,t1), the memory of node \"a\" has already been updated by (a,b,t1), leading to information leakage. This violates the model’s design, as it should not have access to edges at time t1 when the prediction time is t1.\n\n[R1] Temporal Graph Benchmark for Machine Learning on Temporal Graphs\n\n[R2] TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics\n\nMinor: \nFigures 2 and 3 mistakenly include the x-label \"Number of Sampled Neighbors,\" which should only appear in Figure 4.\n\nQUESTIONS:\nQ1. Can LSMU be adopted to other memory-based models, such as TGN, to improve their performance?\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SALoM, a dynamic graph learning framework that addresses three important challenges in temporal graph networks:  \n\nThe effectiveness of capturing long-range temporal dependencies,  \n\nThe difficulty of balancing long/short-term correlations, and  \n\nThe consistency of integrating structural information.  \n\nExperiments on multiple datasets show state-of-the-art performance, with particularly large gains on legislative/voting graphs. SALoM consistently outperforms baselines in both transductive and inductive link prediction tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1(quality) This paper is methodologically rigorous, with claims well-supported by both theoretical analysis and extensive experiments. Comprehensive ablation studies also validate the effectiveness of individual components in a convincing manner. \n\n \n\n2 (clarity) This paper is well-structured, with a clear narrative flow from problem definition to methods and results. This paper also provides sufficient details, such as hyper-parameters and dataset splits. The code is also released in Anonymous GitHub which further contributes to good reproducibility. \n\n \n\n3 (significance) The experimental results of SALoM proposed by this paper clearly outperform typical memory-based and sequence-based baselines, advancing the field by harmonizing existing methods’ advantages while mitigating their limitations. \n\n \n\n4 (originality) Based on Section 2 and Table 1, this paper effectively contrasts SALoM with prior work, highlighting limitations in handling long-term dependency and managing training costs. \n\n \n\n \n\n \n\nWeaknesses: \n\n1(quality) Although efficiency trade-offs are discussed in subsection 4.5, Figure 5 lacks a detailed comparison of Long-Short Memory Updater (LSMU) overhead, such as the routing latency of MoE, versus selective baselines, which is important in real-world development. \n\n \n\n2 (clarity) The neural ODE equations (Equations 1 and 2) and Information Bottleneck (IB) objectives (Equations 17, 18, and 19) could benefit from more intuitive explanations or illustrative examples. For example, a toy case study can be discussed with simple plots to show how ODEs applied in SALoM could mitigate the gradient vanishing issue. \n\n \n\n3 (significance) Concrete examples of deployed applications are encouraged, such as policy change tracking, as they help strengthen the case for real-world significance. \n\n \n\n4 (originality) This paper could better explain how SALoM’s MoE design differs from other MoE-based graph neural networks.\n\nQUESTIONS:\nQuestion 1: Can you please quantify computational costs, such as training runtime and memory space occupation, of LSMU’s expert routing versus existing updaters? \n\n \n\nQuestion 2: For sparse graphs, does co-occurrence encoding degrade due to hash collisions?  \n\n \n\nQuestion 3: Can SALoM handle edge feature dynamics, such as evolving transaction amounts? \n\n \n\nQuestion 4: Please also check the weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SALoM (Structure-Aware Long-Short Memory), a novel framework for temporal graph learning that aims to balance the trade-off between short-range and long-range temporal dependencies—an issue that is often overlooked in prior work. The authors design a memory module enhanced by a Long-Short Memory Updater (LSMU), leveraging a Mixture-of-Experts (MoE) architecture to adaptively combine different temporal signals. Structural information is integrated via co-occurrence encoding and an information bottleneck (IB)-based fusion mechanism. Experimental results across thirteen dynamic graph datasets show that SALoM achieves strong performance in dynamic link prediction tasks, outperforming existing state-of-the-art models, particularly on datasets like USLegis, UNTrade, and UNVote.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nS0: The motivation to simultaneously handle short- and long-range temporal dependencies is well-justified and addresses a practical gap in temporal graph learning.\n\nS1: The design of LSMU using a MoE mechanism (with GRU and CFC experts) is an elegant and potentially effective way to achieve dynamic temporal adaptation.\n\nS2: The experimental section is comprehensive, covering 13 datasets and a broad set of baselines, with ablations on memory modules, fusion strategies, and neighborhood size.\n\nS3: The authors offer a solid discussion on the limitations of prior approaches, such as RNN-based memory methods and neighbor-sequence models, highlighting how SALoM differs in intent and implementation.\n\n**Weaknesses**\n\nW0: While the paper introduces several technical modules (ODEs, IB fusion, hash-based co-occurrence encoding), their individual motivations are not always clearly articulated. Clarifying why each design choice is necessary would improve the exposition.\n\nW1: The overall writing feels verbose and sometimes disorganized, making it harder to follow the technical contributions, particularly in Sections 3.1 and 3.2.\n\nW2: The rationale behind using Neural ODEs (and the CFC variant) for temporal memory modeling could be better justified, especially compared to simpler alternatives like attention or GRUs.\n\nW3: Several inconsistencies and typos were found, including:\n+ In Figures 2–4, the x-axis is uniformly labeled “Number of Sampled Neighbors?”, which appears to be an error.\n+ Appendix B is present only with a heading but lacks any actual content.\n\nW4: From Figure 5, performance degrades as batch size increases, potentially limiting SALoM’s scalability to larger datasets, where larger batches are often necessary for efficiency.\n\nW5: The number of experts (Q) and selected experts (r) in the MoE mechanism are not explored in depth. Understanding their effect on performance and computational cost would be important for users aiming to apply SALoM in practice.\n\nQUESTIONS:\n1. The color and shape coding in Figure 5 could be more distinct. Currently, it is visually cluttered and hard to differentiate across batch size settings and models.\n2. Please complete or remove Appendix B to avoid confusion.\n3. What would happen if the structural encoder simply used a unified co-occurrence hash map (i.e., no distinction between short- and long-range neighbors)? Would this significantly degrade performance?\n4. MoE hyperparameter sensitivity.\n+ What is the impact of setting r = 1 or using only a single GRU/CFC expert? Does performance degrade significantly?\n+ How does increasing the number of experts or varying the r value affect training speed and accuracy?\n+ A brief analysis or ablation on these hyperparameters would provide more clarity on their necessity and practical trade-offs.\n5. Can the authors further justify the choice of using Neural ODEs over more mainstream approaches (e.g., attention with decay) for modeling long-range dependencies?\n\nLIMITATIONS:\n1. The scalability limitations revealed in Figure 5, especially with large batch sizes.\n2. The computational cost associated with MoE routing and ODE solvers (even in closed-form variants).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a dynamic graph learning framework called SALoM, designed to effectively capture both long- and short-range temporal dependencies while incorporating graph structural information. To overcome limitations of existing memory-based and neighbor-based methods, SALoM introduces a Long-Short Memory Updater (LSMU) that leverages a mixture-of-experts architecture to dynamically balance influences from long-term and short-term memory using neural ODEs and GRUs. It also integrates co-occurrence-based structural encoding through an information bottleneck-based fusion module, allowing temporal and structural features to be combined without conflict. Extensive experiments on multiple benchmarks demonstrate SALoM’s superior performance, particularly in dynamic link prediction tasks, where it significantly outperforms prior methods in both accuracy and adaptability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The idea of using mixture of experts for choosing between long and short-term memory modules in interesting and novel for dynamic graphs.\n2. Significant performance improvements in the transudative setting for UN Trade and Vote datasets.\n\nWeaknesses:\n\n1. Weak motivation for some modules: The authors imply that GRUs are for short-term memory and CFC cells for long-term memory. Why are GRUs for short-term memory? Why not use a simpler module for short-term memory like aggregating last K neighbors as done in TGAT [1]. Rather than using an MoE module for fusion, why not concatenate the outputs of both modules? For the co-occurance encoder, why not use a simple message passing architecture instead to capture the structural information? Your architecture is relatively complicated so every module must be motivated with experiments.\n\n2. Slight performance degradation in the inductive setting.\n\n[1] Xu et al. Inductive Representation Learning on Temporal Graphs. ICLR. 2020\n\nQUESTIONS:\nSee weaknesses. Additionally:\n\n1. What is the effect of number of experts with respect to performance?\n2. Why does the performance degrade as the number of historical neighbors increase (Figure 4)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}