Title: Causal Spatio-Temporal Prediction: An Effective and Efficient Multi-Modal Approach
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spatial-temporal prediction; multi-modal data; causal inference
Abstract: Spatio-temporal prediction plays a crucial role in intelligent transportation, weather forecasting, and urban planning. While integrating multi-modal data has shown potential for enhancing prediction accuracy, key challenges persist: (i) inadequate fusion of multi-modal information, (ii) confounding factors that obscure causal relations, and (iii) high computational complexity of prediction models. To address these challenges, we propose E$^2$-CSTP, an Effective and Efficient Causal multi-modal Spatio-Temporal Prediction framework. E$^2$-CSTP leverages cross-modal attention and gating mechanisms to effectively integrate multi-modal data. Building on this, we design a dual-branch causal inference approach: the primary branch focuses on spatio-temporal prediction, while the auxiliary branch mitigates bias by modeling additional modalities and applying causal interventions to uncover true causal dependencies. To improve model efficiency, we integrate GCN with the Mamba architecture for accelerated spatio-temporal encoding. Extensive experiments on 4 real-world datasets show that E$^2$-CSTP significantly outperforms 9 state-of-the-art methods, achieving up to 9.66% improvements in accuracy as well as 17.37%-56.11% reductions in computational overhead.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author proposes E^2-CSTP, a multi-modal spatial-temporal prediction framework that integrates multiple sources of data. The framework has three main components: cross-modal feature fusion, dual-branch causal inference, and spatial-temporal encoding. Experiments are conducted on different datasets, showing significant improvements over the current SOTA methods, achieving a notable improvement in accuracy and computational burden.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n- The paper provides comprehensive empirical studies against 9 baseline methods on 4 datasets on multiple dimensions.\n- The framework's approach to fuse time-series, text, and image data together is a novel approach.\n- Linear-time STED predictor replaces quadratic self-attention, resulting in good improvement on computational efficiency.\n\n\nWeakness\n- The adjacency matrix is built from DeepSHAP importances and without comparison to standard causal-discovery methods. It is stated that the matrix was updated every P epochs, but the author did not mention how the value of P was determined. No ablation test on different values of P\n- For cross-modal feature alignment, the inputs are aligned using deterministic ST matrix thresholds. And the experiment did not test the robustness of this. This suggests the model presumes perfectly matched geo-temporal stamps. Such practice will be sensitive to issues like missing data points or irregular sampling\n- The claimed linear time performance comes after every batch has already spent time and resources on the full BERT and the CNN. But the paper did not address the time performance of this part. \n- The performance is highly sensitive to the manually tuned hyperparameters. Fig 8 and 9 show that suboptimal λ and the β value can degrade the performance on an observable scale. And the author picks up different values for different datasets, showing issue of strong coupling. The generality of this framework is questionable since it has to do a grid search on each new dataset for those values.\n\nQUESTIONS:\nAs weakness\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents E2-CSTP, a novel framework for causal, multi-modal spatio-temporal prediction, aiming to address challenges in multi-modal data fusion, causal confounding, and model efficiency. E2-CSTP integrates cross-modal attention and gating mechanisms to fuse multi-modal inputs and employs a dual-branch design—one branch for spatio-temporal prediction and another for causal intervention to mitigate bias. The model combines Graph Convolutional Networks (GCNs) with the Mamba architecture to enhance computational efficiency. Experimental results on four real-world datasets show up to 9.66% accuracy improvement and 17.37%–56.11% reductions in computational overhead compared to nine state-of-the-art baselines.\n\nHowever, the paper falls short on two key points. First, it naively combines existing tools, offering limited novelty given the abundance of similar works on multi-modal data fusion. Second, if the primary goal is prediction accuracy, the incorporation of causal inference is unnecessary and not well-justified in this context.\n\nSTRENGTHS AND WEAKNESSES:\nThe authors claimed four major contributions in this work. First, they propose a unified multi-modal spatio-temporal prediction framework that integrates environmental images, event-related text, and time-series data using cross-modal attention and adaptive gating. Second, they introduce a dual-branch causal design, where the main branch captures spatio-temporal patterns and the auxiliary branch applies causal interventions to reduce confounding bias. Third, they develop an efficient hybrid model combining GCN for spatial encoding and Mamba for temporal modeling to accelerate inference. Fourth, extensive experiments on four real-world datasets demonstrate up to 9.66% improvement in accuracy and 17.37%–56.11% reductions in computational overhead compared to nine baselines.\n\nThere are several methodological concerns that should be addressed. First, the use of unified binary alignment matrices presents major limitations in handling irregular time series (or longitudinal data) and irregular spatio-temporal data. Second, several key notations are introduced without explicit definitions. For example, what are \\( F_{st} \\) and \\( \\tilde{F}_{st} \\)? These should be clearly defined prior to their use. Similarly, the meaning of \\( N_{st} \\), and the relationship between \\( N \\) and \\( \\{N_i\\} \\), should be clarified. Third, it remains unclear whether the confounding variable \\( S \\) varies across space and time. This is a significant issue, as spatio-temporal systems often involve unobserved spatio-temporal confounders, and causal relationships may themselves vary over space and time.\n\nQUESTIONS:\nThere are several methodological concerns that should be addressed. First, the use of unified binary alignment matrices presents major limitations in handling irregular time series (or longitudinal data) and irregular spatio-temporal data. Second, several key notations are introduced without explicit definitions. For example, what are \\( F_{st} \\) and \\( \\tilde{F}_{st} \\)? These should be clearly defined prior to their use. Similarly, the meaning of \\( N_{st} \\), and the relationship between \\( N \\) and \\( \\{N_i\\} \\), should be clarified. Third, it remains unclear whether the confounding variable \\( S \\) varies across space and time. This is a significant issue, as spatio-temporal systems often involve unobserved spatio-temporal confounders, and causal relationships may themselves vary over space and time.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

comments: SUMMARY:
This ppaper presents a new framework for spatio-temporal prediction. The framework has two branches, a main branch that focuses on spatio-temporal prediction and a secondary branch that models additional modalities and introduces causal interventions. The authors also combined GCN and Mamba for efficiency. They used four datasets to evaluate the proposed framework, demonstrating its improvement in accuracy and computational overhead relative to the SOTA approach.

STRENGTHS AND WEAKNESSES:
Strength.
1. Rationality of design. The key innovation of this framework lies in the designed auxiliary branches and the introduction of the GCN and Mamba architectures, a combination that is intuitively sensible and exemplifies the innovation of hybrid architectures.
2. quality of the paper. This paper is well written and easy to understand. Ideas, task definitions, and methods are described clearly.
3. comprehensive experiments. The baseline selected for the comparison experiments is detailed and innovative. It seems to have enough workload.
4. open project.
Weekness 
1. Although models use causal reasoning to reduce the impact of confounding factors, their internal mechanisms remain relatively complex, making it difficult to intuitively understand the model's predictive process. How do these steps further influence the model's decisions? A visual analysis would be nice.

QUESTIONS:
1. which of the main components of the framework are customized for spatio-temporal prediction? Which parts can be generalized broadly to other tasks?
2. the article mentions that the model employs causal reasoning to reduce the impact of confounding factors, how can it be explained more intuitively how the model distinguishes between causality and confounding factors?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces E²-CSTP, a novel framework for multi-modal spatio-temporal prediction that addresses three key challenges: (1) insufficient fusion of multi-modal data, (2) confounding factors obscuring causal relations, and (3) high computational complexity. The proposed framework integrates cross-modal attention and gating mechanisms for effective multi-modal fusion, employs a dual-branch causal inference approach to mitigate bias, and combines GCN with the Mamba architecture for efficient spatio-temporal encoding. Extensive experiments on four real-world datasets demonstrate that E²-CSTP outperforms nine state-of-the-art baselines in accuracy (up to 9.66% improvement) and computational efficiency (17.37%-56.11% reduction in overhead).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. Comprehensive Multi-Modal Fusion: The framework effectively integrates heterogeneous data (spatio-temporal sequences, text, and images) using cross-modal attention and adaptive gating, enabling richer feature representations.\n2. Innovative Causal Inference: The dual-branch design disentangles true causal relations from confounders, enhancing interpretability and robustness in predictions.\n3. Computational Efficiency: By combining GCN for spatial encoding and Mamba for temporal modeling, the framework achieves linear complexity, significantly reducing computational overhead compared to Transformer-based methods.\nWeaknesses\n\n1. Limited Modality Scope: The framework focuses on text, images, and spatio-temporal data, but other modalities (e.g., audio, sensor networks) are not explored, which could limit its applicability in certain domains.\n2. Dependence on Pre-trained Models: The use of BERT and CNN for feature extraction assumes availability of high-quality pre-trained models, which may not hold for all languages or niche domains.\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nSee above", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}