Kati8/ICLR_2023_2025 huggingface dataset:
authors: Kun Wang
authors: Hao Wu
authors: Yifan Duan
authors: Guibin Zhang
authors: Kai Wang
authors: Xiaojiang Peng
authors: Yu Zheng
authors: Yuxuan Liang
authors: Yang Wang
reviews: 
Comments: This paper explores the application of deep learning to dynamical systems and proposes a new philosophical framework called "NuwaDynamics", which aims to mine data for causal patterns to enhance the explanatory and generalization capabilities of models.  The approach is divided into two phases: Discovery, which identifies the causal components of the data, and Update, which applies the causal model to downstream tasks.  In addition, NuwaDynamics enhances the model's performance in sparse and extreme situations.  Experimental results in the article show that the framework achieves excellent results in several benchmark tests.
Strengths: (1) A conceptually and technically highly innovative paper which seamlessly integrates causal theory with spatio-temporal data mining. The integration of causal theory's environment discovery and augmentation into the popular vision transformer's attention map is ingenious. Coupled with a straightforward mixup augmentation, it achieves a very refined alignment. From my perspective, NuwaDynamics has the potential to revolutionize how models recognize causality in ST data mining realm. This may provide a new approach to solving such problems (data scarcity and the high cost of deploying sensors). 

(2) The paper reads very well, and hardly has any errors or inconsistencies. Information is provided at the right point, is complete and accurate. The split between main paper and supplementary material is good. The narrative and exposition flow well. Meanwhile, the paper is well-structured, I appreciate its visual aids.

(3) Providing a causal intervention perspective for data augmentation seems interesting. In terms of practical implementation, utilizing attention scores without introducing additional complex designs seems both simple and effective. Additionally, I believe the backdoor adjustment mechanism proposed by the authors serves as a robust theoretical foundation. This contributes significantly to enhancing generalization, especially in out-of-distribution scenarios.

(4) Baselines are strong, relevant, discussed well and evaluated fairly. A large number of analytic experiments complete the main findings. Meanwhile, experimental setups are presented accurately and consistently, at an adequate level of detail.
Weaknesses: (1) The paper does not seem to make it clear exactly how the process of obtaining the importance rankings in the patches is done, and it is hoped that the authors will add that detail.

(2) In referring to the fact that DL methods often forgo an explicit understanding of the rules of physics, the practical implications and potential risks of this sacrifice are not explained in detail. More discussion or explanation might have given the reader a better understanding of the consequences of this choice.
Recommendation: 8: accept, good paper
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 4 excellent
Clarity: 3 good
reviews: 
Comments: This paper established a causal concept for spatio-temporal predictions. In particular, it firstly identifies causal regions at the Discovery step. It then augments non-causal patches at the Update stage. By doing so, the model is able to see a broader potential distribution of data, achieving improved downstream performance. Extensive experiments show that the proposed model can benefit many existing frameworks on various tasks.
Strengths: **Originality:** This paper established a causal concept for spatio-temporal predictions and introduced a novel philosophical framework. Its causal perspective is interesting and original. It is also addressing an important challenge faced by AI community which is the interpretability and generalizability. 

**Presentation:** This paper is well written. The flow of the paper is smooth and easy to follow. Figure 1 and Figure 2 are very helpful to illustrate the central idea of the paper. 

**Experiments:** the experiments are extensive, covering a wide range of backbones and tasks. The experimental analysis is well organized with key questions and itemized observations.
Weaknesses: **The experiments are insufficient.** The performed experiments are focused on the comparison between the performance of an original backbone and the performance of adding NuWa. There is a lack of comparison to SOTA performance on the evaluated tasks. Besides, it is also important to compare to other data-augmentation methods (even though they are not using causality), which is also missing in the paper.  

Besides, there is a lack of evaluation on the causal discovery accuracy. Causal discovery always requires a large amount of data to ensure accurate discovery of causal relationships, and it can suffer from highly imbalanced data. How does your model address this issue as you are particularly interested in such regime?
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 2 fair
Clarity: 3 good
reviews: 
Comments: This paper proposes a novel causal framework called NuwaDynamics for spatio-temporal (ST) prediction tasks. The key idea is to leverage self-supervised reconstruction tasks to identify causal regions in the input data, and perform interventions on non-causal regions to expose the model to a broader distribution of data. The framework consists of two main stages: 1) Causal patch discovery, where a Vision Transformer is trained on a reconstruction task and attention maps are used to localize causal patches. 2) Causal model update, where environmental patches are modified through interventions and used to train the downstream model. Experiments on 6 benchmarks demonstrate improved performance over strong baselines.
Strengths: 1. This paper is well-organized and clearly written.
2. Introducing causality concepts into spatio-temporal modeling is novel.
3. The motivation is clear and reasonable.
4. Visualizations demonstrate improved detail and extreme weather perception.
5. The experiments across multiple datasets are thorough.
Weaknesses: 1. It seems that the computational complexity is high due to generating many intervened sequences.
2. Lack of ablation studies to validate design choices.
3. Except for the KTH dataset, the performance gains are relatively low.
4. Lacks of the comparison on computational complexity.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 2 fair
Clarity: 3 good