CausalDiffusion: Causally Related Time-Series Generation through Diffusion Models

Authors: Giuseppe Masi, Andrea Coletta, Novella Bartolini
Primary area: generative models
Keywords: time-series generation, causal discovery, diffusion model, benchmark, dataset, synthetic data

Abstract:
Understanding the intrinsic causal structure of time-series data is crucial for effective real-world interventions and decision-making. 
While several studies address the Time-Series Causal Discovery (TSCD) problem, the lack of high-quality datasets may limit the progress and evaluation of new methodologies. 
Many available datasets are derived from simplistic simulations, while real-world datasets are often limited in quantity, variety, and lack of ground-truth knowledge describing temporal causal relations. 
In this paper, we propose CausalDiffusion, the first diffusion model capable of generating multiple causally related time-series alongside a ground-truth causal graph, which abstracts their mutual temporal dependencies.
CausalDiffusiom employs a causal reconstruction of the output time-series, allowing it to be trained exclusively on time-series data. 
Our experiments demonstrate that CausalDiffusion outperforms state-of-the-art methods in generating realistic time-series, with causal graphs that closely resemble those of real-world phenomena. 
Finally, we provide a benchmark of widely used TSCD algorithms, highlighting the benefits of our synthetic data with respect to existing solutions.

Decision: reject
Reviewer scores: 3, 6, 3, 5

Reviews and discussion:
<thread>
creation date:
1730191839825

thread_type:
official_review

<thread_object>
creator: reviewer_ybqg

<content>
summary:
The author proposes a novel generative framework named CausalDiffusion that generates a causal graph and the time series directly within a diffusion model architecture.

strengths:
1. The generation pipeline is notable for not assuming stationarity, allowing for the generation of diverse graph types.
2. To my knowledge, this is the first work addressing causal time-series generation using diffusion processes.
3. The author provides evaluation of causal graphs in the experiments.

weaknesses:
1. While the method is straightforward, there is excessive focus on dataset descriptions, metrics, and evaluations, leading to a lack of depth in analyzing the rationale behind the method.
2. I am skeptical about the overarching scheme of learning sparse causal graphs while generating time series. This process resembles causal discovery based on generated time series, which are then used to evaluate other causal discovery algorithms. Essentially, the approach is "discovering a causal graph to generate time series, and then testing causal discovery algorithms," but there's no guarantee that the causal graphs learned during this process closely align with true causal graphs present in nature. Still, I am open to debates. If I am convinced, I will be happy to raise my score.

questions:
1. In Lines 230-236, after $\operatorname{DEN}_\theta$ outputs the initial steps, subsequent steps are generated using a linear model:  $\hat{x}_0[l, i]=\mathbf{c}_l^i \cdot \hat{\mathbf{x}}_0 \left[ l - \tau: l-1,:\right]$ 
While I understand that the weights evolve over time, can this generation methodology effectively generalize to accommodate any nonlinearity? A more rigorous theoretical analysis is needed.
2. In Lines 255-257, the authors state that "we aggregate over the time steps by computing the 95th percentile." This suggests that the causal graph $\hat{g}$ may not be a strict causal summary graph. For instance, if the weights along the time axis are $[0, 0, ..., 0, 0, 1]$, the 95th percentile could still be zero. This raises concerns about the causal graph's fidelity to the generated time series.

Minor:
1. The notation $x_0[l,i]$ in Line 227 can be confusing, as the superscripts and subscripts do not align with $c_l^i$. Could the notation be improved for clarity, such as using $x_{(l,i)}^0$?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730570148393

thread_type:
official_review

<thread_object>
creator: reviewer_wxkz

<content>
summary:
Structure learning from time series and diffusion models have been of high interest to a subset of the community. The current work proposes a diffusion model for the generative task of time series generation and causal structure learning. A key motivation for the work seems to be that " ..., this is the first work to incorporate diffusion models for causally related time-series generation." However being first is not a merit on its own but should be accompanied by having a clear motivation i.e. there is no particular motivation or use of a diffusion model beyond being a good generative model. But no specific property of a diffusion model seems to be used beyond the fact that it is a good generative model. 

Opportunities for improvement:
i) In order to justify the interest of the community it would be good if then at least all possible variations or closely related models of the diffusion model class are studied e.g. flow matching, Generator Matching, consistency models etc and evaluated rather than just being first for a special case of diffusion models. Why was the paper not studied with the more general class of flow matching for example? Why not use a consistency model etc? 

ii) The tile or short "CausalDiffusion" is misleading since and potentially an abbreviation with time should be used instead. It just seems to be too optimized for google hits. 

iii) The Markovian condition seems to be a very strong assumption. While it might be true that other works assume it too, it would be good to see some progress in the number of assumptions we need to make when developing new approaches. The same holds true for causal sufficiency i.e. no latent confounding and causal stationarity. These are all bottlenecks for practical deployment. 
At least for the number of assumptions the approach makes, there is no progress to see compared to even very old approaches. Given that these assumptions are very limiting in practical use cases, I would expect to see a clear ablation on the robustness against miss-specification of these assumptions when a new algorithm is introduced. For practical relevance, it is much less interesting to achieve SOTA on a leaderboard compared to being robust against assumption violation. See e.g. [1] where this is done for causal discovery algorithms. Without this analysis or better guarantees it is highly unlikely the approach would actually be used in practice and it should not be up to the reader to perform that analysis. 

iv) Moreover it would be good to see the evaluation not wrt. metrics like dimensionality reduction but wrt. especially real world downstream tasks e.g. by training a policy on top of the learned graphs or a classification task on top of the learned graph. Interpreting UMAPS and t-SNEs should only be done if no other metric is available. 

v) Rather than tables with the top performance it would be better to show the classical training vs performance curves used in deep learning e.g. training time on the x-axis and performance for a metric on the y-axis. Given that there is no finite sample guarantee or any guarantee, but it is a purely heuristic approach it would be better to shift to these plots similarly to how you show training curves in deep learning. 

vi) Where are the results on the Dream3 dataset which is mentioned but does not seem to be evaluated on? 


Unfortunately, in its current form, I do not see a clear motivation or a clear benefit of the work and do not think it is ready to be presented at the conference. I did not generate the review with an LLM but spent significant time reading the paper since I find the problem interesting and hope that the authors find the review helpful. 

[1] Montagna, Francesco, et al. "Assumption violations in causal discovery and the robustness of score matching." Advances in Neural Information Processing Systems 36 (2024).

strengths:
see above

weaknesses:
see above

questions:
One point which is unclear to me but I might have misunderstood that is "using our synthetically generated datasets, highlighting the practical benefits of our data". The benefit as far as I see was simply benchmarking, which could have been achieved with the original data. It would have been a real practical benefit if the model generated synthetic data which could have been used to augment the original real world data for a better performance, but that was not demonstrated. So I unfortunately missed a key contribution or motivation of the work especially since the paper does a very good job in mentioning all the established benchmarks and datasets.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730632879534

thread_type:
official_review

<thread_object>
creator: reviewer_qgem

<content>
summary:
The paper introduces CausalDiffusion, a generative framework for synthesizing time-series with an associated causal graph, addressing a significant limitation in Time-Series Causal Discovery (TSCD). CausalDiffusion is the first framework to integrate diffusion models for generating causally related multivariate time-series data, alongside a ground-truth causal graph that abstracts temporal dependencies. The model employs a tau-lag Vector Autoregressive (VAR) structure within the diffusion model to naturally encode these causal relationships. The experiments demonstrate superior performance in generating realistic time-series data and more accurately recovering causal graphs compared to existing approaches. The paper also introduces new evaluation metrics and benchmarks various causal discovery algorithms on the generated synthetic datasets.

strengths:
1. Novelty in Causal Time-Series Generation: The use of diffusion models to generate causally related time series, coupled with simultaneous causal graph extraction, is a novel approach. Unlike previous methods, which rely on post hoc analysis to infer causal graphs, this framework generates the causal graph directly alongside the time series.

2. Integration of Diffusion with Causal Discovery: The incorporation of VAR structures within the diffusion model enables realistic generation while maintaining explicit causal links. This integration allows CausalDiffusion to support unsupervised learning of causal graphs, making it applicable to a variety of time-series datasets.

3. Thorough Evaluation Across Real and Synthetic Datasets: The authors provide a comprehensive evaluation of the proposed model on both real-world and synthetic datasets, including Hénon, Rivers, and Air Quality Index (AQI) datasets. The results consistently show that CausalDiffusion outperforms other SOTA generative models like CAUSALTIME and CR-VAE in generating realistic time-series data and more accurately recovering causal relationships.

4. Benchmarking for TSCD Algorithms: The paper also uses the generated datasets to benchmark causal discovery algorithms such as DYNOTEARS, PCMCI+, and CUTS+. The extensive comparison across different TSCD methods underscores the practical utility of the synthetic datasets for evaluating and improving causal inference models.

5. Metrics for Causal Evaluation: The introduction of new evaluation metrics, including Granger Causality False Positive Rate (GC-FPR) and Graph False Positive Rate (Graph-FPR), provides more robust tools for assessing the quality of the generated causal graphs.

weaknesses:
1. Comparison Against State-of-the-Art Approaches: The comparison against state-of-the-art methods could be improved by incorporating a broader range of generative models, including more recent GAN-based and attention-based models used in time-series generation. Furthermore, the evaluation is somewhat limited in showcasing how these approaches would fare against real-world causal discovery problems beyond the chosen datasets.

2. Majority Strategy for Causal Graph Extraction: The aggregation strategy for extracting causal graphs based on the strongest 95th percentile coefficients might lead to overlooking personalized or localized causal relationships, which could be critical in diverse or non-stationary data. This limitation might impact the framework's ability to capture more nuanced causality specific to different time points or individuals.

3. Scalability Concerns: Training a diffusion model for high-dimensional time series is computationally demanding. The experiments show that the inference time for generating both synthetic data and causal graphs, especially when using the complete model (OUR W/L2 W/DTW), was substantially longer compared to simpler alternatives such as CR-VAE. This might limit the usability of CausalDiffusion in real-time scenarios or for larger-scale datasets without further optimization.

4. Dependency on Hyperparameter Tuning: The model's performance is highly dependent on hyperparameter tuning, particularly the regularization terms such as the ℓ2-norm and the Dynamic Time Warping (DTW) loss. The authors present several variants of their model, and their results suggest that a significant degree of manual tuning is required to achieve the best performance. This makes the framework less accessible to practitioners without a deep understanding of the hyperparameter effects.

5. Limited Analysis of Real-World Applicability: The evaluation of real-world datasets (e.g., AQI) does not explicitly address potential confounders or latent variables that could affect causality. It is important to assess how latent confounders would impact the generated causal graphs, especially in complex real-world applications like healthcare or finance.

questions:
1. How can the scalability of the CausalDiffusion model be improved to facilitate real-time usage? Could techniques such as implicit diffusion models or parallelization be used to reduce computational overhead?
2. How does the framework handle latent confounders that might not be explicitly represented in the training data?
3. How robust is the graph extraction method to noise in the time series? Would adding a denoising or regularization mechanism help in stabilizing the graph generation process?
4. Beyond the specific datasets used, how generalizable is the model to different time-series domains (e.g., healthcare or financial data)?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730645335328

thread_type:
official_review

<thread_object>
creator: reviewer_qjar

<content>
summary:
the paper proposes a time series datasets, that are generated by using diffusion models.  Ground-truth Granger-causal graphs can also be extracted from the diffusion model. Experiments result show, by using stand metrics and a modified F1 score , the generated samples and graphs are better.

strengths:
- the paper proposes to generate both data and ground truth causal graphs using diffusion models, which is a need in literature in evaluating different dynamic causal models. 

- the generated samples are valided to be better

weaknesses:
- overall, the novelty of the paper seems limited.  It builds on the same approach to an existing work(Yuan and Qiao, 2024), and seems to only add L1/L2regularzations on top (solely for the post processing step to extract the causal graphs). Technically it is standard.

- Moreover, it is not fully clear that graphs extracted, which is based on L1 and L2 regularization, can serve as the ground truth causal graphs. There seems a lack of theoretical justification on this part: under what assumptions can the causal graphs be extracted here, with what guarantees? 
- A lot of the compared algorithms model more than just Granger causality and are still compared. It is not clear what kind of causal graphs are generated. In addition, some heuristic are also used here (95% and 1% sparsity) - not sure these can produce reliable graphs.  A more rigorous justification for why their regularization approach produces valid causal graphs is needed.

Other Comments: 

- L370: only FPR is explained. 
- there are many evaluation metrics, but not mathematically defined. In addition, it would be good to provide a prioritized list of metrics, explaining which are most crucial for evaluating the quality and usefulness of their generated data and graphs. This would help readers understand how to interpret the results more effectively.

questions:
mainly how the causal graphs can be thought as ground truth
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents CausalDiffusion, a framework that generates multivariate time-series data alongside associated causal graphs using diffusion models. Reviewers raised concerns regarding the paper’s motivation and novelty, as well as its reliance on strong assumptions, such as causal sufficiency, which limit its applicability. Additional concerns include the lack of thorough exploration of the model’s scalability and robustness to assumption violations. The experimental evaluation was also noted to lack depth, with limited attention to downstream tasks and real-world datasets. While the authors’ rebuttal addressed some concerns, such as providing theoretical justification for VAR-based graph generation and conducting robustness experiments, several key issues remain unresolved.