Paper: Castor: Causal Temporal Regime Structure Learning (ICLR-2024)

Reviewer comment (Theory):
The significance of the problem setting is unclear:  
- It seems unnatural to assume that the time series data is from several mixed unknown regimes while within each regime the temporal causal relations are stationary. Could the authors give some motivative or real-world examples of this kind of datasets?  
- Usually it's more natural to think of nonstationarity in time series data as changing constantly but gradually, i.e., in the context of this paper, the regime partition is as fine as each single time stamp, while the causal relations (densities, edges weights) change smoothly across different time stamps.  
- Specifically, in this paper, by "time partition" (Definition 2), did the authors assume that each partition is a continent time series block? If yes, it should be stated clearly, instead of a vague "time partition"; if no, why, and any motivations? Also, did the authors assume/constrain any "similarities" between the causal graphs of different regimes?

Author response:
**The significance of the problem setting is unclear:**
- We are committed to providing further motivation for the problem at hand. In our introduction, we highlighted the relevance of our framework to EEG recordings in epilepsy, where distinct regimes (non-seizure, pre-ictal, seizure) form Multivariate Time Series (MTS) sequences. Additionally, Saeed et al.'s work, on causal structure discovery from mixtures of Directed Acyclic Graphs (DAGs) presented at ICML 2020, shows the importance of this setting in learning gene regulatory networks from diverse subtypes in neurological diseases. We believe that there are several other settings where our framework is particularly relevant and attractive, and we will revise the text to clarify these points. 
- Then, we clarify that each partition in our framework corresponds to a MTS block, representing a distinct regime. Visual aids in the form of figures have been added to the appendix to elucidate our setting.

Reviewer comment (Novelty):
Even under this setting, the problem can already be well solved by existing methods:  
- This paper mentions CD-NOD (Huang et al. 2020) but claims that "CD-NOD cannot infer individual causal graphs and necessitates prior knowledge about the number of regimes." But based upon my understanding on CD-NOD, this statement is generally incorrect, and the regime partition is actually readily obtainable from CD-NOD's abstract output.  
- By using the time stamp IDs as the surrogate variable, CD-NOD's phases I and II can output the abstract causal graph where the parents of each variable is identified as the union of all its parents in graphs from different regimes. Then, note that there is a phase III, kernel non-stationary visualization (Fig 12), which generally estimates the variability of the conditional distribution \( p(X_i \mid \text{union.parents}(X_i)) \) over the time index surrogate. The different regime partition is then readily available, i.e., the locations/times where changes happen are detected. Finally, with regimes recovered, any methods like PCMCI can be used in each stationary regime to obtain the graphs of each.  
- So what's the benefit of using the authors' proposed method than just using CD-NOD as above? Or please correct me if I am wrong regarding my understanding on CD-NOD or the problem.

Author response:
**Even under this setting, the problem can already be well solved by existing methods:**

Our statement *"CD-NOD cannot infer individual causal graphs and necessitates prior knowledge about the number of regimes."*
- We sincerely appreciate the insightful remark. Our statement aims to clarify that CD-NOD provides a summary graph for all regimes, which is a more constrained output compared to the comprehensive representation offered by our framework, CASTOR (Full temporal causal graph per regime). As the reviewer correctly noted, CD-NOD can infer change points but lacks the ability to express if regimes are repeated in the Multivariate Time Series due to the absence of temporal causal graphs for each regime. In contrast, CASTOR does not only learn and present the temporal causal graph for each regime but also leverages this information to achieve more accurate causal graphs by grouping together time series blocks from the same causal graph. This approach precisely determines the total number of distinct regimes.
- We argue that learning a full temporal causal graph per regime is more powerful than acquiring a single summary graph. The full temporal causal graph does not only contain the information necessary to reconstruct the summary causal graph but also offers a more accurate evaluation metric. For instance, consider two simple settings with two variables: In setting 1, we have 1->2 in a time-lagged manner and 2->1 instantaneously, resulting in the summary causal graph 1<->2. Now, in setting 2, we have 2->1 in a time-lagged manner and 1->2 instantaneously, resulting in the summary causal graph 1<->2. However, evaluating based on the full temporal causal graph in each setting provides a nuanced and accurate assessment of the differences between them.
- *what's the benefit of using the authors' proposed method than just using CD-NOD as above?* Thank you for your valuable feedback. We initially considered a fair and challenging comparison with methods learning full temporal causal graphs through manual regime splitting. We want to clarify that we had presented comparisons to RPCMCI which is a model for MTS with different regimes. However, in response to your suggestion, we conducted further experiments, comparing with CD-NOD and using phases 1 and 2 from the official Python package causal-learn, providing regime indexes to CD-NOD. The results of these extended comparisons are detailed below.
|    | method | K=2     | K=3      | K=4      |
|----|--------|---------|----------|----------|
| 10 | CD-NOD  | 20.2    | 11.4     | 38.8     |
|    | CASTOR | **100** | **100**  | **97.9** |
| 20 | CD-NOD  | 25.2    | 23.7     | 12.7     |
|    | CASTOR | **100** | **97.2** | **93.4** |
| 40 | CD-NOD  | 0       | 11.3     | 5.57     |
|    | CASTOR | **100** | **99.8** | **99.2** |

Reviewer comment (Theory):
The parametric assumption is quite limited:  
- The authors categorize the proposed CASTOR as score-based methods. Just for the convention, I would suggest to specify it as optimization-based methods (e.g., NOTEARS), to be distinguished from those real score-based methods, like GES.  
- Just like many existing optimization-based methods, this paper assume that the exogenous noise in the additive noise model follows a standard Gaussian distribution (namely, the uni/equal-variance assumption). This, however, is generally untestable and highly impractical in real-world scenarios. When the equal-variance assumption is violated, these optimization-based methods can perform relatively bad.  
- In this paper, is there any way to relax the uni/equal-variance assumption?

Author response:
**The parametric assumption is quite limited:**
- We want to thank you for this remark, part of our future work will be to extend to more complicated settings with more complicated noises.

Reviewer comment (Experiments):
The experimental comparisons are incomplete:  
- The authors only provide experimental comparisons with stationary temporal causal discovery methods PWGC, VARLiNGAM, and DYNOTEARS. Moreover, the VARLiNGAM method is also misspecified for the non-Gaussian distributions.  
- More comprehensive comparisons are needed with at least some non-stationary temporal causal discovery methods, as well as the CD-NOD one discussed above.

Author response:
**The experimental comparisons are incomplete:**
- We have incorporated additional experimental results, conducting comparisons with CD-NOD and Rhino (CASTOR surpasses both models in various settings, achieving over 93% compared to 23% for CD-NOD and 39% for Rhino.). Please refer to the appendix for detailed outcomes ([Comparison with Rhino](https://openreview.net/forum?id=Qqu5mMgIBV&noteId=pdUoJO5frL)). Furthermore, we have included running times for all the models for comprehensive insights.

Reviewer comment (Presentation):
The presentation somewhat lacks clarity:  
- What is the mixing coefficients \( \pi_{t,u}(\alpha) \) in Eq.5? What is such \( \alpha \)? The readers are not expecting a choice of parameters (like the mentioned Samé et al. (2011)), but more from a definition perspective with physical meanings. E.g., in Eq.5, the mixing coefficients should be defined as something related to the regimes partition. Further parameter approximations are then introduced.  
- Why did the authors choose to spell out \( Pa(<t) \) and \( Pa(t) \) separately in expressions like Eq.2?  
- Question: does the proposed method output specific DAGs or equivalent classes? Since with instantaneous relations, usually the exact DAG is unidentifiable non-parametrically. Which is the critical factor that helps the exact identification in this method?

Author response:
**The presentation somewhat lacks clarity:**
- We acknowledge the importance of clarity and have taken steps to address this concern. New figures have been added to the paper, available in the updated appendix, aiming to enhance the reader's understanding.  
## Rebuttal 2 (ID: EHZKsqL3ls)
*Question: does the proposed method output specific DAGs or equivalent classes? Since with instantaneous relations, usually the exact DAG is unidentifiable non-parametrically. Which is the critical factor that helps the exact identification in this method?*

We identify the DAGs. We used in our proof the theorem 1 in the work of Peters et al. (2013),"Identifiability of Gaussian structural equation models with equal error variances."

Reviewer comment (Theory):
Theorem 1 states that each regime is identifiable, and thus the causal structure is also identifiable by giving the correct regime. However, the claim that the regime is identifiable is skeptical. For example, consider a simple normal mixture distribution with two components N(0,1) and N(0,2) (a degenerate case with two regimes), and this theorem seems to suggest that each sample can be identified whether from N(0,1) or N(0,2) which is clearly not true.

Author response:
By addressing the concern raised by the reviewer regarding the correctness of our theorem, we appreciate the opportunity to provide clarification. The example presented by the reviewer, featuring two components with the same mean but different variance, neglects a critical aspect: all our noises originate from $\mathcal{N}(0,1)$. Consequently, variations in the mean of different components of the mixture occur when changing the graphs. This nuanced point ensures that if the means are the same, the underlying graphs are also identical, indicating the same regime. We are hopeful that this clarification will alleviate any doubts about the validity of our theorem.

Reviewer comment (Theory):
Moreover, is the number of regimes identifiable? How to choose the number of regimes.

Author response:
Concerning the inquiry about the choice the number of regimes, our paper explicitly addresses this problem in the learning algorithm. The detailed process is explained in the third section, involving the decomposition of the time series into equivalent windows. Subsequently, we iteratively update these windows by merging some, and canceling others. To enhance clarity, visual explanations, and figures detailing the learning process and CASTOR framework have been included in the appendix.

Reviewer comment (Novelty):
The proposed method lacks novelty by comparing it with the NOTEARS method and the related extensions. See references below. For example, why not consider the nonparametric version of NOTEARS in the DYNOTEARS framework? Can you explain why combining EM is the most efficient way to solve the problem? 

 - Zheng, Xun, et al. "Dags with no tears: Continuous optimization for structure learning." Advances in neural information processing systems 31 (2018).

 - Zheng, Xun, et al. "Learning sparse nonparametric dags." International Conference on Artificial Intelligence and Statistics. PMLR, 2020.

 - Pamfil, Roxana, et al. "Dynotears: Structure learning from time-series data." International Conference on Artificial Intelligence and Statistics. PMLR, 2020.

Author response:
We extend our gratitude to the diligent efforts of the reviewer and value the insightful comments. We are grateful for Reviewer (4Xkb)'s recognition of the importance of our problem and the generality of our framework in addressing both linear and non-linear relations.
The references cited by the author have been duly acknowledged in our paper, and they indeed represent special cases within our framework. We have added a discussion that permits to further clarify the novelty of the proposed framework. 

Notably, the reviewer references DYNOTEARS and its extension to the non-linear setting using the extended version of NOTEARS. We outline that our method encompasses these as special instances. In Equation 3 on page 3 of our paper, we present SEM of our settings. If $K=1$, corresponding to a time series with only one regime, it essentially becomes an extension of DYNOTEARS to non-linear settings. However, it is crucial to highlight that our framework operates in a setting where time series comprise multiple consecutive regimes, with each regime could be seen as MTS with a different causal graph (We employed the EM algorithm to iteratively alternate between regime learning and graph learning). CASTOR learns together the number of regimes, their respective indexes (indicating when each regime starts and ends), and the associated causal graph - none of the methods in the literature is able to address this joint learning problem, to the best of our knowledge.

Reviewer comment (Writing):
I am very confused by the term "heterogeneous time series" in this paper. I cannot find any rigorous definition related to it. The "heterogeneous" in causal inference usually connects to heterogeneity among different individuals described by some variables say confounders. Anyway, without a clear statement, please do not claim such a contribution.

Author response:
Thank you for the clarification. In our framework, heterogeneous time series refers to the fact that the time series data has multiple regimes. Based on our understanding, the term "heterogeneous data" or "heterogeneous time series" is used to denote data from different causal models in causal discovery literature (we provided some references). Your insight is valuable, and we have therefore refrained from using this terminology in the updated version of our manuscript to enhance clarity.
- Saeed, Basil, et al. "Causal Structure Discovery from Distributions Arising from Mixtures of DAGs."  International Conference on Machine Learning. PMLR, 2020.
- Huang, Biwei et al. "Causal Discovery from Heterogeneous/Nonstationary Data." The Journal of Machine Learning Research, 21

Reviewer comment (Evaluation):
An analysis of computational complexity is needed or at least running time should be provided.

Author response:
To address concerns regarding runtime efficiency, we have incorporated comprehensive running time experiments in the appendix. Additionally, new models have been introduced, and all pertinent results are presented in the supplementary materials.

**Setting: 10 nodes and different number of regimes (2,3,4)**

VARLINGAM and CD-NOD (employing Fisher’s Z conditional independence test for faster runs; note that KCI CD-NOD takes over 2000 seconds for 2 regimes, with F1 scores in a similar range) exhibit remarkable speed compared to other methods (less than 4 seconds). However, in terms of scores, both models encounter challenges in effectively learning causal graphs. Notably, CASTOR (around 200 seconds for 4 regimes) runs faster than Rhino (around 2000 seconds for 4 regimes), even though CASTOR learns both temporal causal graphs and regime indexes. 

A fair comparison arises when comparing RPCMCI and CASTOR, as both models learn regime indexes and temporal causal graphs. It's essential to mention that RPCMCI necessitates specifying the number of regimes and the maximum number of transitions, producing only time-lagged relations. CASTOR (200 seconds for 4 regimes) converges more rapidly than RPCMCI (more than 2300 seconds for 4 regimes). This difference in convergence time becomes more pronounced as the settings become more complex.

Reviewer comment (Writing):
The primary weakness of the paper is in its presentation, particularly regarding the explication of the key contribution: the introduction of multiple regimes. More textual emphasis and clarity on the parametrization of the time-series model to account for these multiple regimes would be advantageous.

Author response:
We sincerely appreciate the time and effort invested by the reviewer in evaluating our manuscript, and we are grateful for the insightful comments. We concur with the reviewer's observation that the introduction of multiple regimes adds a novel dimension to existing causal discovery frameworks.

- Regarding the identified weakness in the paper's presentation, we acknowledge the importance of clarity and have taken steps to address this concern. New figures have been added to the paper, available in the updated appendix, aiming to enhance the reader's understanding. Additionally, we are refining the overall presentation and rectifying any typographical errors in the appendix.

Reviewer comment (Theory):
The role and interpretation of $\gamma_{t,u}$ are ambiguous. If $\gamma_{t,u}$ serves as an indicator for time $t$'s association with partition $u$, its continuous nature as presented in Equation 11 raises questions about how to understand the defined likelihood $\log p(X_{0:T})$ defined above Equation 8.

Author response:
- In response to the specific query about $\gamma_{t,u}$, we want to clarify that we learn it, as detailed in Equation 11( E-step). Furthermore, in each iteration, the regime with maximum probability is assigned a value of 1, while others are set to 0.

Reviewer comment (Experiments):
Concerning empirical evaluations, the paper's baselines, namely VARLiNGaM and DYNOTEARS, do not represent the stronger baselines. Including a stronger method, such as Rhino, could significantly enhance the demonstration of CASTOR's efficacy in regime inference.

Author response:
To address the comparison with the Rhino framework, we conducted rigorous evaluations. This involved training Rhino on 50 time series, each with a length of 200, for every regime. The regime indexes were provided to Rhino, and the entire process was carried out following the guidelines outlined in the authors' GitHub repository. We adhered to their README description to ensure accuracy and consistency in our approach.
|     |        |      | $K=2$ |         | $K=3$ |          | $K=4$ |          |
|-----|--------|------|-------|---------|-------|----------|-------|----------|
| $d$ | method | type | SHD   | F1      | SHD   | F1       | SHD   | F1       |
| 10  | Rhino  | I    | 36    | 36.4    | 48    | 37.8     | 64    | 39.1     |
|     |        | L    | 38    | 36.9    | 60    | 33.6     | 85    | 34.7     |
|     | Castor | I    | **0** | **100** | **1** | **97.3** | **1** | **99.3** |
|     |        | L    | **0** | **100** | **0** | **100**  | **2** | **98.4** |

Reviewer comment (Experiments):
Matching or exceeding strong baselines would underscore CASTOR's advantages; alternatively, any performance gap would still furnish useful insights into the framework's relative standing and potential areas for enhancement.

Author response:
We sincerely appreciate the valuable feedback provided by the reviewer. 

In the current experimental setting, we have initiated a comparison between Rhino and CASTOR specifically in linear scenarios. It is worth noting that additional experiments are planned to comprehensively illustrate the advantages of CASTOR. In the preliminary comparison, we want to clarify that Rhino exhibited a superior performance over VARLingam based on F1 score.

Reviewer comment (Writing):
I wonder how should I interpret the generation mechanism for the time across the regime boundaries? For example, if we have two partitions: ${0,1,2,3}$ and ${4,5,6,7}$ with lag $1$ and $3$ respectively. Then, how should I interpret the data generation at $t=4$? What is the corresponding graph and lag I should use?

Author response:
- In response to your observation concerning lag and regimes, our work operates under the assumption that the duration of the regimes is greater than the number of lags.

Reviewer comment (Theory):
How do you obtain eq.11? Is this the analytic solution from the E-step?

Author response:
- In response to the specific query about $\gamma_{t,u}$, we want to clarify that we learn it, as detailed in Equation 11( E-step). Furthermore, in each iteration, the regime with maximum probability is assigned a value of 1, while others are set to 0.

Reviewer comment (Writing):
In appendix, Eq.22, what is $p^{(l)}$?

Author response:
- $p^{(\ell)}$ is the optimal distribution of the regime $\ell$.