Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to discovery causal graphs in term of conditional summary graphs with conditional stationary time series data. Authors first present the framework for learning conditional time series data and then presented a VAE framework to instantiate the framework. Empirical study shows the superior performance over existing baselines. 
strength_and_weaknesses: Strength:
The empirical evaluation is convincing with strong performance.

Weakness:
Presentation could be improved, with many design choices left poorly explained.
Identification result is decoupled from the deep learning framework  with VAE, which makes the theoretical results not really useful. 
The proposed method seem to be adopted from many existing related work. While being applied to a new problem, it is unclear what difficulties in just combing them. 


Comments:
-Figure 1 Scenarios: I found these scenarios seems arbitrary set. For example, in S2, s_t only depends on x_t, but in S3, s_t depends on x_t and x_{t-1} (and s_{t-1}). What about any subset of the dependence in Senario 3? Similarly, in S1 and S3, s_t depends on s_{t-1}  but not in S2. These inconsistency should be explained, if there is a specific choice behind it. 
- Stationarity: the description is not totally clear. If one treat the joint distribution as a mixture distribution (with the states being the mixture variables), would it be stationary as well? If not, do you only assume the non-stationarity comes from s? 
- is there a reason why auto-regressive connections are not considered?
- identiability: it would be good to have a formal statement on this in the paper. In addition, "at in this hidden state regime the
previous theoretical guarantees on identifiability no longer hold due to violations of our assumption" I'm not sure if the identifiability holds in the proposed VAE structure where you find the lower bounds. 
clarity,_quality,_novelty_and_reproducibility: quality: ok

clarity: many improvement needed

originality: the problem is original as far as I can tell.
summary_of_the_review: it is a decent work to address a new problem with decent empirical performance, but technically it is not new and theoretical results are not consistent with the algorithm.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a state-dependent causal learning framework for multivariate time series. The goal seems to learn a state-dependent causal graph between multivariate time series. Since causal structures can change dynamically in practice, the targeted task is practically useful if the state is automatically inferred. In this paper, the authors assume that the states are fully observable. Under this setting, the task is to learn a causal graph for each of the K distinct states.
 
The authors introduce a VAE-based algorithm to learn the probability of getting an edge between a node pair. The model seems to follow an existing work, but the detail is not clearly given. 
 
strength_and_weaknesses: Strength
- Addresses a general problem of state-dependent causal graph learning.
- Proposed a generative model to probabilistically learn the causality graph.
- Conducted empirical evaluation to show a good performance. 
 
Weakness
- The assumption of being able to observe the state variable sounds very strong. The relative novelty to the existing works might not be very significant.
- Presentation can be improved. The main probabilistic model is not fully explained. It is not clear how the authors extend the existing work. 
clarity,_quality,_novelty_and_reproducibility: There seem to be significant issues in clarity and novelty.
 
For the clarity front, the main probabilistic model that looks interesting is not clearly explained. In particular, I do not understand how GNN was applied. GNN is an approach to learning node embedding vectors, given a graph, which is typically assumed to be an undirected graph. In the present setting, however, the graph is unknown and directed. One possibility could be to iterate between graph estimation and GNN, but no clear-cut explanation is given.
 
Overall, it is hard to follow the proposed probabilistic causal learning model, and hence, it is hard to figure out what the true novelty is over the existing work. This is really unfortunate because if such an edge-independent Bernoulli-like model works for causal structure learning, it can be a major achievement.
 
For the novelty front, the assumption of being able to observe the state itself sounds like a very strong assumption. It is not readily clear to me how it is a significant innovation in light of existing works. Also, the novelty of the proposed probabilistic structure learning model is not very clear to me.
 
summary_of_the_review: From what is written above, I cannot endorse the technical validity of the proposed framework. However, I encourage the authors to continue this line of research. If they significantly improve the presentation, in particular how the GNN part and the generative model are designed, it is going to be a good paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors extend the existing work in time series causal discovery from stationary data to conditionally stationary time series data given the state. 
strength_and_weaknesses: The authors assume access to the states at all times. Then one can simply run the existing stationary causal discovery algorithms conditioned on each realization of the state. 

It turns out this is exactly what the authors propose as well. 

Unfortunately, this simple extension might not be a sufficient contribution for an ICLR publication. 
clarity,_quality,_novelty_and_reproducibility: The presentation is clear. Novelty is minor given the existing time-series discovery algorithms. 
summary_of_the_review: Comments on experiments:
"for a fair comparison we evaluate the identification accuracy of the summary
graph such that it only considers the exis tence of an edge rather than capturing the edge type."
Does this mean you are simply checking the graph skeleton? Could you explain why this is more fair?

In the other experiments also, it's not clear if the proposed method shows promise compared to the baseline. 

In Section 4.3, could you explain why causal discovery is important for forecasting, which is a predictive task?
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper considers a particular setting for causal discovery from non-stationary time series. In particular, the non-stationary behavior is modeled as stationarity conditioned on a set of state variables.  The authors show the identifiability of their model, under the assumption that state variables are observed and there is no instantaneous causal effect. They further propose a VAE-based estimation method to estimate the causal graph, but without a consistency guarantee.
strength_and_weaknesses: Strength:

1. This paper considers an interesting and practically useful problem: time-delayed causal discovery from non-stationary data, and models the non-stationary behavior to be stationary conditioned on a set of state variables, which I think is a reasonable assumption.

2. The reported experimental results show that the proposed method has a better performance compared to other methods.

3. The organization and writing are good.

Weakness:

1. The authors only give the identifiability results when the state variables are observed (Scenario 1), which is trivial. Because of this, I fail to find significant contributions of this paper.  

2. From Figure 4, it seems the practical performance of the proposed method decreases quickly with increasing variables and state values. 

 Also, is the proposed method able to handle different time lags?


clarity,_quality,_novelty_and_reproducibility: Clarity: Very good. The paper is well presented.

Novelty: The novelty of this paper is not enough. For theoretical identifiability, it only considers the case when state variables are observed, which can be directly adapted from the previous literature.

Quality: Besides the above two points, the experimental verification fails to show the efficacy of the proposed practical estimation method.
summary_of_the_review: The paper is well presented, but the novelty is limited, especially in theoretical identifiability, and from the experimental results, the performance of the proposed method seems to decrease quickly with increasing variables and state values.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors study an extension of the state of the art in causal discovery for time series from stationary to conditionally stationary time series data given the state. In this setting, as pointed out by the reviewers we could just run the existing stationary causal discovery algorithms conditioned on each realization of the state, which is precisely what the others suggest. I agree with the reviewers that there is not enough novelty here to merit an ICLR publication/presentation.

justification_for_why_not_higher_score: Lack of adequate sophistication/novelty.
Decision: Reject