Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents VCD, a world model that can modularly transfer knowledge between different interventional environments. It causally factories the transition model in the latent space and jointly learns an approximate posterior model, transition model, causal graph, and the intervention targets together with a derived ELBO loss. VCD learns both observational and interventional mechanisms which enable modular adaptation between different interventional environments, as it taking advantage of the invariances of causal mechanism.
strength_and_weaknesses: Strengths:
	
1. Modularity: When encountered with new and unseen environments, VCD can quickly adapt by training the VCD on trajectories in the new environment while fixing the trained parameters for the causal graph and the mechanisms in the undisturbed environment.

2. Sample efficiency: During adaptation, VCD is more sample efficient as compared to the baselines RSSM and MultiRSSM.

3. Extensive experiments are done in the multi-body environment.
	
Weakness:
	
1. Typos in equation 14: p(z^t|o^t)  should be q(z^t|o^t)

2. The experiment is only tested on one simple environment.  Lack of complex environment to demonstrate its claimed effect on higher-dimensional raw image input.

3. The causal graph is learned in the latent space and depends on the dimension d of the latent space. It would be interesting to provide comments on the choice of d, and the relationship between d and the actual number of edges between entities in the multi-body system. 

4. The multi-entity environment only has 4 entities. How will the model scale with a higher number of interacting objects in the task and a higher number intervene objects?
	

clarity,_quality,_novelty_and_reproducibility: Overall the presentation has great clarity and the figures of the experiment results are of good quality. The details of the model are described and the code is provided in the supplemental materials.
summary_of_the_review: Given the above strengths and shortcomings of this paper, I recommend this paper be considered slightly below the acceptance threshold.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a method for latent dynamics modelling. In latent space, the latent variables are assumed to affect variables in the next time step via a sparse graph interaction. The model allows for soft interventions on the mechanisms. The model is trained via the ELBO and the graph structure, causal mechanisms and interventino targets are trained via the method from Brouillard et al. (2020). The method is evaluated on a toy image dataset of moving balls.
strength_and_weaknesses: Strength:
- The paper is slightly more applicable than [1], as it doesn't require each environment to have labelled intervention targets.
- The paper is clearly written.
- The authors have provided code with their paper.

Weakness:
- The empirical evaluation is only done on a toy dataset of coloured circles. This is insufficient to assess the performance of the proposed model.
- The paper has no identifiability guarantees.
- The paper is very similar to [1], but this is not fairly reflected in the text.
- In table 2, it appears like the graph and intervention targets aren't learned very accurately.

[1] Lippe, Phillip, et al. “CITRIS: Causal Identifiability from Temporal Intervened Sequences.” 2022
clarity,_quality,_novelty_and_reproducibility: ### Clarity
The paper is very easy to read.

### Quality
The proposed method seems sensible in theory

### Reproducibility
The provided code appears to make reproducing the results easy.

### Novelty
The method is closely related to prior work. In particular, [1] addresses a very similar problem. The only difference it that the latent causal model is learned via DCDI, so that the different environments don't need different labels on the interventions.
summary_of_the_review: The paper in limited in technical novelty and has neither theoretical guarantees, nor has been convincingly shown to work well in practice.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this work a model for sequences with the following properties is proposed:
- Modular description of the transition function
- Handling of interventions, defined as the modification of a target of the transition function
- Learning of the dependency structure
- Use of latent variables
  
The authors build on top of the formulation from "Differentiable Causal Discovery with Interventional data" (DCDI), which exhibits the above properties, except for the latent variables. The addition of the latent variables introduces intractability that is solved through the use of a variational inference.

strength_and_weaknesses: Strengths:
- The addressed problem is highly relevant. The above-mentioned list of properties seems necessary for artificial intelligence and few works check all these boxes. 
- The experiments show the advantages of the current proposal and even show the emergent property of axis alignment.

Weaknesses:
- I'd like to see more justification as to why this model should be called "causal", as opposed to simply "predictive and modular, while encouraging sparsity". While it seems reasonable that a predictive, modular, sparse model would capture causal dependencies, is there a formal reason why it couldn't capture consistent temporal correlations? What would happen if I were to provide as evidence a video in which entropy increases, but run backwards (so that entropy decreases)? What if I were to show any other video with consistent correlations but lack of causation?
- The experiments are very limited (just one toy example), and the proposed model is not studied in detail:
  - In particular, the dynamics seem pretty deterministic. How does it handle uncertain actions? Can it handle actions in which the target state is multimodal given a source state?
  - How would this model handle situations in which the uncertainty gives rise to multimodality? For instance, imagine that there's a very small wall in the middle of a room and a ball can bounce off it only in a concrete trajectory, but won't if the trajectory deviates slightly. In this case, when predicting in open loop (observations are not fed back into the model), will it be able to handle the multimodality of the predictions properly?
- Other
  - Format violation: More than 9 pages
  - I believe |G| is never defined
clarity,_quality,_novelty_and_reproducibility: Clear, most of the details are deferred to appendices. Novelty is in the use of latent variables and in the analysis of this particular experiment
summary_of_the_review: A step in a good direction, drawing most of the insights from existing work, limited empirical evaluation.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a method for causal representation learning from interventional data and demonstrate how such a model can be adapted to new settings. They consider time-series data generated by a causal graphical model and a rendering function (e.g. to pixels); key assumptions include that there are no instantaneous causal effects and the causal structure is invariant over time steps. They then assume access to data from the observational and multiple interventional distributions. In this setting, a VAE with latent causal structure is trained. The authors argue that such a model should be able to adapt efficiently to data from a new environment provided the sparse mechanism shift hypothesis holds. This is then demonstrated empirically on toy data.
strength_and_weaknesses: **Strengths**:
1. The authors tackle an interesting and largely unsolved problem. Both learning latent causal structure from a different representation and demonstrating the benefits of that for rapid adaptation may be very impactful.
2. The model setup (a VAE with latent causal structure) makes sense.
3. It is great that the authors consider a weak form of supervision: the model is trained only on interventional data without any additional labels. This seems much more easily attainable than labelled data [e.g. 2] or counterfactual data [e.g. 3]
4. It is also great that the authors do not stop at the learning of causal structure, but demonstrate the benefits of that causal structure for more efficient finetuning on new settings.
5. The paper is clearly written and a pleasure to read.
6. The figure design is also good.
6. The authors have provided clean and well documented code along with the paper.

**Weaknesses**:
1. The setup makes some strong model assumptions, in particular that the causal graph is invariant over time steps and that there are no instantaneous causal effects. While the authors point this out, it would be good to stress these assumptions in the introduction and conclusions a bit more and to discuss them: why are they needed? Are they realistic? How could we relax them?
2. Relatedly, I have a hard time thinking of example systems that are compatible with all the assumptions here. It would be great if the authors could provide some motivating examples.
3. Probably my biggest criticism of the paper is that it is not clear what exactly the method should be able to achieve – there is essentially no theoretical discussion. In particular, the authors do not make any claims about identifiability. Up to which equivalence class should the causal variables, the causal graph, and the rendering function be identified? What is the role of the assumptions on the data-generating process for identifiability? What properties do the rendering function and the distributions need to satisfy? Footnote 1 points the reader to Ref. [1], but if the identifiability result of that paper applies here, that should really be discussed more explicitly (which is also important to judge the novelty of this paper).
4. The empirical evaluation also leaves something to be desired. It is okay that there is just a single toy dataset, but the performance in terms of causal representation learning should really be discussed quantitatively. There are many adequate metrics of disentanglement and graph similarity that could be applied here. The numbers that the authors do provide are not particularly impressive. In the appendix, the authors claim that it is "not trivial to obtain a way to map the [ground-truth factors] to a latent dimension". I would politely encourage the authors to have a look at how other representation learning papers deal with that – for instance, they could use a feature importance matrix like in Figure 7 to determine a map between ground-truth factors and learned latents and then compute graph edit distances.
5. The authors cite many relevant works, but missed Refs. [3-5] given below. In particular, it seems that Ref. [5] considers a very similar problem. This is clearly concurrent, so don't take this as a criticism of novelty, but it would make the paper stronger if you could mention and contrast their and your findings.

**References**:
- [1] Lachapelle et al, "Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA", CLeaR 2022, 2107.10098
- [2] Lippe et al, "CITRIS: Causal Identifiability from Temporal Intervened Sequences", ICML 2022, 2202.03169
- [3] Brehmer et al, "Weakly supervised causal representation learning", 2203.16437
- [4] Lippe et al, "iCITRIS: Causal Representation Learning for Instantaneous Temporal Effects", 2206.06169
- [5] Ahuja et al, "Interventional Causal Representation Learning", 2209.11924
clarity,_quality,_novelty_and_reproducibility: **Clarity**: The paper presentation is very good, both in terms of writing and figures. My main gripe is that the lack of theoretical discussion makes it a bit harder to get a clear picture of the method.

**Quality**: The setting is very interesting and the presented VAE setup is solid. However, the method should really be demonstrated either in theory or practice. In its current form, the paper lacks a theoretical analysis and the empirical demonstration is not entirely convincing yet (see above).

**Novelty**: There is definitely novelty in this work, though I am still somewhat unsure how much exactly. There are not a lot of works on causal representation learning from interventional data and certainly not on the demonstration of more efficient adaptation to new settings. At the same time, the lack of theory and limited empirical analysis limits the novelty of this paper. Finally, it would be great if the authors could clarify the relation of their work to Ref. [1] a bit more.

**Reproducibility**: The work appears very reproducible – the authors have attached the code as supplementary material and promise to make it available publicly. The code quality (and level of documentation) looks great. There are also sufficient details in the appendix.

**References**:
- [1] Lachapelle et al, "Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA", CLeaR 2022, 2107.10098
summary_of_the_review: This work has a lot of potential: learning latent causal structure from interventional data on the pixel level could be very useful. The VAE setup makes sense. It is commendable that the authors do not stop at the identification of causal structure, but show that learning it is actually useful for predicting the future of a roll-out and for adapting to new settings.

Unfortunately, in its current form the paper does not demonstrate that the approach works. There should be either a theoretical discussion (like an identifiability result) or a convincing empirical demonstration that the model actually learns the correct causal structure. Right now, the paper has neither, and I can't recommend it for acceptance yet.

I encourage the authors to expand either of these two angles (or, ideally, both). If they can add such a demonstration, this could become a great paper and an impactful contribution to the field.

**Update after rebuttal**: Thanks to the authors for clear answers to my question and for addressing some of my criticisms in the upadted paper version. While the paper is improved, I still believe that it does not sufficiently demonstrate that the approach works. My score remains unchanged.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper describes variational causal dynamics (VCD), an approach that replaces a latent state-space model with a causal transition model that is learnt via a combination of differentiable causal discovery (Brouillard et al., 2020) and a set of independent deep networks that are defined the inferred causal graph. Additionally, they have a simple masking strategy to define the interventional distribution. The whole pipeline is trained via variational inference (optimizing the ELBO). They evaluate the approach of a simple physics-based task involving multiple coloured balls connected by spring forces.
strength_and_weaknesses: Strengths:
 - A very clear and well written paper that combines most of the key ideas from the recent progress in causal representation learning and causal discovery into a latent state-space model. None of the individual components were particularly surprising, but the combined architecture and train procedure is appealing and very well motivated.

Weakness:
 - The experiments were conducted on a very toy dataset. This style of data generating process is common in the recent papers in causal representation learning (although even there, richer datasets like 3DIdent, etc. are becoming more common), but in those papers, the experiments are primarily to ask whether identifiable models are learnable in practice (and to evaluate finite sample / optimization error / etc issues that are hidden in identifiability proofs). Because this paper isn't claiming any theoretical advances (and to be clear, I don't think it needs to!), it leans more heavily on experiments to show the value of these approaches. Given the really strong setup and motivation, I was hoping that the experiments would either provide strong evidence for the value of this approach, or alternatively show where there is need to further improvements. 

   The experiments do provide evidence that there is a benefit to taking advantage of causal structure, but it is not clear whether this generalizes beyond these environments that are specifically designed to show the difference between these approaches. And given that the image based experiments showed relatively modest differences when a large number of trajectories are used (Figure 3), I'm not sure that there will be any significant different in a more natural environment (e.g. an RL-style task). In the adaption experiments with low data regimes, the differences are far more dramatic. But that raises the question: is the advantage of imposing causal structure only about sample complexity, such that if we see enough complexity, the differences are washed out?

 - Causal discovery section: I would have liked to see a quantitative metric like the mean absolute correlation coefficient to measure disentanglement. In this setting you'll need to do a matching based on covariance between the true latents and the model's latents and drop the least correlated latents (to account for differences in size). Dropping the least correlated latents will bias the scores upwards, but they will still be comparable across models that use the same architecture. I would also include a linear disentanglement score ($r^2$ of a linear regression between the true latents and the predicted latents). 

 - The paper draws on a number of areas to compose its architecture  (causal discovery / identifiable representation learning /  independent mechanisms / etc), so I think the experimental section would be significantly improved if it gave guidance on what components need to improve in order to see bigger differences with RSSM and to generalize to richer environments. I.e. were do we need breakthroughs to see the biggest gain in performance? Is it causal discovery that's letting the method down? Or the encoder? Or the parameterization of the independent mechanisms? Something else?
clarity,_quality,_novelty_and_reproducibility: The paper is very clearly written, the methods are natural given the setup. The method is complex because there are a lot of moving pieces, but I think each part is relatively straightforward to reproduce. The authors provided code to reproduce the experiments in the supplement.

Minor:
 - First page, first paragraph, this sentence is a little awkward: "This latter aspect, it is conjectured, is was...". Consider: "It is conjectured that this latter aspect is what...". Alternatively, a better rewording would cite whoever made this conjecture (or a survey) to avoid the passive voice. I.e. "Someone [2004] conjecture that this latter ....")
summary_of_the_review: Overall, I liked the paper - it proposes a natural latent variable model that takes advantage of the recent advances in our understanding of causal representation learning, but I felt like it was let down by an experimental section that didn't do a good enough job of testing the limits of the approach.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a modular world model that can transfer knowledge between different interventual environments. It factorizes the latent space due to causality. The model is trained using VI. The method is evaluated on a toy dataset.

Strength and weaknesses. The idea of modular world models is nice and the model shows also better sample efficiency then standard RSSMs. Yet, as pointed out by reviewer gksh  the approach is very similar to other methods and hence does not offer a lot of theoretical contribution. Further, the experimental evaluation was considered to be not sufficient by almost all reviewers. Authors are encouraged to extend their evaluation to more complex environments and resubmit this interesting line of work to another conference. 
justification_for_why_not_higher_score: Experiments are too simple and limited theoretical novelty.
justification_for_why_not_lower_score: NA
Decision: Reject