Original: Intervention Design for Causal Representation Learning
Revised: Intervention Design for Causal Representation Learning

Original: Abstract
Revised: Abstract

Original: In this paper, we take a first step towards bringing two fields of causality closer together: intervention design and causal representation learning.
Revised: In this paper, we take a first step towards bringing two fields of causality closer together: intervention design and causal representation learning.

Original: Intervention design is a well studied task in classic causal discovery, which aims at finding the minimal sets of experiments under which the causal graph can be identified.
Revised: Intervention design is a well studied task in classic causal discovery, which aims at finding the minimal sets of experiments under which the causal graph can be identified.

Original: Causal representation learning aims at recovering causal variables from high-dimensional entangled observations.
Revised: Causal representation learning aims at recovering causal variables from high-dimensional entangled observations.

Original: In recent work in causal representation, interventions are exploited to improve identifiability, similarly to classic causal discovery.
Revised: In recent work in causal representation, interventions are exploited to improve identifiability, similarly to classic causal discovery.

Original: Hence, the same task becomes relevant in this setting as well: how many experiments are minimally needed to identify the latent causal variables?
Revised: Hence, the same task becomes relevant in this setting as well: how many experiments are minimally needed to identify the latent causal variables?

Original: Based on the recent causal representation learning method CITRIS, we show that for K causal variables, ⌊ log 2 ( K ) ⌋ + 2 experiments are sufficient to identify causal variables from temporal, intervened sequences, which is only one more experiment than needed for classic causal discovery in the worst case.
Revised: Based on the recent causal representation learning method CITRIS, we show that for K causal variables, ⌊ log 2 ( K ) ⌋ + 2 experiments are sufficient to identify causal variables from temporal, intervened sequences, which is only one more experiment than needed for classic causal discovery in the worst case.

Original: Further, we show that this bound holds empirically in experiments on a 3D rendered video dataset.
Revised: Further, we show that this bound holds empirically in experiments on a 3D rendered video dataset.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: Recently, there has been a growing interest in the field of causal representation learning (Brehmer et al., 2022; von
Revised: Recently, there has been a growing interest in the field of causal representation learning (Brehmer et al., 2022; von

Original: Kügelgen et al., 2021; Lippe et al., 2022; Locatello et al., 2020; Schölkopf et al., 2021), which aims at discovering latent, causal factors and their causal relations from highdimensional observations such as images or videos.
Revised: Kügelgen et al., 2021; Lippe et al., 2022a,b; Locatello et al., 2020; Schölkopf et al., 2021), which aims at discovering latent, causal factors and their causal relations from highdimensional observations such as images or videos.

Original: A crucial aspect towards reaching this goal is commonly considered to be the access to interventional data.
Revised: A crucial aspect towards reaching this goal is commonly considered to be the access to interventional data.

Original: An intervention influences the causal mechanism of one or more variables while leaving other mechanisms and the observation function, i e. the way how we perceive these variables, unchanged.
Revised: An intervention influences the causal mechanism of one or more vari- ables while leaving other mechanisms and the observation function, i.e. the way how we perceive these variables, unchanged.

Original: Comparing data under different interventions allows one to find and disentangle the sources of variations, i e. the causal variables, from the high-dimensional observations.
Revised: Comparing data under different interventions allows one to find and disentangle the sources of variations, i.e. the causal variables, from the high-dimensional observations.

Original: In the context of causal representation learning, several works have considered different interventional settings, such as interventions with unknown targets (Brehmer et al., 2022; Locatello et al., 2020; Yao et al., 2022) or observed targets (von Kügelgen et al., 2021;
Revised: In the context of causal representation learning, several works have considered different interventional settings, such as interventions with unknown targets (Brehmer et al., 2022; Locatello et al., 2020; Yao et al., 2022) or observed targets (von Kügelgen et al., 2021;

Original: Lachapelle et al., 2022;
Revised: Lachapelle et al., 2022;

Original: Lippe et al., 2022), in order to guarantee identifiability of the causal variables.
Revised: Lippe et al., 2022a), in order to guarantee identifiability of the causal variables.

Original: However, interventional data can often be expensive, since it requires a specific experiment in which there is a perturbation of the causal system, e.g. a randomized controlled trial.
Revised: However, interventional data can often be expensive, since it requires a specific experiment in which there is a perturbation of the causal system, e.g. a randomized controlled trial.

Original: Hence, a naturally arising question is what is the minimal number of different intervention experiments that suffices for identifying the causal variables.
Revised: Hence, a naturally arising question is what is the minimal number of different intervention experiments that suffices for identifying the causal variables.

Original: In this paper, we answer this question by drawing connections from causal representation learning to the area of intervention design (Eberhardt, 2007; Greenewald et al., 2019;
Revised: In this paper, we answer this question by drawing connections from causal representation learning to the area of intervention design (Eberhardt, 2007; Greenewald et al., 2019;

Original: He and Geng, 2008; Hyttinen et al., 2012a, 2013; Kocaoglu et al., 2017a,b; Shanmugam et al., 2015; Squires et al., 2020), which aims to find the minimal set of experiments that identifies the causal graph for known causal variables.
Revised: He and Geng, 2008; Hyttinen et al., 2012a, 2013; Kocaoglu et al., 2017a,b; Shanmugam et al., 2015; Squires et al., 2020), which aims to find the minimal set of experiments that identifies the causal graph for known causal variables.

Original: As a specific setting, we focus on the recent causal representation learning method CITRIS (Lippe et al., 2022) which leverages data from temporal intervened sequences with known intervention targets.
Revised: As a specific setting, we focus on the recent causal representation learning method CITRIS (Lippe et al., 2022a) which leverages data from temporal intervened sequences with known intervention targets.

Original: Using similar techniques as for intervention design, we show that for CITRIS ⌊ log 2 ( K ) ⌋ + experiments are sufficient to identify the K causal variables, which is exactly one experiment more than needed for causal discovery in the worst case scenario.
Revised: Using similar techniques as for intervention design, we show that for CITRIS ⌊ log 2 ( K ) ⌋ + experiments are sufficient to identify the K causal variables, which is exactly one experiment more than needed for causal discovery in the worst case scenario.

Original: This opens up several opportunities for future work to leverage intervention design methods from causal discovery also in causal representation learning.
Revised: This opens up several opportunities for future work to leverage intervention design methods from causal discovery also in causal representation learning.

Original: Furthermore, to show that this bound holds also empirically, we conduct experiments on the Temporal Causal3DIdent dataset (von Kügelgen et al., 2021; Lippe et al., 2022).
Revised: Furthermore, to show that this bound holds also empirically, we conduct experiments on the Temporal Causal3DIdent dataset (von Kügelgen et al., 2021; Lippe et al., 2022a).

Original: This dataset contains videos of 3D object renderings with 6 causal variables, including the object positions, rotations, colors, and lightning.
Revised: This dataset contains videos of 3D object renderings with 6 causal variables, including the object positions, rotations, colors, and lightning.

Original: As expected based on our theoretical results, with a minimal set of ⌊ log 2 (6) ⌋ + 2 = 4 experiments, CITRIS is able to disentangle the variables well, obtaining an only slightly higher disentanglement error compared to the same model with access to an unlimited number of intervention experiments.
Revised: As expected based on our theoretical results, with a minimal set of ⌊ log 2 (6) ⌋ +2 = 4 experiments, CITRIS is able to disentangle the variables well, obtaining an only slightly higher disentanglement error compared to the same model with access to an unlimited number of intervention experiments.

Original:  RELATED WORK 2.1 INTERVENTION DESIGN
Revised:  RELATED WORK 2.1 INTERVENTION DESIGN

Original: Intervention design, also known as experiment selection, is a well-studied area in causal discovery (Addanki et al., 2020; Eberhardt, 2007; Eberhardt et al., 2005; Greenewald et al., 2019; Hauser and Bühlmann, 2014; He and Geng, 2008; Hyttinen et al., 2012a, 2013; Kocaoglu et al., 2017a,b;
Revised: Intervention design, also known as experiment selection, is a well-studied area in causal discovery (Addanki et al., 2020; Eberhardt, 2007; Eberhardt et al., 2005; Eberhardt and Scheines, 2007; Hauser and Bühlmann, 2014; He and Geng, 2008; Hyttinen et al., 2012a, 2013; Kocaoglu et al., 2017a,b;

Original: Shanmugam et al., 2015; Squires et al., 2020).
Revised: Shanmugam et al., 2015; Squires et al., 2020).

Original: Consider an unknown causal graph G =
Revised: Consider an unknown causal graph G =

Original: ( V, E ) , where each node i ∈ V is associated with a causal variable C i , and each edge ( i,
Revised: ( V, E ) , where each node i ∈ V is associated with a causal variable C i , and each edge ( i,

Original: j ) ∈
Revised: j ) ∈

Original: E represents a causal relation from C i to C j : C i →
Revised: E represents a causal relation from C i to C j : C i →

Original: C j .
Revised: C j .

Original: Given a set of K causal variables C 1 , ..., C K , the goal is to determine the set of experiments { E 1 , ..., E N } which identifies the underlying causal structure G most efficiently.
Revised: Given a set of K causal variables C 1 , ..., C K , the goal is to determine the set of experiments { E 1 , ..., E N } which identifies the underlying causal structure G most efficiently.

Original: In our notation, an experiment E i is defined as a set of causal variables on which interventions are jointly performed.
Revised: In our notation, an experiment E i is defined as a set of causal variables on which interventions are jointly performed.

Original: For example, the observational regime is denoted by E i = ∅ , i.e. all variables are passively observed, and single-target interventions by E 1 = { C 1 } , E 2 = { C 2 } , and so on.
Revised: For example, the observational regime is denoted by E i = ∅ , i.e. all variables are passively observed, and single-target interventions by E 1 = { C 1 } , E 2 = { C 2 } , and so on.

Original: One can show that under causal sufficiency ( i.e. , no latent confounders or selection bias), acyclicity ( i.e. , no cycles in
Revised: One can show that under causal sufficiency ( i.e. , no latent confounders or selection bias), acyclicity ( i.e. , no cycles in

Original: G ), and the causal Markov and faithfulness assumptions ( i.e. , there are no additional independences w.r.t.
Revised: G ), and the causal Markov and faithfulness assumptions ( i.e. , there are no additional independences w.r.t.

Original: the ones encoded in the graph G ), a set of experiments uniquely identifies the causal graph if for every pair of variables ( C i , C j ) , (1) there exists an experiment where C i has been intervened on, but not C j , or vice versa, and (2) there exists an experiment under which both C i and C j are passively observed, i.e. not intervened.
Revised: the ones encoded in the graph G ), a set of experiments uniquely identifies the causal graph if for every pair of variables ( C i , C j ) , (1) there exists an experiment where C i has been intervened on, but not C j , or vice versa, and (2) there exists an experiment under which both C i and C j are passively observed, i.e. not intervened.

Original: The first condition is also referred to as the unordered pair condition , and the latter as the covariance condition (Eberhardt, 2007; Hyttinen et al., 2013).
Revised: The first condition is also referred to as the unordered pair condition , and the latter as the covariance condition (Eberhardt, 2007; Hyttinen et al., 2013).

Original: Using these conditions, Eberhardt (2007) showed that ⌊ log 2 ( K ) ⌋ + 1 experiments are sufficient to guarantee the identifiability of a causal graph with K variables.
Revised: Using these conditions, Eberhardt (2007) showed that ⌊ log 2 ( K ) ⌋ + 1 experiments are sufficient to guarantee the identifiability of a causal graph with K variables.

Original: This bound is for the worst case scenario, and for certain graphs, fewer experiments might be sufficient.
Revised: This bound is for the worst case scenario, and for certain graphs, fewer experiments might be sufficient.

Original: For instance, if all variables are independent, the observational regime alone identifies the whole graph.
Revised: For instance, if all variables are independent, the observational regime alone identifies the whole graph.

Original: While in this work we provide a first application of intervention design for causal representation learning, in the intervention design literature, several other settings can be considered, including (1) where the number of variables that can be subject to an intervention simultaneously is limited (Ghassami et al., 2018; Hyttinen et al., 2013; Kocaoglu et al., 2017a;
Revised: While in this work we provide a first application of intervention design for causal representation learning, in the intervention design literature, several other settings can be considered, including (1) choosing the experiments adaptively after seeing the results of the previous ones (Greenewald et al., 2019; Hauser and Bühlmann, 2014; He and Geng, 2008; Squires et al., 2020), (2) the number of variables that can be subject to an intervention simultaneously being limited (Ghassami et al., 2018; Hyttinen et al., 2013; Kocaoglu et al., 2017a;

Original: Shanmugam et al., 2015), (2) where latent confounders may exist (Addanki et al., 2020; Hyttinen et al., 2012b; Kocaoglu et al., 2017b), or (3) where additional background knowledge about the underlying causal structure is available (Eberhardt, 2008; Greenewald et al., 2019; Hauser and Bühlmann, 2012; Sen et al., 2017).
Revised: Shanmugam et al., 2015), (3) that latent confounders may exist (Addanki et al., 2020; Hyttinen et al., 2012b; Kocaoglu et al., 2017b), or (4) that additional background knowledge about the underlying causal structure is available (Eberhardt, 2008; Greenewald et al., 2019; Hauser and Bühlmann, 2012; Sen et al., 2017).

Original: 2.2 CAUSAL REPRESENTATION LEARNING
Revised: 2.2 CAUSAL REPRESENTATION LEARNING

Original: Causal representation learning aims at learning representations of causal factors in an underlying system from highdimensional observations like images (Brehmer et al., 2022;
Revised: Causal representation learning aims at learning representations of causal factors in an underlying system from highdimensional observations like images (Brehmer et al., 2022;

Original: von Kügelgen et al., 2021; Lachapelle et al., 2022; Lippe et al., 2022; Locatello et al., 2020; Schölkopf et al., 2021).
Revised: von Kügelgen et al., 2021; Lachapelle et al., 2022; Lippe et al., 2022a,b; Locatello et al., 2020; Schölkopf et al., 2021).

Original: One of the first lines of work is Independent Component
Revised: One of the first lines of work is Independent Component

Original: Analysis (ICA) (Comon, 1994; Hyvärinen et al., 2001). ICA tries to recover independent latent variables transformed by some invertible transformation.
Revised: Analysis (ICA) (Comon, 1994; Hyvärinen et al., 2001) trying to recover independent latent variables entangled by some invertible transformation.

Original: ICA was extended to nonlinear transformations by exploiting auxiliary variables under which the latents become conditionally mutually independent (Hyvärinen and Morioka, 2016; Hyvärinen et al., 2019), and combined with deep learning architectures like
Revised: ICA was extended to nonlinear transformations by exploiting auxiliary variables under which the latents become conditionally mutually independent (Hyvärinen and Morioka, 2016; Hyvärinen et al., 2019), and combined with deep learning architectures like

Original: VAEs (Khemakhem et al., 2020a,b; Sorrenson et al., 2020;
Revised: VAEs (Khemakhem et al., 2020a,b; Sorrenson et al., 2020;

Original: Zimmermann et al., 2021).
Revised: Zimmermann et al., 2021).

Original: Further, recent works draw a connection between causality and ICA (Gresele et al., 2021;
Revised: Further, recent works draw a connection between causality and ICA (Gresele et al., 2021;

Original: Monti et al., 2019).
Revised: Monti et al., 2019).

Original: In particular, Lachapelle et al.
Revised: In particular, Lachapelle et al.

Original: Yao et al.
Revised: Yao et al.

Original: (2022) discuss identifiability from temporal sequences.
Revised: (2022) discuss identifiability from temporal sequences.

Original: While both can model interventions in their framework, they do not explicitly exploit the knowledge of the intervention targets and require additional assumptions in terms of sufficient variation.
Revised: While both can model interventions in their framework, they do not explicitly exploit the knowledge of the intervention targets and require additional assumptions in terms of sufficient variation.

Original: Focusing on causal structures in the data, von Kügelgen et al.
Revised: Focusing on causal structures in the data, von Kügelgen et al.

Original: (2021) demonstrate that contrastive learning methods can block-identify causal variables by considering augmentations as interventions on the style of the image, while keeping the content unchanged.
Revised: (2021) demonstrate that contrastive learning methods can block-identify causal variables by considering augmentations as interventions on the style of the image, while keeping the content unchanged.

Original: Locatello et al.
Revised: Locatello et al.

Original: (2020) identify independent latent causal factors from pairs of observations that only differ in a subset of causal factors.
Revised: (2020) identify independent latent causal factors from pairs of observations that only differ in a subset of causal factors.

Original: Brehmer et al.
Revised: Brehmer et al.

Original: (2022) have recently extended this setup to variables with instantaneous causal effects, but require pairs of observations that share the noise term for all variables, except for a single intervened variable, i.e. counterfactual samples.
Revised: (2022) have recently extended this setup to variables with instantaneous causal effects, but require pairs of observations that share the noise term for all variables except one intervened variable, i.e. counterfactual samples.

Original: Finally, CITRIS (Lippe et al., 2022) uses temporal sequences with interventions to identify the minimal causal variables, i.e. the part of a potentially multidimensional causal variable that is influenced by the provided interventions.
Revised: Finally, CITRIS (Lippe et al., 2022a) uses temporal sequences with interventions to identify the minimal causal variables, i.e. the part of a potentially multidimensional causal variable that is influenced by the provided interventions.

Original: Thereby, CITRIS considers the causal variables to have temporal dependencies, but being independent within a time step conditioned on the previous time step.
Revised: Thereby, CITRIS considers the causal variables to have temporal dependencies, but being independent within a time step conditioned on the previous time step.

Original: In this paper, we focus on the relation of CITRIS (Lippe et al., 2022) to intervention design, since its setting is the closest to common intervention design methods by explicitly considering intervention targets for n variables. Still, we note that similar derivations may be possible for other causal representation learning methods, like Lachapelle et al. (2022), under additional assumptions.
Revised: In this paper, we focus on the relation of CITRIS (Lippe et al., 2022a) to intervention design, since its setting is the closest to common intervention de-

Original: C t
Revised: C t

Original: C t···
Revised: C t···

Original: C tK
Revised: C tK

Original: C t + C t + ···
Revised: C t + C t + ···

Original: C t +1 K
Revised: C t +1 K

Original: X t + ε t +
Revised: X t + ε t +

Original: X t ε t
Revised: X t ε t

Original: I t +
Revised: I t +

Original: I t + ···
Revised: I t + ···

Original: I t +1 K
Revised: I t +1 K

Original: R t +
Revised: R t +

Original: Temporal causal relations
Revised: Temporal causal relations

Original: Interventions
Revised: Interventions

Original: Latent confounding
Revised: Latent confounding

Original: Observations
Revised: Observations

Original: C t +
Revised: C t +

Original: C t has as parents a subset of the causal factors at the previous time step C t 1 , . . , C tK , in All causal variables C t t +1 cause the observation X t +1 .
Revised: C t sign methods by explicitly considering intervention targets for K variables. Still, we note that similar derivations may be possible for other causal representation learning methods, like Lachapelle et al.

Original: R t +1 is a confounder between the intervention targets
Revised: (2022), under additional assumptions.

Original:  INTERVENTION DESIGN FOR CAUSAL REPRESENTATION LEARNING
Revised:  INTERVENTION DESIGN FOR CAUSAL REPRESENTATION LEARNING

Original: We first review the setting and assumptions on which CITRIS (Lippe et al., 2022) aims to identify the causal variables from high-dimensional observations.
Revised: We first review the setting and assumptions on which CITRIS (Lippe et al., 2022a) aims to identify the causal variables from high-dimensional observations.

Original: For simplicity, we limit ourselves to the aspects relevant for designing the intervention experiments, and refer to Lippe et al.
Revised: For simplicity, we limit ourselves to the aspects relevant for designing the intervention experiments, and refer to Lippe et al.

Original:  for the full details.
Revised: (2022a) for the full details.

Original: We then discuss the identifiability conditions of CITRIS with respect to the interventions, and bring them into context of common intervention design settings.
Revised: We then discuss the identifiability conditions of CITRIS with respect to the interventions, and bring them into context of common intervention design settings.

Original: Finally, we derive the minimal number of experiments that are sufficient to fulfill these conditions, and show a simple heuristic with which we can create such minimal sets.
Revised: Finally, we derive the minimal number of experiments that are sufficient to fulfill these conditions, and show a simple heuristic with which we can create such minimal sets.

Original: 3.1 TEMPORAL INTERVENED SEQUENCES
Revised: 3.1 TEMPORAL INTERVENED SEQUENCES

Original: CITRIS considers a latent, temporal dynamical system with
Revised: CITRIS considers a latent, temporal dynamical system with

Original: K causal variables, ( C t 1 , C t 2 , ..., C tK ) Tt =1 , as visualized in
Revised: K causal variables, ( C t 1 , C t 2 , ..., C tK ) Tt =1 , as visualized in

Original: Figure 1.
Revised: Figure 1.

Original: The causal graph may contain temporal relations, but no instantaneous effects.
Revised: The causal graph may contain temporal relations, but no instantaneous effects.

Original: This means that we can model each causal factor by C ti =
Revised: This means that we can model each causal factor by C ti =

Original: f i ( pa G ( C ti ) , ϵ i ) for t = 1 , . . .
Revised: f i ( pa G ( C ti ) , ϵ i ) for t = 1 , . . .

Original: , T and i = 1 , . .
Revised: , T and i = 1 , . .

Original: , K , where pa ( C ti ) ⊆
Revised: , K , where pa ( C ti ) ⊆

Original: { C t − 1, . . .
Revised: { C t − 1, . . .

Original: , C t − 1 K } .
Revised: , C t − 1 K } .

Original: The graph structure is thereby time invariant, i.e. , pa G ( C t i ) =
Revised: The graph structure is thereby time invariant, i.e. , pa G ( C ti ) =

Original: pa G ( C 1 i ) for any t
Revised: pa G ( C 1 i ) for any t

Original: = 1 , . .
Revised: = 1 , . .

Original: , T , and all ϵ i are mutually independent noises.
Revised: , T , and all ϵ i are mutually independent noises.

Original: Instead of observing the causal variables directly, we measure high-dimensional observations X t at each time step t .
Revised: Instead of observing the causal variables directly, we measure high-dimensional observations X t at each time step t .

Original: These observations represent an entangled view of all causal factors, C t , and possible noise variables ε t .
Revised: These observations represent an entangled view of all causal factors, C t , and possible noise variables ε t .

Original: To represent interventions, we augment the graph with binary intervention target variables I t ∈ { 0 , 1 } K , where I ti
Revised: To represent interventions, we augment the graph with binary intervention target variables I t ∈ { 0 , 1 } K , where I ti

Original: = 1 if C ti has been intervened upon, and I ti = 0 otherwise.
Revised: = 1 if C ti has been intervened upon, and I ti = 0 otherwise.

Original: In the causal graph, each intervention variable I ti is a parent of its respective causal variable C ti .
Revised: In the causal graph, each intervention variable I ti is a parent of its respective causal variable C ti .

Original: Thereby, CITRIS supports soft interventions (Eberhardt, 2007), which change the conditional distribution, i.e. , p ( C ti | pa ( C ti ) , I ti = 1) ̸
Revised: Thereby, CITRIS supports soft interventions (Eberhardt, 2007), which change the conditional distribution, i.e. , p ( C ti | pa ( C ti ) , I ti = 1) ̸

Original: = p ( C ti | pa ( C ti ) , I ti = 0) , and includes perfect interventions do ( C i =
Revised: = p ( C ti | pa ( C ti ) , I ti = 0) , and includes perfect interventions do ( C i =

Original: c i ) (Pearl, 2009) as a special case.
Revised: c i ) (Pearl, 2009) as a special case.

Revised: In this paper, we consider the general case where interventions could be either soft, perfect, or mixed across variables.

Revised: Further, we assume that the form of intervention, i.e. the way an intervention changes an existing mechanism, can be arbitrary and externally determined.

Original: Further, CITRIS considers a potential latent confounder R t between the intervention targets.
Revised: To model dependencies between interventions, CITRIS considers a potential latent confounder R t between the intervention targets.

Original: In alignment with the intervention design setting, R t could represent the index of the experiment E i which has been performed at time step t .
Revised: In alignment with the intervention design setting, R t could represent the index of the experiment E i which has been performed at time step t .

Original: Moreover, the causal variables are considered to be potentially multidimensional, e.g. representing a 3d position as C i ∈ R 3 .
Revised: Moreover, the causal variables are considered to be potentially multidimensional, e.g. representing a 3d position as C i ∈ R 3 .

Original: As shown by Lippe et al. (2022), multidimensional causal variables are not always identifiable from interventions, since interventions may only affect a subset of the variables dimensions. 
Revised: As shown by Lippe et al. (2022a), multidimensional causal variables are not always identifiable from interventions, since interventions may only affect a subset of the variables dimensions. Instead, we split a causal variable into two parts, s i ( C ti ) = ( s var i

Revised: ( C ti ) , s inv

Original: Thus, we are limited to identifying the intervention-dependent parts of variables, i.e. the minimal causal variables .
Revised: i ( C ti )) where s i is invertible. In this split, s var i ( C ti ) represents the intervention-dependent part of C i and s inv i ( C ti ) the intervention-independent part. We can then write the distribution of a causal variable as:

Revised: [Equation]

Revised: Intuitively, for a multidimensional variable where an intervention may only affect a subset of dimensions, s var i ( C t +1 i )

Revised: represents the dimensions that depend on the intervention, and s inv

Revised: i ( C t +1 i ) the dimensions whose mechanisms are invariant to interventions.

Revised: Under this setting, Lippe et al.

Revised: (2022a) define a minimal causal variable as follows:

Revised: Definition 3.1.

Revised: The minimal causal variable of a causal variable C i with respect to its intervention variable

Revised: I

Revised: i is s var

Revised: i ( C ti ) of the split s i ( C ti )

Revised: =

Revised: ( s var i

Revised: ( C ti ) , s inv

Revised: i ( C ti )) , which maximizes the information content H ( s inv i ( C ti ) | pa ( C ti )) .

Revised: In other words, the minimal causal variable is the smallest part of a causal variable that strictly depends on the provided intervention.

Revised: In practice, the minimal causal variable is commonly the same as the original causal variable if an intervention affects all its dimensions, e.g. an intervention on the position of an object changes the mechanism of the x, y , and z coordinate simultaneously.

Original: 3.2 CONDITIONS FOR IDENTIFYING MINIMAL CAUSAL VARIABLES
Revised: 3.2 CONDITIONS FOR IDENTIFYING MINIMAL CAUSAL VARIABLES

Original: Under the previously described setting, Lippe et al.
Revised: Under the previously described setting, Lippe et al.

Original: (2022) show that the minimal causal variable of a causal factor C i is identifiable if for any other variable C j
Revised: (2022a) show that the minimal causal variable of a causal factor C i is identifiable if for any other variable C j

Original: , i ̸
Revised: , i ̸

Original: = j , the following condition holds:
Revised: = j , the following condition holds:

Original: [Equation]
Revised: [Equation]

Original: Intuitively, we cannot disentangle the causal variables based on interventions if the intervention target variable I t
Revised: Intuitively, we cannot disentangle the causal variables based on interventions if the intervention target variable I t

Original: +1 i is constant or can be replaced by any other target variable I t
Revised: +1 i is constant or can be replaced by any other target variable I t

Original: +1 j .
Revised: +1 j .

Original: To make it more explicit, Equation (1) can be rewritten as the following four conditions:
Revised: To make it more explicit, Equation (2) can be rewritten as the following four conditions:

Original: there exists a time step τ such that C
Revised: there exists a time step τ such that C

Original: i is intervened, i.e. , I τ i = 1 ; 2.
Revised: i is intervened, i.e. , I τi = 1 ; 2.

Original: there exists another time step τ ′ such that C
Revised: there exists another time step τ ′ such that C

Original: i is not intervened, i.e. , I τ ′ i
Revised: i is not intervened, i.e. , I τ ′ i

Original: = 0 ; 3.
Revised: = 0 ; 3.

Original: for all other causal factors C j , there exists a time step ˜ τ when I ˜ τi
Revised: for all other causal factors C j , there exists a time step ˜ τ when I ˜ τi

Original: ̸ = I ˜ τj ; 4.
Revised: ̸ = I ˜ τj ; 4.

Original: for all other causal factors C j , there exists a time step ˆ τ when I ˆ τi
Revised: for all other causal factors C j , there exists a time step ˆ τ when I ˆ τi

Original: = I ˆ τj .
Revised: = I ˆ τj .

Original: The first two conditions ensure that I i is not constant.
Revised: The first two conditions ensure that I i is not constant, which is strictly necessary for Equation (2) to hold.

Original: The last two conditions guarantee that if the intervention target 
Revised: The last two conditions guarantee that if the intervention target I i is not constant, then there cannot exist a deterministic function f for which I τi = f ( I τj ) for all time steps τ . This is sufficient for fulfilling Equation (2),

Original:  I  iis not constant, then there cannot exist a deterministic function f for which I τ i =
Revised: since I t +1 i cannot be determined from I t +1 j and C t and we assume that the minimal causal variable is not the empty set, i.e. C t

Original: f ({++
 i ̸⊥⊥+} I τ j ) for all time steps τ .
Revised: {++
 i ̸⊥⊥+} I t +1 i without conditions.

Revised: At the same time, these conditions are necessary, because if condition 3 or 4 does not hold, we have I ˆ

Revised: τi = I ˆ τj or I ˆ

Revised: τi ̸

Revised: = I ˆ τj for all time steps τ .

Revised: This means that there exists a deterministic function with I τ i

Revised: =

Revised: f

Original:  Instead of considering different time steps, we can also reformulate the conditions in terms of different experiments.
Revised: ( I τ j ) , and hence C t +1 i ⊥⊥ I t +1 i | I t +1 j , violating Equation (2). Hence, conditions 1-4 are both sufficient and necessary with respect to Equation (2). Instead of considering different time steps, we can also reformulate the conditions in terms of different experiments.

Original: Any distribution over intervention targets, p ( I t +1 ) , can be represented by a distribution over all possible combinations of intervention targets, i.e. the experiments E 1 , ..., E N .
Revised: Any distribution over intervention targets, p ( I t +1 ) , can be represented by a distribution over all possible combinations of intervention targets, i.e. the experiments E 1 , ..., E N .

Original: In this case, the regime variable R t +1 can be seen as an experiment indicator, where R t +1 = l denotes that the intervention targets at time step t + 1 follow experiment E l .
Revised: In this case, the regime variable R t +1 can be seen as an experiment indicator, where R t +1 = l denotes that the intervention targets at time step t + 1 follow experiment E l .

Original: Thus, the distribution R t +1 follows the distribution over the experiments.
Revised: Thus, the distribution R t +1 follows the distribution over the experiments.

Original: For a limited sample size, this distribution may correspond to the number of samples we have per experiment.
Revised: For a limited sample size, this distribution may correspond to the number of samples we have per experiment.

Original: On the other hand, the conditional distribution of the intervention targets can then be written as a deterministic function of the regime variable:
Revised: On the other hand, the conditional distribution of the intervention targets can then be written as a deterministic function of the regime variable:

Original: [Equation]
Revised: [Equation]

Original: In other words, I t +1 i
Revised: In other words, I t +1 i

Original: = 1 if C i has been intervened on in the experiment E l (with R t +1 = l ), and otherwise 0 .
Revised: = 1 if C i has been intervened on in the experiment E l (with R t +1 = l ), and otherwise 0 .

Original: Now, considering a set of the experiments { E 1 , ..., E N } , we can rewrite the conditions above as follows:
Revised: Now, considering a set of the experiments { E 1 , ..., E N } , we can rewrite the conditions above as follows:

Original: there exists an experiment E l such
Revised: there exists an experiment E l such

Original: that C i ∈
Revised: that C i ∈

Original: E l ; 2.
Revised: E l ; 2.

Original: there exists an experiment E o such that C
Revised: there exists an experiment E o such that C

Original: i ̸∈ E
Revised: i ̸∈ E

Original: o ; 3.
Revised: o ; 3.

Original: for all other causal factors C j , i ̸
Revised: for all other causal factors C j , i ̸

Original: = j , there exists an experiment E p j
Revised: = j , there exists an experiment E p j

Original: such that C i ∈
Revised: such that C i ∈

Original: E p j and C j ̸∈ E p j , or C i ̸∈ E
Revised: E p j and C j ̸∈ E p j , or C i ̸∈ E

Original: p j and C j ∈ E
Revised: p j and C j ∈ E

Original: p j ; 4.
Revised: p j ; 4.

Original: for all other causal factors C j , i ̸
Revised: for all other causal factors C j , i ̸

Original: = j , there exists an experiment E q j such that C i , C j ∈ E q j , or C i , C j ̸∈ E q j ;
Revised: = j , there exists an experiment E q j such that C i , C j ∈ E q j , or C i , C j ̸∈ E q j ;

Original: Note  the experiments fulfilling these conditions do not need to be mutually exclusive, i.e. an experiment can fulfill both condition 1 and 2.
Revised: Note that the experiments fulfilling these conditions do not need to be mutually exclusive, i.e. an experiment can fulfill both condition 1 and 2.

Original: With this, our goal becomes finding the minimal number of experiments N ′ for a given number of causal variables K that can fulfill these four conditions.
Revised: With this, our goal becomes finding the minimal number of experiments N ′ for a given number of causal variables K that can fulfill these four conditions.

Original: 3.3 DERIVING THE MINIMAL NUMBER OF EXPERIMENTS
Revised: 3.3 DERIVING THE MINIMAL NUMBER OF EXPERIMENTS

Original: Based on the four conditions, we derive the minimal number of experiments in two steps.
Revised: Based on the four conditions, we derive the minimal number of experiments in two steps.

Original: Firstly, following condition 3, we need to ensure that every variable C i has a unique pattern of being intervened or passively observed in different experiments.
Revised: Firstly, following condition 3, we need to ensure that every variable C i has a unique pattern of being intervened or passively observed in different experiments.

Original: In other words, the intervention target variables I i , I j of two causal variables C i , C j cannot be identical for all experiments.
Revised: In other words, the intervention target variables I i , I j of two causal variables C i , C j cannot be identical for all experiments.

Original: For simpler exposition, we express the intervention pattern of a causal variable C i across N experiments as a binary code b i ∈ { 0 , 1 } N , where b il = 1 [ C i ∈
Revised: For simpler exposition, we express the intervention pattern of a causal variable C i across N experiments as a binary code b i ∈ { 0 , 1 } N , where b il = 1 [ C i ∈

Original: E l ] .
Revised: E l ] .

Original: As an example, consider two variables C 1 , C 2 , for which we have the two experiments E 1 = { C 1 } , E 2 = { C 1 , C 2 } .
Revised: As an example, consider two variables C 1 , C 2 , for which we have the two experiments E 1 = { C 1 } , E 2 = { C 1 , C 2 } .

Original: We represent this pattern with the binary code b 1 = 11 for C 1 , since C 1 is intervened in both experiments, and b 2 = 01 for
Revised: We represent this pattern with the binary code b 1 = 11 for C 1 , since C 1 is intervened in both experiments, and b 2 = 01 for

Original: C 2 , since C 2 is only passively observed in E 1 .
Revised: C 2 , since C 2 is only passively observed in E 1 .

Original: Essentially, the binary code b i is a concatenation of the binary intervention targets of a variable C i across experiments E 1 , ..., E N .
Revised: Essentially, the binary code b i is a concatenation of the binary intervention targets of a variable C i across experiments E 1 , ..., E N .

Original: With this representation, the condition that the intervention target variables of two variables, I i , I j , cannot be identical for all experiments, translates to their binary code b i , b j being unique, i.e. different in at least one digit.
Revised: With this representation, the condition that the intervention target variables of two variables, I i , I j , cannot be identical for all experiments, translates to their binary code b i , b j being unique, i.e. different in at least one digit.

Original: To ensure that the binary codes for all variables b 1 , ..., b K are unique, we would need at least ⌈ log 2 ( K ) ⌉ experiments, since a binary code of length L = ⌈ log 2 ( K ) ⌉ can represent 2 L ≥ K different values.
Revised: To ensure that the binary codes for all variables b 1 , ..., b K are unique, we would need at least ⌈ log 2 ( K ) ⌉ experiments, since a binary code of length L = ⌈ log 2 ( K ) ⌉ can represent 2 L ≥ K different values.

Original: However, we need to exclude two codes: (1) the code of all zeros { 0 } L , which violates condition 1, and (2) the code of all ones { 1 } L , i.e. , interventions at all time, violating condition 2.
Revised: However, we need to exclude two codes: (1) the code of all zeros { 0 } L , which violates condition 1, and (2) the code of all ones { 1 } L , i.e. , interventions at all time, violating condition 2.

Original: Removing these two from the available binary codes, we obtain a minimum code length of ⌈ log 2 ( K + 2) ⌉ in order to have K unique, valid codes.
Revised: Removing these two from the available binary codes, we obtain a minimum code length of ⌈ log 2 ( K + 2) ⌉ in order to have K unique, valid codes.

Original: Consider, for example, two variables C 1 , C 2 , for which we have four possible binary codes: { 00 , 01 , 10 , 11 } .
Revised: Consider, for example, two variables C 1 , C 2 , for which we have four possible binary codes: { 00 , 01 , 10 , 11 } .

Original: Removing the code for a variable being always passively observed, 00 , and the code for a variable always being intervened, 11 , we are left with the ⌈ log 2 (2 + 2) ⌉
Revised: Removing the code for a variable being always passively observed, 00 , and the code for a variable always being intervened, 11 , we are left with the ⌈ log 2 (2 + 2) ⌉

Original: = 2 codes { 01 , 10 } .
Revised: = 2 codes { 01 , 10 } .

Original: In a second step, we need to extend this bound to fulfill condition 4, i.e. for all pairs C i , C j , there exists an experiment for which I
Revised: In a second step, we need to extend this bound to fulfill condition 4, i.e. for all pairs C i , C j , there exists an experiment for which I

Original: i = I j holds.
Revised: i = I j holds.

Original: In other words, we need to prevent that b i = ¬ b j for any i, j , with ¬ b representing the one’s complement of a binary code b .
Revised: In other words, we need to prevent that b i = ¬ b j for any i, j , with ¬ b representing the one’s complement of a binary code b .

Original: In the example above, this implies that we cannot use the codes b 1 =and b 2 = 10 for C 1 , C 2 , since b 1 = ¬ b 2 .
Revised: In the example above, this implies that we cannot use the codes b 1 =and b 2 = 10 for C 1 , C 2 , since b 1 = ¬ b 2 .

Original: More generally, any subset of more than 2 L − 1 unique codes of length L must contain a pair b i , b j for which b i = ¬ b j .
Revised: More generally, any subset of more than 2 L − 1 unique codes of length L must contain a pair b i , b j for which b i = ¬ b j .

Original: Hence, we effectively need to double the previous number of codes to ensure that there exist a subset of K unique codes that are not complementarities of each other.
Revised: Hence, we effectively need to double the previous number of codes to ensure that there exist a subset of K unique codes that are not complementary to each other.

Original: Note, however, that this does not affect the two constant codes, all zeros and all ones, since they are invalid in any code space and are the inverse of each other.
Revised: Note, however, that this does not affect the two constant codes, all zeros and all ones, since they are invalid in any code space and are the inverse of each other.

Original: Hence, the minimal code length to find K binary codes that are (1) unique, (2) nonconstant, and (3) not complementarities of each others, is 
Revised: Hence, the minimal code length to find K binary codes that are (1) unique, (2) nonconstant, and (3) not complementarities of each others, is ⌈ log [Equation] [Equation] =

Revised: [Equation]

Revised: +

Original: [Equation]
Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: log

Revised: [Equation]

Original:  This bound simultaneously corresponds to the minimal number of experiments needed to fulfill the conditions 1-4.
Revised: K ) [Equation] + 2 . This bound simultaneously corresponds to the minimal number of experiments needed to fulfill the conditions 1-4, which we summarize in the following proposition:

Revised: Proposition 3.2.

Revised: The minimal number of experiments to fulfill the identifiability condition of Equation (2) is ⌊ log 2 ( K ) ⌋ + 2 , with K being the number of causal variables.

Original: Onesimple way of preventing codes from being complementarities of each other is to add the purely observational regime, i.e. E 0 = ∅ , to an experimental set that already fulfills conditions 1-3.
Revised: One way of preventing codes from being complementary to each other is to add the purely observational regime, i.e. E 0 = ∅ , to an experimental set that already fulfills conditions 1-3.

Original: This ensures that there exist an experimentE l for which
Revised: This ensures that there exist an experiment

Original:  C i , C j ̸∈ E l and thus b il = b jl for all pairs C i , C j .
Revised: E l for which C i , C j ̸∈ E l and thus b il = b jl for all pairs.

Original: A simple algorithm for creating such sets of experiments is shown in Algorithm 1.
Revised: A simple algorithm for creating such sets of experiments is shown in Algorithm 1.

Original: Taking again the example of two variables C 1 , C 2 , we first create all binary codes of length ⌊ log 2 (2) ⌋ + 1 = 2 : B = { 00 , 01 , 10 , 11 } ( i.e. one experiment less than the bound, since we add the observational experiment later).
Revised: Taking again the example of two variables C 1 , C 2 , we first create all binary codes of length ⌊ log 2 (2) ⌋ + 1 = 2 : B = { 00 , 01 , 10 , 11 } ( i.e. one experiment less than the bound, since we add the observational experiment later).

Original: We then remove the code for passively observing a specific causal variable in all experiments:
Revised: We then remove the code for passively observing a specific causal variable in all experiments:

Original: B \ { 00 } = { 01 , 10 , 11 } .
Revised: B \ { 00 } = { 01 , 10 , 11 } .

Original: Next, we add the experiment in which all causal variables are jointly, passively observed:
Revised: Next, we add the experiment in which all causal variables are jointly, passively observed:

Original: B = { 001 , 010 , 011 } .
Revised: B = { 001 , 010 , 011 } .

Original: From these three codes, we pick two codes for the two causal variables C 1 , C 2 .
Revised: From these three codes, we pick two codes for the two causal variables C 1 , C 2 .

Original: Note that any combination of two codes from B is valid, and we could [Table] Require: Number of variables K 1: Create all possible binary codes of length L = 
Revised: Note that any combination of two codes from B is valid, and we could pick them based on some heuristic, for instance minimizing the number of interventions: b 1 = 001 , b 2 = 010 .

Original: ⌊ log 2 ( K ) ⌋ + 1 as set B = { 0 , 1 } L 2: Remove the code of observing a variable passively in all experiments, { 0 } L , from B 3: Extend all codes in B by appending { 0 } , i.e. an experiment where all variables are passively observed 4: From the remaining codes in B , (arbitrarily) pick K unique codes b 1 , ..., b K , one for each causal variable C i 5:

Original: Create experiments by using the codes as binary intervention targets: E l = { C i | i ∈ (cid:74) 1 ..K (cid:75) , b il = 1 } pick them based on some heuristic, for instance minimizing the number of interventions: b 1 = 001 , b 2 = 010 .

Original: We then create the ⌊ log 2 (2) ⌋ + 2 = 3 experiments based on these codes: E 1 = ∅ , E 2 = { C 2 } , E 

Revised: We then create the ⌊ log 2 (2) ⌋ + 2 = 3 experiments based on these codes: E 1 = ∅ , E 2 = { C 2 } , E

Original: = { C 1 } .
Revised: = { C 1 } .

Revised: 1 , ..., b K , one for each causal variable C i 5: Create experiments by using the codes as binary intervention targets: E l = { C i | i ∈ (cid:74) 1 ..K (cid:75) , b il = 1 }

Original: In conclusion, we have shown that we can guarantee to find the minimal causal variables of a set of causal variables
Revised: In conclusion, we have shown that we can guarantee to find the minimal causal variables of a set of causal variables

Original: C 1 , ..., C K with as little as ⌊ log 2 ( K ) ⌋ + 2 experiments, of which one can always be the observational regime.
Revised: C 1 , ..., C K with as little as ⌊ log 2 ( K ) ⌋ + 2 experiments, of which one can always be the observational regime.

Revised: Furthermore, the results generalize to iCITRIS (Lippe et al., 2022b), a recent extension of CITRIS to instantaneous effects, when considering perfect interventions, since both rely on the same intervention condition of Equation (2).

Original: In comparison to the bound derived by Eberhardt (2007) for causal discovery ( ⌊ log 2 ( K ) ⌋ +1 ) in the worst case scenario, we require just exactly one additional experiment to identify the minimal causal variables.
Revised: In comparison to the bound derived by Eberhardt (2007) for causal discovery ( ⌊ log 2 ( K ) ⌋ + 1 ) in the worst case scenario, we require just exactly one additional experiment to identify the minimal causal variables.

Original: The similarity of the two bounds suggests that we can potentially use similar extensions of the causal discovery domain, with minimal adjustments, for causal representation learning, since we may only have to add one more experiment.
Revised: Despite the different setups and goals in causal representation learning and causal discovery, the similarity of the two bounds suggests that we can potentially use similar extensions of the causal discovery domain, with minimal adjustments, for causal representation learning, since we may only have to add one more experiment.

Original: Such extensions include, for example, limiting the number of simultaneous interventions (Hyttinen et al., 2013) or selecting the cheapest set of experiments according to some cost function (Ghassami et al., 2018; 
Revised: Such extensions include, for example, limiting the number of simultaneous interventions (Hyttinen et al., 2013) or selecting the cheapest set of experiments according to some cost function (Ghassami et al., 2018; Kocaoglu et al., 2017a;

Original: Kocaoglu et al., 2017a; Lindgren et al., 2018).
Revised: Lindgren et al., 2018).

Original:  EXPERIMENTS
Revised:  EXPERIMENTS

Original: To verify that CITRIS can operate in a limited experimental setting, we repeat the experiments of Lippe et al.
Revised: To verify that CITRIS can operate in a limited experimental setting, we repeat the experiments of Lippe et al.

Original: (2022) on [Table] [Equation] [Table]
Revised: (2022a) on the Temporal Causal3DIdent dataset, but with a smaller set of interventions.

Original: o - - ✓ - rot_s - - ✓ ✓ hue_o - ✓ ✓ - hue_b - - - ✓ hue_s - ✓ - - the Temporal Causal3DIdent dataset, but with a smaller set of interventions.

Original: The Temporal Causal3DIdent dataset consists of 3D renderings ( 64 × 64 pixels) of an object shape under varying positions, rotations, and lights.
Revised: The Temporal Causal3DIdent dataset consists of 3D renderings ( 64 × 64 pixels) of an object shape under varying positions, rotations, and lights.

Original: For simplicity, we fix the shape to a teapot, which leaves six causal variables that causally interact over time.
Revised: For simplicity, we fix the shape to a teapot, which leaves six causal variables that causally interact over time.

Original: Using our bound derived in
Revised: Using our bound derived in

Original: Section 3.3, we obtain that ⌊ log 2 (6) ⌋ + 2 = 4 intervention experiments are sufficient to identify the variables of the
Revised: Section 3.3, we obtain that ⌊ log 2 (6) ⌋ + 2 = 4 intervention experiments are sufficient to identify the variables of the

Original: Temporal Causal3DIdent Teapot dataset.
Revised: Temporal Causal3DIdent Teapot dataset.

Original: Hence, we sample four intervention experiments following Algorithm 1, and show one example of the experiment set in Table 2. 
Revised: Hence, we sample four intervention experiments following Algorithm 1, and show one example of the experiment set in Table 2. The [Table] [Table]

Original: The first experiment, E 1 , is the observational regime where all variables are passively observed, and the other experiments
Revised: [Equation] - first experiment, E 1 , is the observational regime where all variables are passively observed, and the other experiments

Original: E 2 , E 3 , E 4 cover the needed interventions.
Revised: E 2 , E 3 , E 4 cover the needed interventions.

Original: The data is generated following the same process as in Lippe et al.
Revised: The data is generated following the same process as in Lippe et al.

Original:  For the regime variable, we sample the purely observational experiment E 1 50% of the time, and uniformly between
Revised: (2022a). For the regime variable, we sample the purely observational experiment E 1 50% of the time, and uniformly between

Original: E 2 , E 3 , E 4 otherwise.
Revised: E 2 , E 3 , E 4 otherwise.

Original: Hence, we obtain samples from all experiments, with a bias towards observational data, since this is usually cheaper to obtain.
Revised: Hence, we obtain samples from all experiments, with a bias towards observational data, since this is usually cheaper to obtain.

Original: In Table 1, we show the results of CITRIS on the Temporal Causal3DIdent dataset under different experimental settings.
Revised: In Table 1, we show the results of CITRIS on the Temporal Causal3DIdent dataset under different experimental settings.

Original: The setting “ full experiments ” is the original setup of Lippe et al.
Revised: The setting “ full experiments ” is the original setup of Lippe et al.

Original: (2022), where the intervention targets are independently sampled from a Bernoulli distribution,
Revised: (2022a), where the intervention targets are independently sampled from a Bernoulli distribution,

Original: i.e.
Revised: i.e.

Original: I t +1 i ∼ Bernoulli (0 . 1) .
Revised: I t +1 i ∼ Bernoulli (0 . 1) .

Original: Hence, we effectively obtain sam- ples from all possible experiments.
Revised: Hence, we effectively obtain samples from all possible experiments.

Original: In contrast, the setting “ minimal experiments ” only uses four experiments, as described before.
Revised: In contrast, the setting “ minimal experiments ” only uses four experiments, as described before.

Original: We repeat all experiments with three different seeds and three different minimal sets of experiments (see Appendix A.1 for the specific sets).
Revised: We repeat all experiments with three different seeds and three different minimal sets of experiments (see Appendix A.1 for the specific sets).

Original: The results on the minimal experiment set show that CITRIS is still able to disentangle the causal variables decently, with small degradation in performance compared to the full experiments.
Revised: The results on the minimal experiment set show that CITRIS is still able to disentangle the causal variables decently, with small degradation in performance compared to the full experiments.

Original: In general, we find that the model is more likely to entangle variables with a very similar intervention pattern due to possible local minima.
Revised: In general, we find that the model is more likely to entangle variables with a very similar intervention pattern due to possible local minima.

Original: For example, for the set of experiments in Table 2, one model seed entangled the background hue (hue_b) and the object position (pos_o), which, in terms of interventions, only differ in experiment E 2 .
Revised: For example, for the set of experiments in Table 2, one model seed entangled the background hue (hue_b) and the object position (pos_o), which, in terms of interventions, only differ in experiment E 2 .

Original: Still, CITRIS under a minimal set of experiments considerably outperforms the best baseline model, an iVAE (Khemakhem et al., 2020a), on a full set of experiments.
Revised: Still, CITRIS under a minimal set of experiments considerably outperforms the best baseline model, an iVAE (Khemakhem et al., 2020a), on a full set of experiments.

Original: In conclusion, CITRIS can identify the causal variables even under a minimal set of experiments well, but it is more challenging to optimize due to strong dependencies between intervention targets.
Revised: In conclusion, CITRIS can identify the causal variables even under a minimal set of experiments well, but it is more challenging to optimize due to strong dependencies between intervention targets.

Original:  CONCLUSION
Revised:  CONCLUSION

Original: We show that ⌊ log 2 ( K ) ⌋ + 2 intervention experiments are sufficient to identify K causal variables from highdimensional observations like images for the causal representation learning method CITRIS.
Revised: In this paper, we show that ⌊ log 2 ( K ) ⌋ + 2 intervention experiments are sufficient to identify K causal variables from high-dimensional observations like images for the causal representation learning method CITRIS.

Original: This bound has a strong resemblance to the bound in intervention design, which guarantees the discovery of a causal graph for known variables in just one experiment less than in the bound we present.
Revised: This bound has a strong resemblance to the bound in intervention design, which guarantees the discovery of a causal graph for known variables in just one experiment less than in the bound we present.

Revised: Further, we empirically verify this bound by showing that CITRIS with a minimal set of four experiments disentangles the six causal variables of the Temporal

Revised: Causal3DIdent dataset almost as well as with an unlimited number of experiments.

Original: This suggests that work from the field of intervention designcan also be adapted tothe causal representation learning domain, and the presented work can provide a first step towards this goal. 
Revised: This suggests that adapting further methods from the field of intervention design to causal representation learning holds promise for future work, and the presented work can provide a first step towards this goal. Author Contributions

Revised: P. Lippe and S. Magliacane conceived the idea.

Original: A EXPERIMENTAL DETAILS
Revised: P. Lippe developed the theoretical results, created the datasets, ran the experiments, and wrote the paper. S. Löwe and S. Magliacane provided comments and feedback throughout the writing process. S. Magliacane, S. Löwe, Y. M. Asano, T. Cohen, E. Gavves advised during the project. Acknowledgements

Revised: We thank SURFsara for the support in using the Lisa Compute Cluster.

Revised: This work is financially supported by Qual- comm Technologies Inc., the University of Amsterdam and the allowance Top consortia for Knowledge and Innovation (TKIs) from the Netherlands Ministry of Economic Affairs and Climate Policy.

Revised: A EXPERIMENTAL DETAILS

Original: This section provides additional details on the experimental setting.
Revised: This section provides additional details on the experimental setting.

Original: We first show the specific minimal sets of experiments used in the Temporal Causal3DIdent dataset (Appendix A.1).
Revised: We first show the specific minimal sets of experiments used in the Temporal Causal3DIdent dataset (Appendix A.1).

Original: Next, we provide examples of correlation maps for these experiments.
Revised: Next, we provide examples of correlation maps for these experiments.

Original: A.1 MINIMAL SETS OF EXPERIMENTS
Revised: A.1 MINIMAL SETS OF EXPERIMENTS

Original: The chosen sets of experiments are shown in Figure 2.
Revised: The chosen sets of experiments are shown in Figure 2.

Original: In each set, the experiment E 1 is the observational regime, and
Revised: In each set, the experiment E 1 is the observational regime, and

Original: E 2 , E 3 , E 4 cover all conditions for the interventions.
Revised: E 2 , E 3 , E 4 cover all conditions for the interventions.

Original: The assignment of binary codes to causal variables, as performed in Algorithm 1, is done mostly randomly.
Revised: The assignment of binary codes to causal variables, as performed in Algorithm 1, is done mostly randomly.

Original: To cover all critically different settings, we ensure that for each causal variable, there exists an experimental setting where it is intervened in only one experiment, and that there exists a different experimental setting where it is intervened in two experiments.
Revised: To cover all critically different settings, we ensure that for each causal vari- E E E

Revised: Epos_o - ✓ - ✓ rot_o - - ✓ - rot_s - - ✓ ✓ hue_o - ✓ ✓ - hue_b - - - ✓ hue_s - ✓ - - (a) Experimental setting

Revised: E

Original: For instance, pos_o is intervened in only E 2 for the second experimental setting (Figure 2b), and intervened in
Revised: E E Epos_o - ✓ - - rot_o - - ✓ - rot_s - ✓ ✓ - hue_o - - - ✓ hue_b - ✓ - ✓ hue_s - - ✓ ✓ (b) Experimental setting E

Revised: E

Revised: E

Revised: Epos_o - ✓ - ✓ rot_o - - ✓ ✓ rot_s - - - ✓ hue_o - - ✓ - hue_b - ✓ - - hue_s - ✓ ✓ - (c)

Revised: Experimental setting po s _

Revised: x po s _

Revised: y po s _ z r o t _a l pha r o t _be t a r o t

Revised: s hue_o hue_ s hue_b

Revised: True causal factors pos_o rot_o rot_s hue_o hue_s hue_b

Revised: No variable

Revised: La t en t d i m en s i on s

Revised: [Figure]

Revised: po s _ x po s _

Revised: y po s _ z r o t _a l pha r o t _be t a r o t

Revised: s hue_o hue_ s hue_b

Revised: True causal factors

Revised: [Figure]

Revised: experiment conducted the

Revised: Figure able, there exists an experimental setting where it is intervened in only one experiment, and that there exists a different experimental setting where it is intervened in two experiments.

Revised: For instance, pos_o is intervened in only E 2 for the second experimental setting (Figure 2b), and intervened in

Original: E 2 and E 4 for the first experimental setting (Figure 2a).
Revised: E 2 and E 4 for the first experimental setting (Figure 2a).

Original: A.2 ADDITIONAL RESULTS
Revised: A.2 ADDITIONAL RESULTS

Original: We provide examples of the R 2 correlation matrices between learned latent variables and true causal factors in
Revised: We provide examples of the R 2 correlation matrices between learned latent variables and true causal factors in

Original: Figure 3.
Revised: Figure 3.

Original: The left figure shows the result of CITRIS when trained on the full experiment setting, i.e. intervention targets are independently sampled.
Revised: The left figure shows the result of CITRIS when trained on the full experiment setting, i.e. intervention targets are independently sampled.

Original: The figure on the right, on the other hand, shows the correlation matrix for a minimal set of experiments, specifically the setting of Figure 2a.
Revised: The figure on the right, on the other hand, shows the correlation matrix for a minimal set of experiments, specifically the setting of Figure 2a.

Original: As mentioned in Section 4, the model is still able to disentangle the variables well, but additional correlations between variables with similar intervention patterns can occur.
Revised: As mentioned in Section 4, the model is still able to disentangle the variables well, but additional correlations between variables with similar intervention patterns can occur.

Original: The correlation of 0 .
Revised: The correlation of 0 .

Original: 43 between pos_o and hue_b is one example for such, since pos_o and hue_b only differ in E 2 in terms of interventions.
Revised: 43 between pos_o and hue_b is one example for such, since pos_o and hue_b only differ in E 2 in terms of interventions.