Weakly supervised causal representation learning
Anonymous Author(s)
AffiliationAddress email
Abstract
Learning high-level causal representations together with a causal model fromunstructured low-level data such as pixels is impossible from observational dataalone.
We prove under mild assumptions that this representation is howeveridentifiable in a weakly supervised setting.
This requires a dataset with pairedsamples before and after random, unknown interventions, but no further labels.
Wethen introduce implicit latent causal models, variational autoencoders that representcausal variables and causal structure without having to optimize an explicit discretegraph structure.
On simple image data, including a novel dataset of simulatedrobotic manipulation, we demonstrate that such models can reliably identify thecausal structure and disentangle causal variables
Introduction
Interven’on 𝑥 "𝑥 𝑧 ̃𝑧
The dynamics of many systems can be described in terms ofsome high-level variables and causal relations between them.
Often, these causal variables are not known but only observedin some unstructured, low-level representation, such as the pixels of a camera feed.
Learning the causal representations together with the causal structure between them is a challengingproblem and may be important for instance for applications inrobotics and autonomous driving [1].
Without prior assumptions on the data-generating process or supervision, it is impossible to uniquely identify the causal variables and their causalstructure [2, 3].
In this work, we show that a weak form of supervision is sufficient to identify both the causal representations and the structural causal model between them.
We consider a setting inwhich we have access to data pairs, representing the system before and after a randomly chosen, unknown intervention.
Suchdata may for instance be collected from a video feed of an external agent or demonstrator interacting with a system.
Neitherlabels on the intervention targets nor active control of the interventions are necessary for our identifiability theorem, makingthis setting useful for offline learning.
We prove that with thisform of weak supervision, latent causal models (LCMs)—structural causal models (SCMs) togetherwith a decoder from the causal factors to the data space—are identifiable up to a relabelling andelementwise reparameterizations of the causal variables.
We then discuss two practical methods for LCM inference.
First, we define explicit latent causalmodels (ELCMs) as a variational autoencoder (VAE) [4] in which the causal variables are the latentvariables and the prior is based on an SCM.
While this approach works in simple problems, it can be finicky and is difficult to scale.
We trace this to a major challenge in causal representation learning,namely that it is a chicken-and-egg problem: it can be difficult to learn the causal variables when thecausal graph is not yet learned, and it is difficult to learn the graph without knowing the variables.
To overcome this optimization difficulty, we introduce a second model class: implicit latent causalmodels (ILCMs).
These models can represent causal structure and variables without requiring anexplicit, discrete graph representation, which makes gradient-based optimization easier.
Nevertheless,these models still contain the causal structure implicitly, and we discuss two algorithms that can extractit after the model is trained.
Finally, we demonstrate ILCMs on synthetic datasets, including the new
CausalCircuit dataset of a robot arm interacting with a causally connected system of light switches.
Weshow that these models can robustly learn the true causal variables and the causal structure from pixels.
Related work
Our work builds on the work of Locatello et al.
[5] on disentangled representation learning .
Theauthors introduce a similar weakly supervised setting where observations are collected before and afterunknown interventions.
In contrast to our work, however, they focus on disentangled representations,i. e. (conditionally) independent factors of variation with a trivial causal graph, which our worksubsumes as a special case.
Other relevant works on disentangled representation learning and(nonlinear) independent component analysis include Refs.
The problem of causal representation learning has been gaining attention lately, see the recent reviewby Schölkopf et al.
Lu et al.
[13] learn causal representations by observing similar causal modelsin different environments.
von Kügelgen et al.
[14] use the weakly supervised setting to study selfsupervised learning, using a known but non-trivial causal graph between content and style factors.
Lippe et al.
[15] learn causal representations from time-series data from labelled interventions,assuming that causal effects are not instantaneous but can be temporally resolved.
Yang et al.
[16]propose to train a VAE with an SCM prior, but require the true causal variables as labels.
To the bestof our knowledge, our work is the first to provide identifiability guarantees for arbitrary, unknowncausal graphs in this weakly supervised setting.
Identifiability of latent causal models from weak supervision
𝑧 !
𝑧 " 𝑧 # ̃𝑧 !
̃𝑧 " ̃𝑧 # 𝑧 #$ 𝑧 "$ 𝑧 ! $ ̃𝑧 #$ ̃𝑧 "$ ̃𝑧 !
$ 𝜑 𝑥 %𝑥 ∼ 𝑝(𝑥, %𝑥) 𝜑 𝑔 𝑔′ 𝑔 𝑔 $
I n t e r v e n e
I n t e r v e n e
In this section, we show theoretically thatcausal variables and causal mechanismsare identifiable from weak supervision.
In
Sec.
4 we will then demonstrate how we canlearn causal models in practice by traininga causally structured VAE.
3.1 Setup
We begin by defining latent causal modelsand the weakly supervised setting.
Here,we only provide informal definitions andassume familiarity with common conceptsfrom causality; see Sec.
A in the supplementary materials for a complete and precisetreatment.
In Sec.
B of the supplementarymaterial we discuss limitations of the setupand possible generalizations.
We describe the causal structure between latent variables as a Structural Causal Model(SCM).
An SCM C describes the relationbetween causal variables z 1 , . . .
, z n withdomains Z i and
f i : E i × (cid:81) j ∈ pa i Z j → Z i describe how the value of a causal variable is determined from the associated noise variables, as well as the values of its parents in the graph.
Finally, an SCM includes aprobability measure for the noise variables.
An SCM entails a unique solution s : E
→ Z defined by successively applying the causal mechanisms.
We require the causal mechanisms to be pointwise diffeomorphic, that is, for any value of the parentsz pa i we have that f i ( · ;
z pa i ) is a diffeomorphism.
Then s is also diffeomorphic and thus noisevariables can be uniquely inferred from causal variables.
This simplifies the weakly superviseddistribution, as the only stochasticity comes from the noise variables and the intervention.
The SCMalso entails an observational distribution p C ( z )
(Markov with respect to the graph of the SCM), whichis the pushforward of p E through the solution.
A perfect, stochastic intervention ( I, ( ˜ f i ) i ∈ I ) modifies an SCM by replacing for a subset of thecausal variables, called the intervention target set I ⊂ { 1 , ..., n } , the causal mechanism f
i with a newmechanism ˜ f
i : E i → Z i , which does not depend on the parents.
The intervened SCM has a newsolution ˜
s I : E → Z .
We call interventions atomic if the number of targeted variables is one or zero.
We will reason about generative models in a data space X , in which the causal structure is latent.
Also including a distribution of interventions, we define LCMs:
Definition 1 (Latent causal model (LCM)) .
A latent causal model M = ⟨C , X , g, I , p I ⟩ consists of
• an acyclic, faithful SCM C
• an observation space X ,• a decoder g : Z → X that is diffeomorphic onto its image,
• a set I of interventions on C , and
• a probability measure p I over I .
We define two LCMs as equivalent if all of their components are equal up to a permutation of thecausal variables and elementwise diffeomorphic reparameterizations of each variable, see Fig. 2.
Definition 2 (LCM isomorphism (informal)) .
Let M = ⟨C , X , g, I , p I ⟩ and M ′ =⟨C ′ , X , g ′ , I ′ , p ′I ′ ⟩ be two LCMs with identical observation space.
An LCM isomorphism betweenthem is a graph isomorphism ψ : G ( C )
→ G ( C ′ ) together with elementwise diffeomorphisms fornoise and causal variables that tell us how to reparameterize them, such that the structure functions,noise distributions, decoder, intervention set, and intervention distribution of M ′ are compatible withthe corresponding elements of M reparameterized through the graph isomorphism and elementwisediffeomorphisms.
M and M ′ are equivalent, M ∼ M ′ , if and only if there is an LCM isomorphismbetween them.
Following Ref.
[5], we define a generative process of pre and post interventional data:
Definition 3 (Weakly supervised generative process) .
Consider an LCM M where the underlying
SCM has continuous noise spaces E i , independent probabilities p E i , and admits a solution s .
Wedefine the weakly supervised generative process of data pairs ( x, ˜ x ) ∼ p XM ( x, ˜ x ) as follows:
[Equation]
3.2 Identifiability result
The main theoretical result of this paper is that an LCM M can be identified from p ( x, ˜ x ) up to arelabeling and elementwise transformations of the causal variables:
Theorem 1 (Identifiability of R -valued LCMs from weak supervision) .
Let M = ⟨C , X , g, I , p I ⟩and M ′ = ⟨C ′ , X , g ′ , I ′ , p ′I ′
⟩ be LCMs with the following properties:
• The LCMs have an identical observation space X .
• The SCMs C and C ′ both consist of n real-valued endogeneous causal variables andcorresponding exogenous noise variables, i. e. E i
= Z i =
Z ′ i
=
E ′ i
= R .
• The intervention sets I and I ′
consist of all atomic, perfect interventions, I ={∅ , { z 0 } , . .
, { z n }} and similar for I ′ .
• The intervention distribution p I and p ′I ′ have full support.
Then the following two statements are equivalent:
The LCMs entail equal weakly supervised distributions, p XM ( x, ˜ x ) =
p XM ′
( x, ˜ x ) .
The LCMs are equivalent, M ∼ M ′ .
Let us summarize the key steps of our proof, which we provide in its entirety in Sec.
A in thesupplementary material.
The direction 2 ⇒ 1 follows from the definition of equivalence.
Thedirection 1 ⇒ 2 is proven constructively along the following steps:1.
We begin by defining a diffeomorphism φ = g ′− 1 ◦ g : Z → Z ′ and note that ifz,
˜
z ∼ p ZC ( z, ˜ z ) , the weakly supervised distribution of causal variables of model C , thenφ ( z ) , φ (˜ z ) ∼ p Z ′ C ′ ( z ′ , ˜ z ′ ) .
The distribution over z, ˜ z is a mixture, where each interventiontarget I gives a mixture component; each component is supported on a different ( n + 1) -dimensional submanifold.
Therefore, there exists a bijection between the componentsψ : [ n ] → [ n ] that maps intervention targets I in M to intervention targets I ′ =
ψ ( I ) in
M ′ .
Furthermore, because the joint distribution z, ˜ z is preserved by φ , first mapping withφ , then intervening, Z φ −→ Z
′ I ′ −→ (cid:101)
Z ′ , equals Z I −→ (cid:101)
Z φ −→ (cid:101) Z ′ .
Because I = { i } is a hard intervention, for the map Z φ −→ Z
I −→ (cid:101) Z ′ , ˜ z ′
i ′ is independent ofz ′ .
Thus, in both maps, ˜ z ′ i ′ is independent of z .
This means that for the path through (cid:101) Z , theintervention sample ˜
z i is transformed into ˜ z
′ i ′
independently of z .
For R -valued variables,this statistical independence implies that the transformation is constant in z , and thus φ ( z ) i ′is constant in z j for j ̸
= i .
φ is therefore an elementwise reparametrization.3.
Using this, we can show that ψ is a causal graph isomorphism and that it is compatible withthe causal mechanisms.
This proves LCM equivalence M ∼ M ′ .
Practical latent causal models
Theorem 1 means that it is possible to learn causal structure from pixel-level data in the weaklysupervised setting.
Consider a system that is described by an unknown true LCM and assume thatwe have access to data pairs ( x, ˜ x ) sampled from its probability density.
Then we can train another
LCM with learnable components by maximum likelihood.
Assuming sufficient data and perfectoptimization, this model’s density will match that of the ground-truth LCM.
Our identifiability resultguarantees that the trained LCM then has the same causal variables and causal structure as the groundtruth, up to relabelling.
In the following, we will describe two neural network–based LCM implementations that can betrained on data.
4.1 Explicit latent causal models (ELCMs)
To implement LCMs with neural networks, we use the variational autoencoder (VAE) framework [4].
We first consider an approach where the causal variables ( z, ˜ z ) are the latent variables.
Data ( x, ˜ x ) andlatents ( z, ˜ z ) are linked by a stochastic encoder q ( z | x ) and decoder p ( x | z ) ; unlike the deterministicdecoder from Sec.
this allows us to map high-dimensional data spaces (like images) to lowdimensional causal variables.
The prior p ( z, ˜ z ) encodes the causal structure of the LCM.
It factorizesthe probability density p ( z ) according to the causal graph of the model, with learnable conditionaldensities p
( z i | z pa i ) given by the base densities and structural mechanisms in the SCM.
The SCM isalso able to model interventions and thus defines p (˜ z | z ) .
The model can be trained on the ELBO loss,a variational bound on − log p ( x, ˜ x ) .
For more details, see Sec.
E in the supplementary material.
We call this an explicit latent causal model (ELCM), as the model directly parameterizes all components of an LCM.
In particular, ELCMs contain an explicit representation of the causal graph.
Thegraph can be learned by an exhaustive search over all DAGs or through a differentiable DAG parameterization [18–21] and gradient descent.
[Figure]
In our experiments with ELCMs, which we describe in Sec.
E of the supplementary material, we findthat optimally trained ELCMs indeed correctly identify the causal structure and disentangle causalvariables on simple datasets.
However, jointly learning explicit graph and variable representationspresents a challenging optimization problem.
In particular, we observe that the loss landscape haslocal minima corresponding to wrong graph configurations.
4.2 Implicit latent causal models (ILCMs)
To enable causal representation learning in a more robust, scalable way, we propose a second LCMimplementation: Implicit Latent Causal Models (ILCMs).
ILCMs are also variational autoencoderswith a causally structured prior, but unlike ELCMs, they do not require an explicit graph parameterization and are thus easier to train in practice.
Latents The latent variables in an ILCMs are noise encodings , defined through the inverse solutionfunction as e =
s − 1 ( z ) and ˜
e = s − 1 (˜ z ) .
The pre-intervention noise encoding e is identical tothe SCM noise variables.
The post-intervention noise encoding ˜
e corresponds to the value of the
SCM noise variables that would have generated the post-intervention causal variables ˜
z under theunintervened SCM mechanisms.
ILCMs contain a stochastic encoder q
( e | x ) and decoder p ( x | e ) thatmap data ( x, ˜ x ) to noise encodings ( e, ˜ e ) .
Noise encodings have the convenient property that under an intervention with intervention targets
I , precisely the components e I change value: e i ̸ =
˜
e i ⇔ i ∈ I with probability 1.
We prove thisproperty in Sec.
A of the supplementary material.
This means that from noise encodings e, ˜ e , wecan infer interventions easily.
We use a simple heuristic intervention encoder that assigns higherintervention probability q ( i ∈ I | x, ˜ x ) to a component i the more this component of the noise encodingchanges under interventions:
[Equation]
where µ e ( x ) is the mean function of the noise encoder q ( e | x ) and h is a quadratic function withlearnable parameters.
Both the equality pattern of e under interventions and this heuristic interventionencoder are similar to the ones used for disentangled representation learning in Ref.
Prior Given encoders for noise encodings and intervention targets, let us now write down the priorp ( e, ˜ e, I ) , which encodes the structure of the weakly supervised setting.
The intervention-targetprior p ( I ) and the pre-intervention noise distribution p ( e ) are given by simple base densities, whichwe choose as uniform categorical and standard Gaussian, respectively.
The post-intervention noiseencodings ˜
e follow the conditional probability distributionp (˜ e | e, I ) =
(cid:89)
i/ ∈ I δ (˜ e i − e i ) (cid:89) i ∈ I p
(˜ e i | e ) = (cid:89) i/ ∈
I
δ
(˜ e i − e i ) (cid:89) i ∈ I ˜
[Equation]
(cid:12)(cid:12)(cid:12)(cid:12)
[Equation]
\ i ) .
[Equation]
In the second equality we have parameterized the conditional density p (˜ e i | e ) with a conditionalnormalizing flow consisting of a learnable diffeomorphic transformation ˜
e i (cid:55)→ ¯ z i =
¯ s i (˜ e i ; e \ i ) and abase density ˜ p on ¯ z i , which we choose as standard Gaussian.
How does this prior encode causal structure?
When intervening on i , the map ¯ s i transforms ˜ e i into avariable ¯ z i that is independent of e . This may sound familiar: the causal model gives another suchtransformation.
The noise encoding ˜ e is related to the causal variables via the solution function ˜ z
=s (˜ e ) .
The solution function is defined through recursive application of the causal mechanisms, so that˜ z i = s i (˜ e ) = f (˜ e i , s pa
i (˜ e anc i )) . Because we assume perfect stochastic interventions, the output ˜ z i isindependent from ˜ e anc i = e anc i .
Then, it is easy to show (see Sec.
C of the supplementary material)that for R -valued variables, ¯ z i and ˜ z i are related by an elementwise diffeomorphism.
Because wedesire identifiability up to an elementwise diffeomorphism, we are free to choose ˜ z
i := ¯ z i , and thus¯ s i (˜ e i ; ˜
e \ i ) : = s i (˜ e ) .
By learning to transform ˜ e i into ˜ z
i , we have learned the solution function!
The solution functions can now be also used to infer the unintervened causal variables: z i = s i ( e i ; e \ i ) .
We have thus learned a causal model without ever explicitly modelling a graph.
Becausethe solution function only depends on ancestors in the graph, the learned transformation s i ( e i ; e \ i )should also depend only on ancestors of i .
As each s i is constructed to be a diffeomorphism in its firstargument, jointly they have a triangular structure and thus a diffeomorphism s : e (cid:55)→ z .
In practice,however, the learned solution functions may still depend weakly on non-ancestors.
Therefore, toensure that s always forms a diffeomorphism, at some point in training, we test functional dependenceto infer ancestral dependence, pick a topological ordering of variables conforming to the ancestry,and parameterize the solution functions s i to only depend on earlier variables in the ordering.
The final question is how to implement the first terms in Eq.
(3), which encode that those noiseencodings that are not part of the intervention targets I should not change value under the intervention.
We enforce this in the encoder by setting the non-intervention components of e and ˜
e to the samevalue [similar to 5].
In Sec.
C in the supplementary material this procedure is described in moredetail.
We will refer to this projective noise encoder as q ( e, ˜
e | x, ˜ x, I ) .
Learning Putting everything together, an ILCM consists of an intervention encoder q
( I | x, ˜ x ) , anoise encoder q ( e, ˜ e | x, ˜ x, I ) , a noise decoder p ( x | e ) , and transformations / solution functions s i ( · ; e ) ,see Fig.
All of these components are implemented with neural networks and learnable, see Sec.
C inthe supplementary material for details.
The lower bound on the joint log likelihood of pre-interventionand post-intervention data is given by
[Equation]
The model is trained by minimizing the corresponding VAE loss, learning to map low-level data tonoise variables (with q ) and to map noise variables to causal variables (with s ).
The expectation over
I is computed via summation, but could alternatively be done with sampling.
In practice, we find it beneficial to add a regularization term to the loss that disincentivizes collapseto a lower-dimensional submanifold of the latent space.
For each batch of training data, we computethe batch-aggregate intervention posterior
q I ( I ) =
E x, ˜ x ∈ batch [ q ( I | x, ˜ x )] .
To the beta-VAE loss wethen add the negative entropy of this distribution, weighted with a hyperparameter.
Downstream tasks Despite the implicit representation of causal structure, we argue that ILCMs letus solve various tasks:
• Causal representation learning / disentanglement : ILCMs allow us to map low-level data xto causal variables z by applying the encoder q followed by the solution functions s .•
Intervention inference : It is also straightforward to infer intervention targets from an observed pair ( x, ˜ x ) of pre-intervention and post-intervention data, as this just requires evaluating the intervention-target encoder q I ( x, ˜ x ) .•
Causal discovery / identification : We propose two methods to infer the causal graphs after training an ILCM.
One is to use an off-the-shelf method for causal discovery on thelearned representations.
Since the ILCM allows us to infer intervention targets, we can use
intervention-based algorithms.
In this paper, we use ENCO [20], a recent differentiablecausal discovery method that exploits interventions to obtain acyclic graphs without requiring constrained optimization.
Alternatives to ENCO include DCDI [19] and GIES [22].
Alternatively, we can analyze the causal structure implicitly represented in the learnedsolution functions s i .
We propose a heuristic algorithm that proceeds in three steps.
First,it infers the topological order by sorting variables such that
s i only depends on e j if z i isafter z j in the topological order.
It then iteratively rewrites the solution functions such thatthey only depend on ancestors in the topological order.
Finally, it determines which causalancestors are direct parents by testing the functional dependence of the causal mechanisms.
We describe this algorithm in more detail in Sec.
C in the supplementary material. • Generation of interventions and counterfactuals : The ILCM entails a generative model for pairs of pre- and post-intervention data
It is straightforward to sample from the joint distribution p ( x, ˜ x, I ) , from the conditional p ( x, ˜ x | I ) , or from the conditional p (˜ x | x, I ) .
Experiments
[Table] Finally, we demonstrate latent causal modelsin practice.
Here we focus on implicit LCMs;explicit LCMs are demonstrated in similar experiments in Sec.
E in the supplementary material.
We evaluate the causal graphs learned bythe ILCM models either with ENCO (ILCM-E)or with the heuristic algorithm described above(ILCM-H).
Baselines Since we are to the best of ourknowledge the first to study causal representation learning in this weakly supervised setting,we are not aware of any baseline methods designed for this task.
We nevertheless compare ILCMs to three other methods.
First, we define adisentanglement VAE that models the weakly supervised process, but assumes independent factors of variation rather than a non-trivial causalstructure between the variables.
This baseline issimilar to the method proposed by Ref.
[5], butit differs in some implementation details to bemore comparable to our ILCM setup.
We inferthe causal graph between the learned representations with ENCO (dVAE-E).
We also compareto an unstructured β -VAE that treats x and ˜ x asi. i. d. and uses a standard Gaussian prior.
Finally, for the pixel-level data, we consider a slot attention model [23], which segments the image unsupervisedly into as many objects as there are causalvariables.
The latent representation associated to each object is considered a learned causal variable.
5.1 2D toy experiment
We first demonstrate LCMs in a pedagogical toy experiment with X =
Z = R 2 .
Training data isgenerated from a nonlinear SCM with the graph z 1 → z 2 and mapped to the data space through arandomly initialized normalizing flow.
An ILCM trained in the weakly supervised setting is able to reconstruct the causal factors accurately upto elementwise reparameterizations, as shown in Fig.
In Tbl.
1 we quantify the quality of the learnedrepresentations with the DCI disentanglement score [24].
We find that our LCM is able to disentanglethe causal factors almost perfectly, while the baselines, which assume independent factors of variation,fail as expected.
Both the ILCM and the dVAE baseline infer the intervention targets with highaccuracy.
Finally, we test the quality of the learned causal graphs.
We infer the implicit graph with
ENCO and the heuristic algorithm discussed above.
In both cases, the learned causal graph is identicalto the correct one, whereas the representations found by the dVAE baseline induce a wrong graph.
2 z *    z *
True factors  2 x   x
Data  2 z
z
Learned causal factors
We then turn to pixel-level data and more complexcausal graphs. We test ILCMs on an adaptation of the Causal3DIdent dataset [14], which contains images ofthree-dimensional objects under variable positions andlighting conditions.
We consider three causal variablesrepresenting object hue, the spotlight hue, and the position of the spotlight.
We construct six versions of this dataset,each with a different causal graph, randomly initializednonlinear structure functions, and heteroskedastic noise.
These are mapped to images with a resolution of 64 × 64 ,see Fig.
5 for examples.
ILCMs are again able to disentangle the causal variablesreliably.
The results in Tbl.
1 show that the learned representations are more disentangled than those learned bymethods that do not account for causal structure.
The LCMas well as the dVAE baseline can infer interventions withalmost perfect accuracy.
We demonstrate this in Fig. 5 bycomparing true and inferred interventions.
The ILCMs also learn the causal graphs accurately.
Whenusing ENCO to extract them, the causal graph is alwayscorrect; the heuristic algorithm fails in one instance. Incontrast, the acausal dVAE-E baseline does in most cases not find the correct causal graphs.
5.3 CausalCircuit
[Figure]
While Causal3DIdent provides a good test of the ability to disentanglefeatures that materialize in pixel space in different ways, like throughthe position of lights and the color of objects, the underlying causalstructure we imposed may feel rather ad-hoc.
To explore causal representation learning in a more intuitively causal setting, we introducea new dataset, which we call CausalCircuit.
The CausalCircuit system consists of a robot arm that can interactwith multiple touch-sensitive lights.
The lights are connected with astochastic circuit: a light is more likely to be on if its button is pressedor if its parent lights are on.
The robot arm itself can be seen as part ofthe causal system.
Concretely, we consider the causal graph shown in Fig.
This system is observed from a fixed-position camera, and we generate samples in 64 × 64 × 3resolution with MuJoCo [25], see Sec.
D of the supplementary material for more details.
ILCMs are again able to disentanglethe causal variables reliably and betterthan the acausal baselines, see Tbl. 1.
As shown in Sec.
D of the supplementary material, the slot attentionmodel fails because the lights haveno limited spatial extent and thus arenot well represented by segments ofthe image.
Interventions are identified with high accuracy.
The modelalso correctly learns the causal graphshown in Fig.
6, both when extractedwith ENCO and with our heuristic algorithm.
In Fig.
7 we demonstratehow ILCMs let us infer and manipulate causal factors and reason aboutinterventions.
By studying variations of this dataset, we tested the limitations of our method.
We find that it worksreliably only as long as the causal variables are continuous (that is, when we model the lights with acontinuous intensity).
As soon as we consider discrete states, the assumptions of our identifiabilitytheorem are violated and the model has difficulty disentangling these variables.
Discussion
What makes a variable causal?
One school of thought is that it that causal variables are those aspectsof a system that can be intervened upon [26].
Following this logic, we find it interesting to ask:can we uniquely determine the causal variables underlying a system just by observing the effect ofinterventions?
In this work we have found a partial answer to this question: we have shown in theory and practicethat under certain assumptions, causal variables and their causal structure are identifiable from lowlevel representations like the pixels of a camera feed if the system is observed before and after random,unlabeled interventions.
Our identifiability theorem extends the results by Locatello et al.
[5] fromindependent factors of variation (trivial causal graphs) to arbitrary causal graphs.
LCMs can be implemented as a variational autoencoder.
However, a straightforward, explicitparameterization of the causal structure requires simultaneously learning the variables and the causalgraph.
We found that leads to challenging optimization problems, especially when scaling to largersystems.
As a more robust alternative, we introduced implicit latent causal models (ILCMs), whichparameterize causal structure without requiring an explicit graph representation.
We also discussedtwo algorithms for extracting the learned causal mechanisms and graph after training.
In first experiments, we demonstrated that ILCMs let us reliably disentangle causal factors, identifycausal graphs, and infer interventions from unstructured pixel data.
For these experiments, weintroduced the new CausalCircuit dataset, which consists of images of a robot arm interacting withconnected switches and lights.
The setting we consider is motivated by a potentially useful scenario:
learning causal structure frompassive observations of an agent (or demonstrator) interacting with a causal system.
However, it iscurrently far from practical.
Our identifiability result relies on a number of assumptions, includingthat interventions are stochastic and perfect, that all atomic interventions may be observed, and thatthe causal variables are real-valued.
We discuss these requirements and their potential relaxationin Sec.
B in the supplementary material.
Similarly, our practical implementation has so far beenrestricted to simplified datasets with relatively few, continuous causal variables, and when trying torelax these limitations we saw the model performance decrease quickly.
While more work will berequired to make latent causal models applicable to real-world settings, we believe that our resultsdemonstrate that causal representation learning is possible without explicit labels. Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope? [Yes] (b) Did you describe the limitations of your work?
[Yes] We strive to be transparent
about the limitations of our work.
In the theory section we list the assumptions, in the
experiment section we discuss observed failures
and in the conclusions and in Sec.
B
of the supplementary material we discuss the requirements of our approach again.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See
supplementary material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] We provide
out list of assumptions in Sec. 5 and Sec.
A of the supplementary material and further
discuss them in Sec.
B of the supplementary material.
(b) Did you include complete proofs of all theoretical results?
[Yes] Our identifiability
theorem is proven in Sec.
A of the supplementary material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] Not yet, but we
aim to publish them as soon as we obtain approval to do so.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] We provide these details in Sec.
C of the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No
(d) Did you include the total amount of compute and the type of resources used (e
g., type
of GPUs, internal cluster
or cloud provider)?
[No]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes] We use theCausal3DIdent dataset from von Kügelgen et al. [Table]
(b) Did you mention the license of the assets?
[No]
[Table] (c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table] We strive to publish our CausalCircuit dataset as soon as possible. [Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[N/A] We only consider synthetic data showing simulated objects. [Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable [Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
[Table] (b) Did you describe any potential participant risks, with links to Institutional Review [Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table] (c) Did you include the estimated hourly wage paid to participants and the total amount [Table]
spent on participant compensation?
[N/A]
[Table]