Efficiently Disentangle Causal Representations
Authors: Yuanpeng Li
Joel Hestness
Mohamed Elhoseiny
Liang Zhao
Kenneth Church
Venue:  (2021)
Keywords: causality
representation learning

Abstract: In this paper, we propose a novel approach to efficiently learning disentangled representations with causal mechanisms, based on the difference of conditional probabilities in original and new distributions. We approximate the difference with model's generalization abilities so that it fits in standard machine learning framework and can be efficiently computed. In contrast to the state-of-the-art approach, which relies on learner's adaptation speed to new distribution, the proposed approach only requires evaluating the generalization ability of the model. We provide theoretical explanation for the advantage of the proposed method, and our experiments show that the proposed technique is 1.9-11.0x more sample efficient and 9.4-32.4x quicker than the previous method on various tasks.

Under review as a conference paper at ICLR 2021
EFFICIENTLY DISENTANGLE CAUSAL REPRESENTATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we propose novel approach to efficiently learning disentangled representations with causal mechanisms, based on the difference of conditional probabílities in original and new distributions. We approximate the difference with model's generalization abilities so that it fits in standard machine learning framework and can be efficiently computed. In contrast to the state-of-the-an ap- proach, which relies on learner's is adaptation speed to new distribution, the proposed approach only requires evaluating the generalization ability of the model. We provide theoretical explanation for the advantage of the proposed method, and our experiments show that the proposed technique is 1.9- 1.0x more sample efficient and 9.4-32.4x quicker than the previous method on various tasks. The source code is in supplementary material.
INTRODUCTION
Causal reasoning is a fundamental tool that has shown great impact in different disciplines (Rubin & Waterman 2006 Ramsey et al., 2010 Rotmensch et al. (017). and it has roots work by David Hume in the eighteenth century (Hume 2003) and in classical AI (Pearl 2003). Causality has been
mainly studied from a statistical perspective (Pearl 2009 Peters al.) 2016 Greenland et al. 1999
Pearl 2018) 8) with Judea Pearl's work on the causal calculus leading its statistical development.
More recently, there has been a growing interest to integrate statistical techniques into machine learning to leverage their benefits. Welling raises a particular question about how to disentangle correlation from causation machine learning settings to take advantage the sample efficiency and generalization abilities of causal reasoning (Welling 2015). Although machine learning has achieved important results on a variety tasks like computer vision and games over the past decade (e.g., Mnih et al. (2115;; Silver al. 2017);; Szegedy et al. (217); y Hudson & Manning (2018), )), current approaches can struggle to generalize when the test data distribution is much different from the training distribution (common in real applications). Further, these successful methods are typically "data hungry"", requiring an abundance of labeled examples to perform well across data distributions. In statistical settings, encoding the causal structure in models has been shown have significant efficiency advantages.
In support of the advantages of encoding causal mechanisms, Bengio (2020) recently introduced an approach to disentangling causal relationships in end-to-end machine learning by comparing the adaptation speeds of separate models that encode different causal structures. With this as the baseline, in this paper, we propose more efficient approach to learning disentangled representations with causal mechanisms, based on the difference of conditional probabilities in original and new distributions. The key idea is to approximate the difference with model's generalization abilities so that it fits standard machine learning framework and can be efficiently computed. contrast the ofte-of-the-an art baseline approach, which relies on learner's 's adaptation speed to new distribution, the proposed approach only requires evaluating the generalization ability the model.
Our method is based on the same assumption the baseline that the conditional distribution P(B A) does not change between the train and transfer distribution. This assumption can be explained with atmospheric physics example of learning P(A, B), where A (Altitude) causes B (Temperature) (Peters et al. 2016). The marginal probability of can be changed depending on, for instance, the place (Switzerland to less mountainous country like Netherlands). but P(B)A)

---

Under review as a conference paper at ICLR 2021
as the underlining causality mechanism does not change. Therefore, the causality structure be inferred from the robustness of predictive models on dustof-distributionn (Peters et al.) 2016: 2017).
The proposed method is more efficient by omitting the adaptation process, and is more robust when the marginal distribution complicated. We provide theoretical explanation and experimental verification for the advantage of the proposed method. Our experiments show the proposed technique is 1.9- 11.0x more sample efficient and 9.4-32.4x quicker than measuring adaptation speed on various tasks. We also argue that the proposed approach has less hyper parameters and it is straight-forward to implement the approach within the standard machine learning workflows.
Our contributions can be summarized as follows.
We propose an efficient approach to disentangling representations for causal mechanisms by measuring generalization.
We theoretically prove that the proposed estimators can identify causal direction and disen- tangles causal mechanisms.
We empirically show that the proposed approach is significantly quicker and more sam- ple efficient for various tasks. Sample efficiency important when data size in transfer distribution is small.
2 APPROACH
To begin, we reflect on the tasks and the disentangling approach (as baseline) described by previous work (Bengio et al. (220)). The invariance of conditional distribution for the correct causal direction P(B) A) is the key assumption in their work, and we also follow in this work. We notice that their baseline approach compares the adaptation speed of models transfer data distribution, and hence requires significant time for adaptation. We propose an approach to learn causality mechanisms by directly measuring the changes in conditional probabilities before and after intervention for both P(B) (A) and P(A B). Further, we optimize the proposed approach to use generalization loss rather than a divergence
loss can be directly measured in standard machine learning workflows and we show that it is likely to correctly predict causal direction and disentangle causal mechanisms.
2.1 CAUSALITY DIRECTION PREDICTION
As the first step towards learning disentangled representations for causal mechanisms, start with the binary classification task.
Given two discrete variables A and B, we want to determine whether A causes B, or vice-versa. We assume noiseless dynamics, and A and B do not have hidden confounders.
The training (transfer) data contains samples (a, b) from training (transfer) distribution, Pi (P2). The baseline approach defines models that factor the joint distribution P(A, B) into two causality directions PA-BB(A, (A, B)- PA-B(( PA- B (A) and PB+ = PB (A (B). It then compares their speed of adaptation to transfer distribution.
Intuitively, the factorization with correct causality direction should adapt more quickly to the transfer distribution. Suppose A + B is the ground-truth causal direction. For the correct factorization, they assume that conditional distribution PA-BB (B]A) does not change between the train and transfer distributions, so that only the marginal PA- (-)) needs adaptation. In contrast, for the factorization with incorrect causality direction, both the conditional (AI)A(A(B) and marginal distributions PB-A (BB) need adaptation. They analyze that updating a marginal distribution P(A) is likely to have lower sample complexity than the conditional distribution PB-+ (A) (A(B) (only a part of
PB- B)) because the later has more parameters. Therefore, the model with correct factorization will adapt more quickly, and causality direction can be predicted from adaptation speed.
To leverage this observation, the baseline method defines meta-learning objective. Let CA-B and LD-A A be the log-likeli likeliioddo of PA-I (A, B) and (B-І B), respectively. Their baseline approach optimizes regret, R. to acquire indicator variable R. If 0, the prediction is
2

---

Under review as a conference paper at ICLR 2021
A B, otherwise B A.
R = -oggo((())A (1--aa---------- -1-
(I)
2.2 PROPOSED MECHANISM: OBSERVE THE MODEL'S CONDITIONAL DISTRIBUTION DIVERGENCE
Rather than relying on gradients of end-to-end predictions to identify causal direction, propose to directly observe the divergence of each model's conditional distribution predictions under the intervention of the transfer dataset.
We consider the distribution of a single model's predictions on the training and transfer distributions. These distributions depend on both the marginal and conditional distributions learned by the models. It is expected that a model encoding the correct causal direction will have learned correct conditional distribution, but the marginal distribution might shift from the training transfer datasets. Hence, we would like to ignore changes in the model's predictions that are due to marginal distribution differences and focus directly on the conditional distribution.
Given access to the model's conditional distribution predictions, we could use the KL-divergence to directly measure the conditional distribution differences between the training and transfer datasets. model that encodes the correct causal structure should show no change in its conditional distribution predictions, so the divergence of these predictions should be small. On the other hand, a model that encodes the incorrect causal structure is likely to show significantly large divergence. We tie these observations together in the following proposition:
Proposition I Given two data distributions with the same directed causality between random variables, A and B, the difference of the KL-divergences of their conditional distributions is an unbiased estimator of the correct causality direction between A and B.
This approach shares the same intuition as the baseline approach. The model with correct causal direction should only witness minimal changes in the conditional distribution of its predictions between train and transfer distributions. Thus, the KL-divergence the predictions should not change for models with correct causal structure. The proof for Proposition iis included in Appendix [A]
2.3 PROPOSED SIMPLIFICATION: GENERALIZATION LOSS APPROXIMATES THE DIVERGENCE
Although we can use the KL-divergence the conditional probabilities of two data distributions as an unbiased estimator of causality direction, it requires additional computation to model conditional distribution in transfer distribution. However, in many practical end-to-end learning settings, it is more efficient and straight-forwan to acquire generalization loss, such as the cross-entropy, that can be used to approximate the conditional KL-divergence.
More specifically, we compare the generalization gaps between two causal models to approximate the causality direction. Let the generalization gap, G. be the difference of model's 's losses on the training and transfer datasets: РΛ-B Citansfer Ltrain LA-BB- Here, L A the loss on the specified set. Further, we define a directionality score as the difference of generalization gaps:
Sg = -B-+A GA-BB
We show that, for an appropriately chosen loss, such as cross-entropy. Sg is biased but reasonable estimator of the correct causal direction. When Sg > 0, it likely that + B is the correct causal direction-the generalization gap for the incorrect causal model dominates the score. We formalize this notion in Proposition 2 in Appendix B
Proposition 2 Given two data distributions with the same directed causality between two random variables, and B, the difference of their generalization gaps is a biased estimator of the correct causality direction between A and B.
In practice for the tasks we test in next section, we find that although Sg biased, it always indicates the correct causal direction. An intuitive understanding is that this approach measures how well models trained on train distribution can predict in transfer distribution -teirr generalization ability. Algorithm [ summarizes the process for identifying correct causal model. In Appendix [c describe that the generalization- based approach should converge more quickly than a gradient-based approach, especially in practical settings.
3

---

Under review as a conference paper at ICLR 2021
Algorithm 1 The proposed approach for causality direction prediction.
Train fa- on training data, and get train loss LA
A-+B
2: Train AB-A on training data, and get train loss B-+A"
3: Get transfer loss A- with fA-B transfer data.
4: Get transfer loss BB+A with fB+ on transfer data.
5: Get generalization loss GA B
A
AA-1
6: Get generalization loss GB = B-A L
B-SA'
7: Compute Sş GB-A GA-BB-
8: If Sg >> 0 return A - B, else return return B A.
The baseline approach and our proposed approach are based on the same intuition of unchanging conditional distributions for correct causality direction. However, the baseline approach must observe gradients during transfer distribution training. while the proposed approach looks directly at the changes in model outputs on the transfer data distribution. Informally, we note that these approaches are roughly equivalent by observing the following: For the correct causal direction,
G
Ciranster
Atain
VG = V V.
ptransfer
L.tain
Since Vi = O. Vg.,. = V Ctranster The previous work shows that Vo, Lt
0 for model parameters, 6., representing correct causal structures (i.e., joint distribution), Li should also be small. Similar to that, we expect g. to be small relative to the incorrect causal model.
2.4 REPRESENTATION LEARNING
So far, we assume that the causal variables are already disentangled in the raw inputs. We are interested in extending it to situations where the variables are entangled in input, such as in pixels or sounds. Here, we want to learn representations that disentangle causal variables.
Following the previous work, we suppose the true causal variables (A, B) generate input observation (X, X) with ground truth decoder, D. Then, we want to train an encoder, E, that converts input (X,Y) to a hidden representation, (U, V). Please see Figure I The decoder and encoder are rotation matrices.
X Y
A B
U v
X Y
R(OD)
R(0g)
Then we treat (U,V) as observed input, and build causal modules on them in a way similar to causality direction prediction problem (Section 2.1 If the encoder is learned correctly, expect to obtain (U, V) (A, B) with 0g- = -0p. or (U, addresses this problem with loss function in Equation but extending meta-parameters encoder
[B A) with 0g- = Op. The baseline approach
parameters in the meta-learning framework.
We apply our method learn representation without using adaptation process. Suppose the condi- tional distributions P(V(U) is modeled by fu-v, and P(UVV) by fv Also, the original losses are Lu-v for fu-vV and -V for fv- on train data. We optimize the following objective.
LU-VV + LV + minnnGu-v.,v
is a hyper parameter for interpolation. Note that we compute LU-V, Ly with train data, and Gu-v,Gv with transfer data. When mini-batch, this means that there types of
V
W
W
O
A O
R(OD)
R(0s)
U
(A, B)
Decoder D
(X, Y)
Encoder E
(U,V)
Figure 1: The complete computational graph for representation learning, from Bengio al.
2020
4

---

Under review as a conference paper at ICLR 2021
mini-batches for data from train and transfer datasets, respectively. Train data is used to learn fu-v and fv-U, and transfer data is used to lear encoder fg. Please see Algorithm for details.
Algorithm 2 The proposed approach for representation learning.
Initialize all parameters.
2: for each iteration do
3: Update predictor parameters with train samples, and loss LV-V + LV-
4: Update encoder parameters with transfer samples, and loss À Gu
5: end for
An intuitive understanding is that we want to learn a representation that works well in both causal directions in train distribution, and least one causal direction in transfer data distribution. Therefore, the models are trained well in train distribution, and the representation recovers causal variables, because works on both distributions in least one causality direction.
EXPERIMENTS
We run experiments to show that predicting causal direction and disentangling causal representa- tions can be more efficient with the proposed approach. For fair comparison, follow the same experimental setup as the baseline (Bengio et al. 2020
3.1
CAUSALITY DIRECTION PREDICTION
1.0
100
0.8
80
Accuracy (%6
0.4
40
0.2
20
Proposed
0
100
200
300
400
0
10
20
30
40
Number of episodes
Number of episodes
(a) ((y) (vertical) transfer data along with number (b) Accuracy (vertical) on transfer data with number of episodes (horizontal).
of episodes (horizontal).
Figure 2: Experiments to evaluate efficiency in causality direction prediction. The model is discrete and both and B are dimensional variables (N 10). The proposed approach is green, Baseline approach (Bengio et al. 2020) is orange.
first compare the approaches the same meta-learning setting. For the proposed approach, we replace log- ligeeiill of joint probabilities with the generalization loss. The overall loss for the proposed approach R' is similar to Equation (1) with LA-B = GA-B and CB+ = GB-A-
R' = - log + - (1-o(7))BB-A)
The first case we consider is the discrete model with both A and B being 10 dimensional variables (N 10). For other cases, please refer to Appendix E The model architecture, same in the baseline, has four modules: PA-B( *A)) PA- (BAA).
PB The baseline uses all of them, and the proposed approach only uses the conditionals. We parameterize these probabilities via Softmax of unnormalized tabular quantities, and train the modules independently with maximum likelihood estimator. We then predict causality direction on transfer data with each approach.
For each approach, we fix ground bround-truth causal direction as A + B, and run 100 experiments with different random seeds. We plot the mean value of (n)) in Figure 2a The proposed method (green)
5

---

Under review as a conference paper at ICLR 2021
is above the baseline method (orange). This comparison shows that using generalization loss is more efficient than the log- oogeliiel of joint probabilities. Especially in the first several steps, the proposed approach moves toward the correct more quickly than the baseline approach. This quicker convergence important when the transfer data is expensive. We also observe that for the baseline method, (+) is less than 0.5 for the first several steps, which might be because the factorization with incorrect causality direction has more parameters to be updated, leading to faster adaptation.
Since the proposed approach does need adaptation process, it actually does not need to update (17) with gradients. Instead, we can calculate a score, Sg G GA- to detect causality direction. This score is not directly comparable with (ry) in baseline approach, because they have different scales and semantics. To compare the approaches, we reformulate the causality direction prediction problem as a binary classification problem, and evaluate accuracy of the classification. For each approach, we run 100 experiments with the same setting as above. We then compute the accuracy as the number of experiments with successful prediction over the number of all experiments. For the baseline approach, we count an experiment successful when y 0. For the proposed approach, we count when Sg > 0.
We plot accuracy along with the number of iterations to compare sample efficiency as shown in Figure [ The experimental result demonstrates that the proposed approach (green) achieves 100% accuracy after just 3 iterations with 5.1 ms, while the baseline approach (orange) takes 33 iterations to achieve 100% accuracy with 165.1 ms. Since our method only requires estimating scale value of loss from transfer data, that it is learning / updating all parameters in model during adaptation. The experiments show that our method is 11.0 times sample efficient and 32.4 4 times quicker compared with the baseline approach. Also, the baseline accuracy for the first several steps is less than 50%. This is consistent with the previous observation that the (77) < 0.5 for the first several steps.
sample efficient than the original method that requires
3.2 REPRESENTATION LEARNING
Encoder Angle [n/2 rad]
Encoder Angle [n/2 rad]
0.4
0.4
0.2
0.2
Angle
0.0
0.2
0.2
100
600
Iterations
Time (sec)
(a) 0 values (xT/2) with number of iterations.
(b) values (xx/2) with computation time in seconds.
Figure 3: Experiments to evaluate efficiency in representation learning with exactly the same setting. The curve of the proposed approach (green) is above that of baseline approach (orange), indicating the proposed method have both better sample and time complexity.
We evaluate representation learning in the same setting as in previous work. We hope finding correct causality direction helps learning causal representations. We set the correct causal direction as A > B. With sample (A, B), a fixed decoder D = R(Op) converts it to an observation (X, Y), where a rotation matrix. We then use both approaches to learn encoders
R(0g) that map (X, Y) to representation (U, V). We set decoder parameter Op T/4, so that the expected encoder parameter should be /T or -744.
For both approaches, we use the same experiment setting and hyper parameters. In each iteration, the baseline approach uses transfer samples to adapt the model by learning gradient descent for several steps, and obtain regret to update encoder. On the other hand, directly use loss instead of analyzing with statistical test, because this can be neurally integrated with neural network training for disentangled representation leaming.

---

Under review as a conference paper at ICLR 2021
We plot the relation of I values along with the number of iterations and time to compare sample and time efficiency. The result for sample efficiency is in Figure Ba The proposed approach (green) reaches near TT / after around 220 iterations. Baseline approach (orange) reaches near π x/4 after around 410 iterations. The proposed approach is around 1.9 times sample efficient compared with the baseline approach. The result for time efficiency is in Figure 3b The proposed approach (green) reaches near / in around 34 seconds. Baseline approach (orange) reaches near π in around 319 seconds. The proposed approach 9.4 times quicker than the baseline approach.
The results show that proposed approach is both sample and time efficient for the representation learning task. Especially, the time efficiency shows the proposed method's advantage that it does not need multiple steps of adaptation in each iteration. It even does not need to compute gradient with back propagation or update model parameters.
4 DISCUSSIONS
To further understand the proposed approach, we discuss alternative metrics for distribution dif- ferences, and the working conditions of the proposed approach and the baseline. We also extend experiments on real data and neural network models.
Alternative Metrics The proposed approach may even work when using other metrics that are easily accessed in standard machine learning workflows. Many loss functions share properties of distance metrics and smoothness conditions that make them useful for comparing the generalization ability of different models. Such metrics may be biased but can also be used assess causality direction. We describe and experiment with metrics such as the KL-divergence loss and the gradient norm in Appendix [ Empirical tests show that both alternative metrics can identify causality direction in the prediction task. One may choose these alternative metrics depending on the workflow.
Robustness We compare the proposed and the baseline approach on the working conditions. One potential problem of the baseline approach is the dependency on marginal distributions. This not only requires more samples and computation. but when the marginal distribution for the correct factorization P(A) is complicated, it may produce incorrect causality predictions (suppose -> B is the correct causality direction). We show an example for such a case in Appendix G We design marginal distribution of cause variable P(A) with a complicated hidden structure, which makes it harder to adapt after intervention than the marginal distribution of effect variable P(B), and conditional distribution P). We use a mixture model for P(A) with K dimensional hidden variables C, and do not change models for other distributions. The result shows that the proposed approach is more robust than the baseline one in this case.
Real Data In addition to comparing the proposed approach with the previous work the same setting to understand fundamental mechanism for causality learning, we also evaluate whether the propose method extends naturally to broader situations. We use real data of temperature and altitude (Mooij al. 2016). We scaled the data scaled to have unit variance. We apply the proposed approach with both linear models and neural networks. We also add noise to the data and observe the changes on performance. The details can be found in Appendix H The results show that the proposed approach works for the real data, and also applies for both linear models and neural networks. It resists some level of noise (standard deviation 1), but not when the noise is too large.
RELATED WORK
Our work is closely related to causal reasoning, disentangled representation learning and its applica- tion domain adaptation and transfer learning.
Causal Reasoning There is rich literature causal reasoning based on pre-conceived formal theory since Pearl's 's seminal work on do-calculus (Pearl 1995 2009). The major approach is structure learning in Bayesian networks. Discrete search and simulated annealing reviewed in (Heckerman ct al. 1995). Minimum Description Length (MDL) principles are commonly used to score and
search (Lam & Bacchus
1993 Friedman & Goldszmidt, 1998). Bayesian Information Criterion
7

---

Under review as a conference paper at ICLR 2021
(BIC) is also used search when models have high relative posterior probability (Heckerman et al. 1995).
Prior work uses observation, but does not consider interventions, and focus likelihood learning hypothesis equivalence classes (Heckerman et al. 1995). Later, approaches have also been proposed to infer the causal direction only from observation (Peters al. 2017), based not generally robust assumptions on the underlying causal graph. The impact of interventions on probabilistic graphical models is also introduced in (Bareinboim & Pearl 2016). Another early work (Peters et al.) 2016) uses intervention to detect causal direction, but it does not address representation learning, which an important problem in modern neural network, and the main topic of this paper. Further, our method should be more efficient than the fast approximation method (Peters et al., (2016), which needs to fit a model and takes many iterations. contrast, we only run one forward pass on fixed model.
Recently, meta-learning approach adopted to draw causal inferences from purely observational data. Dasgupta al. (2119) focuses on training agent to learn causal reasoning by meta reinforcement learning: Bengio et al. (0200) focuses on predicting causal structure based on how fast a learner adapts to new distributions. In contrast, we propose an efficient approach for disentangling causal mechanisms, based on the difference of conditional probabilities in original and new distributions, assuming causal mechanisms hold in both distributions. More recently, generative models have been augmented with causal capabilities to learn to generate images and also to plan (Kocaoglu et al.. 2018 Kurutach al. 2018).
Disentangled Representation Learning This work discovers the underlying causal variables and their dependencies. This is related disentangling variables (Bengio al. (2013) and disentangled representation learning (Higgins et al. (217)). Some work point out that assumptions, including priors or biases should be necessary discover the underlying explanatory variables (Bengio et al. [2013] Locatello et al. 2018). Another work (Locatello et al. 2018) reviews and evaluates disentangling and discusses different metrics.
A strong assumption of disentangling is that the underlying explanatory variables are marginally independent from each other. Many deep generative models (Goodfellow et al. (0166) and independent component analysis models (Hyvärinen et al.) [204]] Higgins et al. [201 Hyvarinen al. (2018) are built on this assumption. However, in realistic cases, this assumption is often not likely to hold. Some recent work addresses compositionality learning (Li al. 2019 2020) without such assumption, but they do not address how to use causal mechanisms for representation learning.
Domain Adaptation and Transfer Learning Previous work finds a subset of features to best predict a variable of interest (Magliacane et al., 2018). However, the work is about feature selection, and our work about causality direction and representation learning.
Other work also examine incorrect inference and turn the problem into a domain adaptation prob- lem (Johansson et al.] (016)). Counterfactual regression is proposed to estimate each effect from observation (Shalit et al. 2017). Another approach is to find a subset which makes the target indepen- dent from the variable sel selection (Rojas-Carull Carulla et al., 2018). To do that, they assume the invariance of conditional distribution in train and target domains.
Also, competition may be introduced to recover a set of independent causal mechanisms, which leads to specialization (Parascandolo et al. 2017).
CONCLUSION
In this paper, we propose efficient simplification to recent technique to disentangle causal representations. Unlike the baseline approach which requires significant adaptation time and so- phisticated comparisons of the models' adaptation speed, the proposed approach directly measures generalization ability based on the divergence of conditional probabilities from the original to transfer data distributions. We provide theoretical explanation for the advantage of the proposed method, and our experiments demonstrate that the technique is significantly more efficient than adaptation speed based approach in causality direction prediction and representation learning tasks. We hope this work in causal representation learning will be helpful for more advanced artificial intelligence.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27):345--335, 2016.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence. 35(8):1798-182 2013.
Yoshua Bengio. Tristan Deleu, Nasim Rahaman, Nan Rosemary Ke, Sebastien Lachapelle, Olexa Bilaniuk, Anirudh Goyal, and Christopher Pal. A meta-transfer objective for learning to disentangle causal mechanisms. In International Conference on Learning Reprexentations, 2020. URL
lttts:: net
Sergei Bernstein. The theory of probabilities, 1946.
Ishita Dasgupta, Jane Wang, Silvia Chiappa, Jovana Mitrovic, Pedro Ortega, David Raposo, Edward Hughes, Peter Battaglia, Matthew Botvinick, and Zeb Kurth-Nelson. Causal reasoning from meta-reinnfrrcemnt learning. arXiv preprint arXiv:/901.08/65 2019.
Nir Friedman and Moises Goldszmidt. Learning bayesian networks with local structure. In Learning in graphical models, pp. 421-459. Spriierr 1998.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http
7/www. deep waw.oogooooo..oooo.o..ooooooooo. org
Sander Greenland, Judea Pearl, James M Robins, al. Causal diagrams for epidemiologic research. Epidemiology, 10:37-48, 1999.
David Heckerman, Dan Geiger, and David M Chickering. Learning bayesian networks: The combi- nation of knowledge and statistical data. Machine learning, 20(3):197-243, 1995.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, volume 3, 2017.
Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58:13-30, 1963.
Drew A Hudson and Christopher D Manning. Compositional attention networks for machine reasoning. arXiv preprint 1803.v:800.03:30 2018.
David Hume. A treatise of human nature. Courier Corporation, 2003.
Aapo Hyvärinen, Juha Karhunen, and Erkki Oja. Independent component analysis, volume 46. John Wiley & Sons, 2004.
Aapo Hyvarinen, Hiroaki Sasaki, and Richard E Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. arXiv preprint arXiv:1805.0865 2018.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In International conference on machine learning, pp. 3020-3029, 2016.
Murat Kocaoglu, Christopher Snyder, Alexandros G. Dimakis, and Sriram Vishwanath. CausalGAN: Learning causal implicit generative models with adversarial training. In International Confer- ence Learning Representations, 2018. URL /tpp://e////e wwtp:/neeettrrmmm.e/
BJE 4xw0w
Harold W. Kuhn and Albert W. Tucker. Nonlinear programming. In Berkeley Symposium, pp. 481-492 Berkeley: University of California Press, 1951.
Thanard Kurutach, Aviv Tamar, Ge Yang, Stuart J Russell, and Pieter Abbeel. Learning plannable representations with causal infogan. In Advances in Neural Information Processing Systems, pp. 8733-8744, 2018.

---

Under review as a conference paper at ICLR 2021
Wai Lam and Fahiem Bacchus. Using causal information and local measures to learn bayesian networks. In Proceedings of the Ninth international conference on Uncertainty in artificial intelligence, pp. 243-250. Morgan Kaufmann Publishers Inc., 1993.
Yuanpeng Li, Liang Zhao, Jianyu Wang, and Joel Hestness. Compositional generalization for primitive substitutions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4284-4293, 2019.
Yuanpeng Li, Liang Zhao, Kenneth Church, and Mohamed Elhoseiny, Compositional continual language learning. In International Conference Learning Representations, 2020. URL https
//openreview net. wott:/ffummiiim i Dn
Francesco Locatello, Stefan Bauer, Mario Lucic, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled represen- tations. arXiv preprint arXiv: 12359, 2018.
Sara Magliacane, Thijs van Ommen, Tom Claassen, Stephan Bongers, Philip Versteeg, and Joris M Mooij. Domain adaptation by using causal inference predict invariant conditional distributions. In Advances in Neural Information Processing Systems, pp. 10846-10856, 2018.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, al. Human-level control through deep reinforcement learning. Nature, 5 518(7540):529, 2015.
Joris M Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bemhard Schölkopf. Distin- guishing cause from effect using observational data: methods and benchmarks. The Journal of Machine Learning Research, 17(1): 1004.204. 2016.
Yurii Nesterov. Introductory Lectures on Convex Optimization. Springer US, 2004.
Giambattista Parascandolo, Niki Kilbertus, Mateo Rojas-Carulla, and Bernhard Schölkopf. Learning independent causal mechanisms. arXiv preprint arXiv: 2017.
Judea Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669-688, 1995.
Judea Pearl. CAUSALITY: MODELS, REASONING, AND INFERENCE. Cambridge university press, 2003.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl. Does obesity shorten life? or is it the soda? on non-manipulable causes. Journal of Causal Inference, 6(2), 2018
F. Pedregosa. G. Varoquaux, A. Gramfort, V. Michel, B. Thirion. O. Grisel. M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-288 2830, 2011.
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 78(5):947-1012, 2016.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations foundatioo and learning algorithms. MIT press, 2017.
Joseph D Ramsey, Stephen José Hanson, Catherine Hanson, Yaroslav O Halchenko, Russell A Poldrack, and Clark Glymour. Six problems for causal inference from fmri. neuroimage, 49(2): 1545-1558, 2010.
Mateo Carulla, Bernhard Schölkopf, Richard Turner, and Jonas Peters. Invariant models for causal transfer leaminn. The Journal of Machine Learning Research, 19(1):1309-1344 2018.
Maya Rotmensch, Yoni Halpern, Abdulhakim Tlimat, Steven Horn,, and David Sontag. Learning a health knowledge graph from electronic medical records. Scientific reports, 7(1):5994, 2017.
10

---

Under review as a conference paper at ICLR 2021
Donald B Rubin and Richard P Waterman. Estimating the causal effects of marketing interventions using propensity score methodology. Statistical Science, pp. 206-220222222 222, 2006.
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: general- ization bounds and algorithms. In Proceedings of the 34th International Conference on Machine Learning-Volle Volume 70, pp. 3076-3085. JMLR. org, 2017.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. Nature, 550(7676):354, 2017.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception- resnet and the impact of residual connections on learning. In Thirty-First AAAI Conference on Artificial Intelligence, 2017.
Max Welling. Are ml and statisties complementary? In IMS-ISBA Meeting on 'Data Science in the Next 50 Years, 2015.
Dong Yin, Ashwin Pananjady, Max Lam, Dimitris Papailiopoulos, Kannan Ramchandran, and Peter L. Bartlett. Gradient diversity: a key ingredient for scalable distributed learning. In International Conference Artificial Intelligence and Statistics (AISTATS), 2018.

---

Under review as a conference paper at ICLR 2021
A PROOF THAT KL-DIVERGENCE CAN BE USED TO IDENTIFY CAUSALITY DIRECTION
Here, we prove that we could use conditional -i--eieeeee to detect causality direction.
Proposition Given two data distributions with the same directed causality (e.g.. - B) between two random variables, A and B, the difference of the KL-divergences of their conditional distributions (i.e., ()) is an unbiased estimator of the correct causality direction.
Proof.
First, let P1 and P2 be the distributions such that P1 + P2. but their conditional distributions the same. Let (-I(-ll) be conditional KL-divergence. CrossEntropy A A- 1-BBPP. (P,, P2) ) is the condi-
tional cross cross-entropy, Ev(a,b) Pi(a, b) log (y(b(a) = EA log P, (B[A). Let H.+B(() be conditional entropy.
By the Maft-MeMilllan theorem, we have
DA-B ( (BBAA)
EA.B~P, log P2 A) log Pi (B[AA)]
EAA.~~AI--Oo log Pi P1 A)] BA log P2(B]A)
CrossEEnrrpp ( Pi) - -A
Without loss of generality, suppose A - B is the correct causality direction. Since Pi (BBAA) = (2)B)A), we have that CrossEntropy
HA-B((2)) and
DA-B = CrossEntropy A- P2 P1) H (
A
= 0
Finally, if we define directionality score, Spa. as the difference of KL-divergence of the two distributions, we can identify the incorrect causal direction by greater divergence. Namely,
Spa. = DB-+A - DA-B
CrrssEntropy R- (P2, P1) HB--A(P)) -
> = B
Here we use the property that entropy is less than cross entropy when the two distributions are not equal.
B PROOF THAT GENERALIZATION Loss CLOSELY APPROXIMATES DIVERGENCE
Here, we prove that we can replace conditional KL-divergence with standard generalization loss calculation to approximate the causal directionality score, Sp in Proposition | This approximation introduces bias into the score, but we expect this bias to be small relative to the difference of conditional KL-divergences of the incorrect causal model. Thus, can be used as good estimator of causality direction.
Proposition 2. Given two different data distributions, Pi and Ps, with the same directed causality between random variables, and B, consider the generalization loxs defined as the conditional cross-entropy of a given distribution:
L.. A CrossEntropy, A- (P,,P))
EABBB. log Pi (B(AA)
And define the generalization gap as G. = L L.
Finally, define the modified causality
score, Sg GB-1A GA-BB If Spa. DB-1 DA-BB defined in Proposition / then SG = Spm (H (H(A)) Sg is a biased estimator of which model has correct causality direction.
12

---

Under review as a conference paper at ICLR 2021
Proof. Let (-..) be the joint entropy. The generalization gap is
AA-B = L A -
CrossEntropy P2, P1) HA-B(PP)
(CrossEntroyya-nBB - + Ha-nn(P)) - HA-
DA-B + -H-nn(Pa) -HA-n((P)
DA-B + (B) -((((())))))
=D (H(P2AA H(P((A)) - - -
-DA-B [H(P((A,
H(PI(A, (H(P(AA) - ((P) = -DABB + AH (A(A)
9B-A = DB-A + AH (H(
Here (H B). (A) and (H(B) are the change in entropy from P1 to P2 for (A, B), A and B, respectively. Further, the modified causality score is the difference of these generalization gaps:
Sg = GB+AA - GA-B
[DB-A (H B) (H [DA-B AH(A,B) AH(A))
[DB+A DA-B] [BH(B) (H(A)
=SDu- (BH(B)- (A)
Without loss of generality, suppose + B is the correct causal direction, then DA
Sg = DB+A (B) (A)]
When Sq > 0 (and [AH(B) - AH A)) is small), the prediction of causality direction is correct.
Informally, we expect the bias term, [BH(B) (H))), to be small relative to the divergence, DR-A, real application settings. If + B, then A and B will have some (positive or negative) correlation-ther will vary in correlated way under intervention on A. Unless the conditional distribution, (B]AA (2 A), contains large portions with high entropy, it is likely that a change in A will cause commensurate change in B. Thus, it is likely that (H z (AH(B)), so it is unlikely that (A) and (B) are both large and have opposite sign.
Further, in cases where (A) is large, but (B is small, it is frequently indicator that the selected P. are "lucky" (uncommon) or the causality between and B is weak. Consider the example in which (Pi (A)) is small, but (P)( is large. This case occur if H (B))) is small, but is unlikely, because the (low-entropy) conditional distribution must impose the disorder on Pi (B). is easy see that the reverse situation (H (B) A)) is large) implies that the causality between A and B is weak, such that causality direction prediction may be difficult anyway.
Finally, if want to relax the assumption that either A + or + A (e.g., a null-hypothesis that A and B are not causally related), we could introduce a significance test to decide whether we can accept the selection of causal direction from either method. Such a significance test would need be mindful of the magnitude of any biases introduced in the approach.
C GENERALIZATION LOss IS LIKELY TO BE MORE PRACTICAL
Here, we explain how generalization loss is likely to be more practical than gradient-based approaches for identifying model with correct causal direction. Both the baseline and proposed approaches rely on convergence of estimators toward a ground truth model, but gradient-based approaches require much stronger assumptions to guarantee convergence, and they are more susceptible to practical challenges like tuning optimizer hyperparameters.
First, both gradient- and generalization- based approaches have the same asymptotic convergence characteristics, with the prediction eтror proportional to _ where is the number of steps or samples. Many theorems state that gradient- based approaches can have asymptotic convergence to within an error based on number of gradient descent steps, k: E = O ) These proofs rely on many assumptions that cannot be guaranteed in real-world datasets, such as convexity, regularity,
13

---

Under review as a conference paper at ICLR 2021
smoothness of error surface, using optimal settings of optimization parameters for best-case step sizes, etc. (Kuhn Tucker Nesterov 2004: Yin et al. 2018). These bounds can also tend to be loose for real data distributions.
On the other hand, non-pann cor--mmmetric estimators, such as the sample average generalization loss proposed in this work, do not have model, so their convergence rates depend only the structure of the dataset (e.g., first or second moments). For instance, early estimator convergence results show that after summarizing, 71 samples. the error in an estimator can be within of the true value: é O (Bernstein, 1946 Hoeffding, 1963). The constants in these bounds are just the moments that characterize the dataset (e.g., mean, varannce). Like the gradient-based convergence rates, these non con-parametric convergence rates can also be loose for real data distributions.
Thus, in practical settings, it is likely that generalization-bain based approach (non-parametric estimators) will converge more quickly on causality identification in real-world datasets. The proposed approach does not require further model optimization fine-tuning), it is not susceptible to poorly chosen hyperparameters. Further, it is unlikely to encounter complex error surface topology. Thus, in practice, it is likely converge more quickly than gradient-based causality identification techniques.
D REPRESENTATION LEARNING
We run representation learning experiments for both approaches on a GeForce GTX TITAN Z with PyTorch. We use the original implementation for the baseline approach, and extend it for the proposed approach. The hyperparameters are the same as the original setting (Bengio et al. 2020).
E MORE EXPERIMENTS ON CAUSALITY DIRECTION PREDICTIONS
100
100
80
80
Accuracy (%)
Accuracy(%6
60
60
0
100
Number episodes
Number episodes
(a) (ry) (vertical) on transfer data along with number (b) Accuracy (vertical) on transfer data with computa- of episodes (horizontal).
tion time in seconds (horizontal).
Figure 4: Experiments evaluate efficiency in causality direction prediction. The model discrete and both A and B are 100 dimensional variables (N = 100).
We would like to extend causality direction prediction experiments to high dimensional variable and continuous variables. We still follow the experiment settings in previous work (Bengio et al., 2020). and compare the proposed and the baseline approaches. We run all the experiments of causality direction prediction with Intel Core i9 9920X 12-Core Processors.
For high dimensional variable, experiments with N = 10, but only changed the N. The result for sample efficiency is in Figure 4al Similar to the case of N 10, the proposed approach (green) is significantly more efficient than the baseline approach (orange) in both sample.
experiments with N 100. We use the same setting the
For continuous variable, used the same setting the previous work (Bengio al. 2020). For the proposed method, we only use one sample for each episode. The plot shows that the proposed approach (green) is more efficient than the baseline approach (orange) in sample number (Figure 4b).
14

---

Under review as a conference paper at ICLR 2021
F ALTERNATIVE METRICS
The proposed approach may even work when using other metrics that are easily accessed standard machine learning workflows. Many loss functions share properties of distance metrics and smoothness conditions that make them useful for comparing the generalization ability of different models. Such metrics may be biased but can also be used to assess causality direction. We describe and experiment with metrics such as the KL-divergence loss and the gradient norm. Empirical tests show that both alternative metrics can identify causality direction in the prediction task. One may choose these alternative metrics depending on one's 's workflow.
For KL-divergence, define the metric as the follows, and show an example result in Figure Sa
Spa. = P|
For gradient L2 norm, we define the metric as the follows, and we show example result in Figure 5b
S1 L2
A-B
L2
06 06
00 B-
Note that the model optimization algorithm does not have special requirement gradient, such as gradient clipping.
These results indicate that other loss functions can behave similarly to the conditional cross-entropy when identifying causality direction. The initial samples already show a large gap between the correct and incorrect causal models, and after just few tens of examples, the difference close convergence.
We expect that the same intuition applies for these other loss metrics. In particular, these loss metries both satisfy distance metric properties of non-negativity and the triangle inequality. Further, they have useful continuity and smoothness characteristics that suggests any biases they may have should behave like conditional cross- cross-entropy. Our empirical results confirm these intuitions.
0.150
1.2
1.0
0.125
0.100
0.075
0.050
0.025
0.0
0.000
40
Number examples (x100)
Number examples (x100)
(a) Difference of KL-divergence Spa. along with the (b) Difference of L2 gradient norm St, along with the number of transfer samples in the proposed approach. number of transfer samples in the proposed approach.
Figure 5: Experiments for other metrics The model is discrete and N 10. Curve (blue) is median 100 runs, with 25-75% quantiles intervals, and it is significantly above zero (red), indicating these metrics are good indicators for causality learning.
G ROBUSTNESS
To compare the robustness of the proposed approach with the baseline, we design P(A) with complicated hidden structure, so that it is harder adapt after intervention than P(B) and P(A For P(A). we use mixture model with K dimensional hidden variables C, and keep other distributions unchanged.
P) Σ
P(A(CE)))P(Cs::))
1=
15

---

Under review as a conference paper at ICLR 2021
where 8 RK RKXN We set N M 10, 200. For the baseline approach, we plot the improvement of log likelihood during adaptation along with the number of samples (x 100) in Figure ba The model with correct causality direction (blue) adapts slower than that of incorrect direction (red), which means the fundamental intuition of baseline approach does not apply in this situation. For the proposed approach, we plot the difference of generalization losses Sg = -B-A GA-B along with the number of samples (x 100) in Figure 6bb The difference (blue) is significantly larger than zero (red). This is natural because the proposed approach does not use P(A), so that it is not influenced by whether P(A) has complicated hidden structure. This experiment shows that the proposed approach is more robust than the baseline one in this case.
1.2
-.4
4.5
0.6
4.8
0.4
-5.0
A-0
0.2
5.2
9
0.0
0
20 40 60 80
20
40 60 80
Number examples (x100)
Number examples (x100)
(a) Incorrect result from baseline approach. Log likeli- (b) Result of the proposed approach. The difference hood during adaptation along with the number of sam- of generalization losses Sg = -B- GA- along ples (x 100). The correct causal model (blue) adapts with the number of samples (x10)). The difference slowly than that of the incorrect model (red). (blue) is significantly larger than zero (red).
Figure 6: Experiments to evaluate robustness. A + B is the correct causality direction. Here, N 10, .M 10, K 200. Curves are median over 100 runs, with 25-75% quantiles intervals. The result shows that the models with correct causalities are slow to adapt in baseline approach (a), which means the baseline approach does not work, but the proposed approach (b) works.
H REAL DATA EXPERIMENTS
100
Accuracy (%)
75
50
25
0
0.0
0.5
10.050.0 Noise stdev
1.0 5.0
Figure 7: Accuracy when adding noise.
We altitude and temperature dataset (Mooij al. 2016), where altitude (A) causes temperature (B). We generate train and transfer datasets the following way. We first divide the data to two sets Dj and D2, according to their A values. samples in D2 has larger equal A values than those in Dz. We respectively shuffle and separate D, and D2 to two subsets with ratio of 9:1, generating DJ. DR. D D3. We than merge sets generate train data (D) U D2) and transfer data (D} U D.).
We use both linear regression model with Lg loss and neural network model with two layers. The hidden layer has 100 nodes with ReLU activation. We use scikit-learn (Pedregosa al. 2011 for
16

---

Under review as a conference paper at ICLR 2021
implementation. We experiments 1,000 times with different random seeds, and the predicted causal directions are always correct.
For different noise levels, temperature experiment with the linear model. the success rate start to drop to 99.8 + 0.1% at 0.5, and becomes 96.0 + 0.4% at 1.0, 63.9 + 1.8% at 5.0, 55.3 = 1.2% at 10.0, and 50.1 + 1.3% at 50.0 (Figure 7 This means the method is robust to some noise (around or less), but does not work when the noise is too large.
Gaussian noise with different standard deviations. In the altitude-
17


Reviews:
[{"review_id": "tqRjEngdEI", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an efficient approach to learn disentanglement representation causal mechanism, based on a generalization loss between a training data set and transfer data sets. Empirical studies show it achieves better sample and computation efficiency than a previous work (Bengio et al., 2020). \n\nPros: \n- Improvement over the existing baseline is significant\n- Theoretical statements on the results, particularly on biased and unbiased estimators, offer good understanding for the approach. \n\nCons:\n-  Currently, I find the setting and results on real datasets rather weak. It would be great if authors could demonstrate such a causal direction identification problem in some real setting, with the distribution shift occurrence between different sets of data. \n- There are no comparison with other methods. It would be great if authors could show baseline methods used in (Mooij et al., 2016) perform, by considering the train alone and train+transfer data together. In (Mooij et al., 2016) datasets, representation learning are not needed; for other datasets when representation learning is needed, existing criterions of these baseline methods could replace the generalization loss. With these results, it would be better to judge the effectiveness of generalization loss. \n\n\nOther Thoughts:\n- Besides the marginal distribution shift, P(A|B) and P(B|A) may both change significantly, even one of them is the correct causal direction, as the underlying distribution may shift.  Can authors comment how their approach could handle such situations?\n- In representation learning, is the decoder never used?\n- since Section 2.4 is only a small part of paper, it may also be worth to also test the causal direction identification without the representation learning part, for example, in linear cases. \n- The statement \"this work in causal representation learning will be helpful for more advanced artificial intelligence\" is rather vague and pompous. \n\nRating: \nTo me the rating is borderline and I'm not yet convinced it is above the acceptance threshold, hence I left it as 5. Hopefully authors' rebuttal could address my concerns. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "jRXOnX_mi2L", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents a novel approach for determining the causal direction between two random variables $A$ and $B$. The approach is based on the assumption that the conditional distribution $P_{A \\rightarrow B}(B \\mid B)$ does not change between the train and transfer distribution. As a result, a model that predicts the correct causal direction $A \\rightarrow B$ should generalize better from the train to the transfer distribution compared to a model predicting the wrong causal direction $B \\rightarrow A$. While previous work has proposed to use this insight to determine the causal direction by comparing the adaptation speed, this paper proposes to directly measure and compare the generalization performance. The results indicate that the proposed approach leads to the same performance in terms of causal relation prediction, but that it is more sample efficient and faster.\n\n\n\nStrengths:\n* The paper presents an interesting approach for determining the causal direction between two random variables. While the generally accepted wisdom is that a better causal model will lead to better generalization, their approach follows the reverse logic: if a causal model achieves better generalization, it is interpreted as the better (i.e. correct) causal model. The paper builds on this logic proposed by Bengio et al. (2020) and proposes a more efficient measure for the generalization performance of a model.\n* The results show a clear advantage of the proposed approach in terms of sample efficiency and speed.\n* The presented proofs seem correct and are presented in a clear way.\n\n\n\nWeaknesses:\n* The paper builds heavily on the previous approach presented by Bengio et al. (2020). While this is not a problem per se, the paper should be written in such a way that it can stand alone. In the current version, it is not possible to follow the experimental results without consulting the paper by Bengio et al. (2020). It would be very helpful to add a description of the data and models used, as well as the measures compared, e.g. what is $\\sigma(\\gamma)$?\n    * Consulting the paper by Bengio et al. (2020), it seems that the results in Figure 2 can only be interpreted as correct if the model also predicts the correct causal direction. I assume this is the case for the presented models, but it is not stated anywhere.\n* As far as I understand (Algorithm 2, line 3), one model is trained to optimize the loss for both causal directions simultaneously. That approach seems rather counterintuitive to me. I would imagine that this could lead to the model learning a conditional distribution that does not match either causal direction well, effectively rendering the proposed approach unworkable. Wouldn’t it be better to train one model for each causal direction separately and to compare the generalization performance between these models?\n* It would be interesting to see how the proposed approach compares to other methods for causal discovery.\n* The paper could be improved in terms of clarity:\n    * There are a few grammatical errors, especially the articles “the” and “a” are missing quite often.\n    * The structure could be reformatted to be more efficient. At the moment, the introduction spends a lot of time on a very general introduction of causality. I would recommend reducing this and instead, focus more on the proposed approach. Based on the current introduction, I could understand how the previous approach worked, but not the proposed one.\n    * The title should be more concrete. \n\n\n\nOther comments:\n* What happens if $S_G = 0$?\n* There is a mismatch in Figure 1.b): While the caption states that the x-axis shows the “computation time in seconds”, the x-axis in the plot is labeled as “Number of episodes”.\n* The paper could benefit from a clearer statement of all the assumptions that are necessary for the presented approach. For example, I would expect the approach to rely on noiseless dynamics and a fully observed setting without hidden confounders.\n* Regarding noiseless dynamics, the paper presents an experiment in the appendix showing how performance degrades when increasing the standard deviation of additive noise. For this experiment, it would be interesting to know the standard deviation of the underlying data, otherwise, it is unclear how the noise relates to that.", "extra_scores": {}, "extra_text": {}}, {"review_id": "bL9_GR3Uzj", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes an approach for learning a representation U(X,Y), V(X,Y) of data (X,Y) such that U and V are causally meaningful, and U causes V. The approach relies on observing data from two domains, P and Q, where P(V| U) = Q(V |U) (reflecting the causal structure). The approach is a modification of Bengio et al. The main new idea is that the objective function can be tweaked by replacing a KL divergence term with a term involving domain-shift induced generalization errors.\n\nThe paper covers an interesting subject, and the idea to directly use domain generalization error learn causal structure is exciting. However, the paper is clearly not yet baked. The writing is generally poor, and the key ideas have not been formalized. The main ideas of the paper are unclear, as are the validity of the core insights. In particular, there is a fundamental confusion between estimators and estimands.  The paper will require extensive revision and formalization before it's ready for public consumption.\n\nBelow I including some free form thoughts I had below. These give a flavor of my issues with the paper, and will hopefully provide some direction for the authors. However, I stress that these issues are only examples. The paper requires extensive revision.\n\n1. the phonemes/acoustics example just shows that structured learning may be beneficial, it doesn't rely on causality\n2. the explanation of equation 1 is unclear (or, possibly, wrong). Presumably, the actual aim is to compare the likelihoods of two distinct models corresponding to A->B and B<-A. The paper argues that the higher complexity model will have lower likelihood. But this is false in general; a very flexible model will simply memorize the training data.\n3. in general, the paper suffers from a confusion in the notation between population parameters and finite-sample estimators. The notation generally suggests the former, but the prose, appealing to sample-complexity handwaving, suggests the later. \n4. Is proposition 1 meant to be a theorem? The text doesn't reference any estimator, much less an 'unbiased' one.\n(update: I read the appendix, and the intended statement is simply, \"If P(A|B)=Q(A|B), but P(B|A)!=Q(B|A) then 0 = KL(P(A|B),Q(A|B)) < KL(P(B|A),Q(B|A))\"\n5. In section 2.3, \\script{L} has changed from denoting log-likelihood to denoting risk (incorrectly called loss in the prose)\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This submission builds on recent work by Bengio et al. (2020) to learn disentangled representations of causal mechanisms. The main innovation in this work is that the authors propose to use compare the generalization gap between a model A->B and its causal inverse B->A to determine the true causal mechanism. 

Reviewers  are in consensus that this paper in its current form is not ready for publication. Aside from issues with writing, and comparatively weak experimental results, the reviewers noted significant technical problems in the writing and proofs. As noted by R3, Proposition 1 mentions "unbiased estimator of correct causality direction", which is a nonsensical combination of words in absence of a definition of the causality direction as a random variable. Moreover the proof in the appendix only shows that that 0 = KL(P(A|B),Q(A|B)) < KL(P(B|A),Q(B|A)) when P(A|B) = Q(A|B) and P(B) != Q(B). As R3 also notes, the authors also do not clearly distinguish between the true data distributions (which are not known), datasets that are sampled from these distributions, and approximations to these distributions that are learned from finite sample sets.  

The metareviewer would suggest that the authors either improve the level of precision of the formalization of their approach, and/or more explicitly position it as an empirical study, in which case more substantive experimental evaluation would strengthen the contributions.
Decision: Reject