Variational Causal Autoencoder for Interventional and Counterfactual Queries
Authors: Pablo Sanchez Martin
Miriam Rateike
Isabel Valera
Venue: NeurIPS 2021 Submitted (2021)
Keywords: Causality
Inference
Counterfactual
Intervention
Variational Autoencoder
Graph Neural Networks

Abstract: We propose the Variational Causal Autoencoder (VCAUSE), a novel class of variational graph autoencoders for causal inference in the absence of hidden confounders, when only observational data and the causal graph are available. Without making any structural assumptions, VCAUSE mimics the necessary properties of a Structural Causal Model (SCM) to provide a framework  for performing interventions (do-operator) and  abduction-action-prediction steps. As a result, and as shown by our empirical results, VCAUSE provides a practical and accurate pipeline for estimating the interventional and counterfactual distributions of diverse SCMs. Finally, we apply VCAUSE to evaluate counterfactual fairness in classification problems and also to learn accurate and fair classifiers.

Variational Causal Autoencoder for Interventional and Counterfactual Queries
Anonymous Author(s)
Affiliation Address email
Abstract
We propose the Variational Causal Autoencoder (VCAUSE), novel class of variational graph autoencoders for causal inference the absence of hidden con- founders, when only observational data and the causal graph are available. Without making any structural assumption, VCAUSE mimics the necessary properties a Structural Causal Model (SCM) to provide framework for performing inter- ventions (do-operator) and abduction-ace aactio-ctoo---- -et--aatttooiiittiiiii prediction steps. As a result, and as shown by our empirical results, VCAUSE provides practical and accurate pipeline for estimating the interventional and counterfactual distributions diverse SCMs. Finally, we apply VCAUSE to evaluate counterfactual faimess in classification problems and also to learn accurate and fair classifiers.
2
6
10
Introduction
"1
12
Predicting causal effects of actions (interventions) is a central problem scientific research in a broad variety of fields [ 5 2 (1], and machine learning is exception 44 As example,
13
14
fundamental machine learning questions-such as faimess increasingly being formulated as causal queries.
and interpretability
17
16
Research on causal reasoning has predominantly focused on causal discovery, a.k.a. structure learning, aimed at discovering the underlying causal graph from data (see, e.g., [15 30 40 60))
18 An alternative line work instead aims to answer causal queries under different assumptions, e.g., assuming access to partial causal knowledge 17 M
to randomized trial [16]. Here, focus on the latter line of research, that is, on answering the following two types of causal questions: interventional queries, e.g., "What is the effect of universal unconditional basic income of Ik EUR on the health of the population?": and counterfactual queries, e.g., "Had Kim received an unconditional basic income of Ik EUR, what would have been the effect on Kim's health?".
20
22
23
24
Unfortunately, predicting causal effects from observational data alone is in general difficult and often requires strong and impractical causal assumptions. In this context, the Structural Causal Model (SCM) B3) is a framework that allows to answer causal queries from observational data, but requires complete causal knowledge. That is, knowledge not only the parent-children (cause-effect) relationship between every pair of observed variables (i.e., on the causal graph), but also on how these relationships are (i.e., the structural equations). As consequence, randomized controlled
26
28
30 studies are today still considered to be the gold standard for estimating causal effects. Unfortunately, often expensive conduct, unethical, directly impossible.
31 real world experiments
32 In this work, we aim at answering the above causal queries, when only observational data and
33 the causal graph are available. Note that the causal graph can often be inferred from domain
34 knowledge [62] or via one the numerous approaches for causal discovery (27 [4]]. We assume causal sufficiency, i.e., that there are no hidden confounders, which are unobserved variables that
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

36
affect more than one observed variable. We propose the novel Variational Causal Autoencoder (VCAUSE). a variational graph autoencoder that leverages the causal graph structure and yields accurate estimates of the observational, interventional and counterfactual distributions induced by an unkonwn causal model.
37
38
39
40
Importantly, we provide the necessary conditions for the design of the encoder and decoder graph neural networks (GNNs), so that the resulting VCAUSE behaves like an SCM. As result, and without making any assumptions on the true structural equations, VCAUSE provides a practical framework perform interventions (do-operator) and abduction cattiiionnattinn--ee preeiictinni steps, which are necessary to evaluate interventional and counterfactual queries.
41
42
43
44
45
We evaluate the performance of the proposed VCAUSE model in extensive experiments using observational data from different SCMs, with diverse causal graphs and structural equations. Our experiments show that VCAUSE outperforms competing methods 17 18 at estimating not only the mean of the intervenionallcounefffaaaaa distribution, but also the overall distribution, as shown
46
48
19 by the quality of its samples (in terms of Maximum Mean Discrepancy, MMD). We finally show a use-case in which VCAUSE is used to assess counterfactual fairess of different classifiers trained on
50
51 the German Credit dataset [TO as well as learn accurate and counterfactually fair classifiers.
52
Related work. There are numerous works on causal discovery 15
18
27
30
60 [33. In addition, extensive work focuses interventional and/or counterfactual queries 54 using non non--nn-aanne parammerreti methods 32 46 17 and more recently, tractable probabilistic models (59]
58
55
Moreover, deep generative models enjoying increasing attention for causal queries in complex Existing approaches focus on estimating the Average Treatment Effect (ATE) by
56
data 31 K5
assuming fixed causal graph that includes the treatment variable IT
45
discovering
58 and intervening on the causal latent structure of the (e.g.. image) data 19
35
48
[56; or
59
on addressing interventional and/or counterfactual queries by fitting conditional model for each observed variable given its causal parents [I] 17 36 38 In the most recent work related
60
to ours [ the authors propose CAREFL, an autoregressive normalizing flow (ANF) for causal discovery and queries, which focuses on bi-variable scenarios with affine relationships between 63 observed and unobserved variables. In our experiments, we compare VCAUSE with CAREFL (as well as (177) in more general settings. Finally, up to the best of our knowledge, GNNs have previously been used for causal discovery (58 61, but have not yet been exploited to address counterfactual and interventional queries, like VCAUSE does.
61
62
64
65
66
Background
67
68
In this section, we first provide a brief overview on structural causal models (SCMs) and then introduce the main building block of VCAUSE, i.e., variational graph autoencoders (VGAEs).
69
70
2.1
Structural causal models
An SCM M = () determines how set of d endogenous (observed) random variables X == {X : is generated from a set of exogenous (unobserved) random variables := (U,..... Ua}
71
72
(with prior distribution p(U)) via the set of structural equations = (X, := f. (Xpa(i), U,) 1-1 Here Xpa(i) refers to the set of variables directly causing X,, i... parents of i. Every SCM M is associated with a directed acyclic graph (DAG): a causal graph G == (X, E). for which the nodes (vertices) correspond to endogenous variables X and the directed edges E account for the causal parent-child relationship between variables 39 Given an SCM, there are two types of causal queries
73
75
76
77
78 of general interest: interventional queries, c.g., "What would happen to the population X, variable X, would be set to fixed value a?"; and counterfactual queries, e.g., "What would have happened to
80 a specific factual sample x", had X, been set to a value a?".
81 More in detail, interventional queries aim to evaluate changes in the causal world, or equivalently,
82 manipulations of a subset of the endogenous variables I S [d] = {l....,d} at the population
83 level. Interventions on an SCM M are often represented with the do-operator do(X, = a;) and
84 lead to new distribution over the set of endogenous variables p(X do(X, = ax)), which is referred to as the interventional distribution. In G an intervention removes incoming edges to node
86 i and sets X; = a (see Figure lcp. counterfuctual query for a given factual instance x aims to
estimate what would have happened had X1 instead taken value a. This effect is captured by the
2

---

88 counterfactual distribution p x² doo(o1 a)), which can be computed using the abduction- 89 action- prediccion approach by Pearl [9 Refer to Section for further details on the computation of the interventional and counterfactual distributions.
~ P(U)
h =1
X1
Ui
X1
Ui
U/2
f2 X₂ = =2X++
a X3- 3X1 4X2 Un
X₂
X3
Us
X3
(a) SCM M = [}(
(b) 9 without intervention
(c) 9 with intervention
Figure 1: Example of (a) a triangle SCM M with d X = 3 endogenous variables; (b) correspond- ing causal graph G and (c) illustration of an intervention do(XX2 a) on the causal graph. Green arrows highlight the direct causal path from X1 to X3, and red arrows the indirect causal path via X2.
91
2.2
Variational Graph Autoencoder and Graph Neural Networks
Variational Autoencoders (VAEs) 20 are powerful latent variable models based on neural networks (NNs) for jointly i) learning complex and expressive density estimators p(X)
93
po Z) (Z)dZ where the likelihood function (a.k.a. decoder) is parameterized using NN with parameters 0; and ii) performing approximate posterior inference over the latent variables Z using a variational distribution ka. encoder) 4e (Z X) parameterized using NN with parameters 6. The parameters 0 and are usually learned by maximizing lower bound on the log-evidence
94
95
97
52).
98
Variational Graph Autoencoders (VGAEs) 21 extend VAEs to account for graph-structur in- formation on the data (58). VGAEs define a (potentially multidimensional) latent variable Z; per observed variable X;, i.e., Z == [Zi..... Zz}. Additionally, VGAEs rely on an adjacency ma- trix A, which is used by two Graph Neural Networks (GNNs), one for the encoder and one for the decoder, to enforce structure
99
100
101
the posterior approximation (Z X, A) and the likelihood Po X Z. A). More in detail, A E {0, 1] encodes the graph structure among the observed that Auj = 1 there a directed edge from X, to X,, and A,₃ otherwise. Hence, A determines which variables X; influence Zj (i,j E [d]), and vice versa.
103
104
variables X == (X1,...X,,
106
Graph Neural Networks (GNNs) have generated a lot of attention during the last years, they achieved sigmificant improvements in graph representation learning 2 12
107
43 57 While the taxonomy of GNNs is immense (53]. in this work we focus on message passing GNNs which allow
44
108
109
us to work with directed graphs. In its most general form, a message- passsing GNN calculates the output h for node in layer l in three steps: compute the set of incoming messages arriving to node from its neighbors Nr- {X, Ag 1} using a message function fm (a NN with parameters O%,,), that is {ml DEN, = - (1/-1 , ( j E W.): ii) combine the set of messages into a single ((mm))))
110
111
112
113
message M/- == (fm b)) using an aggregation function f (e.g. adding up the messages); and iii)
update the node state h == f"(h (hi--1 MM: (,,), using an update function f" (a NN with parameters g As a result, the output hi can be written as
114
115
h = f (n!-1, f (
116
Note that the above expression assures that the output for each node is computed using information from its neighbors N₂ according to A. Moreover, if the GNN has N₁ hidden layers, then the output for node i not only depends on its direct neighbors N,, but also on its neighbors up to order
117
118
No. + (hops). As example, if N₁. = 0 then the output for each node only depend on its direct neighbors (parents). If instead N₂, = 1, then the output for each node depends on 2-hop neighbors (grand-parents). For detailed description of GNNs, please refer to Appendix A
120
3 Observational, interventional and counterfactual distributions
123 In this section, we introduce the observational, interventional and counterfactual distributions (trig- 124 gered by any intervention txxx, = a+)) that are induced from an SCM M := {p(U), F). Specifi- cally, we summarize the main properties of an SCM that will allow us to propose novel class of
3

---

128
VGAEs, the variational causal autoencoders (VCAUSE), to compute accurate estimates of these distributions using observational data and a known causal graph. To this end, we assume the absence of hidden confounders, i.e., we assume that p(U) = P(.))
127
128
129
Observational distribution. The SCM M determines the observational distribution p(X) over the set of endogenous variables X {X X2), which satisfies causal factorization 44 i.e. p(X) =
130
I 1 p(X, X p) That is, after marginalizing out the exogenous variables U, the distribution of each endogenous variable X; depends only on its parents, i.e., X xut(t). The observational distribution
131
132
133
can alternatively be written only in terms of the exogenous variables U as
p (X) - (UU)U
134
where F U + X corresponds to the set of structural equations, equivalent to F, that directly transform the exogenous variables U into the endogenous variables X. Let us denote by an(i) the set of indexes of the ancestors of i, and an" (i) == an(i) U (2). Then, the causal factorization induced by the SCM M leads to the following property of F(U):
135
136
137
Property 1. Each endogenous variable X; can be expressed as a function of its exogenous variable U, and the ones of all its causal ancestors, i.e. F(U) == {X; = Su(UU, i E (n)))) This, together with the causal sufficiency assumption, implies that X, is statistically independent of Un Vj & an" (i).
139
141
142
Interventional distribution. As stated in Section 2.1 interventions a set of variables I can be performed using the do-operator, which can be seen as a mapping do(X, = a) : M + M
143
(p( U) )T) where F = (fz j & I] U {as i I). As above, we can represent the resulting set I> in terms of only the exogenous variables U, so that can write the interventional distribution as:
144
145
of intervened structural equations F = {f, 1j u | E
p(X do dX aa)= / F (U)))UUU)
Assuming an intervention do(Xz = a) on M, then the resulting structural equations F (T( satisfy:
147
Property 2. After intervention do(X] α)) M, all the causal paths from Uj Vj E an (i) to X, that include an intervened variable in XI (i.e., the causal paths where XT, is a mediator) are severed in F while the rest of causal paths remain untouched.
149
151
The above property illustrated in Figure D where we can easily observe that after an intervention do a), the indirect causal path (in red) from X1, and thus from U1, to X3 via X2 is severed, while the direct path (in green) remains.
153
154
Counterfactual distribution. Assuming the SCM M = (p( F} be known, the following three steps defined by Pearl 33 allow us to compute counterfactuals x as: i) Abduction: infer the values of the exogenous variables U for factual sample x"", i.c., compute p(U xx) ii) Action: intervene with do(X, = a): M H M = (p(U), FT); and iii) Prediction: use the posterior
155
156
157
distribution P x and the new structural equations F to compute p(xCT |x"). The prediction step can be alternatively computed using the new set of structural equations F defined in terms of the exogenous variables U, so that we can write the counterfactual distribution as:
158
160
CF xx do - a)) - f FZ (UU)(U
x x))dU.
(4)
Importantly, the resulting posterior distribution p(U xx)) satisfies:
181
162 Property 3. In the abduction step, statistical independence implies that conditioned on the endoge- nous variables of the factual sample xF, each exogenous variahle Us is independent of the factual
163
value x... i + and the variable X, is not a parent of X i.e. j&paa (i) == pa(i) U {i}.
4 Variational Causal Autoencoder (VCAUSE)
186
In this section, we present a novel variational causal graph autoencoder (VCAUSE) to approximate the observational, interventional and counterfactual distributions given in 2 B and respectively.

---

168
While the underlying SCM M is unknown, we assume access to: the causal graph Ç and observational ie., ii.d. samples of the observational distribution induced by M..
data {x, /
189
Definition 4.1. (VCAUSE) Given causal graph 9 over set of endogenous variables X which establishes the set of parents pa" (i) for each variable X; (including the i-th node). A variational causal graph autoencoder (VCAUSE) is defined by:
171
172
173
A causal adjacency matrix A, which is a d x d binary matrix with elements As =
174
j E pa" paa(1), i.e., when i = j or j is a parent of i. Otherwise, A:j 0.
175
II: pl Z.) over the set of latent variables Z = (Z1,.... Za). A decoder (x Z, A). which is GNN (parameterized by 0) that takes as input the set of latent variables Z and the causal adjacency matrix A, and outputs the parameters of the likelihood pe [X| Z, A).
A prior distribution p(Z)
177
179
An encoder ge (Z [ X, which is a GNN (parameterized by 6) that takes as input the endogenous variables X and the causal adjacency matrix A, and outputs the parameters of the posterior approximation (o A).
181
Given observational data {xn J==1, one may learn the parameters 0 and é that best estimate the density p(X). We here rely on the partially importance weighted auto-encoder (PIWAE) 44
182
183
184
Next, we discuss how to design VCAUSE such that it is able to capture the observational, inter- ventional, and counterfactual distribution induced by an unknown SCM. Importantly, we derive the necessary conditions on the design of both the encoder and decoder GNNs such that VCAUSE fulfills the SCM properties introduced in Section
185
186
187
4.1
Observational distribution
VCAUSE approximates the observational distribution in using the generative model as 189
d
p(X)= = / po Z, A)) (Z)
Pa (X, Z, P(Z)dZZ
(5)
190
Figure depiccss this generative process. If we compare ( with the true observational distribution
191
in 2b. we observe that the latent variables Z play a similar role the exogenous variables U,
192
and the decoder pe [X] A) plays similar role to the structural equations F. Yet, we remark that Z does not need to correspond to the exogenous variables, i.e. p( + p()), in order for (5)
193
194
to provide a good approximation of the observational distribution in 2 In fact, standard VAEs perform accurate density estimation using observational data, without the need for capturing causal information. However, in this paper, we seek to ensure that our observational distribution induced
195
196
197
by VCAUSE complies causal factorization (Property D.. To that end, we need to make sure that (X; Z, A) pe (X; (z). That is, X, depends only on Z, if j or X; is an ancestor of X, in the causal graph. To fulfill this property, the GNN of the decoder should satisfy the following:
199
Proposition 1. (Causal factorization). VCAUSE satisfies causal factorization, po(X A) II Po. X; (g)) if and only if the number of hidden layers in the decoder is greater or equal than 8- 1, with 5 being the longest shortest directed path between any two endogenous nodes.
201
202
203
The above proposition (proved in Appendix B is based on the fact that, in a GNN with N₁ hidden layers (and N₂ + layers in total), the output for the i-th node depends its neighbors of up to N₁ + hops. As an example, consider the following chain causal graph: X; - X₂ - X3. such that 5 = 2. In the decoder, the first layer yields hidden representation for the 3-rd node h (Z)). Zz) that only depends on Z₂ and Zz- Thus, we need a second layer for its output h3g f(h,, Za) (()(, Z2), Za) to depend on Zi (note that X is an ancestor of X3).
205
206
207
208
4.2
Interventional distribution
210
VCAUSE approximates the interventional distribution in B as (illustrated Figure 3
p(X do = a)) ~ / po | Ap A
(6) A + AZ {Ay} VigIA U (Ay = 0} wez.j. This ensures that X, for i e I is independent of Z, for all
211 where the do-operator is performed on the causal adjacency matrix as
5

---

d
CKN
zz
~ pl
d
p(Z)
GNN
X
GNN
GNN 26
GNN
A
(c) Counterfactual Figure VCAUSE generation (a) observational, (b) interventional, and (c) counterfactual samples. The 'hat' in X and &CF indicate that they are sample estimates of the true random variables.
(a) Observational
(b) Interventional
j A i. Note that in order for 6 to be able to approximate the interventional distribution in G an intervention on a variational causal autoencoder should satisfy Property 2 i.e.:
214
Proposition 2. (Causal interventions). VCAUSE can capture causal interventions and only if the number of hidden layers in its decoder is greater than or equal to - 1, with y being the longest directed path between any two endogenous nodes in G.
216
217
218
To illustrate this, deepicts how messages hehhhhhddddlaaa- decoder GNN corresponding to the causal graph 9 in Figure [ (Iriangle with 7 2), both (a) without and (b) with an intervention on X2. We high- light in green the direct messages (sent via direct causal path in 9), and in red the indirect mes- sages (sent via indirect causal path in 9) from
exchanged in
X
21
+u
220
+/22
X
Z2
<gg
222
223
224
2x
X
23
+g
225
226
Z1 to X3. Observe that, similarly to Figure ! in
(a) Original
(b) Intervened
227
(a) there is an indirect path (via h2) from Z1 to
Figure 3: VCAUSE decoder (a) with and (b) with- out intervening on X₂. Arrows indicate message passing in the GNN corresponding to direct (green) and indirect (red) causal paths in Figure
228
X3, while in (b) this path is severed. Hence, the
229
hidden layer (h, h2. h2) allows differentiate
between direct and indirect paths and thus to capture interventional effects.
231
232
As the condition in Proposition is more restrictive than the one in Proposition I in order for VCAUSE to be able capture observational and interventional distributions, should satisfy that:
233
234
Design condition The decoder GNN of VCAUSE has at least as many hidden layers as 7 - 1, with y being the longest directed path in the causal graph G.
4.3
Counterfactual distribution
237
VCAUSE approximates the counterfactual distribution in as (illustrated in Figure 2c
do((x x).x²)= px( (2 Jugz (2,) Jiez A (Z x
(2 |x dZ,
action
abduction
prediction
where F represents a sample from X for which seek to compute the distribution over counterfac- tual CF Note here that two different passes of the encoder are necessary: one for the abduction 239
step of the factual instance qs F x" A); and another one for the action step (intervention) with of a; Vi (we remark that the of the values in x do not affect the overall counterfactual computation). We then evaluate the likelihood, making sure that the resulting counterfactual sample xC only depends the (2,") )igz C Z and [ Jiez C z Importantly, in order for VCAUSE to be able to approximate the counterfactual distribution, we need its abduction (and action) step(s) to comply with Property 3 i.e.:
240
9 (Z
242
243
244
245
Proposition& 3. (Abduction). The abduction step of an observed sample x fx1,.... ..a) in variational causal outoencoder satisfies that for all the posterior of Zu is independent on the subset {z, hepe" (i) C if and only if the encoder GNN has no hidden layers.
247
249
The above result (proved in Appendix B) can shown by the message passing algorithm computed by the encoder GNN, and leads to the second condition that VCAUSE should satisfy by design:

---

Table I: Evaluation of the observational and interventional distributions generated by VCAUSE with different numbers of hidden layers No. All metrics are shown in percentage (%).
collider (6 = 1,7 = 1)
triangle (6 = 1.7 7 2)
chain (6 = 2, 2)
N₂
MMD Obs. (%)
MMD Int)r.(%)
MMD Ohs.(%)
MMD Inter.(%)
MMD (bs.(%)
MMD Inter.(%)
1.37 0.54
0.90 + 0.19
2.20 + 0.74
4.03 - 0.42
5.58 + 1.01
8.07 + 0.53
0.86 0.34
0.95 0.28
1.05 0.38
- 0.31 2.35 + 0.35
1.56 1 0.4
1.0 + 0.50
0.91 + 0.16
1,20 + 0.63
2.33 + 0.29
1.67 + 0.61
1.46 + 0.29
251
Design condition 2: The encoder GNN of VCAUSE has no hidden layers.
Note that while the above condition may look restrictive and limiting the capacity of our encoder, may choose arbitrarily complex NNs to model the message f and update I functions, as well as one or more aggregation functions f e.g., sum, mean or max, model the encoder g
253
254
255
4.4
Practical considerations
256
Next, we briefly discuss practical implementation considerations to handle complex causal models, which often appear in real world applications- see the causal graph of the German Credit dataset [T] X for an example. For further details on VCAUSE implementation, refer to Appendix
257
258
in Section
259
Heterogeneous endogenous variables: In general GNNs parametrized such that the parameters of the message fd and update l functions are shared for all the nodes and edges in the graph.
280
However, similarly in the structural causal equations F, can define different message function
em for every edge in the causal graph by assuming a different set of parameters 6miz per edge in { Similarly, we can also assume different update functions f:" for each node i, by considering different
282
263
264
update parameters Ous for each node. This allows us to use different functions for each node, and
265
thus model heterogeneous endogenous variables, in terms of their continuous/diserete distribution, and also of their structural equations, e.g., linear/non-lineat
266
Heterogenous causal nodes: So far, we have modeled each endogenous variable X, as a node in the causal graph G. and thus in the VCAUSE GNNs. However, in some application domains the relationships between a subset of variables may be unknown, or they may be affected by hidden confounders, leading to an undirected path between them. In such cases, the subset of (k)) variables
268
269
270
271
modeled multidimensional and potentially heterogeneous node X, [X........ Xxx, J. Note that all the variables in the multidimensional node X, share the same latent random variable Z,
272
5 Evaluation
274
In this section, we conduct extensive experiments to evaluate the performance of VCAUSE at estimating the outcomes of causal queries. Please refer to Appendix for complete description of the experimental set-up. Moreover, to case the reproducibility of our experiments, our code is publicly available at htttp:://github. com/ XXXX/ XXXXX
275
278
277
278
Datasets. We consider different synthetic causal graphs that differ in the number of nodes d, diameter
5, and longest path y: synthetic collider (d 3, 8 = 1, = 1), M-graph (d 3, d = 1, 7 1), triangle (d = 3. 5 = 1. 7 = 2), chain (d = 3, 5 = 2, 7 = 2), and semi-synthetic loan (d = 7, 5 2,
280
7 3) from 17. For all of the synthetic datasets (i.e., except loan), we consider three different types of structural equations with increasing complexity: linear additive noise (LIN), non-linear additive noise (NLIN) and non-additive noise (NADD).
282
283
284
Metries.
We evaluate the observational distribution using the Maximum Mcan Discrepancy (MMD) as distance-meastue between the true and estimated distributions as a whole, i.c., the lower the MMD the better the distributions match. For the interventional distribution, we additionally report the estimation squared error for the mean and for the standard deviation (MeanE and StdE respectively)
K
286
288
for the children of the intervened variables. For the counterfactual distribution we report the mean square error (MSE) well as the standard deviation of the squared error (SSE) between the true and the estimated counterfactual value. We compute all results over 10 independent runs.
290
Validating VCAUSE design conditions. In a first step we empirically validate our design choices
252
for the VCAUSE encoder and decoder. We show how the number of hidden layers N₁, in the decoder
affect the quality of the estimation of the observational and interventional distributions for three
7

---

Table 2: Performance of different methods at estimating the observational, interventional and counter- factual of different SCMs. All metrics are shown in percentage (%)
Obs.
Interventional
Counterfactuals
SCM
Model
MMD (%)
MMD (%)
MeanE (%)
StdE (%)
MSE (%)
SSE (%)
MultiCVAE
1.0710.88
4.92:12.00
0.31
24.39-0.20
15.52:14.69
12.78.15.07
CAREFL
5.51=0.80
3.63++0.22
0.18=0.05
50.10=00.99
6.18=0081 0.11
VCAUSE
1.26:+0.68
0.21+0.26
0.65=0.12
24.51=0.09
11.68+0.69
7.62+0.42
tranngle
NADD NLIN
MultiCVAE
1.15+0083
7.21+3.90
0.57=0.29
17.58=0026
12.92+4.11
10.03+5..33
CAREFL
5.37.11.8
8.15.10.76
1.14
1..48.11.36
8.03
8.95
VCAUSE
1.55:10.90
6.26
0.15
17.41-10.09
12.
8.171.0.64
MuliCVAA
2.15:00.88
43.6312.73
0.18
1..1
24.45.11.62
38.2333388
CAREFL
6,14
74.84+14.78
2.59=3.76
112.65=
8,32-10,93
39.82:10.88
VCAUSE
2.54
8.87+1.52
0.09=0.04
20.94+1.72
10.36:10.78
17.82+1.20
MuliiCVAE 76.18=12.61
188.35+9.005
16.84=5.64
60.29+3.39
72.41+4.75
38.69+1.16
loan
CAREFL 9.28++2.15
9.54+1882
3,55=2.48
28.94=1.15
32.54+0.21
17.68+0.34
VCAUSE
1.09:0.24
1.41
0.40=0.09
9.58=00006
30.06::0.14
14.22+0.111
294
SCMs, with different values of longest shortest directed path d and longest directed path y. In Table I we observe that as expected: i) the collider (6 7 1) does not need any hidden layer to provide accurate estimate of both the observational and interventional distributions. contrast, the triangle
296
(6 7 2), which according to Proposition 2 needs at least one hidden layer to get more accurate estimate of the interventional distribution (while an improvement in the observational is not evident). Finally, as stated by Propositions [ and 2
298
the chain (6 y 2) requires at least one hidden layer to accurately approximate both the observational and interventional distributions.
300
301
5.1
Estimating interventional and counterfactual distributions
In the following we evaluate the potential of VCAUSE to model interventional and counterfactual queries. We consider interventions of the form do(r; = ax) for several values a, on both root and non-root nodes. Here report the results for the triangle and loan graphs. Refer to Appendix E for the remaining results.
303
305
306
Baselines. We compare our VCAUSE with two competing methods: i) MultiCVAE, which trains a conditional VAE for each endogenous variable that is not root node in the causal graph [ and ii) CAREFL I8]. which relies on autoregressive causal flows to estimate counterfactual queries.
307
308
309
Results for interventional distributions. Table 2 (middle columns) reports the MMD, MeanE, and
310
StdE for the interventional distribution. Here we can observe that VCAUSE consistently outperforms other methods in terms of MMD. Note that the three methods provide comparable results capturing
311
312
the mean of the interventional distribution (MeanE) (except for the more complex loan graph, where
VCAUSE outperforms the others). However, can also be seen that CAREFL and MultiCVAE
313
314
often fail to capture the standard deviation of the interventional distribution (StdE), while VCAUSE provides a more accurate estimate of the overall interventional distribution.
315
316
Results for the counterfactuals. Table 2 also reports the results for the counterfactual dis- tribution. Here, we first observe that MultiC-
CAREFL
VCAUSE
Ground truth
317
318
319
VAE slightly underperforms the other two mod- els. Second, we observe that CAREFL provides
320
321
more accurate estimates than VCAUSE in terms
0
322
of MSE, which may be explained by the fact that
Figure 4: Example of counterfactuals for a fac- tual x F from the test set of the triangle NLIN and do(x1 a).
323
CAREFL performs exact inference. However,
324
CAREFL presents high variance in its results
325
(see SSE). Note that to perform interventions,
326
CAREFL sets the parents of the intervened vari-
327
ables to zero, which may not completely severe the causal paths to the intervened nodes. In contrast, as further illustrated in Figure 4 VCAUSE leads to consistent counterfactual estimations across fac- tual samples and interventions. Figure Bl also shows that CAREFL. fails severely for some intervention
329
values, despite of intervening root node.
8

---

Use case: counterfactual fairness
331
332
Finally, we showcase the practical use of VCAUSE for assessing counterfactual faimess and also for
training a counterfactually fair classifier. To this end, we use the German Credit dataset publicly avail- able at the UCI repository (5).. We rely on the causal model with the following random variables X proposed in |6 (see Figure 5 sensitive feature S {sex}. and non-sensitive features C {age).
334
336 R credit amount, repayment history) and H = {checking account, savings, housing). Then, we aim to predict the binary feature Y =credit risk} from X. See Appendix F for further details.
338
Counterfactual fairness. Let S C X be a sensitive attribute (e.g.. gender), then classifier h X Y is considered c-counterfactuall fair [24] if:
P( = y do(S - = do(S = a'), sc Vx + a, y. 339
A classifier is counterfactually fair (c 0), if, given factual x" with sensitive attribute S had its sensitive attribute been different = a', the classifier prediction would remain the same. As VCAUSE allows to generate counterfactual samples,
341
thus use it to audit the fairness level of 343 a classifier. Moreover, we can use the VCAUSE encoder to learn a fair classifier hYCAUSE Z - Y. which takes as input the latent variables generated by VCAUSE without the one of the sensitive attribute Zs. Following (24]. we compare our VCAUSE fair classifier lyCcAUSE with: i) a full model
345
348
hall X Y that takes input the complete variable set; ii) an unaware model Nanaw X Y
that takes as input all variables but the sensitive one; iii) and a fair model htair (X:]Sgan" (1) - Y
347
348
that takes input all non-descendant variables of the sensitive attribute.
349
The results for logistic regression (LR) and support vector machine (SVM) classifiers are summarized in Table 3 Note that VCAUSE correctly ranks the different methods based on their unfairness level, showing that the full classifier is consistently less fair than the unaware and the fair classifiers, respectively. Moreover, the VCAUSE classifier leads to fair classifier, while keeping the fl score comparable to the unfair classifier. Therefore, VCAUSE does not only allow us to audit counterfactual fairness but also provides practical approach to train accurate and fair classifiers.
Results.
351
353
Table Evaluation of counterfactual (un)fairness. All metrics are shown in %. Lower/Larger values of unfairness/fl score are better.
Metric
Classifier
full
unaware
fair
VCAUSE
R
LR
71.07
68.33
50.00
74.81
fl-score (%)
SVM
74.60
72.44
64.71
70.40
Causal graph for variables X of the German Credit dataset 6
Figure
LR
5.93
2.25
0.16
0.85
unfairness (%)
SVM
6.07
2.68
0.20
1.00
Conclusion
356
357
In this work, we have proposed VCAUSE a variational causal autoencoder based on GNNs that: i) is specially designed to capture the properties of SCMs; ii) inherently handles heterogeneous causal graphs and data; and iii) provides accurate estimates of interventional and counterfactual distributions for SCMs of different complexities. As demonstrated by extensive experiments, VCAUSE provides accurate results for a wide variety of interventions in diverse SCMs Icading to significantly more robust results than competing methods [1 18]. Finally, we have shown practical use-case of VCAUSE in a problem of increasing interest for the machine learning community, namely, fairness in classification. In particular, have shown how use VCAUSE to both assess counterfactual fairness and to train counterfactually fair classifiers.
359
361
363
365
Moreover, our work opens up many interesting venues for future work. First, we have assumed known causal graph and the absence of hidden confounders, it would important to evaluate the sensitivity of VCAUSE to the violation of these assumptions in order to avoid its misuse. We also plan to extend VCAUSE to handle hidden confounders and to perform efficient causal discovery. Second, it would be interesting to perform ablation studies on the limitations of available GNNs architectures (55] for the VCAUSE encoder and decoder; as well as on how the performance of GNNs deteriorates as increase the length of the causal path and thus the required number of hidden
367
369
371
373
layers [2 Finally, it would be interesting to apply VCAUSE to other causal questions recently
discussed in the machine learning literature, such as preeeevii causal inference 226 or
375
explainable machine learning [7

---

References
378
377
[1] Ahmed M Alaa and Mihaela van der Schaar. 2017. Bayesian inference of individualized treatment effects using multi-task Gaussian processes. arXiv preprint arXiv: 0280/ (2017).
378
[2] Jasmijn Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Sima' an. 2017. Graph convolutional encoders for syntax-aware neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Vol.
380
381
382
33 Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. 2016. Importance weighted autoencoders. In Proceedings of the International Conference onl enrming Represenlations (ICLR), Vol. 4.
[4] Daniel C Castro, Ian Walwer, and Ben Glocker. 2020. Causality matters in medical imaging. Nature Communications 11 (2020).
385
386
[5] Denis Charles, Max Chickering, and Patrice Simard. 2013. Counterfactual reasoning and learning systems: The example of computational advertising. Journal of Machine Learning Research 14 (2013).
387
388
389
[6] Silvia Chiappa. 2019. Path-specific counterfactual faimess. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, Vol. 33.
390
391
[7] Alexander Chudik, Kamiar Mohaddes, M Hashem Pesaran, Mehdi Raissi, and Alessandro Rebucci. 2020. Economic consequences of Covid-19) 19: A counterfactual multi-country analysis. vaxeu.org (2020).
393
394
[8] Gabriele Corso. Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković. 2020. Prin- cipal neighbourhood aggregation for graph nets, In Advances in Neural Information Processing Systems (NeurIPS), Vol. 33.
395
396
[9] Saloni Dash, Vineeth N Balasubramanian, and Amit Sharma. 2020. Evaluating and miti- gating bias in image classifiers: A causal perspective using counterfactuals. arXiv preprint arXiv:2009.0822 (2020).
388
400
100]
Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. ics. ics.uci. www.:uii..uuuuua.. http:////aaaaasssssttalooo/ https///sstttlllgeeeeeeeaeteeeee
https://archive
401
402
[11] Sergio Garrido, Stanislav S Borysov, Jeppe Rich, and Francisco C Pereira. 2020. Estimating causal effects with the neural autoregressive density estimator. arXiv preprint arXiv:2008.07283 (2020).
403
404
[12] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry, In Proceedings of the International Conference on Machine Learning (ICML), Vol. 34. PMLR.
406
[13] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. The Journal of Machine Learning Research (JMLR) 13 (2012).
409
410
411
[14] William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Representation learning on graphs: Methods and applications. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering (2017).
412
413
414
[15] Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. 2008 Nonlinear causal discovery with additive noise models. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 21.
416
417
[16] Maximilian Ilse, Patrick Forré, Max Welling, and Joris M Mooij. 2021. Efficient causal inference from combined observational and interventional data through causal reductions. arXiv preprint arXiv:2103.04786 (2021).
418
419
[17] Amir-Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. 2020. Algorithmic recourse under imperfect causal knowledge: A probabilistic approach. arXiv preprint arXiv:2006.068) (2020).
421
10

---

423
[18] Ilyes Khemakhem, Ricardo Monti, Robert Leech, and Aapo Hyvarinen. 2021. Causal au- toregressive flows. In Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 24. PMLR.
424
425
426
[19] Hyemi Kim, Seungjae Shin, JoonHo Jang, Kyungwoo Song, Weonyoung Joo, Wanmo Kang, and I1-Chul Moon. 2021. Counterfactual fairness with disentangled causal effect variational autoencoder. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35.
427
428
429
[20]
Diederik P Kingma and Max Welling. 2014. Auto-encoding variational bayes. In Proceedings of the International Conference Learning Representations (ICLR), Vol. 2.
[21] Thomas N Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint 431
432
1rXivv1611.07308 (2016).
433
[22] Murat Kocaoglu, Christopher Snyder, Alexandros G. Dimakis, and Sriram Vishwanath. [n.d.]. CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training. In Pro- ceedings of the International Conference on Leurning Representations (ICLR), year=2018..
434
435
436
[23] Noemi Kreif and Karla DiazOrdaz. 2019. Machine learning in policy evaluation: New tools for causal inference. arXiv preprint 1931v:1903 (2019).
438
[24]
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 30.
439
[25]
Matt J Kusner, Chris Russell, Joshua R Loftus, and Ricardo Silva. 2018. Causal interventions for fairness. arXiv preprint arXiv:1806.0223 (2018).
441
[26] Matt J Kusner, Yu Sun, Karthik Sridharan, and Kilian Q Weinberger. 2016. Private causal inference. In Proceedings of the Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 19. PMLR.
443
445
[27] Felix Leeb, Yashas Annadani, Stefan Bauer, and Bernhard Schölkopf. 2020. Structured representation learning using Structural autoencoders and hybridization. arXiv preprint :.Xiv:2006.0796 (2020).
446
447
448
[28] Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
419
450
[29] Christos Louizos. Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. 2017, Causal effect inference with deep latent-variable models. In Advances in Neural Informa- tion Processing Systems (NeurlPS), Vol. 30.
452
454
[30] Ricardo Pio Monti, Kun Zhang, and Aapo Hyvärinen. 2020. Causal discovery with general non-linear relationships using non-lineat ICA. In Proceedings of the Uncertainty in Artificial Intelligence (UAI), Vol. 36. PMLR.
455
456
457
[31] Raha Moraffah, Bahman Moraffah. Mansoorch Karami, Adrienne Raglin, and Huan Liu. 2020. CAN: A causal adversarial network for learning observational and interventional distributions. arXiv preprint arXiv:2008. 11376 (2020).
458
459
[32]
Krikamol Muandet, Motonobu Kanagawa, Sorawit Saengkyongam, and Sanparith Marukatat. 2018. Counterfactual mean embeddings. arXiv preprint arXiv: / 08058805588888 08845 (2018).
461
[33] Ignavier Ng, Shengyu Zhu, Zhitang Chen, and Zhuangyan Fang. 2019. A graph autoencoder approach to causal structure learning. arXiv preprint arXiv: T. 7.07420 (2019).
463
464
[34] Sebastian Nowozin. 2018 Debiasing evidence approximations: On importance-weighted autoencoders and jackknife variational inference. In Proceedings of the International Conference on Learning Representations (ICML), Vol. 35. PMLR.
485
486
[35] Álvaro Parafita and Jordi Vitrià. 2019. Explaining visual models by causal attribution. arXiv preprint 1909. 08891 (2019).
467
11

---

489
[36]
Álvaro Parafita and Jordi Vitrià. 2020. Causal inference with deep causal graphs arXiv preprint Stti::20006083880 (2020).
470
471
[37] Álvaro Parafita and Jordi Vitria. 2019. Explaining visual models by causal attribution. In International Conference on Computer Vision Workshop (ICCVW).
[38]
Nick Pawlowski, Daniel Coelho de Castro, and Ben Glocker. 2020. Deep structural causal models for tractable counterfactual inference. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 33.
474
475
478
[39] Judea Pearl. 2009, Causal inference in statistics: An overview. Statistics surveys (2009).
477
[40] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen, 2016. Causal inference by using invariant prediction:
identification and confidence intervals. Journal of the Royal Statistical Society, Series B (Statistical Methodology) (2016).
479
480
[41] Tom Rainforth, Adam Kosiorek, Tuan Anh Le, Chris Maddison, Maximilian Igl. Frank Wood, and Yee Whye Teh. 2018. Tighter variational bounds are not necessarily better. In Proceedings of the International Conference on Machine Learning (ICML), Vol. 35. PMLR.
482
483
[42] Vineeth Rakesh, Ruocheng Guo, Raha Moraffah, Nitin Agarwal, and Huan Liu. 2018. Linked causal variational autoencoder for inferring paired spillover effects. In Proceedings of the International Conference on Information and Knowledge Management (CIKM). ACM.
484
485
486
[43] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE transactions on neural networks 20 (2008).
487
488
[44] Bernhard Schölkopf. 2019. Causality for machine learning. arXiv preprint arXiv:1911.1052 (2019).
[45] Patrick Schwab, Lorenz Linhardt, and Walter Karlen. 2018. Perfect match: A simple method for learning representations for counterfactual inference with neural networks. arXiv preprint arXiv: 1810.0065000 (2018).
481
482
493
[46] Uri Shalit, Fredrik Johansson, and David Sontag. 2016. Bounding and minimizing counterfactual error. arXiv preprint arXiv: 03976 (2016).
494
[47] Uri Shalit, Fredrik D Johansson, and David Sontag. 2017. Estimating individual treatment effect: Generalization bounds and algorithms. In Proceedings of the International Conference on Machine Learning (ICML), Vol. 34. PMLR.
496
[48] Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, and Tong Zhang. 2020. Disentangled generative causal representation learning. arXiv preprint arXiv:2010.02637 (2020).
499
500
501
[49] Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. 2006. A linear Non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research 7 (2006).
502
503
504
[50] Ilya Shpitser, Thomas S. Richardson, and James M. Robins. 2011. An efficient algorithm for computing interventional distributions in latent variable causal models. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), Vol. 27.
506
507
[51] Bob Siegerink, Wouter den Hollander, Maurice Zeegers, and Rutger Middelburg. 2016. Causal Inference in law: An epidemiological perspective. European Journal of Risk Regulation 7, 1 (2016).
509
510
[52] George Tucker, Dieterich Lawson, Shixiang Gu, and Chris J Maddison. 2018. Doubly repa- rameterized gradient estimators for monte carlo objectives. arXiv preprint 121.0:1810 (2018).
511
512
[53] Matthew James Vowels, Necati Cihan Camgoz, and Richard Bowden. 2020. Targeted VAE: Structured inference and targeted learning for causal parameter estimation. arXiv preprint arXiv:2009.13472 (2020).
514
12

---

516
[54]
Antoine Wehenkel and Gilles Louppe. 2021. Graphical normalizing flows. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 24. PMLR.
517
518
[55]
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions Neural
620
Networks and Learning Systems (2020).
521
[56] Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. 2020. VausalVAE: Disentangled representation learning via neural structural causal models. arXiv preprint arXiv:2004.08697 (2020).
523
524
[57] Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting, In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Vol. 27.
525
526
527
(58) Yue Yu, Jie Chen, Tian Gao, and Mo Yu. 2019. DAG-GNN: DAG structure learning with graph neural networks. In Proceedings of the International Conference on Machine Learning (ICML),
528
Vol. 36. PMLR.
[59] Matej Zečević, Devendra Singh Dhami, Athresh Karanam, Sriraam Natarajan, and Kristian Ker- sting. 2021. Interventional sum- con-product networks: Causal inference with tractable probabilistic models. arXiv preprint arXiv:2 102. /0440 (2021).
531
600
Kun Zhang, Biwei Huang, Jiji Zhang. Clark Glymour, and Bernhard Schölkopf. 2017. Causal discovery from nonstationarryheterggenn nation. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI),
534
data: skeleton estimation and orientation determi-
535
536
Vol. 26.
[61] Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Garnett, and Yixin Chen. 2019. D-VAE: variational autoencoder for directed acyclic graphs. Advances in Neural Information Processing Systems (NeurIPS), Vol. 32.
538
[62] Min Zheng and Samantha Kleinberg. 2019. Using domain knowledge to overcome latent variables in causal inference from time series. In Proceedings of the Machine Learning for Healthcare Conference (MLHC). PMLR.
541
543
[63] Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. 2018. DAGS with NO TEARS: Continuous optimization for structure learning. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 31.
544
545
13

---

Checklist
548
547
1. For all authors...
548
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
549
550
[Yes] Our contributions are: i) identifying the properties of an SCM that allow us to propose our model (Section B) ii) proposing the Variational Causal Autoencoder (VCAUSE) for answering causal queries given observational data and the causal graph (Section T. iii) evaluating the estimation of interventional and counterfactual distributions as a whole on different SCMs (Section 55) iv) showing how VCAUSE can be used for counterfactual faimesss evaluation and classification (Section 3
552
554
556
(b) Did you describe the limitations of your work?
557
[Yes] As stated throughout the paper, our approach relies on two causal assumptions, i.e. causal sufficiency and access to the true causal graph. As these assumptions may be seen as a practical limitations, in Section 7 we discuss our plans to address them in future work. Moreover, Section 7 states that our approach inherits the limitations of GNNs, and thus propose future ablation studies to investigate the scalability of the proposed approach to more complex causal graphs.
$59
581
(c) Did you discuss any potential negative societal impacts of your work?
564
[Yes] The proposed approach allows for causal reasoning under certain assumptions, which we make explicit to avoid its misuse. Moreover, as discussed in Section 7 we consider the relaxation of such assumptions as future work.
566
567
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
569
[Yes] We have read the ethics review guidelines and ensured that our paper conforms to them.
2. If you are including theoretical results... 571
572
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] In section El we explicitly state the assumptions that we make throughout the paper as well as the necessary conditions for the proposed approach to yield reliable results.
573
574
575
576
(b) Did you include complete proofs of all theoretical results?
[Yes] Appendix B provides complete proofs for all the theoretical results in the paper.
578
3. If you ran experiments...
579
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)?
580
581
[Yes] We upload the code with the supplementary material and will be releasing the code on GitHub.
582
583
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
584
[Yes] Appendix D provide a detailed description of the experimental set-up, the im- plementation and validation of the methods and the computation of the performance metrics.
585
506
587
588
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
589
580
[Yes] In Tables I and 2 we report the average and standard deviation for 10 different runs.
591
582
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix D However, note that experiments are light- and we do not require any GPU,
583
594
595
596
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
597
(a) If your work uses existing assets, did you cite the creators?
598
[Yes] Our implementation the baselines in Section 3 is built on the publicly available code provided by [17 and [8] The semi-synthetic dataset loan is taken from [77. In
600
Section we use the publicly available German Credit dataset from the UCI repository
10
14

---

602
(b) Did you mention the license of the assets?
603
[Yes] See Appendix D We rely on research code from 17 and [8] provided under
MIT License.
(c) Did you include any new assets either in the supplemental material URL?
606
[Yes] Our code can be found in the supplemental material and we will make publicly available GitHub.
608
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating?
809
610
[No] Does apply. The only real world data we use the publicly available German Credit dataset from the UCI repository [IO) which is published in anonymized form.
(c) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content?
613
[N/A] Does not apply.
5. If you used crowdsourcing conducted research with human subjects...
616
(a) Did you include the full text of instructions given to participants and screenshots, applicable?
618
[N/A] Does not apply.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable?
620
[N/A] Does not apply.
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation?
623
[N/A] Does not apply.
15


Reviews:
[{"review_id": "XQRRxv__rXO", "reviewer": "Reviewer_oFSi", "summary": "The paper introduces Variational Causal Autoencoder (VCAUSE) for causal inference in the absence of hidden variables. It is designed to capture the properties of SCM. \n", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths. The paper is well written. \n\nWeaknesses. The authors state that the VCAUSE was developed for causal inference, however, they also state that they assumed that the causal graph is known in the current contribution. If it is the case, why there is the need for the causal inference? \n\nAlthough the paper is quite clearly written, there are some flows. First, the role of U (exogenous) variables is not clear: is is assumed that there is 1 latent variable per 1 observed variable, as shown on Figure 1. Then, these U variables are not mentioned further in the paper. The VCAUSE (Definition 4.1) is defined without them. It is stated that \"the latent variables Z play a similar role to the exogenous variables U\" but it is very vague, and I do not see a clear correspondence. \n\nPropositions 1 and 2 are important but I guess no novel, it should be a known result in deep learning.  \n\nI would appreciate more explanations on the design condition 2. I see that it follows from Prop. 2 but it is unclear why to use neural networks without hidden layers. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "T6QsCGhm83q", "reviewer": "Reviewer_ENhm", "summary": "The paper leverages the power of variational graph autoencoders and graph neural networks and proposes a novel framework, namely Variational Causal Autoencoder, to answer the interventional query at a population level and counterfactual query at an individual level. The paper further studies the counterfactual fairness as a use case under the development. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, the paper is well written and read naturally. There are two key assumptions in the framework: 1) there are no hidden confounders in the causal graph; 2) the causal graph is available. As also said by the author at the end, both are a bit strong. Despite the two assumptions, the model first approximates the posterior of an exogenous vector Z in d dimensional for an endogenous sample X in d dimensional as well, using a graph neural network. After interventional changes or counterfactual changes on Z, the modified Z can be decoded back to the endogenous variable space using a GNN to obtain its interventional appearance or counterfactual appearance. The experiments are conducted in synthetic/semi-synthetic datasets with d = 7 at most.  \n\n-Originality\nThe framework that combines GNN and VGAE is novel but not surprising due to its strong assumptions. It is natural to model a graph-like data structure using GNN. The VGAE, on the other hand, is well studied in terms of manipulation of its latent space. \n\n-Quality\nThe quality of the paper is good overall. It is well written and easy to follow. However, the experiments conducted are mainly from synthetic/semi-synthetic datasets. And the dimenson of data sets is relatively low (7 at most). \n\n-Clarity\nThe paper is good in terms of clarity on presenting the ideas, formulating the question, and conducting experiments. \n\n-Significance\nAs said above, the method is novel but carrying two strong assumptions, which is a limitation. Thus, its significance is fair to a minor problem. \n\n-Concerns\nMy main concerns are:\n1) The no-hidden-confounders assumption is quite strong. \n2) The data sets are synthetic/semi-synthetic. The dimension is relatively low. It seems that only continuous variables are considered, no discrete variables. \n3) It might be interesting to see the study on the dimension of Z_i (i.e. the parameter m). Does that have an effect on overall results?\n4) The decoder might induce bias on the reconstructed sample. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See main reviews for limiations. There is no societal impact. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "N00wgG23YwE", "reviewer": "Reviewer_7HjM", "summary": "The authors propose a framework for the use of variational graph autoencoders for causal inference in settings without hidden confounders when observational data and the causal graph are available. The framework aims to mimic the properties of the matching Structural Causal Model (SCM), including the ability to evaluate observational, interventional and counterfactual queries. To evaluate their proposed framework, the authors perform an experimental evaluation in estimating the outcomes of causal queries on multiple small synthetic causal graphs and in assessing counterfactual fairness.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**:\n- The question of how to accurately answer causal queries with machine learning models from observational data given knowledge of the underlying causal graph is of practical relevance in some use cases (see limitations outlined in \"Significance\" below), and not yet extensively studied in literature. \n- The application of graph variational autoencoders to this setting is to the best of my knowledge novel. However, the manuscript's exposition is relatively sparse in details as to why this direction of research would be expected to lead to performance gains.\n\n**Quality**:\n- Given that VCAUSE is proposed as a broadly-applicable tool for answering causal queries from observational data when the underlying causal graph is available, I am concerned that the presented evaluation is not representative enough to substantiate claims that VCAUSE is in general better able to estimate observational, interventional and counterfactual distributions. In particular, the experimental evaluation only evaluated graphs with 3 and 7 nodes, whereas in practice significantly larger graph sizes are commonly encountered (e.g., in biology where structures with 100s to 1000s of nodes are not uncommon). I believe an evaluation on a larger set of diverse settings with varying graph properties (number of nodes, connection sparsity, ...) would be necessary (i) to substantiate broad claims in the general case and (ii) to better understand the relative performance of VCAUSE under varying assumed graphical structures.\n\n- A better understanding of the importance of the properties of the modelled SCM on VCAUSE would be particularly important because Table 2 indicates that the relative performance of VCAUSE appears to depend strongly on the underlying SCM.\n\n- Several relevant reference baselines are missing from the experimental comparison (Table 2), e.g. learning the parameters of an SCM directly using neural networks [1], additive noise models [2] which can be parameterized by different regressor classes (see e.g. [3]) and non-linear structural equation models (SEMs) [4].\n\n- Different approaches were used for hyperparameter optimization for each of the compared baselines (Appendix D.2). It is therefore unclear if differences in observed performance (Tables 2 and 12) are due to inherent properties of the models or the different hyperparameter optimization strategies and budgets. The same hyperparameter optimization strategy with a fixed hyperparameter optimization budget should have been employed to enable a fair comparison of methods on equal grounds.\n\n- In addition, there are several VCAUSE hyperparameters for which only the final selected values are presented (Appendix D.2) but no details are given as to *how* they were selected (learning rate, epochs, K, latent variable dimension, number of neurons, ...). Presumably, these values are the result of manual optimization - if so, this should be transparently stated in the manuscript including information as to whether or not held-out performance results were used to optimize these hyperparameters (risk of overfitting).\n\n- The application of VCAUSE to use cases in counterfactual fairness (Section 6) is interesting but no state-of-the-art baselines are presented for comparison. It is therefore unclear if VCAUSE performs better, equally well or worse in the counterfactual fairness setting than other methods that parameterise an SCM. In addition, uncertainty and statistical significance metrics for the experiments in Section 6 are missing.\n\n**Clarity**:\n\n- In terms of clarity, my main consideration is that the motivational element appears to be missing in the paper's story line. Why do we expect that mimicking an SCM using a graph VAE is a more efficacious approach than alternatives (and therefore embark on this line of research)?\n\n- In Table 2 (main) and Table 12 (appendix), the caption indicates that all metrics are given in percentages. However, the authors do not state what reference values were used to compute the percentages. In addition, even if the reference values were indicated, I would generally prefer to be shown the absolute values, since some metrics are not on a linear scale (e.g. MSE) which makes interpretation as percentages difficult.\n\n- The computational characteristics of VCAUSE and how they compare to existing alternatives are missing. Compute time details are in general missing from the manuscript (the Paper Checklist refers to Appendix D but no compute time information is provided therein).\n\n**Significance**:\n- The overall impact of the presented work is limited because of the restrictive assumptions (availability of a faithful causal graph and no hidden confounding) that hold true in a very limited set of settings and can in practice not be verified from data alone [5]. For practical use cases that VCAUSE would be considered for, there therefore exists the risk that the assumptions would likely be at least partially violated.\n\n- In addition, the manuscript's claims are only substantiated with experimental evidence for small directed acyclic graphs (DAGs), and it is therefore currently unclear if VCAUSE offers relative benefits in settings with larger numbers of nodes.\n\n- Given the absence of several baselines in the experimental comparison, I am unsure if the complexity of processing the causal graph through a graph neural network (GNN) is necessary. What motivates this approach over simpler existing ones (e.g. parameterising the SCM using neural networks [1] or additive noise models [2]) that would justify the added complexity? What is the inductive bias introduced by VCAUSE that one would expect to lead to performance benefits?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "**Societal Impact**: \n- I see no immediate reason for concern.\n\n**Questions/suggestions for the authors**:\n- In Section 2.1, the authors state that \"Every SCM M is associated with a directed acyclic graph (DAG) [...].\" This is, to the best of my knowledge, not true - SCMs are not by definition acyclic and can in the general case include cycles [6]. Is VCAUSE only applicable to acyclic settings? If yes, please add the acyclicity assumption to the manuscript.\n\n- Please add the missing compute time details and a comparison of computational considerations between VCAUSE and existing methods.\n\n- Introduction, 3rd paragraph: the sentence \"[...] but also on how these relationships are (i.e., on the structural equations).\" is missing a verb, e.g. \"how these relationships are mediated (e.g., in the structural equations)\".\n\n- In Table 2 (main) and Table 12 (appendix), the caption indicates that all metrics are given in percentages. Please provide absolute values instead (or at least the reference values used to compute the percentages).\n\n- In Table 2 (main) and Table 12 (appendix), some values are shown in bold print. However, it is not stated what the bold print indicates. Does the bold print indicate statistical significance? If so, compared to what reference?\n\n- Given that the authors claim \"(...) *significantly* more robust results than competing methods\" in the presented conclusion, the statistical significance of all results should be reported in the manuscript (p-values) - in particular for Table 2.\n\n- Line 325 to 327: \"Note that to perform interventions, CAREFL sets the parents of the intervened variables to zero, which may not completely severe the causal paths to the intervened nodes. \" - I assume CAREFL sets the *contribution of* the parent variables (and not the parent variables themselves) to the intervened node to zero (which would correspond to Pearl's do-operator). Can the authors explain why this would not sever all causal paths to the intervened nodes? If all incoming contributions are zeroed, to my understanding no other causal paths to the intervened node can exist.\n\n- Line 327: \"which may not completely severe [should be *sever*] the causal paths to the intervened nodes.\"\n\n[1] Ke, Nan Rosemary, et al. \"Learning neural causal models from unknown interventions.\" arXiv preprint arXiv:1910.01075 (2019).\n\n[2] Peters, Jonas, et al. \"Causal discovery with continuous additive noise models.\" (2014).\n\n[3] Khemakhem, Ilyes, et al. \"Causal autoregressive flows.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2021.\n\n[4] Monti, Ricardo Pio, Kun Zhang, and Aapo Hyvärinen. \"Causal discovery with general non-linear relationships using non-linear ica.\" Uncertainty in Artificial Intelligence. PMLR, 2020.\n\n[5] Stone, Richard. \"The assumptions on which causal inferences rest.\" Journal of the Royal Statistical Society: Series B (Methodological) 55.2 (1993): 455-466.\n\n[6] Bongers, Stephan, et al. \"Foundations of structural causal models with cycles and latent variables.\" arXiv preprint arXiv:1611.06221 (2016).", "review_text": "", "extra_scores": {"ethical_concerns": "I have no concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "2dgLqRELs3C", "reviewer": "Reviewer_bpui", "summary": "This work introduces the variational causal autoencoder (VCAUSE), a model for estimating interventional and counterfactual distributions from observational data and a known causal graph. VCAUSE builds on graph neural networks (GNNs) to propagate information across the causal graph, enabling simulation of interventions and abduction of exogenous noise for counterfactual inference.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality\n- From my limited knowledge of the graph deep learning and causal discovery literatures (where I know GNNs have been used), this application of a GNN-based VAE for modelling a causal graphical model seems novel and inspiring.\n- On the other hand, it is unclear what limitations of the existing methods this work is aiming to address. Specifically, readers are missing a compelling description of the advantages of the GNN-based VCAUSE over CVAE-based (e.g. Karimi et al., 2020) or flow-based (Khemakhem et al., 2020) approaches. Is it more general, more scalable, easier to define/implement, more stable to train...?\n- Other comments:\n  - CAREFL (Khemakhem et al., 2021) seems somewhat mischaracterised in the related work section: it is not evident in the text that CAREFL is not limited to two variables, and that it can model non-linear mechanisms between endogenous/observed variables.\n  - It would be relevant to relate your work to Pawlowski et al. (2020). Their deep SCM framework is a generalisation of Karimi et al. (2020)'s MultiCVAE—supporting abduction-action-prediction for high-dimensional variables (e.g. images) with VAE-based, flow-based, or GAN-based causal mechanisms.\n    > Pawlowski, N., Castro, D. C., Glocker, B. Deep Structural Causal Models for Tractable Counterfactual Inference. In _NeurIPS_ 2020.\n\n### Quality\n- The theoretical portion of this work is interesting, principled, and well developed. I appreciated the care in formulating and proving the model design requirements to match the behaviour of a given causal graph.\n- However, the empirical benchmarking (Section 5) was limited to small, synthetic tabular datasets. While I understand the value of toy experiments in canonical conditions to validate basic model capabilities, the presented results did little to convince the reader to favour the proposed method over the baselines in realistic settings (e.g. larger, more complex graphs; low- and high-dimensional variables; noisy real-world data).\n- The fairness experiment (Section 6) is not very informative due to the choice of baselines. In absolute terms, the VCAUSE-based fair classifier seems to perform well fairness-wise on this German Credit dataset. However, it has more expressive power (computational layers) than the shallow baselines, so it does not look like a fair comparison. Furthermore, no other counterfactual inference methods (e.g. MultiCVAE) were evaluated for fairness assessment, which makes the conclusions from this experiment unclear.\n- The authors claim VCAUSE can 'accurately' estimate counterfactual distributions of unknown SCMs. Yet, this is impossible by definition! Counterfactuals are unidentifiable without knowledge of the structural equations because they cannot be falsified based on factual data alone—observational or even experimental (see e.g. Pearl, _Causality_, 2009, Ch. 7). Such evaluation is possible only in synthetic settings with access to a simulator (i.e. the true data-generating process). These claims are misleading and need rephrasing (see below).\n- Estimating interventions in VCAUSE (Section 4.2) seems to incur a reconstruction error on the intervened variables (i.e. $do(\\mathbf{X}_\\mathcal{I} := \\boldsymbol{\\alpha})$ results in $\\mathbf{X}_\\mathcal{I}\\neq\\boldsymbol{\\alpha}$), though the implications are never discussed in the paper.\n\n### Clarity\n- The paper is overall very well written and well structured. The authors present thorough additional theoretical and experimental details in the supplement, along with sharing the code.\n- No justification is presented for the use of PIWAE for training. Why is it necessary? Does the model fail to converge with the vanilla ELBO?\n- Minor suggestions:\n  - The distinction between $\\mathbf{F}=\\\\{f_i\\\\}$ and $\\tilde{\\mathbf{F}}=\\\\{\\tilde{f_i}\\\\}$ is confusing and could be made a lot more explicit in the text.\n  - [Eqs. (2–4)] As written, the integrals are computing expectations, not the intended pushforward distributions.\n  - [Eq. (6) & L237] These expectations are 'double-counting' the $\\mathbf{Z}$ variables. Either make them double integrals with $\\iint\\cdots d\\mathbf{Z}\\\\,d\\mathbf{Z}^\\mathcal{I}$ or explicitly specify the indices of $\\mathbf{Z}$ over which $p$ and $q_\\phi$ are being evaluated.\n  - The sudden switch from using $\\mathbf{U}$ to $\\mathbf{Z}$ in Section 4 was jarring. Consider explaining early the differences between these two sets of variables.\n  - It would be helpful to include visualisations of the distributions entailed by (at least some of) the SCMs used in the experiments to give a sense of the tasks' difficulty and encoded assumptions.\n\n### Significance\n- The work presents interesting ideas that may be further extended and applied to varied types of data.\n- Nevertheless, there does not appear to be strong enough discussion or evidence to convince readers to adopt this methodology rather than existing ones.\n----\n### Update after discussion\nUnder the assumption that the authors will include the discussed clarifications in the final version of the paper, I am increasing my recommendation by 1.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- The paper is missing a discussion of the implications of causal modelling with deep learning. How can we tell that the model is right? What can happen when people misuse the model, i.e. apply it outside the built-in assumptions? Who may be harmed by such misuse, and in what scenarios?\n- The multiple claims of 'accurate' counterfactual inference may mislead practitioners to blindly trust the counterfactual predictions of this model, without knowing that this assumes the neural networks faithfully match the functional form (not just the outputs!) of the true causal mechanisms, and that the accuracy cannot be objectively measured in real-world settings. Any work dealing with counterfactuals must clearly acknowledge such limitations to reduce risks of misuse and over-interpretation, especially in high-stakes applications.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Two reviewers recommend accepting the submission and two reviewers recommend rejecting the submission. Reviewer 7HjM raises several important concerns: 1) the motivation for the work is missing from the story line, 2) the synthetic experiments show that the proposed method performs better some times and worse others, but without explanation as to why, and 3) the real data experiment does not justify the claims of the proposed method being a broadly-applicable tool. Reviewer bpui also points out limitations of the experiments, as well as many positive aspects of the work, which I agree with. However, I find the submission’s merits are not sufficient to warrant acceptance to a top-tier machine learning conference in light of aforementioned limitations.
Decision: Reject