Deep Counterfactual Estimation with Categorical Background Variables
Authors: Edward De Brouwer
Venue: NeurIPS 2022 Accept (2022)
Keywords: counterfactuals
causal inference
causality
healthcare
time series

Abstract:  Referred to as the third rung of the causal inference ladder, counterfactual queries typically ask the "What if ?" question retrospectively. The standard approach to estimate counterfactuals resides in using a structural equation model that accurately reflects the underlying data generating process. However, such models are seldom available in practice and one usually wishes to infer them from observational data alone. Unfortunately, the correct structural equation model is in general not identifiable from the observed factual distribution. Nevertheless, in this work, we show that under the assumption that the main latent contributors to the treatment responses are categorical, the counterfactuals can be still reliably predicted.
 Building upon this assumption, we introduce CounterFactual Query Prediction (\method), a novel method to infer counterfactuals from continuous observations when the background variables are categorical. We show that our method significantly outperforms previously available deep-learning-based counterfactual methods, both theoretically and empirically on time series and image data. Our code is available at https://github.com/edebrouwer/cfqp.

Deep Counterfactual Estimation with Categorical Background Variables
Edward De Brouwer
ESAT-STADIUS
KU Leuven
edward. werrrdddbbowweeeeseeeee www.eeeeeaaa..eu.e.........m kuleuven.
Abstract
Referred to as the third rung of the causal inference ladder, counterfactual queries typically ask the "Whal if question retrospectively. The standard approach estimate counterfactuals resides in using structural equation model that accurately reflects the underlying data generating process. However, such models are sel- dom available in practice and one usually wishes to infer them from observational data alone. Unfortunately, the correct structural equation model is in general not identifiable from the observed factual distribution. Nevertheless, in this work, we show that under the assumption that the main latent contributors the treatment responses are categorical, the counterfactuals can be still reliably predicted. Build- ing upon this assumption, we introduce CounterFactual Query Prediction (CFQP), a novel method to infer counterfactuals from continuous observations when the background variables are categorical. We show that our method significantly out- performs previously available geeeplleannige based counterfactual methods, both theoretically and empirically on time series and image data. Our code available at /https://githee //gtthub///ub...g www. henn//debrwwwer//// cfap
Introduction
Counterfactual queries aim at inferring the impact of treatment conditioned another observed treatment outcome. Typically, given an individual, a treatment assignment, and a treatment outcome, the counterfactual question asks what would have happened that individual, had it been given another treatment, everything else being equal. An illustrative and motivating example is the case of clinical time series. Based on the observation of the outcome of treatment A on particular patient, counterfactual queries ask what would have been the outcome for this patient, had it been given treatment B instead. Notably, counterfactual prediction differs from interventional prediction, which is also referred to counterfactual potential outcomes [23] and constitutes the second rung of the causation ladder [19] Counterfactual predictions are retrospective, as they condition on observed treatment outcome. In contrast, interventional predictions are prospective as they only condition on observations obtained before treatment assignment.
Much more than a statistical curiosity, counterfactual reasoning reflects complex cognitive abilities that deeply ingrained in the human brain [22 and emerges in the early stages of cognitive development [77. The ability to reason counterfactually can indeed help to identify causes of outcomes retrospectively, has been suggested to be central in the formation of rational intention. and supports key theories of human cognition [X The importance of counterfactual reasoning in the human cognitive process has thus motivated researchers to endow artificially intelligent systems with the same ability (21. However, counterfactual inference is not possible from observational and interventional data alone
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Counterfactual reasoning, therefore. requires making several assumptions to overcome this limitation. The most popular one assumes knowledge of the underlying structural equation model that describes the data generating process [21 or a specific functional form thereof [1 S 21 Unfortunately, this assumption rarely met in practice, especially in high-dimensional data such as time series or images.
This led to the development of deep structural equation models that attempt to model the structural equations with neural networks [1 24 However, despite their ability to model high- high-dimensional data, these approaches fail to provide theoretical guarantees for the reconstruction of counterfactuals. Indeed, they focus on modeling the factual distribution which, without further assumption, can, unfortunately, lead to erroneous counterfactual distributions.
In this work, we bridge the gap between the classical structural equation model assumptions and deep-learning based architectures. By assuming that the treatment and observables are continuous and that the hidden variables that contribute most to the treatment response are categorical, we can rely on recent results in identifiability of mixture distributions R to show that we can approximately recover the counterfactuals using arbitrary parametrie functions (i.e. deep neural networks) to model the causal dependence between variables. This allows to infer counterfactuals on high-dimensional data such as time series and images. Generally, this work explores the assumptions that can lead to approximate counterfactual reconstructions while controlling the discrepancy between the recovered and the true counterfactual distributions.
Besides the general appeal of endowing machine learning architectures with counterfactual reasoning abilities, an important motivation for our work is the counterfactual estimation of treatment effects in clinical patient trajectories. In this motivational example, one wishes to predict the individual treatment effect retrospectively. Based on the observed treatment outcome of a particular patient, we want to predict what would have been the outcome under different treatment assignment. The ability to perform counterfactual inference on patient trajectories has indeed been identified as a potential tool for improving long and costly randomized clinical trials [S]
Contributions
We provide a new set of assumptions under which the counterfactuals are identifiable using arbitrary neural networks architecture, bridging the gap between structural equation models and deep learning architectures.
We derive a new counterfactual identifiability result that motivates a novel counterfactual reconstruction architecture.
We evaluate our construction on three different datasets with different high -igh-dimensional modalities (images and time series) and demonstrate accurate counterfactual estimation.
2 Background
2.1 Problem Setup: Counterfactual Estimation
We consider the general causal model M = V,) depicted in Figure la consisting of background variables U, endogenous variables X and Y and the set of structural functions F. Background variables U = (Ux, UT, U., W} are hidden exogenous random variables that determine the values of the observed variables V (X, T, Y}. Covariates X E X' represent the information available before treatment assignment, T E T is the treatment assignment and Y E ) is the observed response to the treatment. We refer to the space of probability measures on y as P()). Observed variables V generated following the structural equations F === (fx, fr, fy, fe). such that X fx (U,, W), (UT, U. = fe(W), ). Y fy (X, U.). We further assume strong ignorability (i.e. no hidden confounders between T' and Y).
Using notations introduced in Pearl et al. (21. we define the potential response of variable Y to an action do(T t) for particular realization of U = u as (u). Our goal is to predict the counterfactual response, for new treatment assignment (7 1'), conditioned on an observed initial treatment response. That is, the probability of observing a different treatment response under treatment I', after observing treatment response y for covariate and treatment i. The probability density function of counterfactual y' then writes:

---

p(Y = X Y y t) p(Y = = =y)
J. p(v(uu)= 1/p(U = = == y): (1)
and we refer to the counterfactual probability measure as vr y, t). Equation suggests a natural three step procedure for computing the probability of counterfactual. First, the abduction step infers the density of U conditioned on the observed treatment outcomes, covariates and treatments: X x, T Y y). Second, in the action step, one sets the new treatment in the causal model (do(T = )) Lastly, the prediction step, one can propagate the values of = u and in the causal graph, using F, to compute p(Y.: (u) y').
p(U
l' =
In practice, only have access to set of N observations of variables X, Y and T. We refer to this dataset as D = (X. T, Y) where X = {a; (t, :1 ..., J. Importantly, we don't have access to counterfactual examples (i.e. tuple (x, y, L, 1,, D), such that direcity learning map (I, U,,, - yu is excluded.
1,... N}, Y {br : 1....nn) n) and
2.2 General Non-identifiability of Counterfactuals
Because the background variables U are hidden, the above three-steps procedure requires knowledge of the structural functions F [Jxx, frr,/ Jy Indeed, one can show that there exist multiple structural functions F that would lead to the same observed joint density p(X, Y, T) but would lead to incorrect counterfactual probabilities [99 20 The correct causal model is thus in general non-identifiable. leading to non-identifiability of the counterfactual probability. We specify what is meant by the identifiability of counterfactuals in the following definition.
Definition 1 (Identifiability of Counterfactuals). Let be metric on P(Y), uy Y. the true counterfactual probability measure and By [X, T) the estimator of the counterfactual probability measure with N data points. Counterfactuals are p-identifiable at threshold ó if, for all t, E TT, X,y y V.
lim (., y. 8r (z. s N
(z.
W
W
Ux
Ux
X
Ue
X
Uz
UT
UT
Un
T
Y
T
Y
(a) General Bayesian network for the treatment counterfactual problem. X, Y and 7'' are observed while Ux, Ur, W and U. are hidden background variables.
(b) Bayesian Network embodying the hidden cat- egorical background variable assumption. Ue is split background categorical variable Uz and continuous hackground variable Un-
Figure 1: Graphical model representations of the causal model M. We assume strong ignorability and continuous treatments T, observables and responses Y.
3

---

2.3 Causal Model Assumptions for Counterfactual Idenfiability
Despite the general non-identifiability structural equation models laid out above, we propose plausible assumptions that one can build upon to identify counterfactuals reliably. We first assume X and Y are continuous (potentially high dimensional) variables (such as images or time series). The treatment assignment 7' is also assumed continuous, and X' x 'I is connected space. Our first central assumption posits that the hidden variable Ue factorizes into a categorical and continuous variable.
Assumption (Categorical Background Variable). The background variable Ue decomposes into categorical latent variable Uz E [K] {I,.,, K} and an independent exogenous continuous variable Un.
This assumption depicted in the graphical model of Figure Ib and embodies the intuition of different hidden groups that drive the treatment response. For instance, treatment could have different responses depending on the stage of the disease a patient finds themself in. The disease stage is unobserved yet correlated with the observed covariates X (through W).
Due to the categorical nature of variable Uz, one can write the conditional density of Y as a mixture model:
P(Uz=ul f
D (,,) uz, u.) = yl
P(Y = v x x
Σ uyE(L...
We define as the probability density function of the conditional treatment response generated by fY, Uz and Un y is thus mixture probability density function with mixture components 7/ (())) and weights wk.
K Σ www
Y XXX,T~>(XTT)+
(2)
Without loss of generality, we assume that U, (0, In the case of additive noise, the con- ditional distribution of Y becomes mixture of Gaussians: >Xx, T) Σ (M (X, T), EE), where l* are functions mapping X and to the mean of the mixture components and we consider different variances for each k. We now proceed with the next assumptions.
Assumption 2 (Continuity). The moments of the probability density functions 7 (z, t) exist and continuous functions X and T::
(x 1) = Ey [Y]] E C(x,)) (,
Assumption (Clusterability). For each (x, t) (N,T), the density 7(x, t) is clusterable and the expected deviation of each (xxx, t) is bounded by constant 8 E IR That is, vk [K], Vx, e (X x T), with px(x,t) Ey
By [IIX (x(x,)))22) 2
(4)
In our motivating clinical example, Assumption 2 reflects that the probability of specific treatment response changes continuously over the set of observed covariates and treatments. In particular, the expected treatment outcome for partieular patient varies continuously with the treatment assignment, which is common assumption, e.g. in clinical practice [
'The mixture components defined such that for any subset A Vd have SA (X, (y)ddy I[y (X, Uz = k. U,) AAddPUU,). The mixutre weights are defined P(Uz k).

---

Assumption posits clusterability of the mixture components "ee for which rigorous mathematical definition is given in Appendix y It is motivated by recent results on the identifiability of mixtures models 3 Intuitively, it supposes that patients with the same observed covariates and treatment assignment but different hidden group will show different treatment outcomes. We also bound the expected deviation of the mixture components "i that characterize the inter-group variability in the treatment response for a particular patient and treatment outcome.
3 Methods
3.1
Identifiability and Counterfactuals Reconstruction
For fixed point (X = xr T 1), Equation 2 is finite mixture model, for which identifiability results are available 3 Notably, these results guarantee identifiability up to a permutation of the latent class
assignment (-) : [K] + [K]. That is, there exists some permutation (+)) [K] + [y] such that
(k),a)() 22 7/k (zz t), (oy(k)) (k) RJ Wk. where ? and & are the estimated density functions and weights. However, does not entail identifiability of the counterfactuals in the sense of definition [] Indeed,
the action step of the counterfactual strategy from Section 2.1 requires consistent permutation o across the whole domain (A' T) in order to reuse the inferred class assignments Üz at a specific point (X = x, T 1) to predict the counterfactual at another point (X x; = 1') - with a different treatment assignment. Nevertheless, using the assumptions from the previous section, we can still ensure the identifiability of the counterfactuals as the following result confirms:
Result 3.1 (Identifiabilty of Counterfactuals with Categorical Background Variables), Let X, T and Y be continuous random variables generated according to the graphical model of Figure with the domain of X and T being connected. Let W.(...) be the first Wasserstein distance on P(Y), ve (X, (X,Y,
Y. T) the probability distribution of Yp | S,YYT and Dy (X, Y,T) T) its estimator from N
observed data points. If Assumptions 7 and 3 hold, for each (x. f), the counterfactual distribution W -identifiable expectation at threshold d:
lim Ey Eyaz(z,) (,), (x, 8
N
In the special case when the noise response is additive, we have
lim W (X,,,,
N -
The proof is given in Appendix This result gives bound on the distance between the inferred and true counterfactual distributions in the asymptotic regime. Importantly, it does not restrict the dimension of X' and V, and is thus valid on challenging data modalities such time series or images.
Continuity of distribution and complexity The result above holds asymptotically in the number of available samples. In the additive Gaussian case, the sample complexity for learning a K-mixture model with Y E IRd within total variation distance is Ö(K d /e) 4. Fortunately, the continuity assumption (Assumption 2 saves us from having to learn an individual mixture at each point (X xz T t), by jointly learning the continuous moments functions μl, (X) A better sample complexity bound can then be derived with further assumptions on μ (X,T).
3.2 CFQP: CounterFactual Query Prediction
Equipped with those theoretical results, we introduce CFQP, counterfactual prediction model based on a neural Expectation- Maximiiaioiiiiiiiioonn mechanism. The basic building block of CFQP is base-model m(x, t). that predicts the treatment response y based on covariates and treatment assignment. For each latent category we learn base-moder that approximates the individual treatment response in that category: m& t) z f4 (Ir, t). Our theoretical results require the true number of classes of Ko to be known in advance. Yet, this is rarely the case in practice, and we describe our architecture for an arbitrary number of classes K. The learning of the base-models follows three steps: joint initialization, an expectation phase, and a maximization phase. The overall process is depicted in Figure We also present a pseudo-code description of the procedure in Algorithm]]
5

---

Initialization
Learning individual models
mi (x,
Y
(i f
:
mo (at 1)
Clustering
mk (x,
mg(a,8)
every Appochs
Figure 2: Training procedure of CFQP. We first initialize common model my. Second, we cluster the different available data points into K clusters. We then use this clustering to train K individual models (Maximization step). The clustering is updated every A epochs (Expectation step).
Initialization We first train single common base-model my all data points X, T and Y. Because discards the variability of the hidden variable Uz. this model will approximate the conditional average treatment response (z E X x. T T=
Expectation - Clustering The clustering step ensures that each base-model my is trained with the data from the corresponding latent category. We distinguish between an initial clustering and an update clustering stage.
The initial clustering assignment happens after the initialization step has converged. We then the clusterability assumption (A [Assumtton 3 to drive the assignment and use K-means clustering based on the residuals |!y (1, 2)
The update clustering step happens at regular intervals to improve the quality of the cluster assignment the performance of each individual base-model increases. A point (z, y, t) is assigned to cluster arg minlly m2(2, I).
Maximization Learning individual base-models Once the dataset D = (X, T. Y) is clustered K groups, we train the individual models my on the corresponding cluster. This corresponds the maximization step of the aupectation-maximzznn scheme. Every A epochs, we update the cluster assignment until convergence.
Counterfactual prediction For data point (x.,ty). first infer the cluster assignment k: arg miglinlly (x, We predict the counterfactual for treatment t' y mx( t').
Related Work
Causal perspectives in machine learning have gained significant traction in the past years 26
Among them, one distinguishes between causal discovery approaches, aiming at discovering the causal relations between variables 100 9 and causal inference, aiming building treatment effects estimators from data [18]. Our work belongs to the latter. A common way to estimate counterfactuals
posit a specific structural causal model 220 its specific functional form [5), therefore ensuring identifiability. Synthetic controls are an example of this strategy, assuming an underlying linear structure [. However, these approaches are by definition restrictive terms of the expressivity of the structural equations. Deep learning approaches for counterfactual estimation have been recently proposed to address this issue 24 but without identifiability guarantee. For the discrete case, assumptions have been proposed to bridge this gap, such monotonicity generalization

---

Algorithm 1: CFQP Training
Data: X, X,Y,T, the number of latent clusters K, number of epochs eman, the update- peeeeeperood A. Result: list K hidden models my.
Initialize single base model mg at random.
for epoch + 0 to Cmaa do
Train My minimizing Lu (mo((, Y) end
Compute residuals r r= (X, T Y). Assign the residuals of each training sample into K' clusters (initial). Initialize each m2 with mo. for epoch +-
Emar do for i <- 0 to A do
Train my minimizing Cs = E [mm (X2, Tk) - Ya) for each cluster.
end
Compute residuals {(m, (x, T) Y) k 1.... K}.
Assign the residuals of each training sample into K clusters (update).
end
Return trained models my.
thereof 16 13
Nevertheless, because they focus addressing the discrete case, they are not directly applicable to high-dimensional data such as time series, which a motivation for our work.
Our work builds upon the literature on the identifiability of finite mixture distributions where guarantees have been achieved in the Gaussian g and non-parametric cases 3 among others.
Experiments
5.1
Datasets
Treatment outcomes are often complex and high-dimensional. We thus evaluate our proposed model architecture high-dimensional data modalities: time series and images. As counterfactual inference evaluation on real-world data a complex and ongoing research area, we use synthetic datasets inspired by case studies from the literature. Details about the data generation are given in Appendix
5.1.1 Counterfactual Image Transformation
To explore the capacity of our model to operate on images, we generate dataset inspired by [17] building upon modification MNIST images. Covariates X are original MNIST images and the treatment T is spatial rotation applied to the image. Uz is a coloring of the image and the treatment outcome Y the resulting colored and rotated MNIST image. Un either an additive Gaussian noise at each pixel Gaussian blur with random kernel.
5.1.2 Counterfactual Longitudinal Treatment Effect Prediction
Next, we explore the performance our model on time series data. As per our motivational example, goal is here infer counterfactuals from clinical trajectories with hidden patient class. X and are multi-dimensional time series, Uz is latent patient group and T a continuous treatment assignment. We used two different datasets: harmonic oscillator and cardiovascular model.
Harmonic oscillator. We simulate the angular positions of two coupled harmonic oscillators. The treatment consists of applying different gradual offsets to the time series. We consider three hidden groups (K, 3) that modulate the treatment response differently.
Cardiovascular simulator. We use a simulator of the cardiovascular system proposed 29 to predict the impact of fluid intake on blood pressure. Fluids are commonly administered in intensive care treating severe hypotension. However, the individual patient response is difficult to assess priori, it depends clinically hidden variables. We consider here two hidden groups of patients

---

with distinct responses to fluid intake. X and Y contain the arterial and venous blood pressure time series before and after fluid intake. The treatment the amount fluid injected. We add process noise by either considering an additive Gaussian noise on the observed treatment response or by introducing noise in the fluid injection process, leading to a non-linear and complex perturbation.
5.2 Baselines
We compare our approach against multiple counterfactual inference models from the literature:
Diff-SCM [24], recently introduced diffusion based counterfactual model; Deep-SCM 177. deep counterfactual inference architecture building on normalizing flows Synthetic Controls (SC) M. well known linear method for counterfactuals estimation and Deep-ITE, deep individual treatment effect estimator such as proposed in 27 using direct predictions from X and T.
Table 1: Test MSE of the counterfactual reconstructions for the different datasets.
Additive Noise
Non Additive Noise
Model
Harmonic Oscillator
colored-MNIST
Cardiovascular
Harmonic Oscillator
colored-MNIST
Cardiovascular
Deep-IIE
0.001
0.174
0.017
0.321
SC 0
0.177 + 0.131
0.020 0.0001
1.610 - 0.141
0.167 0.004
0.020 0.001
1.628 0.114
Deep-SCM 17
0.124 = 0.005
0.011 0.001
0.405
0.042
0.123 +: 0.005
0.011 = 0.001
0.121
0.012
Diff-SCM
0.082 0.023
0.008 + 0.004
0.036
0.106
0.009
0,311 0.073
CTQP (ours)
0.013 1. 0.001
0.001 1 0.001
0.077
0.050
0.009
0.001
0.002 0.001
0.188 1 0.114
5.3 Counterfactuals Prediction
We evaluate our approach in terms of the qual- ity of the counterfactual reconstructions. We generate counterfactuals (t' t') for each observa- tion (I.y,t) in the test set, resulting in a tuple (I.Y,t,t' yy).. We evaluate the MSE between the true counterfactual y' and the estimated one ÿ' as MSE, = 1/vl 5'112. We set the number of clus- ters K as an hyper-parameter and select the value that result in lowest validation error. The results are presented in Table [ We use additive and non- additive noise variants for each dataset. For each dataset and each variant, we see that our approach outperforms the other baselines, demonstrating its experimental effectiveness.
True
8
COFF
8
6
g
g
8
8
8
g
8
8
T-20
T-30
T=40
In Figure also report examples of counter- factual reconstructions on the image dataset. Each row corresponds counterfactual estimations for a particular method (top is the truth), and each col- umn represents a different treatment assignment T" (the left column is the factual T). Because it cannot model the categorical latent variable. the Deep-ITE model gets the expected individual treat- ment effect right but fails accurately estimate the counterfactuals. For Diff- SCM, we observe that, despite providing an accurate reconstruction for most of the treatment assignments, the lack of guarantees on the deep structural equation model leads to potential incorrect reconstructions (e.g. at T' 30). To assess the quality of the counterfactual images reconstructions, we also evaluate the structural similarity index and report the results in Appendix A.I.I We observe that CFQP outperforms all basclines on that metric.
Figure 3: Example of reconstructed counterfac- tuals on the image dataset for different methods (best seen in color). Here, Uz corresponds the color of the digit and T to the rotation angle. X is the non-rotated and non-colored image while Yr is the colored and rotated image. First col- umn is the observed factual treatment outcome. Subsequent columns show the counterfactual re- constructions for different values of T", First row is the ground truth.

---

5.4 Robustness Analysis
We further complement our experimental section with a comprehensive robustness analysis. We first investigate the impact of misspecifying the number of latent classes (i.e. when Kg o K). We then investigate the impact of correlation between X and Z with increasing strength. Finally, we investigate the robustness of our approach to different noise responses.
5.4.1 Number of latent classes
We study the evolution of performance when the number of classes is misspecified (Ko * K). In Figure 4a| we show the reconstruction eπrоr on the validation set (i.e.. on the factual data) and the counterfactual reconstruction error of CFQP for different values of K on the image dataset with true number of latent classes Ku- = 6. The validation MSE is lowest for K = 6 and K = 7, which corresponds to optimal test reconstruction error. This hyper-parameter can thus easily be tuned by monitoring the treatment prediction performance on the factual data. In Appendix [.2] we further report the quantitative results for the other datasets function of the number the clusters K.
5.4.2 Correlation between X and Uz
Our approach holds even when the latent background variable Uz and the covariates X are correlated. When this the case, information from X can be used to infer the value of Uz. In Figure 4b we study the impact of the correlation strength & between X and Uz. More details about the definitio of p are to be found Appendix B We compare the performance of CFQP with a Deep-ITE model. We observe that the counterfactual performance of our model remains constant, regardless of the correlation strength. However, as the correlation becomes more important, Uz becomes a deterministic outcome of X and Deep-ITE eventually converges to the performance of CFQP.
8.0175
CFGP
Deap-ITE
0.025
0.0150
0.0135
MSE
8.0100
0.010
9.0030
0.005
3,0000
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
0.0
0.2
0.4
0.6
0.8
1.0
Number of Clusters K
Confounding level
(a) Reconstruction MSE in function of the number of groups. The true number of groups is 6.
(b) Evolution of the reconstruction MSE in function of the correlation strength between X and Uz.
Figure 4: Robustness analysis for the number of latent classes (left) and correlation strength (right).
5.4.3 Non-additive Noise Responses
Table 2: Fluids PEHE
Our model architccture is motivated by the additive noise case. Yet, our theoretical results hold for arbitrary distributions. In Table I we reports the performance of the different models under non-additive noise responses. We observe an overall small degradation of performance of all methods but CFQP still outperforms baselines, as predicted by our identifiability results.
Model
Cardiovuscular PEHE
Deep-ITE
12
0.594 + 0.160
SC 0I
0.258 +:: 0.016
Diff-SCM 24
0.377 + 0.049
CFQP (ours)
0.143 + 0.108
5.5 Retrospective Individualized Treatment Effect Estimation
We explore the ability of our model to infer individualized treatment effects retrospectively. Based on observed treatment outcomes, we predict the difference in treatment response between two treatment

---

regimes for single individual. The metric of interest is the Precision in Estimation of Heterogeneous Effects PEHE [ For two treatment regimes t' and t", we write
PEHE(L, - --VEEXX (Yr- Yv) (Y, - Yx)) X, XXY,T
(5)
In Table we report PEHB(LL = 0.5, 1" 0.8) for test patients in the Cardiovascular dataset with different fluid intake. We observe that the methods leveraging information about treatment outcomes, CFQP, SC and Diff-SCM outperform the classical individual treatment effect estimators (Deep-ITE).
5.5.1 Impact of the clustering algorithm
Different clustering Our approach relies on an initial clustering step as described in Section 222
strategies can considered for this step. In Appendix K-means and Gaussian mixture models. Only minor deviations are observed in terms of performance between both approaches.
[..c [35] we investigate the difference between
Conclusion
Estimating counterfactuals from observational studies is of the most challenging tasks in causal inference. In this work, we proposed a set of reasonable assumptions that allow computing counterfac- tuals on high-dimensional data while harnessing the power of modern machine learning architectures. Based on these assumptions, we derived a new counterfactual model and demonstrated favorable experimental performance. In particular, ual treatment effects a posteriori in clinical patient trajectories. Nevertheless, the set of assumptions proposed here is not unique, and depending on the specific applications, others might be deemed more relevant. Indeed, the space of assumptions that allow to approximately recover counterfactuals with controlled level of error is potentially very rich research direction. Other model architectures compatible with our set of assumptions are also possible.
showed that this approach could be used to infer individ-
Potential negative societal impacts Counterfactual inference has been used in clinical settings and for assessing the impact of public policies [2], among others. However, as shown in this paper, its correctness hinges on assumptions whose validity is not always met, potentially leading misleading conclusions.
Edward is funded by FWO-SB grant. Edward would also like to thank Zeshan Hussain, David Sontag and Adam Arany for their comments preliminary versions of this manuscript.
Acknowledgements
10

---

References
[1] A. Abadie. Using synthetic controls: Feasibility, data requirements, and methodological aspects. Journal of Economic Literature, 59(2):391 -455,, 2021.
[2] A. Abadie, A. Diamond, and Hainmueller. Synthetic control methods for comparative case studies: Estimating the effect of california's tobacco control program. Journal of the American statistical Association, 105(490):493-50, 2010.
[3] B. Aragam, C. Dan, E. P. Xing, and P. Ravikumar. Identifiability of nonparametric mixture models and bayes optimal clustering. The Annals of Statistics, 48(4):2277-2302, 2020.
[4] H. Ashtiani, S. Ben-David, N. Harvey, C. Liaw, A. Mchrabian, and Y. Plan. Nearly tight sample complexity bounds for learning mixtures of gaussians via sample compression schemes. Advances in Neural Information Processing Systems, 31, 2018.
[5] A. Balke and J. Pearl. Counterfactuals and policy analysis in structural models. arXiv preprint arXiv: 1302. 4929, 2013.
[6] E. D. Brouwer, J. González, and S. Hyland. Predicting the impact of treatments over time with uncertainty aware neural differential equations. In AISTATS 2022, February 2022.
[7] R. M. Byrne. Counterfactual thought. Annual review of psychology, 67:135-157, 2016.
[8] S. Dasgupta. Learning mixtures of gaussians. In 40th Annual Symposium on Foundations of Computer Science (Cat. No. 99CB37039), pages 634-644. IEEE, 1999.
[9] E. De Brouwer, A. Arany, Simm, and Moreau. Latent convergent cross mapping. In International Conference Learning Representations, 2020.
[10] C. Glymour, Zhang, and Spirtes. Review of causal discovery methods based on graphical models. Frontiers in genetics, 10:524, 2019.
[11] J. I.. Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1)21172240, 2011.
[12] F. Johansson, U. Shalit, and D. Sontag. Learning representations for counterfactual inference.
In International conference on machine learning, pages 9020-3029. PMLR, 2016.
[13] G. Lorberbom, D. D. Johnson, C. J. Maddison, D. Tarlow, and T. Hazan. Learning generalized gumbel-max causal mechanisms. Advances in Neural Information Processing Systems, 34:
2021.
[14] X. Monnet, P. E. Marik, and J.-L. Teboul. Prediction of fluid responsiveness: an update. Annals of intensive care, 6(1):1-11, 2016.
[15] T.-L. Nguyen, G. S. Collins, P. Landais, and Y. Le Manach. Counterfactual clinical prediction models could help to infer individualized treatment effects in randomized controlled trials an illustration with the international stroke trial. Journal of clinical epidemiology. 125:47-56. 2020.
[16] M. Oberst and D. Sontag, Counterfactual off-policy evaluation with gumbel-max structural causal models. In International Conference Machine Learning, pages 4881-4890 PMLR, 2019.
[17] N. Pawlowski, D. Coelho de Castro, and B. Glocker. Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33:857-869, 2020.
[18] Pearl. Statisties and causal inference: review. Test, 12(2):281-345, 2003.
[19] J. Pearl. Causal inference in statistics: An overview. Statistics surveys, 3:96-146, 2009.
[20] J. Pearl. 7.1 causal and counterfactual inference. The Handbook of Rationality, page 427, 2021.

---

[21] J. Pearl et al. Models, reasoning and inference. Cambridge, UK: CombridgeUniveesssyyrrsss 19:2, 2000.
[22] N. J. Roese and M. Olson. What might have been: The social psychology of counterfactual thinking. Psychology Press, 2014.
[23] D. B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5);688, 1974.
[24] Sanchez and S. A. Tsaftaris. Diffusion causal models for counterfactual estimation. In First Conference on Causal Learning and Reasoning, 2021.
[25] B. Schölkopf. Causality for machine learning. In Probabilistic and Causal Inference: The Works of Judea Pearl, pages 765- 8044004.22 2022.
[26] B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio. Toward causal representation learning. Proceedings of the IEEE, (09(5):11--634, 2021.
[27] U. Shalit, F. D. Johansson, and D. Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference Machine Learning, pages 0076-3085. PMLR, 2017.
[28] W. Starr. Counterfactuals. In E. N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Summer 2021 edition, 2021.
[29] Zenker, Rubin, and G. Clermont. From inverse problems in mathematical physiology to quantitative differential diagnoses. PLOS Comput Biol, 3(1 1):4204, 2007.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] The assumptions are cleary
defined in section 2 The conclusion also relfects on the limitations of our work.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] Last paragraph in the conclusion discusses the potential negative societal impacts of coun- terfactual inference. The main risk we identify is blind acceptance of the assumptions required for correct inference, which can lead to misleading conclusions in assessing treatment effects as posteriori.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] Our paper conforms to the ethics review guidlines.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] The full set of assumptions is given in Section 2.3
(b) Did you include complete proofs of all theoretical results? [Yes] The proofs of our results given in Section of the Appendix.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material as URL)? Yes] The code is available at the following url: [https: : /tt.////ooomuuus 4open. scw.nce/rr/ccqpp
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] We provide the training details in Section B of the Appendix
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] Error bars are reported from running the experiments times (5 folds)
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] The total amount of compute and the type of resources used is available in the Appendix Section B
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] The authors of the code bases used for implementing the baselines are credited both in the code base and cited in the text. The list of software packages used is available at https //anonymous. 4open. science/r haticee///qq/////e hat.a//lleesssssee txt
(b) Did you mention the license of the assets? [Yes] The licenses of all software packages used in our experiments is available at https::// //nooymouss 4open science/r) cffp/licenses. txt
(c) Did you include any new assets either the supplemental material or as URL? [Yes] We include an anonymized repository containing the code used for the experiments. It available at lattps://anonynin
4open. science/r/cfqp)
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A] We are not using any personal data.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] None of our datasets contains personal data.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] No research with human subjects
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, applicable? [N/A] No research with human subjects
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A] No research with human subjects
13


Reviews:
[{"review_id": "ztV1sKEJqK", "reviewer": "Reviewer_dSw5", "summary": "The authors tackle the limitation of pre-existing neural network-based structual equation methods which only enable to model the factual distribution, not counterfactual, such that it often leads to incomplete ounterfactual distributions. The authors focus on the counterfactual inference of treatment effects on patient trjectories in healthcare setting. They validate the proposed model on three image and time-series datasets and the results show the model outperforms its beaseline in accuracy of counterfactual estimation.", "questions": "Q1) Based on the experimental results (Figure 3) with the authors' argument, it's a bit hard to understand that the lack of guarantees on pre-existing deep structural equation models tend to generate potential incorrect reconstructions. The shown results are relatively weak to support their argument. Can authors show more convincing resutls on that?\n\nQ2) The authors need to describe how their assumption on strong ignorability works on time-series data. The authors said they perform counterfactual estimation of treatment on patient's trajectory in which the observed variable Y can be time-wisely influenced by multiple treatments or other confounding factors. Considering the fact that the patient in a hospitcal (a general ward/ICU) usually recieves multiple treatments, I wonder how the authors handle this problem.\n\nFor improvement, the authors need to describe how they address counterfactual estimation on time-series data setting.\n\nQ3) It's hard to see the benefit of spliting Uε to a categorical variables and continuous variables. Can authors describe more on that in detail?\n\nQ4) In the experiment on cardiovascular simulator, what kind of dataset does the authors use? They should describe more explanations about the dataset and explain  how they preprocess data when handling clinical records (such as time points for fluid intake, length of sequence) and how a training set of patients are selected from the entire clinical database (e.g., the selected patients show normal range of blood pressure? or what types of other clinical events do these patients have in their EHRs?). Also, more detailed explanations about qualitative analysis on cardiovascular experiments will be helpful for showing the effectiveness of their method. For example, how clinical meanful is the estimated patient trajectory from counterfactuals?", "limitations": "The authors addressed the limitations.\n", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The tackled goal is an interesting topic in the field of counterfactual inference and estimation. The proposed methodology is well described and the quality of writings is good.\n\nHowever, it's a bit hard to understand the benefit of proposed methodology and, although interesting, it’s hard to say the introduced technique is quite new as the proposed method seems to be a simple extension from pre-existing method (structural causal model). Also, the paper lacks detailed explanation and proof for their argument that pre-existing structural causal equation models fail to provide theoretical guarantees for reconstruction of counterfacturals. Although the authors show some qualitative result (Figure 3) for supporting their argument, it's not quite convincing. It will be helpful if the authors are able to provide more results and explanations for this.\n\nFurthermore, even though the authors argue that the proposed method provides a counterfactual reconstruction architecture on images and time series, the experimental results do not seem to be good enough to show benefits of their method. Especially for time-series experiment, more diverse qualitative and quantitative experiments should be performed.\n\n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "oukpWu6ruhX", "reviewer": "Reviewer_Md9L", "summary": "This paper proposes a counter-factual model with the assumption that the traditional exogenous variable is decomposed into independent categorical and continuous variable.\nHere, only the categorical variable is governed by top-most exogenous variable, and due to the nature of categorical selection, the authors proposed clustering-based two-step counter-factual prediction model.\n\n", "questions": "- Is there any difference on the performance when using GMM instead of k-means clustering? \n- Could you explain about Figure 3 more precisely? For example, the digit in the second row & the third column is drawn by intervening the color, am I correct? What is $T'$? Considering Figure 1, $U_z$ and $T$ are not linked. However, it seems that other factors, such as rotation (and probably also the noisy-ness, but not perfectly sure due to the low-resolution of MNIST data itself), have changed after intervening the color of digit in Figure 3, which is not desirable. What causes such phenomenon? Probably, the size of $T'$?\n- $U_\\eta$ can be some other factors. For example, let $T$ be some other underlying natures of digit data (ex. thickness, slope), and set $U_\\eta$ as rotation (since it is enforced at data generation stage). I think it also can be one example of the causality. (Please let me know if I'm missing something.) Will the intervention cause the change of rotation in this case? What was the intuition behind of setting $U_\\eta$ as noise?\n- With the reconstructed counter-factual images, how does the digit classification result change? (with simple classifier)", "limitations": "The authors have discussed the limitations and potential negative societal impact in the main paper.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- The paper is well-written and well-organized. \n- Also, the quantitative results seems promising.\n- The experiment is conducted two simple datasets and one real-world dataset. I would recommend conducting counter-factual image generation task on some other image dataset, perhaps CelebA, which can directly visualize and compare the effect and performance of proposed method against baselines.\n- Setting $U_\\eta$ as noise is a little bit unnatural. Regarding this, see questions below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "i8yRhKaZyC1", "reviewer": "Reviewer_Ddey", "summary": "The paper deals with the problem of counterfactual identification and estimation in treatment effect estimation problems from observational datasets. The task of counterfactual identification is challenging as we need access to the structural equations, which is a strong assumption in typical real-world datasets. The authors tackle these issues by making the assumption of discrete/categorical latent variables that affect the final outcome distribution and provide novel identification results under the ($\\rho$, $\\delta$) definition. They propose the approach CFQP, which takes inspiration from the discrete latent assumption and trains a mixture model using expectation maximization. They show the gains of the proposed approach against the existing works in the literature on semi-synthetic datasets; MNIST, harmonic oscillator, and cardiovascular simulator. ", "questions": "-  How does the proposed approach perform when there is a mismatch in the total number of latent categories for the harmonic oscillator and the cardiovascular dataset? The authors only show the effect of error in modeling the total latent categories for the MNIST dataset. Further, the authors should report results for the approach CFQP when they tune the total latent categories ($\\hat{K}$) using validation data (instead of selecting $\\hat{K}=K$) and include the same across different settings in Table 1. \n\n- A standard way to define counterfactual identification (https://causalai.net/r60.pdf) would be as follows: 'if two different SCM $M_1, M_2$ have the same causal graph and generate same observational/interventional distributions, then a counterfactual query Q is identifiable iff $M_1(Q) = M_2(Q)$'. How does the ($\\rho$, $\\delta$) definition considered by the authors compare with it? Does the identification result in the paper imply there can still be some SCMs that are not distinguishable from each other considering the proposed assumptions, but they yield different results on the counterfactual query?", "limitations": "I think the authors have adequately addressed the limitation and potential negative societal impact of their work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- Given that we would never observe structural causal models (SCM) for a majority of realistic datasets, the task of counterfactual identification without access to SCM is an extremely relevant problem. The paper provides identification results towards the same under the discrete latent variable assumption, which is novel to the best of my knowledge.\n\n- The identification results are significant as the assumptions introduced in the paper for counterfactual identification are practically motivated. Many practical scenarios can have discrete latent variables affecting the outcome distribution, and additionally, there are no restrictions in terms of the dimensionality of treatment and outcome.\n\n- The proposed approach (CFQP) has simplicity in the sense of being inspired from mixture models, and obtains substantial improvements over the prior approaches.\n\n- The paper is overall well written with good explanations for all the assumptions and nice presentations of the empirical results.\n\n**Weaknesses**\n\n- The proposed approach (CFQP) required the knowledge of the true number of latent categories (K). While the authors present ablation studies when there is a mismatch with the true total latent categories, further empirical analysis might be needed. \n\n- The identification result holds for the ($\\rho$, $\\delta$) definition stated in the paper, which might not lead to perfect identification.\n\nPlease refer to the question section below for further details.\n\nNote: I do think the paper makes good contributions and I am happy to raise my score if the authors answer my concerns with the evaluation studies.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2AEQkSqhh_R", "reviewer": "Reviewer_PhWG", "summary": "The paper proposes a series of assumptions that permit identifiability in cases with categorical background variables. It further proposes CFQP a neural net based algorithm for counterfactual estimation based on the aforementioned assumptions and an EM method", "questions": "- What is the justification of gaussian assumptions and not something like $g(u^{*}_{\\eta})$ where is a learnable function and u* an approximation \n- What is the approximate complexity of the proposed NN method? How does the it scale as the number of treatments, outcomes and categories scale ? \n- What is the accuracy of the method in the cases where the ground truth is known ? in the image scenario what is the SSIM ? ", "limitations": "The authors have explicitly stated their assumptions and provided an ethical discussion", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Originality : the paper appears to provide a novel and original methodology both in terms of identifiability results and NN methodology\n- Clarity : adequate clarity\n- Significance: Potential for impact in the wider causal ML community\n- Related works : Not enough discussion of related works and how this work relates to it\n- Experimentation: Only MSE and some images are shown - no accuracy for counterfactuals where ground truth exists - synthetic ones \n- Methodology: good but assumption of Gaussian distributions is not supported. See questions for more\n\nDisclaimer: I did not check the validity of the proofs ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept