Learning Disentangled Representations for CounterFactual Regression
Authors: Negar Hassanpour
Russell Greiner
Venue:  (2020)
Keywords: Counterfactual Regression
Causal Effect Estimation
Selection Bias
Off-policy Learning

Abstract: We consider the challenge of estimating treatment effects from observational data; and point out that, in general, only some factors based on the observed covariates X contribute to selection of the treatment T, and only some to determining the outcomes Y. We model this by considering three underlying sources of {X, T, Y} and show that explicitly modeling these sources offers great insight to guide designing models that better handle selection bias. This paper is an attempt to conceptualize this line of thought and provide a path to explore it further.
In this work, we propose an algorithm to (1) identify disentangled representations of the above-mentioned underlying factors from any given observational dataset D and (2) leverage this knowledge to reduce, as well as account for, the negative impact of selection bias on estimating the treatment effects from D. Our empirical results show that the proposed method achieves state-of-the-art performance in both individual and population based evaluation measures.

Published as a conference paper at ICLR 2020
LEARNING DISENTANGLED REPRESENTATIONS FOR COUNTERFACTUAL REGRESSION
Negar Hassanpour & Russell Greiner Department of Computing Science University of Alberta Edmonton, Alberta, T6G 2F8, CANADA (hassanpo, rgreiner) Qualberta.ca
ABSTRACT
We consider the challenge of estimating treatment effects from observational data; and point out that, in general, only some factors based on the observed covariates contribute to selection the treatment T, and only some to determining the outcomes Y. We model this by considering three underlying sources of { X, T, Y and show that explicitly modeling these sources offers great insight to guide designing models that better handle selection bias in observational datasets. This paper an attempt to conceptualize this line of thought and provide a path to explore it further.
In this work, we propose an algorithm to (1) identify disentangled representations the above-mentioned underlying factors from any given observational dataset and (2) leverage this knowledge to reduce, as well as account for, the negative impact of selection bias on estimating the treatment effects from D. Our empirical results show that the proposed method achieves ofte-oo-theean art performance in both individual and population based evaluation measures.
INTRODUCTION
As we rely more and more on artificial intelligence (AI) to automate the decision making processes, accurately estimating the causal effects of taking different actions gains an essential role. prominent example is precision medicine i.e., the customization of health-care tailored to each individual patient which attempts to identify which medical procedure E T will benefit a certain patient x the most, in terms of the treatment outcome E R. Learning such models requires answering counterfactual questions (Rubin) 1974] Pearl 2009) such as: "Would this patient have lived longer [and by how much], had she received an alternative treatment?".
For notation: dataset D [xs,, t, M4] J used for treatment effect estimation has the following format: for the ith instance (e.g., patient), we have some context information
X C R (e.g., age. BMI, blood work, etc.), the administered treatment t, chosen from a set of treatment options T (e.g., (0: medication, 1: surgery)), }), and the respective observed outcome y4 y (e.g., survival time; y S 1 as a result of receiving treatment fę. Note that D only contains the outcome of the administered treatment (aka observed outcome: 3/1), but not the outcome(s) of the alternative treatment(s) (aka counterfactual outcome(S): y for & E TT) (t,)), which are inherently unobservable. ti.
For the binary-treatment case, we denote the alternative treatment as -t;
Pearl 2009) demonstrates that, in general, causal relationships can only be learned by experimenta- tion (on-line exploration), or running Randomized Controlled Trial (RCT), where the treatment assignment does not depend on the individual X see Figure (a) In many cases, however, this is expensive, unethical, even infeasible. Here, we are foreed approximate treatment effects from off-line datasets collected through Observational Studies. In such datasets, the administered treatment depends on some all attributes of individual see Figure ((b) Here, as Pr( T X ) o Pr( T)),
we say these datasets exhibit selection bias (Imbens & Rubin 2015). Figure illustrates selection bias an example (synthetic) observational dataset.

---

Published as a conference paper at ICLR 2020
features
features
T treatment
y9 outcome
Y1 autcome
T treatment
yo outcome
y1 natcome
(a) Randomized Controlled Trial
(b) Observational Study
Figure I: Belief structure for randomized controlled trials and observational studies. Here, yd (Y') is the outcome of applying T= treatment#0 (#1) to the individual represented by X.
Here, we want to accurately estimate the Individual Treatment Effect (ITE) for each instance i- i.e. to estimate e; = y/- v. We frame the solution learning the function X T + y that can accurately predict the outcomes (both observed y1* 4t as well as counterfactuals f/s "h, given the context information z, for each individual. As mentioned earlier, there are two challenges associated with estimating treatment effects:
(i) The fact that counterfactual outcomes are unobservable (i.e., not present in any training data) makes estimating treatment effects more difficult than the generalization problem in the supervised learning paradigm. This is an inherent characteristic of this task.
(ii) Selection bias in observational datasets implies having fewer instances within each treatment arm at specific regions of the domain. This sparsity, in turn, would decrease the accuracy and confidence of predicting counterfactuals at those regions.
This paper addresses the second challenge by in- vestigating the root causes of selection bias, by dis- secting and identifying the underlying factors that can generate an observational dataset D, and lever- aging this knowledge reduce, as well as account for, the negative impact of selection bias on esti- mating the treatment effects from D. this work,
(e.gg. survival time)
[=1 (e.g., surgery) (e.g..
-t=1 [if got surgery)
-0=0 (if got medication)
we borrow ideas from the representation learning literature (Bengio et al., 2013) in order to reduce selection bias and from the domain adaptation lit-
(e.g., age)
erature (Shimodaira) (2000) in order to account for the remainder selection bias that (might) still exist after its reduction.
Figure 2: An example observational dataset. Here, treat heart disease, doctor typically prescribes surgery (t=1) 1) to younger patients (+) and medication (t 0) older ones (+). Note that instances with larger (resp., smaller) I values have higher chance be assigned to the t = (resp., 1) treatment arm; hence we have selection bias. The counterfactual out- comes (only used for evaluation purpose) are illustrated by small
Our analysis relies the following assumptions: Assumption 1: Unconfoundedness (Rosenbaum)
& Rubin 1983) There are no unobserved
founders (i.e., covariates that contribute to both treatment selection procedure as well determina- tion of outcomes). Formally, { teT 1 T X. Assumption 2: Overlap (Imbens) 2004)- Every individual should have non-zero chance of be- ing assigned to any treatment arm. That is,
(+) for -/= (0).
Pr( [T===|XX = + 0 VI E T, Vr E X. These two assumptions together are called strong ignorability (Rosenbaum & Rubin 1983). Imbens & Wooldridge (2099) showed that strong ignorability is sufficient for ITE to be identifiable.
Without loss of generality, we assume that the random variable X follows a(n unknown) joint prob- ability distribution Pr( X Гr A, T), treatment T follows Pr(T]D r, A), and outcome y" follows (Y" Y. where П, A, and Y represent the three underlying factors that generate obser-
Examples for: (1') rich patients receiving the expensive treatment while poor patients receiving the cheap one although outcomes of the possible treatments are not particularly dependent on patients' wealth status: (A) younger patients receiving surgery while older patients receiving medication; and (T) genetic information that determines the efficacy of medication, however, such relationship is unknown the attending physician.
2

---

Published as a conference paper at ICLR 2020
vational dataset D. The respective graphical model is illustrated in Figure Conforming with the statements above, note that the graphical model also suggests that selection bias is induced by factors I and A, where A represents the confounding factors between T and Y.
We argue that ex- plicit identification of the underlying factors (T,A,T) } in observational datasets offers great insight to guide designing models that better handle selection bias and consequently achieve better performance in terms of estimating ITEs. In this paper, we propose a model, named Dis- entangled Representations for CounterFactual Regression (DR-CFR), that is optimized to do exactly that. We also present experiments that demonstrate the advantages of this perspective; and show empirically that the proposed method outperforms tftte-of-the-ar models in variety of data generation scenarios with different di- mensionality of factors; see below.
Main contribution:
T
A
T
T
YO
YI
Underlying factors of X; r (T) are factors that partially determine only T (Y) but not the other random variable: and A are confounders; Selection bias is induced by factors T and A.
Figure 3:
2 RELATED WORKS
Selection bias in observational datasets is equivalent to domain adaptation scenario where model is trained on a "source" (observed) data distribution, but should perform well on a "target" (counterfactual) one. Learning treatment effects from observational datasets is closely related to "off-policy learning from logged bandit feedback" - cf., (Swaminathan & Joachims) 2015a), whose goal is learning optimal policy that selects the best personalized treatment for each individual. A common statistical solution re- re-weighting certain data instances to balance the source and target distributions. The majority of re-weighting approaches belong to the Inverse Propensity Weighting (IPW) family of methods- - of., (Austin) 201 Bottou et al. 2013 Swaminathan & Joachims, 2013c). While IPW methods are unbiased, they suffer from high variance, Swaminathan Joachims (2015b) proposed the Counterfactual Risk Minimization (CRM) principle to alleviate this issue. In summary, re-weighting is an attempt to account for the selection bias.
Johansson et al. (2016) is among the pioneer works that explored ways to use techniques from representation learning (Bengio et al., 2013) to reduce the selection bias. (Shllit et al.) (2017) present refined version of (Johansson et al. 2216)'ss method that learns common representation space (x) = e by minimizing the discrepancy (Mansour et al. 2009) (hereinafter "disc") between the conditional distributions of à given t == versus Φ, given =1. That is,
disc ((#(2))) (0(r.))
(1)
which is (effectively) regularization term that attempts reduce selection bias in the learned representation. On top of this representation learning network, they trained two regression networks h(()) one for each treatment arm (1 E (0, 1)) that predict the outcomes.
Hassanpour & Greiner (2019) argued that the learned representation cannot and should not remove all the selection bias, as the confounders not only contribute choosing treatment but also to determining the respective outcomes As a result, where there are confounders (which is a common situation), even would exhibit some selection bias, although less than that the original domain They built on the work of (Shalit et al. 2017) by introducing context-aware importance sampling
2 While Hassanpour & Greiner 2019 presented graphical model similar to investigate the nature selection bias. N.b., they did not implement the idea of learning disentangled representations for counterfactual regression; instead, their method [like (Shalit et al. 2017)] learns a common representation e that can represent only the confounders, but not the other factors. Our approach extends theirs by providing algorithm that can learn disentangled representations of the underlying factors from observational datasets.
Figure 3 they only used
3

---

Published as a conference paper at ICLR 2020
weights, that attempt to account for the mentioned remainder selection bias. These weights
W, 1+ Pr( 0: ht,) Pr( Φ: (t)
Pr(4)) 1 x (4: 1+ Pr( Pr(()) (44 (1)
are designed to enhance performance of estimating both factual as well as counterfacual outcomes terms, respectively), where (t; (x) is the probability of assigning the observed 4 conditioned on the learned context d;-
(by the 1 and pr( p()
Note that both (Shalit et al. 2017 and (Hassanpour & Greiner 2019) use & to model the concatenation of factors A and (see Figure 3). Although it does make sense that there should be no discrepancy between conditional distributions of Y, the A factor should model the confounding factors, which by definition, must embed some information about treatment assignment. This would result in a positive discrepancy between conditional distributions of Á that should not be minimized. Thus, minimizing Equation with respect to & can lead to problematic results as it discards some of the confounders.
Yao et al. 2018)) proposed the Similarity preserved Individual Treatment Effect (SITE) method, which extends Shalit et al. (207))s framework by adding a local similarity preserving component. This component acts regularization term, that attempts retain the same neighhourhood relationships in the learned representation space as exhibited in the original space, by matching the propensity scores Pr( t= 1 x) and Pr( t | ) This, however, results learning sub-optimal representations when I' # as SITE tries to keep instances whose I's are far apart, also far apart in Φ. In other words, this component penalizes reducing selection bias in by not discarding the irrelevant information present in r even when it does not hurt the outcome estimation at all.
Our work has many similarities (Kuang et al.) 2017p, who decomposed X into subsets: con- founding and adjustment variables, which are similar to our A and Y factors respectively. They then used an optimization algorithm for identifying these variables, to ultimately find an unbiased estimate of the Average Treatment Effect (ATE). We extend their work in three ways: (i) In addition to confounders and adjustment variables, we also identify the factors that determine the treatment and have effect the outcome (i.e. I). (ii) Unlike (Kuang al., 2017) that take linear approach by tagging the raw features as either confounders or adjustment variables, our proposed method has the capacity to learn [non-lineat representations of the underlying factors. (iii) Our method facilitates estimating both ATE as well ITE, whereas (Kuang et al., 2017) cannot provide estimates of ITES.
LEARNING DISENTANGLED REPRESENTATIONS
We assume, without loss of generality, that any dataset of the form {X, T, Y is generated from three underlying factors { (r, A, Y as illustrated in Figure Observe that the factor I (resp., Y) partially determines only T (resp., Y), but not the other variables; and A includes the confounding factors between T and Y. This graphical model suggests that selection bias is induced by factors T and A, It also shows that the outcome depends on the factors A and Y. Inspired by this graphical model, our model architecture incorporates the following components:
Three representation learning networks; one for each underlying factor: I(x), A(x), and T(x).
Two regression networks; one for each treatment arm: 1 A (xr), ((x)) and h (A(x), T(x)),
Two logistic networks: To (t [xx), A(x)) to model the logging policy aka behaviour policy in Reinforcement Learning; cf., (Sutton & Barto 1998) and T A (2:)) to design weights that account for the confounders' impact.
3 Note that the assumption of unconfoundedness still holds; here is why:
Short: Observing either X A blocks the path from T to Y, which supports the unconfoundedness assumption. Long: Once the representation networks are learned from the observational data, we can compute the latent factors (1', A, Y from X only. Therefore, although these factors are not explicitly observed, they are effectively observed, in that they are derived directly from the observed X, and so should not be categorized as "unobserved confounders". For example, the latent factor for "zip code" in X is "socio-cconomic status" (perbaps in A). In other words, "sncio-cconomic status" be inferred from "zip code" which can be viewed proxy for it.
4

---

Published as a conference paper at ICLR 2020
We therefore try to minimize the following objective function:
N
J[, T, mo)
1
Σ (t., C[ ye, h² (A(x)) I(x)))
(3)
+ disc ((T()) [1) (o) (T(
(4)
+ 3 log To (4, [(x,), (c.)))
(5)
A Reg(T A, 1 /1, m))
(6)
where w/ (tt, (zx)) is the re-weighting function: L[ h² (2;). T (2;)) the prediction loss for observed outcomes (aka factual loss); (dscc(((())):aee)))_ given 1 versus given 1= log nO is the cross entropy loss of predicting the assigned treatments given the learned context; and Reg( is the regularization term for penalizing model complexity. The following sections elaborate on each of these terms.
calculates the discrepancy
between conditional distributions
3.1 FACTUAL Loss: C[ [v (x)),, [(x))]
Similar to (Johansson et al. 2016 Shalit et al.) 2017| Hassanpour & Greiner 2019 Yao et al. 2018 we train two regression networks htd and lt", one for each treatment arm. As guided by the graphical model in Figure the inputs these regression networks are the outputs of the & (xx and Y (ar) representation networks and their outputs the predicted outcomes for their respective treatments.
Note that the prediction loss can only be calculated on the observed outcomes (hence the name factual loss), as counterfactual outcomes are not available in any training set. This would be an L2-loss for real-valued outcomes and a log-loss for binary outcomes. By minimizing the factual loss, ensure that the union of the learned representations A(x) and (z) retain enough information needed for accurate estimation of the observed outcomes.
3.2 RE- GE-WEIGHTING FUNCTION: w (t,, A(x))
follow (Hassanpour & Greiner 2019)'s design for weights as re-stated in Equation 2 with the modification that we employ A to calculate the weights instead of d. Although following the same design, we anticipate our weights should perform better in practice than those in (Hassanpour & Greiner 2019) as: (i) confounders
discarded due minimizing the imbalance loss (because our disc is defined based on Y, not $); and (ii) only the legitimate confounders are used to derive that have not contributed to treatment selection (i.e., Y).
the weights (ie., A), not the
Notably, the weights design in Equation 2 is different from the common practice re-weighting techniques (e.g., IPW) in that the weights are calculated based on all factors that determine T (i.e., as well as A). However, we argue that incorporation of the weights might result in emphasizing the wrong instances. other words, since the factual loss is only sensitive factors A and Y, and not I, re-weighting L according to I' would yield wrong objective function to be optimized.
3.3 IMBALANCE Loss: disc (IT(z))) disc((T(z)))
(T(r)))
According to Figure 3
should be independent of 7 due to the collider structure at Y. Therefore,
TLT
Pr (((TT) = Pr(T)
Pr( T=0) = Pr( Pr(T|T=1)
(7)
We used Maximum Mean Discrepancy (MMD) (Gretton et al. 2012) to calculate dissimilarity between the two conditional distributions of Y given t= versus t= 1.
By minimizing the imbalance loss, we ensure that the learned factor Y embeds no information about T' and all the confounding factors are retained in A. Capturing all the confounders in A and only in is the hallmark of the proposed method, as we will use for optimal re- re-weighting of the factual loss term (next section). Note that this differs from Shalit al. (2017)'s approach in that they do not distinguish between the independent factors A and Y; and minmizzin the loss defined on only one factor & which might erroneously suggest discarding some of the confounders in A.
5

---

Published as a conference paper at ICLR 2020
3.4 CROSS ENTROPY LOss: - -log [ [t))
We model the logging policy logistic regression network parameterized by WW by as follows: To( t (b) = [1 + (2t- w-Wo+bo ']_ where và is the concatenation of matrices T and A. Minimizing the cross entropy loss enforces learning I' and in a way that allows To( (o( to predict the assigned treatments. In other words, the union of the learned representations of I and A retain enough information to recover the logging policy that guided the treatment assignments.
4 EXPERIMENTS
4.1 BENCHMARKS
Evaluating treatment effect estimation methods is problematic on real-world datasets since, as mentioned earlier, their counterfactual outcomes are inherently unobservable. A common solution to synthesize datasets where the outcomes of all possible treatments are available, then discard some outcomes to create a proper observational dataset with characteristics (such as selection bias) similar to real-world one cf.. (Beygelzimer & Langlord 2009; Hassanpour & Greiner 2018). In this work, we use two such benchmarks: our synthetic series of datasets as well as publicly available benchmark: the Infant Health and Development Program (IHDP) (Hill 2011).
4.1.1 SYNTHETIC DATASETS
We generated our synthetic datasets according to the following process, which takes as input the sample size N; dimensionalities [mr, ma, mr) E 2+(3); ; for each factor L E {T, A, T the means and covariance matrices (JLL Σz); and a scalar Ç that determines the slope of the logistic curve.
For each latent factor I E (T,A,T)
Form L by drawing N instances (each of size m2) from W(z,, EL), Concatenate Г, A, and Y to make the covariates matrix X [of size Nx Concatenate and to make V [of size (mp++ma)
Concatenate A and Y to make [of size N x (m& +mr]]
For treatment T:
Sample mm+ tuple of coefficients 0 from (0,
Define the logging policy as To( t= 2= I+exp(-Cxj* where =V
For each instance sample treatment t; from the Bernoulli distribution with parameter To (o(t=1
For outcomes Yd and y1:
-Sample m. +my tuple of coefficients yo and d from (0, m
Define yº = ( o o d 0.5) 10 / (m +m)) +e and y² = ($o 4) (00/((mmtmmm)))) +s, where E is a white noise sampled from (0, 0.1) and the symbol for element-wise (Hadamard/Schur) product.
We considered all the viable datasets mesh generated by mr, MA. my E {0. 4, 8}. This creates 24 scenarios" that consider all possible situations in terms of the relative sizes of the factors r, A, and Y. For each scenario, we synthesized five datasets with various initial random seeds.
4.1.2 INFANT HEALTH AND DEVELOPMENT PROGRAM (THDP)
The original RCT data was designed to evaluate the effect of specialist home visits on future cognitive test scores of premature infants. Hill (01T)) induced selection bias by removing a non-random subset of the treated population to create realistic observational dataset. The resulting dataset contains 747 instances (608 control, 139 treated) with 25 covariates. We run our experiments on the same
benchmark (100 realizations of outcomes) provided by and used in (Johansson al. 2016 Shalit
There are not 2 =27 scenarios because we removed the three tuples: (0. 00, (4.0, 0, and (8. 0. 0), as
any scenario with A 3=1 Y= would generate outcomes that pure

---

Published as a conference paper at ICLR 2020
X
X
Xxc
Xmg
Xm
Xmy+
r
r
Xme+ting
Xmy+ra
Xm
Xmy
Xm
(a) Slice weights matrix connects (the vari- ables X belonging to I') to the first layer of the representation network that attempts to identify r). The size of this slice is my K.
(b) Slice of the weights matrix that connects (the variables in X not belonging to I') to [the first layer of the representation network that attempts to iden- tify r). The size of this slice is (ma + K.
Figure 4: Visualization of slicing the learned weights matrix in the first layer of the representation network (number of neurons: K) for identifying I' (best viewed in color).
r
T
A
Y ~ -Y
4.4.4
4.4.4
4.4.4
0
0.00
0.07
0.07
4.4,8
B.05
BLOS
0,00
8:5
0.33
0,01
0,03
8.8.4
4_8.4
8.8.4
4.8.4
8,8.4
4.8.4
8,4.8
4,8.8
8,4_8
4.8.8
8,4,8
4,8.8
(a) Identification of
(b) Identification of A
(c) Identification of T
Figure 5: Radar charts that visualize the capability DR-CFR in identifying the underlying factors r, A, and T. Each vertex the polygons is identified with the factors' dimension sequence iitr_m__me of the associated synthetic dataset. The polygons' radii are scaled between 0: and quantify the average weights of the first slice (in dotted magenta) and the second slice (in eyan).
et al. 2017p. Outcomes of this semi-synthetic benchmark simulated according response surfaces provided in the -on-Parrmcttic Causal Inference (NPCI) package (Dorie 2016).
4.2 RESULTS AND DISCUSSIONS
4.2.1
EVALUATING IDENTIFICATION OF FACTORS (T,A,
First, we want to determine if the proposed method is able to identify the variables that belong to each underlying factor. To do so, look at the weight matrix in the first layer of each representation network, which is of size (mr-tma +mr) K, where K is the number of neurons in the first hidden layer of the respective representation network. For example, check if identified properly, we partition the weights matrix into two slices, shown in Figure 4 and calculate the average of each slice. The first slice [referred to as Sr; highlighted Figure 4(a) pertains '''s ground truth variables in X" and the second slice [S-r; Figure 4(b) pertains to "variables in X that do not belong to T Constructing Sa S-A. Sr. and S-r follow a similar procedure.
If the proposed method achieves good identification, then we expect the average of the absolute values weights in Sp should be higher than that S-r: this same claim should hold for (Sa. S-A) and (Sr. S-r) well. Note that only the relative relationships between the average weights in either of the slices matter; since this analysis aimed at checking whether, for example, for identifying I, its respective representation network has indeed learned emphasize "I's ground truth variables
7

---

Published as a conference paper at ICLR 2020
0_0_4
CFR
0_0_4
8_8_8
0.8
0
8_8_8
0.8
D_0_8
CFR-ISW
0_4_0
SITE
8_8_0
0_4_4
DR-CHR
8_8_8
0.4
0_8
8
0_4_8
8_4
0_8_0
8_4
0_8_0
8
0
840
0.8
8_0_8
0_8_8
808
0_8_8
8_0_4
4_0_4
8_0_4
4_0_4
4_8_8
4_0_6
4_8_6
1_0_8
4.8.4
4.4.0
4.8.4
4_4
4_8_0
4_4
4_4_
4_4_8
4_4_8
Figure Radar charts for visualizing the PEHE performance results on the synthetic datasets. Training sample size on the left chart is 2,500 and on the right chart is 10,000. Each vertex on the polygons is identified with the factors' dimension sequence (mr (mr_ma, ____mr)) of the associated group of datasets. The polygons' radii are scaled between 0 0.8 to quantify the PEHE values (i.e. the closer to the centre, the smaller the PEHE). The dashed purple curve illustrates the results of the proposed method.
in X more than the other variables in X. Figure illlsstrates the identification performance of DR-CER according to this analysis; showing empirically that the proposed method successfully identifies all the three underlying factors, for all synthetic datasets.
4.2.2 EVALUATING ESTIMATION OF TREATMENT EFFECTS
Given a synthetic dataset (that include both factual as well as counterfactual outcomes), one can evaluate treatment effect estimation methods with two types of performance measures:
Indivddual-based "Precision in Estimation of Heterogeneous Effect" PEHE = V N En (é:-e;)
where è; = 5/! 5 is the predicted effect and e; = y/ yf is the true effect.
Population based: "Bias of the Average Treatment Effect" CATE = |TE ATE where ATE N D 1 +/2 - Dy in which y and v/ are the true outcomes for the respective treatments and ATE is calculated based the estimated outcomes.
In this paper, we compare performances of the following treatment effect estimation methods:
CFR: CounterFactual Regression Shalit et al. 2017).
CFR-ISW: CFR with Importance Sampling Weights (Hassanpour & Greiner 2019
SITE: Similarity preserved Individual Treatment Effect (Yao al., 2018).
DR-CFR: Disentangled Representations for CFR our proposed method.
Figure visualizes the PEHE measures in radar charts for these four methods, trained with datasets of size N 2,500 (left) and N 10,000 (right). As expected, all methods perform better with observing more training data; however, DR-CFR took the most advantage by reducing PEHE the most (by 0.15, going down from 0.60 0.45), while CFR, CFR-ISW, and SITE reduced PEHE by 0.07, 0.08, and 0.08 respectively.
Table I summarizes the PEHE and CxTE measures (lower is better) for all scenarios, in terms of mean and standard deviation of all the 24 x datasets, in order to give unified view on the performance.
Note that all four methods share the same core code-base: based on CFR (developed by Johansson et al. (206)) and Shalit et al. (207)) and they share very similar model architectures. To allow for fair comparison, searched their respective hyperparameter spaces, constrained ensure that all had the same model complexity.
8

---

Published as a conference paper at ICLR 2020
Table 1: Synthetic datasets
Table 2: IHDP datasets (100 with N
(21x5 with N = 10,000)
Methods
PEHE
СATE
Methods
PEHE
EATE
CFR
0.61 (0.05)
0.021 (0.018)
CFR
0.81 (0.30)
0.13 (0.12)
CFR-ISW
0.58 (0.06)
0.017 (0.009)
CFR-ISW
0.73 (0.28)
0.11 (0.10)
SITE
0.63 (0.05) 0.035 (0.039)
SITE
0.73 (0.33)
0.10 (0.09)
DR-CFR
0.45 (0.11)
0.013 (0.006)
DR-CFR
0.65 (0.37)
0.03 (0.04)
PEHE and CATE measures (lower is better) represented in the form of "mean (standard deviation)"
DR-CER achieves the best performance among the contending methods. These results are statistically significant based on the Welch's 's unpaired t-test with a 0.05. Table summarizes the PEHE and cATE measures the IHDP benchmark. The results are reported in terms of mean and standard deviation over the 100 datasets with various realizations of outcomes. Again, DR-CFR achieves the best performance (statistically significant for CATE) among the contending methods.
5 FUTURE WORKS AND CONCLUSION
The majority of methods proposed to estimate treatment effects including this work fall under the category of discriminative approaches. A promising direction to consider developing generative models, in an attempt to shed light on the true underlying data generating mechanism. Perhaps this could also facilitate generating new, virtual, yet realistic data instances - similar to what is done in computer vision. [ooiizos et al. (2017)'s method is a notable generative approach, which uses Variational auto-Encoder (VAE) to extract latent confounders from their observed proxies. While that work is interesting step in that direction, it is not yet capable of addressing the problem of selection bias. We believe that our proposed perspective on the problem can be helpful to solve this open question. This is left to future work.
In this paper, we studied the problem estimating treatment effect from observational studies. We argued that all factors in the observed covariates X might contribute the procedure of selecting treatment T, or more importantly, determining the outcomes Y. We modeled this using three underlying sources of X, T, and Y, and showed that explicit identification of these sources offers great insight to help us design models that better handle selection bias in observational datasets. We proposed an algorithm, Disentangled Representations for CounterFactual Regression (DR-CFR), that can (1) identify disentangled representations of the above-mention (2) leverage this knowledge to reduce as well as account for the negative impact of selection bias on estimating the treatment effects from observational data. Our empirical results showed that the proposed method achieves state-of-the-arr performance both individual and population based evaluation measures.
underlying sources and
ACKNOWLEDGEMENTS
The authors gratefully acknowledge financial support from Natural Sciences and Engineering Re- search Council of Canada (NSERC) and Alberta Machine Intelligence Institute (Amii). We wish to thank Dr. Pouria Ramazi and Shivam Raj for fruitful conversations, and Dr. Fredrik Johansson for publishing/maintaining the code-base for the CFR method online. We also would like to thank the ICLR 2020 anonymous reviewers, as well as Dr. Kun Kuang and Tianle Liu, for their valuable reviews, which helped improve this paper.
REFERENCES
Peter C Austin. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3):399-424, 2011.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE TPAMI, 35(8): 1798-1888, 2013.

---

Published as a conference paper at ICLR 2020
Alina Beygelzimer and John Langford. The offset tree for learning with partial labels. In ACM SIGKDD. ACM, 2009.
Léon Bottou, Jonas Peters, Joaquin Quinonero Candela, Denis Xavier Charles, Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Y Simard, and Ed Snelson. Counterfactual reasoning and learning systems: The example of computational advertising, JMLR, 14(1), 2013.
Vincent Dorie. NPCI: Non-parametrics for causal inference, 2016. vdorie/npci
https://githut com/
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. JMLR, 13(Mar):723-77,, 2012.
Negar Hassanpour and Russell Greiner. A novel evaluation methodology for assessing off-policy learning methods in contextual bandits. In Canadian AI, pp- pp- 31-44, 2018.
Negar Hassanpour and Russell Greiner. Counterfactual regression with importance sampling weights. In IJCAL, pp. 5880-5887, 7 2019.
Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):2172240, 2011.
Guido W Imbens. Nonparametric estimation of average treatment effects under exogeneity: A review.
Review of Economics and Statistics, 86(1):4-29, 2004.
Guido W. Imbens and Donald B. Rubin. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, 2015.
Guido W Imbens and Jeffrey M Wooldridge. Recent developments in the econometrics of program evaluation. Journal of Economic Literature, 47(1):5-86, 2009.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In JCML, pp. 3020-3029, 2016.
Kun Kuang, Peng Cui, Bo Li, Meng Jiang, Shiqiang Yang, and Fei Wang. Treatment effect estimation with data-driven variable decomposition. In AAAI, 2017.
Christos Louizos, Uri Shalit, Joris Mooij, David Sontag. Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In NeurIPS, pp. 6446-6456. 2017.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.34300 2009.
Judea Pearl. Causality. Cambridge University Press, 2009.
Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 1983.
Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5):688, 1974.
Uri Shalit, Fredrik D. Johansson, and David Sontag. Estimating individual treatment effect: General- ization bounds and algorithms. In ICML, pp. 3076-3085, 2017.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning And Inference, 90(2), 2000.
Richard Sutton and Andrew G Barto. Reinforcement Learning: An Introduction, volume 1. MIT
Press Cambridge, 1998.
Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization. JMLR, 16, 2015a.
Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: Learning from logged bandit feedback. In /CML., 2015b.
10

---

Published as a conference paper at ICLR 2020
Adith Swaminathan and Thorsten Joachims. The self nelf-normalized estimator for counterfactual learning. In NeurlPS, 2015c.
Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. Representation learning for treatment effect estimation from observational data, In NeurIPS, pp. 2633-2643, 2018.


Reviews:
[{"review_id": "rJxIX7EWjS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n   The authors consider the problem of estimating average treatment effects when observed X and treatment T causes Y. Observational data for X,T,Y is available and strong ignorability is assumed. Previous work (Shalit et al 2017) introduced learning a representation that is invariant in distribution across treatment and control groups and using that with treatment to estimate Y. However, authors point out that this representation being forced to be invariant still does not drive the selection bias to zero. A follow up work (Hassanpour and Greiner 2019) - corrects for this by using additional importance weighting that estimates the treatment selection bias given the learnt representation. However, the authors point out even this is not complete in general, as X could be determined by three latent factors, one that is the actual confounder between treatment and outcome and the other that affects only the outcome and the other that affects only the treatment. Therefore, the authors propose to have three representations and enforce independence between representation that solely determines outcome and the treatment and make other appropriate terms depend on the respective latent factors. This gives a modified objective with respect to these two prior works.\n\nThe authors implement optimize this joint system on synthetic and real world datasets. They show that they outperform all these previous works because of explicitly accounting for confounder, latent factors that solely control only outcome and treatment assignment respectively. \n\nPros:\n  This paper directly addresses the problems due to Shalit 2017 that are still left open. The experimental results seems convincing on standard benchmarks. \n\nI vote for accepting the paper. I don't have many concerns about this paper.\n\nCons:\n  - I have one question for the authors - if T and Y(0),Y(1) are independent given X is assumed, then how are we sure that the composite representations (of the three latent factors) are going to necessarily satisfy ignorability provably ?? I guess this cannot be formally established. It would be great for the authors to comment on this.\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "SkeKSvde9r", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a new way of estimating treatment effects from observational data, that decouples (disentangles) the observed covariates X into three sets: covariates that contributed to the selection of the treatment T, covariates that cause the outcome Y and covariates that do both. The authors show that by leveraging this additional structure they can improve upon existing methods in both ITE and ATE \n\nThe main contributions of the paper are:\n* Highlighting the importance of differentiating between treatment and outcome inducing factors and proposing an algorithm to detect the two\n* Creating a joint optimisation model that contains the factual loss, the cross entropy (treatment) loss and the imbalance loss\n\nOverall, I like the paper quite a lot, I find it well-written and clearly motivated with a very nice experimental section that it is designed around understanding the behaviour of the proposed model.\n\nIn terms of suggestions, I think it will be very interesting to link the approaches using invariant causal representations with existing work in the Counterfactual Risk Minimization [1] literature and to mutualise the experimental setup. \n\n[1] Swaminathan, Adith, and Thorsten Joachims. \"Counterfactual risk minimization: Learning from logged bandit feedback.\" International Conference on Machine Learning. 2015.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJx2t271cB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe paper proposes an algorithm that identifies disentangled representation to find out an individual treatment effect. A very specific model that tries to find out the underlying dynamics of such a problem is proposed and is learned by minimizing a suggested objective that takes the strengths of previous approaches. The method is demonstrated in a synthetic dataset and IHDP dataset and shown to outperform other previous methods by a large margin.\n\nMy initial review was negative, but I changed my mind after reading a few papers in this area. It seems that explicit learning of underlying factors that are described in (Hassanpour & Greiner, 2019) is a nice idea and works well. My only concern is that the paper has a lot of overlap with (Hassanpour & Greiner, 2019), even using identical figures. I am not sure whether it is OK.\n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new way of estimating treatment effects from observational data. The text is clear and experiments support the proposed model.
Decision: Accept (Poster)