Partial Identification of Treatment Effects with Implicit Generative Models
Anonymous Author(s)
AffiliationAddress email
Abstract
We propose a new method for the problem of partial identification, the estimationof bounds on the treatment effects from observational data.
Although studiedusing discrete treatment variables or in specific causal graphs (e.g., instrumentalvariables), partial identification has been recently explored using tools from deepgenerative modeling.
We propose a new method for partial identification of average treatment effects (ATEs) in general causal graphs using implicit generativemodels comprising continuous and discrete random variables.
We leverage averagetreatment derivatives , the partial derivatives of response functions, to prove thatour algorithm converges to tight bounds on ATE.
Our empirical results show thatusing average treatment derivatives leads to tighter and more stable bounds thanmethods that directly optimize the ATE when treatments are continuous.
In thecase of discrete treatments, our derived bounds match those from bespoke solutionsfor partial identification.
Introduction
Estimating average treatment effects (ATEs) is a common task that arises in fields involving decisionmaking, such as healthcare and economics.
In the presence of the gold-standard randomized controlledtrial (RCT) data, one can compare the outcome variable between treated and control groups to makedecisions.
But RCTs can be costly to set up and run and are, in many circumstances, infeasible.
Consequently, communities are using observational data to assist in decision-making.
Identification of treatment effects from observational data is tied to the structure of the causal graph.
For example, the treatment T and outcome Y in Figure 1b are confounded by an unobserved randomvariable, making it impossible to find the causal effect of T on Y only from observational data.
Onthe other hand, Figure 1c is identifiable, and one can adjust for confounders using the Back-doorformula [Pearl, 2009].
Even in identifiable settings, non-parametric estimations such as Back-dooradjustment formula can point-identify the ATE only with additional assumptions such as positivity,i.e., P ( T = t | X ) > 0 for all values of covariate X .
Observational data is finite, high-dimensional,and consequently can suffer from severe violations of such assumptions [D’Amour et al., 2021].
In lieu of the challenges of point-identification, there has been a recognition that decisions can bejustified using reliable bounds on the ATE rather than its exact value.
For an oncologist treatinga cancer patient, knowing that a drug has a significant, positive reduction in the patient’s risk of progression may suffice as a rationale to prescribe that drug.
This problem is known as partialidentification [Manski, 2003].
Most existing methods for bounding the ATEs are only applicablein discrete/binary treatment variables [Makar et al., 2020, Zhang et al., 2021, Duarte et al., 2021,
Guo et al., 2022].
There has been recent interest in continuous treatment settings.
However, suchmethods are applicable for special causal graphs such as the instrumental variables (IV) setting[Gunsilius, 2020, Kilbertus et al., 2020] or make parametric assumptions on the distribution of
[Figure]
T a r g e t , Y
[Figure]
treatment-response functions [Padh et al., 2022].
An exception is the work by Hu et al.
[2021] whichprovides a non-parametric approach for partial identification using generative adversarial networks(GANs).
However, they only provide convergence guarantees for the special case of IV causal graphs.
Using the framework of structural causal models (SCMs) and causal graphs, one can see partialidentification as a constrained optimization problem, where the objective, i.e., maximizing/minimizingthe ATE, can be written as a post-intervention function of exogenous noise (a.k.a response function)and the constraint is to match the generated samples with the observational distribution.
This naturallyleads to using generative neural networks such as neural causal models (NCMs) [Xia et al., 2021].
We find that directly solving the ATE optimization using flexible generative models such as GANscan lead to non-informative and degenerate solutions.
The flexibility afforded by generative modelssuch as GANs allows them to deviate significantly from the true response curve in the neighborhoodof intervention points to maximize/minimize the ATE while continuing to generate samples akin tothe data distribution.
Figure 1e (bottom left) showcases a typical solution to the ATE optimization.
Our insight is that the ATE between any two points can be seen as an integral over the derivatives ofthe response function w.r.t.
the treatment variable within that interval.
Rather than directly optimizingthe ATE, we optimize the partial derivatives of the response function, a quantity that we refer toas the average treatment derivative (ATD), which is also known as the average partial effect in theliterature [Powell et al., 1989, Wooldridge, 2005, Rothenhäusler and Yu, 2019].
The solution to theoptimization of ATD can be shown to be a solution to the original partial identification problem.
Byoptimizing the ATD, the model is required to maximize/minimize the partial derivatives for all pointswithin the interval of interest, avoiding extreme local solutions and resulting in tighter bounds asshown in Figure 1e (bottom right).
Our contributions are as follows:
• We formally define the partial identification of average treatment derivatives as a distributionallyconstrained optimization problem, where we choose Wasserstein distance as our constraint metric.•
For the class of linear SCMs, we prove that the solution to our optimization problem converges tooptimal bounds on the true value of ATD in infinite data for general causal graphs.•
We show that the solution to partial identification of ATDs can be used to find informative boundson the value of ATE under uniform treatment assignment within the interval.
We introducea practical algorithm to solve the distributionally-constrained optimization problem using theLagrange multiplier formulation with alternating optimization.
We empirically show that our
algorithm results in tighter and more stable bounds than methods that directly optimize the ATE.
Problem Setup & Background
We introduce the definitions and assumptions we will use throughout the paper.
Consider the observeddata as (possibly continuous) random variables V
= { X 1 , · · · , X m , T, Y } ∈ R d , where T , Y , and{ X 1 , · · · , X m } denote the treatment variable, target variable, and covariates, respectively.
Data generating model.
Our approach will be based on the framework of Structural Causal Models(SCMs).
An SCM is a tuple M =
( V , U , F , P U ) , where each observed variable V i ∈ V is adeterministic function of a subset of variables pa
( V i ) ⊆ V and latent variables
U V
i ⊆ U , i.e.,
V i =
f V i ( pa ( V i ) , U V i ) where f V i ∈ F , V i ̸∈ pa
( V i ) (1)
The only source of randomness are latent variables U with probability space (Ω , Σ , P U ) .
Thisinduces a probability law over the observed variables P M .
We may omit the subscript M and denotethe observational probability distribution by P throughout the text.
Given M , one can construct agraph with nodes V ∪ U and directed edges from nodes in pa ( V i ) ∪ U V i to V i .
We call this graph thecausal graph corresponding to SCM M and denote it by G M or simply G .
Note that G is acyclic andwe assume it is known.
For random variable V in the SCM M , let V M ( u ) be its deterministic valueafter fixing a realization u of latent variables U .
The causal effect of treatment T on target Y is:
Definition 1 (Causal Effect) .
Let Y M ( T = t ) ( u ) be the value of Y by fixing U = u and changing thefunction f T to a constant function f T = t in M .
Then, we call the random variable Y M ( T = t ) the 83 causal effect of treatment T = t on target Y .
We may simplify the notation and write it as Y t if the
SCM M and treatment variable T are known from context.
Note that Y T ( u ) ( u )
=
Y ( u ) .
When T is continuous, then we can view { Y t : t ∈ supp ( T ) } as a stochastic function defined on(Ω , Σ , P U ) .
This is referred to as the response function, partial dependence plot, and dose-responsecurve in the literature [Zhao and Hastie, 2021, Ritz et al., 2015, Chernozhukov et al., 2018].
Average treatment effect, average treatment derivative, and partial identification.
Our goal is toestimate bounds on the effectiveness of a treatment regime on a population from the observationaldistribution P and the causal graph G .
In the continuous treatment case, where there is no "on"/"off"notion of treatment, we can compare the average causal effect of an arbitrary treatment (dosage)to the average causal effect at a fixed point T = t 0 .
For example, to indicate the effect relativeto not prescribing any treatment, we can choose t 0 = 0 .
This quantity is known as the averagetreatment effect, average level effect, or average dose effect in the literature on continuous treatmentsetting
[Hirano and Imbens, 2004, Kennedy et al., 2017, Callaway et al., 2021].
Definition 2 (Average Treatment Effect) .
For SCM M , the average treatment effect (ATE) at T = dw.r.t.
a fixed point T = t 0 is defined as
[Equation]
Note that estimating the ATE and finding bounds on it only depends on the value of the averageresponse function in T = d and T = t 0 .
As pointed in Gunsilius [2020], this quantity can takearbitrary values if we do not make any assumptions on the set of response functions.
Here, we assumethe partial derivative of the response function w.r.t.
the treatment, i.e., ∂Y t /∂t exists and is a boundedcontinuous function.
We then define the average treatment derivative as the following:
Definition 3 (Average Treatment Derivative) .
For the treatment regime f T in SCM M , we define theaverage treatment derivative (ATD) as
[Equation]
Estimating the ATD can be seen as a proxy for the effectiveness of the prescribed treatment,where we consider the population-level average effect of an infinitesimal increase in the treatment/dosage [Rothenhäusler and Yu, 2019].
In this work, however, we leverage the regularity of thisquantity to achieve smoother solutions to the ATE estimation.
We will expand on this in section 4.
Note that we cannot readily use eq. 2 (or eq. 3) to estimate the ATE (or ATD), as we only have accessto the observational distribution P and the causal graph G and not the latent distribution P U .
In fact,
ATEs are generally non-identifiable, i.e., there exist multiple SCMs with the same causal graph G andgenerated distribution P that result in different values of ATE.
For some graphs, however, one can use non-parametric identification algorithms like do -calculus to identify the causal effect from theobservational distribution [Pearl, 2009].
In practice, even for identifiable causal graphs, we cannotpinpoint the true ATE due to the uncertainty caused by sampling variation and finite sample errors.
Instead, we are interested in finding a tight set of possible solutions that will contain the true valueof ATE (or ATD) with high probability.
This is known as the partial identification problem in theliterature [Manski, 2003].
More formally, the partial identification of ATDs/ATEs is defined as:
Definition 4 (Partial Identification of ATD/ATE) .
Partial identification of ATD is the solution to thefollowing optimization problem:
[Equation]
where M is the set of all SCMs on random variables V .
We denote the solution to the above problemas ( ATD , ATD ) .
Similarly, we can define the partial identification of ATEs by replacing ATD M ′ with
ATE M ′ ( d ) in eq.
We refer to the solution to the latter problem as ( ATE ( d ) , ATE ( d )) .
Implicit generative models.
To solve the partial identification problem, we use the expressive powerof generative models to satisfy the distribution constraint in eq. 4.
Choosing distance measures suchas Jensen-Shannon divergence or Wasserstein metric results in models such as GANs or Wasserstein
GANs (WGANs)
[Goodfellow et al., 2014, Arjovsky et al., 2017].
The typical way to implementthese models is to solve a minimax game between the generator and a discriminator.
However,adding the ATD minimization/maximization term to the minimax loss function will result in unstabletraining.
Instead, in our practical algorithm, we will use Sinkhorn Generative Networks (SGNs) thatuse Sinkhorn divergence S ϵ , a differentiable ϵ -approximation of Wasserstein metric, as the distancemeasure between generated and true samples [Cuturi, 2013, Genevay et al., 2018, Feydy et al., 2019].
Due to the differentiability of Sinkhorn divergence, we will only need to train a generator networkenabling us to sidestep much of the unstable minimax training in (W)GANs.
Related Work
This work builds upon partial identification and generative causal models.
Partial identification.
Finding informative bounds on treatment effects has been well-studied inthe existing literature ([Robins, 1989, Manski, 1990, Evans, 2012, Ramsahai, 2012, Richardsonet al., 2014, Miles et al., 2015, Finkelstein et al., 2021, Zhang and Bareinboim, 2021a,b]).
Balkeand Pearl [1997] find the tightest possible bound for the discrete instrumental variable setting byconverting it to a linear programming problem.
For the backdoor setting and binary treatments, Makaret al.
[2020] provide probabilistic upper/lower bounds on potential outcomes in the finite sampleregime.
Recently, Zhang et al.
[2021] and Duarte et al.
[2021] independently describe a polynomialprogramming approach to solve the partial identification for general causal graphs.
They both usethe notion of canonical SCMs to map the latent variables to the space of functions from treatment
T to outcome Y .
Though they show their polynomial programming formulation finds the optimalbound, their approach is only applicable to discrete random variables with small support.
In fact, thetime complexity of their algorithm grows exponentially with the size of the support set of variables,making their algorithm intractable for continuous settings.
Gunsilius [2019] extends the commonly-used linear programming approach to partial identificationof IV graphs with continuous treatments.
They use a stochastic process representation of the variablesand solve the linear programming via sampling.
However, their method suffers from stability issues,as discussed in Kilbertus et al.
[2020] and is only applicable for the IV setting.
Kilbertus et al. Padh et al.
[2022] parameterize the space of response functions by assuming a multivariate Gaussiandistribution over them. They assume the response functions are linear combinations of a set ofbasis functions.
Then, they match the observed distribution while minimizing/maximizing the ATE.
However, they do not provide any theoretical guarantees on the tightness of their derived bounds.
Most similar to our work is Hu et al.
[2021] who use generative adversarial networks (GANs) to matchthe observed distribution and search for response functions with maximum/minimum ATEs.
Theyprovide convergence guarantees for the instrumental variable causal graph with linear models.
Theirapproach is also based on the minimax game between generator and discriminator, which can resultin unstable training.
Our work differs in a few important ways.
We focus on partial identificationof average derivatives and use that to find bounds over the ATE.
Using this approach, we show that our derived bounds converge to the optimal bounds for linear SCM with general causal graphs,including both identifiable and non-identifiable settings.
We use Sinkhorn divergence, a differentiableapproximation of Wasserstein distance, to train our implicit generative models.
Empirically, we findthat this avoids the unstable training of GANs.
Guo et al.
[2022] studied the partial identificationof ATE with noisy covariates.
Their work is similar to our approach in that we both use a similarrobust optimization formulation.
However, they focus on identifiable causal graphs, where one canuse adjustment formulas such as the Back-door formula and make parametric assumptions on thejoint distribution of observed variables.
Generative causal models.
[Goudet et al., 2017, Yoon et al., 2018, Kocaoglu et al., 2018, Sauerand Geiger, 2021] use generative models to capture a causal perspective on evaluating the effectof interventions on high-dimensional data such as images.
They do not consider the problem ofbounding treatment effects.
Xia et al.
[2021] introduced Neural Causal Models (NCMs) that leveragesthe universal approximability of neural networks to learn the SCM.
Although it is not generallypossible to learn the true SCM by training on the observational data, they prove that NCMs canbe used to test the identifiability of causal effects and propose an algorithm to estimate identifiablecausal effects.
Their work’s theory and empirical instantiation are in the context of discrete randomvariable datasets.
Our work can be seen as an extension of NCMs for partial identification with bothcontinuous and discrete random variables.
Partial Identification using Implicit Generative Models
We explain our method to solve the partial identification problem in Def. 4 using implicit generativemodels.
In subsection 4.1, we describe partial identification of ATDs as a constrained optimizationproblem using G -constraint generative models [Xia et al., 2021].
Then, in subsection 4.2, we showthat the solution to this constrained optimization problem converges to the optimal bounds on the
ATD in infinite data samples.
We prove our results for linear SCMs with general causal graphs,i.e., both identifiable and non-identifiable settings.
Next, we propose our approach to extend thepartial identification of ATDs to ATEs.
Finally, we describe a practical algorithm to solve ourdistributionally-constrained optimization problem in subsection 4.3.
4.1 G -constraint generative models
To solve the partial identification problem, we need to search over the set of all possible SCMs M .
This is generally not feasible, as there is no constraint on the distribution of the latent variables P U ,as well as the function family F .
Instead, we parameterize the space of all SCMs that are consistentwith causal graph G using neural networks.
More specifically, we use G -constraint generative models:
Definition 5 ( G -constraint Generative Models (Def. 7 in Xia et al.
For a given causalgraph G , a G -constraint generative model is a tuple M θ G =
( V , ˆ U , F θ , ˆ P ˆ U ) , where each V i ∈ V isgenerated from
[Equation]
where pa ( V i ) is the observed parents of node V i in G and ˆ U C ∈ U is the latent noise correspondingto maximal C 2 -Component C ⊆ V containing node V i , i.e., each pair of variables in C have commonlatent parent nodes.
In addition, ˆ P ˆ U ∼ Unif (0 , 1) for each ˆ U ∈ ˆ U .
G -constraint generative models make the search over the set M feasible by limiting it to generativemodels with uniformly distributed latent variables that are consistent with causal graph G .
Infact, in their Theorem 3, Xia et al.
[2021] show that for any discrete SCM M ∗ with causal graph
G , there exists a G -constrained generative model M θ G that generates the same causal effect, i.e., 206 Y
M ∗ ( T = t ) =
Y M θ G ( T = t ) a.s.
Though their proof technique only works for SCMs with discretevariables, we believe a similar approach can be used to show their results for general SCMs.
Here, wedo not prove the expressiveness of G -constrained generative models for continuous SCMs.
Instead,for simplicity and completeness of our theoretical results in subsection 4.2, we assume that the true SCM is a G -constrained generative model itself.
In our experiments, we empirically show that ourresults hold even for SCMs with different latent distributions, such as Gaussian noise.
Assumption 1.
The true SCM M is a G -constrained generative model.
In other words, there exist θsuch that M = M θ G .
Under Assumption 1, we reformulate the problem in eq.
4 using generative models, i.e.,
[Equation]
In practice, we never have access to true distribution P as we only observe a finite number of samplescorresponding to the empirical distribution P n = 1 n (cid:80) ni[-=
-]
=1 δ v ( i ) for a given dataset { v (1) , · · · , v ( n ) } .
Also, the observed variables may be biased due to noisy measurements.
Therefore, we reformulatethe problem in eq.
6 as a constrained optimization problem.
We choose the 1-Wasserstein metric asour distance measure, which naturally results in generative models such as WGANs.
We will stateour theory in subsection 4.2 based on this metric.
However, in subsection 4.3, we will propose apractical algorithm that uses Sinkhorn divergence, a differentiable approximation of 1-Wassersteindistance, for more stable results.
Our constrained optimization problem is as follows:
[Equation]
where α n is a hyper-parameter that specifies the level of tightness of the bounds.
We denote thesolution to eq.
7 as ( ˆ ATD , ˆ ATD ) .
In the case of noisy measurements, we need domain knowledge ofhow noisy the data is to determine the value of α n .
Otherwise, we can use the asymptotic convergencerate of empirical Wasserstein distance to choose an appropriate value of α n
[Weed and Bach, 2019].
As our theoretical results are concerned with the infinite-sample case, we will assume that there existvalues of α n such that the true distribution lies within the Wasserstein ball.
Assumption 2.
For each n ∈ N , there exist α n > {+
+} such that W 1 ( P, P n ) ≤ α n .
4.2 Theoretical guarantees and extension to ATEs
Assumptions 1 and 2 ensure that the bound derived by eq.
7 contains the true value of ATD.
However,we do not know how informative/tight the derived bounds are.
In fact, one can always return( −∞ , + ∞ ) as one solution to partial identification.
This part gives theoretical guarantees that ouralgorithm can result in tight bounds over ATD.
In particular, we focus on linear SCMs and show that,under the infinite number of samples, our algorithm converges to the optimal bound ( ATD , ATD ) forboth identifiable and non-identifiable causal graphs.
See Appendix A for the proof.
Definition 6 (Linear SCMs) .
SCM M =
( V , U , F , P U ) is linear, if
[Equation]
Theorem 1 (Tight Bounds) .
Assume the dataset { v (1) , · · · , v ( n ) } is generated from a linear SCM.
Then, under assumptions 1, and 2, the solution to the constrained optimization problem in eq.
7converges to the optimal bound over the ATD in infinite samples, i.e., ( ˆ ATD , ˆ ATD ) → ( ATD , ATD ) .
Up until now, we have only focused on partial identification of ATDs.
Here, we discuss how to extendour results to find bounds on ATEs.
A naive solution is to replace ATD with ATE in eq.
and directlyoptimize it.
However, as demonstrated in the experiments, this approach can result in non-informativebounds.
In fact, Gunsilius [2020] gives a counterexample of a generative function that can match theobservational distribution in the IV setting and produce arbitrary values of ATE.
Instead, we claim that we can use the same generative model trained for partial identification of ATDto bound the value of ATE.
In particular, we define a new objective function by uniform interventionon the treatment, which we call UniformATD , and show that the solution to partial identification of
UniformATD matches the solution to partial identification of ATEs.
Definition 7 (UniformATD) .
For an SCM M , we define the uniform average treatment derivative(UniformATD) at interval [ t 0 , d ] as
[Equation]
Now, we state our result on using average derivatives to solve partial identification of ATEs:
Corollary 1.
Let θ ∗ be the solution to the partial identification of UniformATD at interval [ t 0 , d ] .
Then, θ ∗ is also a solution to the partial identification of ATE ( d ) .
If the true SCM M is linear, thenthe bound is tight, i.e., ( ˆ ATE ( d ) , ˆ ATE ( d ))
→ ( ATE ( d ) , ATE ( d )) as n → ∞ .
The proof is given in Appendix B.
Remark.
Note that the UniformATD objective equals the ATE up to a scale factor.
Therefore, one canargue that optimizing both will essentially result in the same solution.
However, the main benefit of
UniformATD is that we can approximate it by distributions with continuous differentiable log-densityfunctions such as Gaussian distribution or uniform distribution with Gaussian tails.
This will replacethe pathological quantity ATE with a pathwise differentiable approximation resulting in smoothersolutions.
See Example 2 Chernozhukov et al. [2018] for more discussion. The full details of ourimplementation of this approximation are given in Appendix C.
4.3 Our algorithm
We describe our algorithm to solve the optimization problem in eq.
We will focus on finding ˆ ATD,a similar approach can be taken for ˆ ATD.
A general strategy is to convert the constrained problem toits unconstrained version using the method of Lagrange multiplier:
[Equation]
As the Wasserstein distance is not differentiable, we cannot directly use gradient descent to solveeq.
A common approach is to use the dual formulation of Wasserstein distance W 1 ( P M θ G , P n ) =
[Equation] P
[Equation] [Equation] ϕ
[Equation] v )] − E P M
[Equation] G [ q ϕ ( v )] and solve eq.
10 using WGANs, a similar solution usedin Hu et al.
However, this min-max-max formulation can result in unstable bounds as weshow in our experiments.
Instead, we use Sinkhorn divergence, a differentiable approximation to
Wasserstein distance, as the measure of distance between distributions and directly solve the following
Lagrangian:
[Equation]
To solve eq.
11, we need to evaluate ATD M θ G and calculate its gradient w.r.t.
θ .
As we are using
G -constrained generative models, we can calculate the value of Y M θ G ( T = t ) ( u ) by hard intervention
T = t , i.e., fixing the output of function f θT as t and computing Y through a topological order ofcalculations.
Then, we estimate ATD M θ G as follows:
[Equation]
where { T ( i ) } ni =1 are samples from the treatment variable, and { u ( i ) } ni =1 are the latent variablesgenerated from a uniform distribution.
To choose an appropriate value of α n , we first train ourgenerator without the ATD term until the Sinkhorn loss converges to some value and use that as ourchoice of α n .
We then continue our training by adding the ATD term.
We note that using algorithms such as projected gradient descent to solve the constrained optimizationproblem requires us to project the weights of our network into the Wasserstein (Sinkhorn) ball ineach step.
This can be computationally infeasible, and current methods are mainly focused on specialloss functions [Mohajerin Esfahani and Kuhn, 2018, Li et al., 2019, Wong et al., 2019].
Instead, weconsider an alternating optimization procedure, in which we alternate between updating the gradientsfor the ATD and the Sinkhorn loss.
The full details of our algorithm, its extension to ATEs, and thealternating optimization are described in Appendix C.
Experiments
We run our partial identification algorithms on a variety of simulated settings.
We mainly focus on thesynthetic data generating processes as the ground truth must be known to evaluate our derived boundsproperly.
Our primary goal is to show that using average treatment derivatives instead of directlyoptimizing the average treatment effect will result in tighter and more stable bounds.
First, we runour algorithm to estimate bounds on the value of ATDs for both identifiable and non-identifiablecausal graphs.
We show that, as the number of samples increases, our algorithm converges to tightbounds over the true value of ATD (Figure 2).
We then focus on partial identification of ATEs and [Figure] [Figure] demonstrate that using partial derivatives of the response function leads to more informative boundsthan methods that directly optimize the ATE (Figure 3).
Finally, as a sanity check, we run our methodon two binary datasets, where the optimal bounds are known and show that our approach can reachthe optimal solution.
Therefore it is applicable for both discrete and continuous datasets.
Additionalexperiments are provided in Appendix E
[Figure]
5.1 Datasets and Baseline
Continuous Setting.
We generate a linear SCM with three-dimensional covariates from a multivariate
Gaussian distribution for the Back-door causal graph (Figure 1c).
We also simulate a quadratic SCMwith nonlinear interaction between the covariates and treatment.
For the Front-door setting (Figure 1d),we similarly generate an SCM where the target variable is a quadratic function of the mediator.
Inaddition, we consider two different SCMs for the IV setting (Figure 1b) based on the strength of theinstrument and the confounding.
In particular, we generate a linear SCM with a weak instrument(small correlation between the treatment and the instrument) and strong confounding (high correlationbetween the hidden confounders and the target variable).
We also consider a nonlinear dataset with astrong instrument and weak confounding.
Finally, for the leaky mediation causal graph (Figure 1a),we generate a two-dimensional linear dataset.
Discrete Setting.
We consider the binary IV dataset described in Duarte et al.
[2021], where thetrue value of ATE is not identifiable, but the optimal bound is known.
We also use the Front-doorbinary dataset in Zhang et al.
[2021] where the causal effect is identifiable.
The full details of ourdata generating processes for both continuous and discrete settings can be found in Appendix D.
GAN Baseline.
Our baseline is the algorithm in Hu et al.
[2021] that directly optimizes the value of
ATE using GANs.
We use their default hyper-parameters with a tolerance of 0 . 0001 .
Similar to theirexperimental setup, we consider 50 intermediate solutions where the distance is within the toleranceand compute the bounds using the mean and one-sided confidence intervals.
Our implementationdetails, as well as hyper-parameters can be found in Appendix F.
[Table]
5.2 Results
Bounding average treatment derivatives.
We generate data with sample sizes N ={ 500 , 1000 , 2000 , 5000 } from the nonlinear Front-door and linear Back-door SCMs (identifiable),as well as linear IV with strong confounding and leaky mediation settings (non-identifiable).
Werun our algorithm with ten different random seeds for each setting/sample size.
Then, we choosethe five runs with the lowest tolerance parameter α n and choose the upper (lower) bound as themaximum (minimum) value of the ATD within these five runs.
Figure 2 shows our derived bounds.
As expected, the algorithm is able to point-identify the value of ATD for identifiable scenarios as thenumber of samples increases (Figures 2 (a) and (b)).
In non-identifiable cases, our algorithm leads totight bounds containing the true value of ATD by increasing the number of samples as depicted in
Figures 2 (c) and (d).
This is in line with our results in Theorem 1.
Bounding average treatment effects.
Here, we aim to demonstrate the effectiveness of using partialderivatives for bounding the ATE compared to the direct optimization approach.
We consider fourdifferent settings and run our algorithm for 10 different values of treatment { t i } 10 i =1 in each setting.
We compute the value of ATE w.r.t.
a fixed point t 0 , chosen as the maximum treatment value in thesamples.
For each value of T , we generate N = 5000 samples and run each experiment five times.
Then, we select the maximum (minimum) value of ATE within the five runs as the upper (lower)bound.
We follow the same procedure for the GAN baseline.
To find the bounds on ATE using our approach, we uniformly intervene on the interval between t 0 and 342 t i and maximize/minimize the partial derivatives.
Figure 3 shows the effectiveness of this approachin comparison to the GAN baseline.
Our algorithm produces stable and tight bounds containing thetrue value of ATE, while the GAN baseline, which relies on the direct optimization of ATEs, resultsin unstable loose bounds that may not include the true value of the treatment effect.
Binary treatments.
To showcase the generality of our framework, we study two datasets with binarytreatments.
Here, the partial derivatives do not exist, so we directly optimize the value of ATE.
Notethat, in the discrete setting, the network can’t generate arbitrary large values in the intervention pointswithout violating the distributional constraint.
Table 1 shows our derived bounds and compares themto the optimal bounds.
In the identifiable Front-door causal graph, we find a tight bound over the truevalue of ATE.
In the non-identifiable IV setting, our bound includes the optimal bound with a smallgap.
Conclusion, Limitations and Future Work
Our work introduces a novel method to estimate average treatment effects from observational data.
Specifically, we propose optimizing the average treatment derivative, which in turn can be used toestimate the average treatment effect in treatment response curves.
Empirically we find that the useof our method recovers known bounds on treatment effects in the discrete case and outperforms othermethods based on implicit models for partial identification in the continuous case.
There remain several limitations of this work.
Our work builds on the constrained optimizationproblem defined by Xia et al.
[2021] instantiated in the context of the ATD.
Developing new methodsfor function maximization/minimization approaches under distributional constraints remains animportant direction for future work.
Our work primarily uses carefully designed synthetic datasets toevaluate our method under different constraints on the data distribution.
A larger-scale evaluation ofour approach on real-world benchmarks will better help us assess the method’s practicality.
Finally,the theory of our work is restricted to the linear SCM scenario; consequently, practitioners mustexercise caution when deploying this method when there is a nonlinear structure posited among therandom variables.
Paper checklist
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope
[Yes
We have
listed the
key contributions of our work in the
introduction In the experimental section we verify our theoretical contributions andperform a comparison to the closest baseline.
Our theoretical results are in idealizedlinear settings though we do experiment beyond that in our work.
We highlight thelimitations of our theoretical results and explain the rationale for the approximationsthat we make in practice
(b) Did you describe the limitations of your work?
[Yes] We explicitly highlight the
limitations of our work in the context of related work, in our exposition of the
theoretical
results and the
computation necessary for our method. Our theoretical results make explicit the assumptions that we make and full proofs are presented in
the
supplementary material. (c) Did you discuss any potential negative societal impacts of your work?
[Yes]
Our paper explicitly tackles the problem of bounding treatment effects from observational data.
We have highlighted that our theory is focused on the case of linear models and provided empirical results supporting the method’s utility in the general, non-linear case.
If the method is deployed in high-risk settings such as healthcare, there may betimes where the method fails to estimate bounds correctly from real-data in non-linear settings.
We have encouraged practitioners to excercise caution in this regard.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] We include the
complete proofs of our results in the supplementary material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g. data splits hyperparameters, how they
were chosen)?
[Yes]
Our supplemental material makes clear the training procedurefor
our
method, the main sets of hyperparameters and any heuristics for optimizing
the
models
We
will release code to reproduce our results.
Our work made no use ofparticipants
(c) Did you report error bars (e g., with respect to the random seed after running exper- iments multiple times)?
[Yes] Our method finds maximum/minimum values on the
quantity of interest.
Instead of reporting error bars, we consider maximum/minimumvalues among multiple runs with different random seeds.
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Experiments were run on two a
single machine with 2 Nvidia RTX A5000 GPUs, 2 RTX A6000 cards GPUs, 96 CPUs
and 188GB of RAM and on a larger internal cluster comprising hundreds of GPUs.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] Our evaluation is primarily on
synthetic data for which we have cited the original authors who created them.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A] We use publicly available datasets that do not
contain unanonymized personal data.