Partial Identiﬁcation of Treatment Effects with Implicit Generative Models
Abstract
We consider the problem of partial identiﬁcation, the estimation of bounds on the treatment effects from observational data.
Although studied using discrete treatment variables or in speciﬁc causal graphs (e.g., instrumental variables), partial identiﬁcation has been recently explored using tools from deep generative modeling.
We propose a new method for partial identiﬁcation of average treatment effects (ATEs) in general causal graphs using implicit generative models comprising continuous and discrete random variables.
Since ATE with continuous treatment is generally non-regular, we leverage the partial derivatives of response functions to deﬁne a regular approximation of ATE, a quantity we call uniform average treatment derivative (UATD).
We prove that our algorithm converges to tight bounds on ATE in linear structural causal models (SCMs).
For nonlinear SCMs, we empirically show that using UATD leads to tighter and more stable bounds than methods that directly optimize the ATE.
Introduction
Estimating average treatment effects (ATEs) is a common task that arises in ﬁelds involving decisionmaking, such as healthcare and economics.
In the presence of the gold-standard randomized controlled trial (RCT) data, one can compare the outcome variable between treated and control groups to make decisions.
But RCTs can be costly to set up and run and are, in many circumstances, infeasible.
Consequently, communities are using observational data to assist in decision-making.
Identiﬁcation of treatment effects from observational data is tied to the structure of the causal graph.
For example, the treatment T and outcome Y in Figure 1b are confounded by an unobserved random variable, making it impossible to ﬁnd the causal effect of T on Y only from observational data.
On the other hand, Figure 1c is identiﬁable, and one can adjust for confounders using the Back-door formula [Pearl, 2009].
Even in identiﬁable settings, non-parametric estimations such as Back-door adjustment formula can point-identify the ATE only with additional assumptions such as positivity, i.e., P ( T = t | X ) > 0 for all values of covariate X .
Observational data is ﬁnite, high-dimensional, and consequently can suffer from severe violations of such assumptions [D’Amour et al., 2021].
In lieu of the challenges of point-identiﬁcation, there has been a recognition that decisions can be justiﬁed using reliable bounds on the ATE rather than its exact value.
For an oncologist treating a cancer patient, knowing that a drug has a signiﬁcant, positive reduction in the patient’s risk of progression may sufﬁce as a rationale to prescribe that drug.
This problem is known as partial identiﬁcation [Manski, 2003].
Most existing methods for bounding the ATEs are only applicable in discrete/binary treatment variables
[Makar et al., 2020, Zhang et al., 2021, Duarte et al., 2021,
Guo et al., 2022].
There has been recent interest in continuous treatment settings.
However, such
[Figure]
r
[Figure] t [Figure] methods are applicable for special causal graphs such as the instrumental variables (IV) setting [Gunsilius, 2020, Kilbertus et al., 2020] or make parametric assumptions on the family of treatmentresponse functions [Padh et al., 2022].
An exception is the work by Hu et al.
[2021] which provides a non-parametric approach for partial identiﬁcation using generative adversarial networks (GANs).
However, they only provide convergence guarantees for the special case of IV causal graphs.
Using the framework of structural causal models (SCMs) and causal graphs, one can see partial identiﬁcation as a constrained optimization problem, where the objective, i e., maximizing/minimizing the ATE, can be written as a post-intervention function of exogenous noise (a.k.a response function) and the constraint is to match the generated samples with the observational distribution.
This naturally leads to using generative neural networks such as neural causal models (NCMs)
[Xia et al., 2021].
We ﬁnd that directly solving the ATE optimization using ﬂexible generative models such as GANs can lead to non-informative and degenerate solutions.
The ﬂexibility afforded by generative models such as GANs allows them to deviate signiﬁcantly from the true response curve in the neighborhood of intervention points to maximize/minimize the ATE while continuing to generate samples akin to the data distribution.
Figure 1e (left) showcases a typical solution to the ATE optimization.
Our insight is that the ATE between any two points can be approximated as an integral over the derivatives of the response function w.r.t.
the treatment variable.
Rather than directly optimizing the ATE, we optimize the partial derivatives of the response function, a quantity that we refer to as the uniform average treatment derivative (UATD).
By optimizing the UATD, the model is required to maximize/minimize the partial derivatives for all points within the treatment support, avoiding extreme local solutions as shown in Figure 1e (right).
Our contributions are as follows
: • We formally deﬁne the partial identiﬁcation of average treatment effects as a distributionallyconstrained optimization problem, where we choose Wasserstein distance as our constraint metric.
• For the class of linear SCMs, we prove that the solution to our optimization problem converges to optimal bounds on the true value of ATE in inﬁnite data for general causal graphs.
• We use the solution to partial identiﬁcation of UATDs to ﬁnd informative bounds on the value of ATE.
We introduce a practical algorithm to solve the distributionally-constrained optimization problem using the Lagrange multiplier formulation with alternating optimization.
We empirically 2 Average treatment derivative is also known as average partial effect the
show that our algorithm results in tighter and more stable bounds than methods that directly optimize the ATE.
Problem Setup & Background
We introduce the deﬁnitions and assumptions we will use throughout the paper.
Consider the observed data as (possibly continuous) random variables V
= { X 1 , · · · , X m , T, Y } 2 R d , where T , Y , and { X 1 , · · · , X m } denote the treatment variable, target variable, and covariates, respectively.
Data generating model.
Our approach will be based on the framework of Structural Causal Models (SCMs).
An SCM is a tuple M =
( V , U , F , P U ) , where each observed variable V i 2 V is a deterministic function of a subset of variables pa
( V i ) ✓ V and latent variables
U V i
✓ U , i.e.,
[Equation]
The only source of randomness are latent variables U with probability space ( ⌦ , ⌃ , P U ) .
This induces a probability law over the observed variables P M .
We may omit the subscript M and denote the observational probability distribution by P throughout the text.
Given M , one can construct a graph with nodes V [ U and directed edges from nodes in pa ( V i ) [ U V i to V i .
We call this graph the causal graph corresponding to SCM M and denote it by G M or simply G .
We assume G is acyclic and known.
Moreover, We will assume each node in the graph is 1 -dimensional.
For random variable
V in the SCM M , let V M ( u ) be its deterministic value after ﬁxing a realization u of latent variables
U .
The causal effect of treatment T on target Y is:
Deﬁnition 1 (Causal Effect) .
Let Y M ( T = t ) ( u ) be the value of Y by ﬁxing U = u
and changing the function f T to a constant function f T = t in M .
Then, we call the random variable Y M ( T = t ) the causal effect of treatment T = t on target Y .
We may simplify the notation and write it as Y t if the
SCM M and treatment variable T are known from context.
Note that Y T ( u ) ( u )
=
Y ( u ) .
When T is continuous, then we can view { Y t : t 2 supp ( T ) } as a stochastic function deﬁned on ( ⌦ , ⌃ , P U ) .
This is referred to as the response function, partial dependence plot, and dose-response curve in the literature [Zhao and Hastie, 2021, Ritz et al., 2015, Chernozhukov et al., 2018].
Average treatment effect, average treatment derivative, and partial identiﬁcation.
Our goal is to estimate bounds on the effectiveness of a treatment regime on a population from the observational distribution P and the causal graph G .
In the continuous treatment case, where there is no "on"/"off" notion of treatment, we can compare the average causal effect of an arbitrary treatment (dosage) to the average causal effect at a ﬁxed point T = t 0 .
For example, to indicate the effect relative to not prescribing any treatment, we can choose t 0 = 0 .
This quantity is known as the average treatment effect, average level effect, or average dose effect in the literature on continuous treatment setting [Hirano and Imbens, 2004, Kennedy et al., 2017, Callaway et al., 2021].
Deﬁnition 2 (Average Treatment Effect) .
For SCM M , the average treatment effect (ATE) at T =
d w.r.t.
a ﬁxed point T = t 0 is deﬁned as
[Equation]
Note that estimating the ATE and ﬁnding bounds on it only depends on the value of the average response function in T =
d and T = t 0 .
As pointed in Gunsilius [2020], this quantity can take arbitrary values if we do not make any assumptions on the set of response functions.
Here, we assume the partial derivative of the response function w.r.t.
the treatment, i.e., @ Y t / @ t exists and is a bounded continuous function.
We then deﬁne the average treatment derivative as the following:
Deﬁnition 3 (Average Treatment Derivative) .
For the treatment regime f T in SCM M , we deﬁne the average treatment derivative (ATD) as
[Equation]
Estimating the ATD can be seen as a proxy for the effectiveness of the prescribed treatment, where we consider the population-level average effect of an inﬁnitesimal increase in the treatment/dosage [Rothenhäusler and Yu, 2019].
In this work, however, we leverage the regularity of this quantity to achieve smoother solutions to the ATE estimation.
We will expand on this in section 4.
Note that we cannot readily use eq. 2 (or eq. 3) to estimate the ATE (or ATD), as we only have access to the observational distribution P and the causal graph G and not the latent distribution P U .
In fact,
ATEs are generally non-identiﬁable, i.e., there exist multiple SCMs with the same causal graph G and generated distribution P that result in different values of ATE.
For some graphs, however, one can use non-parametric identiﬁcation algorithms like do -calculus to identify the causal effect from the observational distribution [Pearl, 2009].
In practice, even for identiﬁable causal graphs, we cannot pinpoint the true ATE due to the uncertainty caused by sampling variation and ﬁnite sample errors.
Instead, we are interested in ﬁnding a tight set of possible solutions that will contain the true value of ATE (or ATD) with high probability.
This is known as the partial identiﬁcation problem in the literature [Manski, 2003].
3 More formally, the partial identiﬁcation of ATDs/ATEs is deﬁned as:
Deﬁnition 4 (Partial Identiﬁcation of ATD/ATE) .
Partial identiﬁcation of ATD is the solution to the following optimization problem:
[Equation]
where M is the set of all SCMs on random variables V .
We denote the solution to the above problem as ( ATD , ATD ) .
Similarly, we can deﬁne the partial identiﬁcation of ATEs by replacing ATD M 0 with
ATE M 0 ( d ) in eq.
We refer to the solution to the latter problem as ( ATE ( d ) , ATE ( d )) .
Implicit generative models.
To solve the partial identiﬁcation problem, we use the expressive power of generative models to satisfy the distribution constraint in eq. 4.
Choosing distance measures such as Jensen-Shannon divergence or Wasserstein metric results in models such as GANs or Wasserstein
GANs (WGANs)
[Goodfellow et al., 2014, Arjovsky et al., 2017].
The typical way to implement these models is to solve a minimax game between the generator and a discriminator.
However, adding the ATD minimization/maximization term to the minimax loss function will result in unstable training.
Instead, in our practical algorithm, we will use Sinkhorn Generative Networks (SGNs) that use Sinkhorn divergence S ✏ , a differentiable ✏ -approximation of Wasserstein metric, as the distance measure between generated and true samples [Cuturi, 2013, Genevay et al., 2018, Feydy et al., 2019].
Due to the differentiability of Sinkhorn divergence, we will only need to train a generator network enabling us to sidestep much of the unstable minimax training in (W)GANs.
Related Work
This work builds upon partial identiﬁcation and generative causal models.
Partial identiﬁcation.
Finding informative bounds on treatment effects has been well-studied in the existing literature ([Robins, 1989, Manski, 1990, Evans, 2012, Ramsahai, 2012, Richardson et al., 2014, Miles et al., 2015, Finkelstein et al., 2021, Zhang and Bareinboim, 2021a,b]).
Balke and Pearl [1997] ﬁnd the tightest possible bound for the discrete instrumental variable setting by converting it to a linear programming problem.
For the backdoor setting and binary treatments, Makar et al.
[2020] provide probabilistic upper/lower bounds on potential outcomes in the ﬁnite sample regime.
Recently, Zhang et al.
[2021] and Duarte et al.
[2021] independently describe a polynomial programming approach to solve the partial identiﬁcation for general causal graphs.
They both use the notion of canonical SCMs to map the latent variables to the space of functions from treatment
T to outcome Y .
Though they show their polynomial programming formulation ﬁnds the optimal bound, their approach is only applicable to discrete random variables with small support.
In fact, the time complexity of their algorithm grows exponentially with the size of the support set of variables, making their algorithm intractable for continuous settings.
Gunsilius [2019] extends the commonly-used linear programming approach to partial identiﬁcation of IV graphs with continuous treatments.
They use a stochastic process representation of the variables and solve the linear programming via sampling.
However, their method suffers from stability issues, as discussed in Kilbertus et al.
[2020] and is only applicable for the IV setting.
Kilbertus et al.
[2020], Padh et al.
[2022] parameterize the space of response functions by assuming them as linear combinations of a set of ﬁxed basis functions.
Then, they match the ﬁrst two moments of observed distribution while minimizing/maximizing the ATE.
However, they do not provide any theoretical guarantees on the tightness of their derived bounds.
Most similar to our work is Hu et al.
[2021] who use generative adversarial networks (GANs) to match the observed distribution and search for response functions with maximum/minimum ATEs.
They provide convergence guarantees for the instrumental variable causal graph with linear models.
Their approach is also based on the minimax game between generator and discriminator, which can result in unstable training.
Our work differs in a few important ways.
We focus on partial identiﬁcation of average derivatives and use that to ﬁnd bounds over the ATE.
Using this approach, we show that our derived bounds converge to the optimal bounds for linear SCM with general causal graphs, including both identiﬁable and non-identiﬁable settings.
We use Sinkhorn divergence, a differentiable approximation of Wasserstein distance, to train our implicit generative models.
Empirically, we ﬁnd that this avoids the unstable training of GANs.
Guo et al.
[2022] studied the partial identiﬁcation of ATE with noisy covariates.
Their work is similar to our approach in that we both use a similar robust optimization formulation.
However, they focus on identiﬁable causal graphs, where one can use adjustment formulas such as the Back-door formula and make parametric assumptions on the joint distribution of observed variables.
Generative causal models.
[Goudet et al., 2017, Yoon et al., 2018, Kocaoglu et al., 2018, Sauer and Geiger, 2021] use generative models to capture a causal perspective on evaluating the effect of interventions on high-dimensional data such as images.
They do not consider the problem of bounding treatment effects.
Xia et al.
[2021] introduced Neural Causal Models (NCMs) that leverages the universal approximability of neural networks to learn the SCM.
Although it is not generally possible to learn the true SCM by training on the observational data, they prove that NCMs can be used to test the identiﬁability of causal effects and propose an algorithm to estimate identiﬁable causal effects.
Their work’s theory and empirical instantiation are in the context of discrete random variable datasets.
Our work builds upon NCMs for partial identiﬁcation with both continuous and discrete random variables.
Partial Identiﬁcation using Implicit Generative Models
We explain our method to solve the partial identiﬁcation problem in Def.
4 using implicit generative models.
In subsection 4.1, we describe partial identiﬁcation of ATDs as a constrained optimization problem using G -constraint generative models [Xia et al., 2021].
Then, in subsection 4.2, we show that the solution to this constrained optimization problem converges to the optimal bounds on the
ATD in inﬁnite data samples.
We prove our results for linear SCMs with general causal graphs, i.e., both identiﬁable and non-identiﬁable settings.
Next, we propose our approach to extend the partial identiﬁcation of ATDs to ATEs.
Finally, we describe a practical algorithm to solve our distributionally-constrained optimization problem in subsection 4.3.
4.1 G -constraint generative models
To solve the partial identiﬁcation problem, we need to search over the set of all possible SCMs M .
This is generally not feasible, as there is no constraint on the distribution of the latent variables P U , as well as the function family F .
Instead, we parameterize the space of all SCMs that are consistent with causal graph G using neural networks.
More speciﬁcally, we use G -constraint generative models:
Deﬁnition 5 ( G -constraint Generative Models (Def. 7 in Xia et al.
For a given causal graph G , a G -constraint generative model is a tuple M ✓ G =
( V , ˆ U , F ✓ , ˆ P ˆ U ) , where each V i 2 V is generated from
[Equation]
[Equation] ) where pa ( V i ) is the observed parents of node V i in G and ˆ U C 2 U is the latent noise corresponding to maximal C 2 -Component C ✓ V containing node V i , i.e., each pair of variables in C have common latent parent nodes.
In addition, ˆ P ˆ U ⇠ Unif (0 , 1) for each ˆ U 2 ˆ U .
G -constraint generative models make the search over the set M feasible by limiting it to generative models with uniformly distributed latent variables that are consistent with causal graph G .
In fact, in their Theorem 3, Xia et al.
[2021] show that for any discrete SCM M ⇤ with causal graph
G , there exists a G -constrained generative model M ✓ G that generates the same causal effect, i.e., Y
M ⇤ ( T = t ) =
Y M ✓ G ( T = t ) a.s.
Their proof technique, however, only works for SCMs with discrete variables.
Here, we do not prove the expressiveness of G -constrained generative models for continuous
SCMs.
Instead, for simplicity and completeness of our theoretical results in subsection 4.2, we assume that the true SCM is a G -constrained generative model itself.
In our experiments, we empirically show that our results hold even for SCMs with different latent distributions, such as Gaussian noise.
Assumption 1.
The true SCM M is a G -constrained generative model.
In other words, there exist ✓ such that M = M ✓ G .
Under Assumption 1, we reformulate the problem in eq.
4 using generative models, i.e.,
[Equation]
In practice, we never have access to true distribution P as we only observe a ﬁnite number of samples corresponding to the empirical distribution P n = 1 n
[Equation]
i ) for a given dataset { v (1) , · · · , v ( n ) } .
Also, the observed variables may be biased due to noisy measurements.
Therefore, we reformulate the problem in eq.
6 as a constrained optimization problem.
We choose the 1-Wasserstein metric as our distance measure, which naturally results in generative models such as WGANs.
We will state our theory in subsection 4.2 based on this metric.
However, in subsection 4.3, we will propose a practical algorithm that uses Sinkhorn divergence, a differentiable approximation of 1-Wasserstein distance, for more stable results.
Our constrained optimization problem is as follows:
[Equation]
where ↵ n is a hyper-parameter that speciﬁes the level of tightness of the bounds.
We denote the solution to eq.
7 as ( ˆ ATD , ˆ ATD ) .
In the case of noisy measurements, we need domain knowledge of how noisy the data is to determine the value of ↵ n .
Otherwise, we can use the ﬁnite-sample convergence rate of empirical Wasserstein distance to choose an appropriate value of ↵ n
[Weed and
Bach, 2019].
As our theoretical results are concerned with the inﬁnite-sample case, we will assume that there exist values of ↵ n such that the true distribution lies within the Wasserstein ball.
Assumption 2.
For each n 2 N , there exist ↵ n > [-
-]0 such that W 1 ( P, P n )  ↵ n .
4.2 Theoretical guarantees and extension to ATEs
Assumptions 1 and 2 ensure that the bound derived by eq.
7 contains the true value of ATD.
However, we do not know how informative/tight the derived bounds are.
In fact, one can always return ( 1 , + 1 ) as one solution to partial identiﬁcation.
This part gives theoretical guarantees that our algorithm can result in tight bounds over ATD.
In particular, we focus on linear SCMs and show that, under the inﬁnite number of samples, our algorithm converges to the optimal bound ( ATD , ATD ) for both identiﬁable and non-identiﬁable causal graphs.
See Appendix A for the proof.
Deﬁnition 6 (Linear SCMs) .
SCM M =
( V , U , F , P U ) is linear, if
[Equation]
Theorem 1 (Tight Bounds) .
Assume the dataset { v (1) , · · · , v ( n ) } is generated from a linear SCM.
Then, under assumptions 1, and 2, the solution to the constrained optimization problem in eq.converges to the optimal bound over the ATD in inﬁnite samples, i.e., ˆ ATD ! ATD and ˆ ATD ! ATD.
Up until now, we have only focused on partial identiﬁcation of ATDs.
Here, we discuss how to extend our results to ﬁnd bounds on ATEs.
A naive solution is to replace ATD with ATE in eq.
7 and directly optimize it.
However, as demonstrated in the experiments, this approach can result in non-informative bounds.
In fact, ATE with continuous treatments is a non-regular quantity [Kennedy et al., 2017].
Instead, we claim that we can use the same generative model trained for partial identiﬁcation of ATD to bound the value of ATE.
In particular, we deﬁne a new objective function by uniform intervention on the treatment, which we call uniform average treatment derivative (UATD), and show that the solution to partial identiﬁcation of UATD matches the solution to partial identiﬁcation of ATEs.
Deﬁnition 7 (UATD) .
For an SCM M , we deﬁne the uniform average treatment derivative (UATD) at interval [ t 0 , d ] as
[Equation]
[Figure]  [Figure] [Figure]
5  [Figure]  5
Treatment, T
T a r g e t , Y true response upper bound treatment density ˜ µ
Now, we state our result on using average derivatives to solve partial identiﬁcation of ATEs:
Corollary 1.
Let ✓ ⇤ be the solution to the partial identiﬁcation of UATD at interval [ t 0 , d ] .
Then, ✓ ⇤ is also a solution to the partial identiﬁcation of ATE ( d ) .
If the true SCM M is linear, then the bound is tight, i.e., ˆ ATE ( d ) ! ATE ( d ) and ˆ ATE ( d ) ! ATE ( d ) as n ! 1 .
The proof is given in Appendix B.
Remark.
ATD and ATE are generally different quantities and ﬁnding bounds on one does not necessarily results in bounds on the other.
In fact, ATE is not pathwise differentiable for continuous treatments [Díaz and van der Laan, 2013, Kennedy et al., 2017, Chernozhukov et al., 2018] (See Appendix C).
We deﬁne UATD as a quantity to relate ATE and ATD.
The formal deﬁnition of UATD in
Def. 7 is the same as ATE up to a scale factor, and one would expect to see similar issues with ATE here as well.
Therefore, we approximate UATD with a version that, instead of a uniform distribution of treatment within interval [ t 0 , d ] with zero density outside, the treatment distribution has continuous differentiable non-zero density deﬁned over the whole support of T .
More concretely,
ATE
[Equation]
where µ is a the uniform measure within interval [ t 0 , d ] and ˜ µ is its approximation with a non-zero density over the full support of T .
Choosing ˜ µ trades off between regularity of the response function and the approximation error.
The more ˜ µ is close to the uniform measure µ , the more irregularity we allow in the response curve and the less informative the bound will be, while the objective function is closer to ATE.
Adding more density outside of interval [ t 0 , d ] imposes regularity on the response curve (Figure 2).
It is important to note that we use the formal deﬁnition of UATD (with uniform treatment distribution) in Corollary 1 since, in linear SCMs, it is not possible for the generator model to attain arbitrarily large values in points t 0 and d as the derivative of linear functions remains ﬁxed outside of interval [ t 0 , d ] .
This is why we state our theoretical results based on the uniform intervention.
4.3 Our algorithm
We describe our algorithm to solve the optimization problem in eq.
We will focus on ﬁnding ˆ ATD, a similar approach can be taken for ˆ ATD.
A general strategy is to convert the constrained problem to its unconstrained version using the method of Lagrange multiplier:
[Equation]
As the Wasserstein distance is not differentiable, we cannot directly use gradient descent to solve eq.
A common approach is to use the dual formulation of Wasserstein distance W 1 ( P M ✓ G , P n ) = max
|| q  ||
L  1 E P n [ q  ( v )]  E P M ✓ G [ q  ( v )] and solve eq.
11 using WGANs, a similar solution used
[Figure]
in Hu et al.
However, this min-max-max formulation can result in unstable bounds as we show in our experiments.
Instead, we use Sinkhorn divergence, a differentiable approximation to
Wasserstein distance, as the measure of distance between distributions and solve the following:
[Equation]
To solve eq.
12, we need to evaluate ATD M ✓ G and calculate its gradient w.r.t.
✓ .
As we are using
G -constrained generative models, we can calculate the value of Y M ✓ G ( T = t ) ( u ) by hard intervention
T = t , i.e., ﬁxing the output of function f ✓ T as t and computing Y through a topological order of calculations.
Then, we estimate ATD M ✓ G as follows:
[Equation]
[Equation] where { t ( i ) } ni =1 are samples from the treatment variable, and { u ( i ) } ni =1 are the latent variables generated from a uniform distribution.
To choose an appropriate value of ↵ n , we ﬁrst train our generator without the ATD term until the Sinkhorn loss converges to some value and use that as our choice of ↵ n .
We then continue our training by adding the ATD term.
We note that using algorithms such as projected gradient descent to solve the constrained optimization problem requires us to project the weights of our network into the Wasserstein (Sinkhorn) ball in each step.
This can be computationally infeasible, and current methods are mainly focused on special loss functions [Mohajerin Esfahani and Kuhn, 2018, Li et al., 2019, Wong et al., 2019].
Instead, we consider an alternating optimization procedure, in which we alternate between updating the gradients for the ATD and the Sinkhorn loss.
The full details of our algorithm, its extension to ATEs, and the alternating optimization are described in Appendix D.
Experiments
We run our partial identiﬁcation algorithms on a variety of simulated settings.
We mainly focus on the synthetic data generating processes as the ground truth must be known to evaluate our derived bounds properly.
Our primary goal is to show that using uniform average treatment derivatives instead of directly optimizing the average treatment effect will result in tighter and more stable bounds.
We ﬁrst run our algorithm to estimate bounds on the value of ATDs for both identiﬁable and non-identiﬁable causal graphs.
We show that, as the number of samples increases, our algorithm converges to tight bounds over the true value of ATD (Figure 3).
We then focus on partial identiﬁcation of ATEs and demonstrate that using partial derivatives of the response function leads to more informative bounds, while being valid (Figures 4a, 4b).
Additionally, as a sanity check, we consider two binary-treatments datasets where the optimal bounds are known and show that our approach can reach the optimal solution.
We also test our method on an ACIC dataset, a case study with real-world covariates, to illustrate the performance of our algorithm on higher-dimensional datasets.
We include these additional experiments in Appendix E.
Our implementation details, as well as hyper-parameters can be found in Appendix F.
5.1 Datasets and Baseline
Datasets.
We consider various data generating processes for different causal graphs, including a linear
SCM with three-dimensional covariates and a quadratic SCM with nonlinear interaction between
[Figure] [Figure]
t ,
Y
E [ Y t ]  E [ Y t 0 ]
Our Bounds
GAN Bounds
Real samples
[Figure]
the covariates and treatment for the Back-door causal graph (Figure 1c), as well as a nonlinear
SCM for the Front-door setting (Figure 1d).
For the IV graph (Figure 1b), we use four different
SCMs, two linear and two nonlinear datasets, based on the strength of the instrument variable and the confounding.
Finally, we generate a two-dimensional linear dataset for the leaky mediation causal graph (Figure 1a).
The full details of our data-generating processes can be found in Appendix G.
Baselines.
Our ﬁrst baseline is the algorithm in Hu et al.
[2021] that directly optimizes the value of
ATE using GANs.
We use their default hyper-parameters with a tolerance of 0 . 0001 .
Similar to their experimental setup, we consider 50 intermediate solutions where the distance is within the tolerance and compute the bounds using the mean and one-sided conﬁdence intervals.
We also compare our algorithm with the moment-matching method in Padh et al.
They parameterize the ﬁrst two moments of the generated distribution and match them with the observed samples.
They choose response functions as linear combinations of a ﬁxed number of basis functions.
We consider the neural basis functions for our experiments with their default hyper-parameters.
In particular, we train a 3-hidden layer MLP with 64 neurons in each layer to learn the target variable given the treatment.
We then use the activation of the k th neuron in the last hidden layer as the k th basis function.
Instead of maximizing/minimizing E [ Y T = t ] , we consider E [ Y T =
t ]  E [ Y T = t 0 ] as our goal is to bound the ATE between two points.
We leave other implementation details untouched.
5.2 Results
Bounding average treatment derivatives.
We generate data with sample sizes N = { 500 , 1000 , 2000 , 5000 } from the nonlinear Front-door and linear Back-door SCMs (identiﬁable), as well as linear IV with strong confounding and leaky mediation settings (non-identiﬁable).
We run our algorithm with ten different random seeds for each setting/sample size.
Then, we choose the ﬁve runs with the lowest tolerance parameter ↵ n and choose the upper (lower) bound as the maximum (minimum) value of the ATD within these ﬁve runs.
Figure 3 shows our derived bounds.
As expected, the algorithm is able to point-identify the value of ATD for identiﬁable scenarios as the number of samples increases (Figures 3 (a) and (b)).
In non-identiﬁable cases, our algorithm leads to tight bounds containing the true value of ATD by increasing the number of samples as depicted in
Figures 3 (c) and (d).
This is in line with our results in Theorem 1.
Bounding average treatment effects.
Here, we aim to demonstrate the effectiveness of using partial derivatives for bounding the ATE compared to the direct optimization approach.
We consider six different settings and run our algorithm for 10 different values of treatment { t i } 10 i =1 in each setting.
We compute the value of ATE w.r.t.
a ﬁxed point t 0 .
For each value of T , we generate N = 5 ,samples and run each experiment ﬁve times.
Then, we select the maximum (minimum) value of ATE within the ﬁve runs as the upper (lower) bound.
We follow the same procedure for the baselines.
To ﬁnd the bounds on ATE using our approach, we sample from a distribution with uniform density within [ t 0 , t i ] and Gaussian tails outside of [ t 0 , t i ] , and maximize/minimize the partial derivatives.
Figure 4a shows the effectiveness of this approach in comparison to the GAN baseline.
Our algorithm produces stable and tight bounds containing the true value of ATE, while the GAN baseline, which relies on the direct optimization of ATEs, results in unstable loose bounds that may not include the true value of the treatment effect.
Figure 4b compares our algorithm with the method in Padh et al.
First, we ﬁnd that their method do not contain the actual response curve in the nonlinear IV setting (left).
This is not a surprising result; since they consider a ﬁxed set of basis functions, the set of possible response curves is more restricted than ours.
If the basis functions are not carefully designed to capture the true form of the response curve, the resulting bounds can be invalid, thus not containing the actual value of ATE even if there are regimes where the approach provides a tighter ﬁt.
On the other hand, our approach considers a larger family of response functions, enabling us to both capture shape and variation in the response curve while bounding the effects.
Conclusion, Limitations and Future Work
Our work introduces a novel method to estimate bounds on average treatment effects from observational data.
Speciﬁcally, we propose optimizing the average treatment derivative, which in turn can be used to estimate the average treatment effect in treatment response curves.
Empirically we ﬁnd that the use of our method recovers known bounds on treatment effects in the discrete case and outperforms other methods based on implicit models for partial identiﬁcation in the continuous case.
There remain several limitations of this work.
Our work builds on the constrained optimization problem deﬁned by Xia et al.
[2021] instantiated in the context of the ATD.
Developing new methods for function maximization/minimization approaches under distributional constraints remains an important direction for future work.
Our work primarily uses carefully designed synthetic datasets to evaluate our method under different constraints on the data distribution.
A larger-scale evaluation of our approach on real-world benchmarks will better help us assess the method’s practicality.
Moreover, the theory of our work is restricted to the linear SCM scenario.
We have also made regularity assumptions throughout the paper, including Assumption 1, that the true SCM can be modeled using implicit generative models with uniform confounding distribution, as well as the approximation of
UATD with regular treatment distributions.
Consequently, practitioners must exercise caution when deploying this method when there are nonlinear or irregular structures among the random variables.
Finally, we have stated our results (Theorem 1 and Corollary 1) without quantifying the ﬁnite-sample estimation uncertainty.
One can use the existing theory of ﬁnite-sample convergence of empirical
Wasserstein distance in Weed and Bach [2019] to extend our results with high-probability bounds.
Acknowledgments
We thank Tom Ginsberg and Michael Cooper for many helpful discussions.
We also thank David
Alvarez Melis for his suggestion on using Sinkhorn divergence instead of Wasserstein distance.
This research was supported by NSERC Discovery Award RGPIN-2022-04546 and a CIFAR AI Chair.
Resources used in preparing this research were provided, in part, by the Province of Ontario, the
Government of Canada through CIFAR, and companies sponsoring the Vector Institute.
Paper checklist
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] We have listed the key contributions of our work in the introduction.
In the experimental section we verify our theoretical contributions and perform a comparison to the closest baseline.
Our theoretical results are in idealized linear settings though we do experiment beyond that in our work.
We highlight the limitations of our theoretical results and explain the rationale for the approximations that we make in practice.
(b) Did you describe the limitations of your work?
[Yes] We explicitly highlight the limitations of our work in the context of related work, in our exposition of the theoretical results and the computation necessary for our method.
Our theoretical results make explicit the assumptions that we make and full proofs are presented in the supplementary material.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] Our paper explicitly tackles the problem of bounding treatment effects from observational data.
We have highlighted that our theory is focused on the case of linear models and provided empirical results supporting the method’s utility in the general, non-linear case.
If the method is deployed in high-risk settings such as healthcare, there may be times where the method fails to estimate bounds correctly from real-data in non-linear settings.
We have encouraged practitioners to excercise caution in this regard.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] We include the complete proofs of our results in the supplementary material.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Our supplemental material makes clear the training procedure for our method, the main sets of hyperparameters and any heuristics for optimizing the models.
We will release code to reproduce our results.
Our work made no use of participants.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Our method ﬁnds maximum/minimum values on the quantity of interest.
Instead of reporting error bars, we consider maximum/minimum values among multiple runs with different random seeds.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] Experiments were run on two a single machine with 2 Nvidia RTX A5000 GPUs, 2 RTX A6000 cards GPUs, 96 CPUs and 188GB of RAM and on a larger internal cluster comprising hundreds of GPUs.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] Our evaluation is primarily on synthetic data for which we have cited the original authors who created them.
(c) Did you include any new assets either in the supplemental material or as a URL?
[-Since Wmetrizes P 1 , we have W 1 ( P n , P ) → 
-]
[No] (d) Did you discuss whether
[-Hence, θ 
-]
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] We use publicly available datasets that do not contain unanonymized personal data.