BAYESIAN STATISTICS 9, pp. 569–610.
J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid,
D. Heckerman, A. F. M. Smith and M. West (Eds.)c Oxford University Press, 2011

Transparent Parametrizations ofModels for Potential OutcomesThomas S. Richardson,

Robin J. Evans

University of Washington, USA
thomasr@uw.edu,
rje42@uw.edu

James M. Robins
Harvard School of Public Health, USA
robins@hsph.harvard.edu
Summary
We consider causal models involving three binary variables: a randomized
assignment Z, an exposure measure X, and a final response Y . We focus
particular attention on the situation in which there may be confounding of
X and Y , while at the same time measures of the e⇤ect of X on Y are ofprimary interest. In the case where Z has no e⇤ect on Y , other than throughZ, this is the instrumental variable model. Many causal quantities of interest
are only partially identified. We first show via an example that the resulting
posteriors may be highly sensitive to the specification of the prior distribution
over compliance types. To address this, we present several novel “transparent”
re-parametrizations of the likelihood that separate the identified and nonidentified parts of the parameter. In addition, we develop parametrizations
that are robust to model mis-specification under the “intent-to-treat” null
hypothesis that Z and Y are independent.
Keywords and Phrases: Bounds; Continuous Covariates; Exclusion
Restriction; Instrumental Inequality; ITT-Null-Robust; Model
Mis-Specification; Parametrization; Prior Sensitivity.

1. INTRODUCTION
The potential outcomes model for causal inference is a well-established frameworkfor formalizing causal assumptions and modelling causal e⇤ects; see Neyman (1923);
Rubin (1974). However, in many contexts, the causal estimands of interest are not
identified by the observed data. Even in the asymptotic limit, there may remain arange of values for a parameter of interest that are logically possible, rather than asingle point. Such a parameter is partially identified, and is entirely non-identified
if, in the limit, the data impose no restriction at all.
It is often argued that identifiability is of secondary importance in a Bayesiananalysis provided that the prior and likelihood lead to a proper joint posterior for all
the parameters in the model. Following Leamer (1978), Gustafson (2005) and Greenland (2005), we argue that partially identified models should be re-parameterized so

570

T. S. Richardson, R. J. Evans and J. M. Robinsthat the complete parameter vector may be divided into point-identified and entirely
non-identified subvectors. Such an approach facilitates “transparency”, allowing areader to see clearly which parts of the analysis have been informed by the data. In
addition, it makes it simpler for someone to incorporate their own prior beliefs thatmay di⇤er from those of the analyst.
In this paper, we first motivate the approach by considering a simple instrumental variable model for a randomized trial with non-compliance, in which the“instrument” Z, the exposure measure X, and the final response Y are all binary.
We then extend this approach to the analysis of a randomized encouragement design, though still with binary treatment and response, under a variety of di⇤erent
assumptions.
In Section 5, we develop novel variation independent smooth parametrizations
that permit this approach to be applied in the context of continuous or discrete
baseline covariates. As the response Y is binary, these parametrizations are ofnecessity complex and somewhat non-intuitive.
In Section 6, we consider a successfully randomized controlled trial in which thefollowing both hold: (i) the random assignment of Z and (ii) the exclusion restrictionthat Z has no e⇤ect on Y except through its e⇤ect on X. The exclusion restriction
is guaranteed to hold in a placebo-controlled double-blind trial in which the active
treatment is which the active treatment is without side-e⇤ects and unavailable topatients in the control arm. Under this model, the sharp null hypothesis of no causale⇤ect of X on Y implies the Intent-To-Treat (ITT) null hypothesis that Z and Y are
independent both marginally and conditional on the baseline covariates V . We provide a second transparent parametrization with the following important robustness
property: under the ITT null, a Bayes estimator of the conditional covariance of Z
and Y given V converges to zero, even when both the estimates of the distributionp(x | y, z, v) and of p(y | z = 0, v) are inconsistent due to model mis-specification.
This is important because mis-specification is inevitable whenever V has continuouscomponents.
The paper is organized as follows: In Section 2, we introduce the notation andthe basic potential outcomes model that we consider throughout. In Section 3, we
motivate our approach via a simple example, and show how the method applies. In
Section 4, we describe eight causal models and explicitly characterize the induced
model for the joint distribution of the observed data implied by each model. In
Section 5, we extend the approach to incorporate baseline covariates. In Section 6,
we modify the approach presented in Section 5 to preserve consistency under the
ITT null.
2. BASIC CONCEPTS
Throughout this paper we consider potential outcomes models involving three binary
variables, X, Y and Z. Here:Z is a treatment, presumed to be randomized, e.g., the assigned treatment;
X is an exposure subsequent to treatment assignment;
Y is the response.
For Z, we will use 1 to indicate assignment to drug, and 0 otherwise. For X, we
use 1 to indicate that the drug is received and 0 if not. For Y , we take 1 to indicate
a desirable outcome, such as survival.

Transparent Parametrizations of Models for Potential Outcomes

571

The potential outcome Xz is the treatment a patient would receive if assigned
to Z = z. We follow convention by referring to the four compliance types as shownin Table 1. We will use tX to denote a generic compliance type, and DX the set ofsuch types.
Table 1: Compliance types describing the potential outcomes Xz ; see Imbens
and Rubin (1997).
Xz=0

Xz=1

0
0
1
1

0
1
0
1

Compliance Type
Never Taker
Complier
DefierAlways Taker

NT
CO
DE
AT

Similarly, we consider the four potential outcomes Yxz with x, z ⇣ {0, 1} for Y .
These describe the outcome for a given patient if they were to be assigned to Z = z,
and then were exposed to X = x. For a given individual, we will refer to the 4-vectorof values taken by the variables (Y00 , Y01 , Y10 , Y11 ) as their response type, tY . We use
DY to indicate the set of such types, of which there are 24 = 16 in general, though
we will often consider models in which some of these are assumed to be identical.
Since we suppose the potential outcomes are well-defined, if Z = z, then X = Xz ,similarly if X = x and Z = z, then Y = Yxz . This is referred to as the “consistency
assumption” (or axiom).
2.1. Notation
Let tX ⇧ p(tX ) denote the marginal probability of a given compliance type tX ⇣DX , andX ⇧ { t X | t X ⇣ DX }denote a distribution on DX . Similarly, we use tY |tX ⇧ p(tY | tX ) to denotethe probability of a given response type within the sub-population of individualsof compliance type tX , and Y |X to indicate a specification of all these conditional
probabilities:
Y |X ⇧ { tY |tX | tX ⇣ DX , tY ⇣ DY }.
We will use to indicate a joint distribution p(tX , tY ) on DX ⇤ DY .

We use ⇤tij for the probability of recovery for a patient of a given complianceX
type tX , under an intervention that sets X = i and Z = j:⇤tij ⇧ p(Yx=i,z=j = 1 | tX ), for i, j ⇣ {0, 1} and tX ⇣ DX .
X
In places, we will make use of the following compact notation for probability distributions:p(yk |xj , zi )p(xj |zi )p(yk , xj |zi )Finally, we use⇧⇧⇧p(Y = k | X = j, Z = i),p(X = j | Z = i),p(Y = k, X = j | Z = i).

k to indicate the simplex of dimension k.

572

T. S. Richardson, R. J. Evans and J. M. Robins

2.2. Randomization Assumption
We will make the randomization assumption that the distribution of types (tX , tY )is the same in both the Z = 0 and Z = 1 arms:
Z◆◆ {Xz=0 , Xz=1 , Yx=0,z=0 , Yx=1,z=0 , Yx=1,z=0 , Yx=1,z=1 }.

(1)

A causal graph corresponding to the model given by (1) is shown in Figure 1.
4

T. S. Richardson, R. J. Evans & J. M. Robins
tX , tYZ

X

Y

Figure1: Graphical representation of the model given by assumption (1). The
Figure 1: Graphical representation of the model given by assumption (1). The
shaded nodes are observed. In this model tX takes 4 states, while tY takes 16.

shaded nodes are observed. In this model tX takes 4 states, while tY takes 16.

3. A SIMPLE MOTIVATING EXAMPLE

The potential outcome Xz is the treatment a patient would receive if assigned

Pearl (2000) and Chickering and Pearl (2000) use potential outcomes to analyze
to Z = z. We follow convention by referring to the four compliance types as shown
the data in Table 2, which arise from a double-blind placebo-controlled randomized
trial ofinCholestyramine;
seetXEfron
and aFeldman(1991). Complianceoriginally
Table 1. We will use
to denote
generic compliancetype, and DX was
the setofmeasured as a percentage of prescribed dosage consumed; this measure was thensuch types.
dichotomized
by Pearl. Similarly, the response was also dichotomized to indicate areduction in cholesterolof at least 28 units.
Table 1: Compliance types describing the potential outcomes X
zTable 2: Lipid/Cholestyramine data; originally considered by Efron and Freeman (1991); dichotomized by Pearl. There are two structural zeros.
z
0
0
0
0

Xz=0

x
y
0
0
0 0 1
1
0
0
1
1

Xz=1

Compliance Type

1

count
z
x
y
158
1
0
0
014
Never
1 Taker
0
1
0
1
1
0
1
Complier
0
1
1
1
172
0
Defier1

1

Always Taker

count
52
NT12
23
CO
78
DE165
AT

Table 3: Response types under the exclusion restriction (2); see Heckerman andShachter (1995).
Similarly we consider the four potential outcomes Yxz with x, z {0, 1} for Y .
YResponse
x=0·
x=1·
These describe the Youtcomefora given patient
if theyType
were to be assigned to Z = z00
Never RecoverNR
and then were exposed to X = x. For a given individual we will refer to the 4-vector
01HelpedHE
Hurt
HU
of values taken by the1variables0(Y00 , Y01, Y10 , Y11 ) as their response
type, tY . We use
1
1
Always RecoverAR
DY to indicate the set of such types, of which there are 24 = 16 in general, thoughwill often outcomesconsider models
in whichof these aresince
assumedto be identical.
Thewepotential
analysisheresome
is simplified
subjects
in the control
arm had noaccesstotreatment.
Hence,
Z=0
implies
X
=
0,
so
there
are only two
Since we suppose the potential outcomes are well-defined, if Z = z then X = X ,zsimilarly if X = x and Z = z then Y = Yxz . This is referred to as the ‘consistency

Transparent Parametrizations of Models for Potential Outcomes

573

compliance types (NT, CO). Since it is a double-blind randomized trial, Pearl also
assumes that Z has no e⇤ect on Y other than through X, or more formally:Yxz = Yxz⇧

for all x, z, z ⇧ ⇣ {0, 1}.

(2)

In this case, there are only four response types tY ; see Table 3. Consequently, there
are eight combinations for (tX , tY ) ⇣ {NT, CO} ⇤ {HE, HU, AR, NR}.
When equation (2) holds, we will use Yx· to refer to Yx,z=1 = Yx,z=0 . Similarly,
i·we let
6 ⇤t ⇧ P (Yx=i · = 1 | tX ). T. S. Richardson, R. J. Evans & J. M. Robins
XtX , tYZ
Figure 2:X

YGraphical representation of the IV model given by assumptions (2)Figure2:. Graphicalrepresentationof theIV tmodel
given
and (1)In this modeltX takes 4 states,while
4. by assumptions (2)Y takesand (1).
In thismodel
takes 4 quantity
states, whiletY takesthe
4. (global) average causal
Pearl(2000)
takes
as histXprimaryof intereste⇤ect of X on Y :A causal graph corresponding to the model given by (1) is shown in Figure 1.

ACE(X ↵ Y ) ⇧ E[Yx=1·

Yx=0· ] = (HE )(HU ).

Pearl proposes analyzing
modelMOTIVATING
by placing a EXAMPLE
prior distribution over p(tX , tY )3. AtheSIMPLEand then using Gibbs sampling to sample from the resulting posterior distribution
for ACE(X↵ Yand). Henotes thatthe resulting
to thePearl (2000)
Chickeringand Pearl(1996) useposterior
potential appears
outcomessensitive
to analyze
prior distribution and suggests that a sensitivity analysis be used.
from
double-blind placebo-controlled
8 the data in Table 3 which arises T.
S.a Richardson,
R. J. Evans &randomized
J. M. Robins
trial of Cholestyramine; see Efron and Feldman (1991). Compliance was originally
Unit & perturbed unit priors on potential outcomesDir(1,...,1,1.2,1,0.8)
dichotomized
by Pearl. Similarly the response Posteriors
wasDir(1/8,...,1/8,3/16,1/8,1/16)
also
dichotomized to indicate a
Posteriors on ACE(X ! Y)
on ACE(X ! Y)5

5

Prior and posterior on ACE(X ! Y) for Lipid data

4

Prior and posterior on ACE(X ! Y) for Lipid data

Uniform & perturbed uniform priors on potential outcomesmeasured
as a percentage of prescribed dosagePriorsconsumed;
this measure was thenPriors on ACE(X ! Y)
on ACE(X ! Y)
Dir(1/8,...,1/8)
4

Dir(1,...,1)

3

Dir(1/8,...,1/8)
Dir(1/8,...,1/8,3/16,1/8,1/16)

Densityfrom empirical distribution

Density3

Dir(1,...,1)
Dir(1,...,1,1.2,1,0.8)
reductionin cholesterol of at least 28 units.
Bounds on ACE(X ! Y)

Bounds on ACE(X ! Y)from empirical distribution

2

2

The potential outcomes analysis here is simplified since subjects in the control
1

1

arm had no access to treatment. Hence Z = 0 implies X = 0, so there are only two00compliance types (NT, CO). Since it is a double blind randomized trial Pearl also
assumes that Z has no effect on Y other than through X, or more formally:

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

ACE(X ! Y)

0.0

0.5

1.0

ACE(X ! Y)

Figure 3: Prior to posterior analysis for ACE(X ↵ Y ) for the Lipid data;
Yxz = Yare
all x, linesz, z ⇤indicate{0, 1}. bounds on the ACE(2)priors are green; posteriors
verticalxz red; forevaluated
at thedistribution.
TickACE(Xmarks indicate
Figure3: Prior
to empiricalposterioranalysis forY )respective
for the medians.
Lipid Data;
See text for further details.
In this case there are only four response types tY ; see Table 2. Consequently therepriors are green; posteriors are red; vertical lines indicate bounds on the ACE

are eight
combinations
for sensitivity.
(tX , tY ) ⇤ {NT,AR,redNR}.
NoteFigure3 illustrates
this
TheCO}
solid {HE,greenHU,andlines
in (2)
the leftplot
show,respectively,
the
prior
and
posterior
for
ACE(X
↵
Y
)
under
a
uniform
may
be
false
even
in
a
placebo-controlled
double-blind
trial,
ifdue
toside-effects
evaluated at the empirical distribution. Tick marks indicate respective mediDir(1, . . . , 1) on the distribution (tX , tY ); the dashed green and red lines indicateof the active treatment, a patient can guess with greater than the randomization

ans. See text for further details.

probability (here 0.5) the arm to which they have been assigned.
When equation (2) holds we will use Yx· to refer to Yx,z=1 = Yx,z=0 . Similarly

plot show, respectively, the prior and posterior for ACE(X
we let ti· ⇥ P (Yx=i · = 1 | tX ).
XY ) under a uniform

Dir(1, . . . , 1) on the distribution (tX , tY ); the dashed green and red lines indicate
the corresponding prior and posterior after increasing the parameter corresponding

574

T. S. Richardson, R. J. Evans and J. M. Robins

the corresponding prior and posterior after increasing the parameter corresponding
to (NT,HE) to 1.2, while reducing that for (NT,NR) to 0.8, but leaving all others
at 1. If the model were identified, we would expect such a change in the prior tohave little e⇤ect (the smallest observed count is 12). However, as the plot shows,
this perturbation makes a considerable di⇤erence to the posterior.
Experts whom we consulted, noting the fact that there was relatively little priorsupport in the range dominated by the posterior, hypothesized that the sensitivity
might be due to an insu⌃ciently di⇤use prior. It was suggested that a “unit information” prior should be used instead. The right plot in Figure 3 shows the priorand posterior for the ACE resulting from a Dir(1/8, . . . , 1/8) and under a prior inwhich the parameter for (NT,HE) is increased to 3/16 while that for (NT,NR) is
reduced to 1/16. The plot shows that the more di⇤use prior on (tX , tY ) has succeeded in increasing the spread of the prior for ACE(X ↵ Y ), but this has come at
the expense of multi-modality in the posterior, and greater prior sensitivity: notice
the di⇤erence between the posterior medians (indicated at the base of the plot).
On closer inspection, the sensitivity should not be surprising, since the observed
data contain no information allowing us to learn about the ratio of (NT,HE) to(NT,NR): patients who are of type “Helped” (HE), and “Never Recover” (NR) willboth have Yx=0 = 0; they only di⇤er with respect to their values of Yx=1 . However,patients who are “Never Takers” will never expose themselves to treatment, sothese potential outcomes are never observed (at least not without instituting a newexperimental protocol that eliminates non-compliance). Of course, the proportion ofpatients who are of type “Helped” (rather than “Never Recover”) is directly relevant
to ACE(X ↵ Y ).
3.1. Separating the Identified from the Unidentified
Figure 4 provides a graphical depiction of the functional relations between the pai·i·rameters parametrizations
the observed
distributionp(y, x|z).
Transparentof modelsfor potential
outcomes
9
X , ⇤CO , and ⇤NT , andp(y|x = 0, z = 0)

0·
CO

0·
NT

p(y|x = 0, z = 1)⇥X

p(x|z = 1)

1·
NT

1·
CO

p(y|x = 1, z = 1)

Figure 4:

A graph representing the functional dependencies in theanalysis of the simple IV model with no Always Takers or Defiers.
Figure 4:RectangularA graph representing
the functional
dependencies
in theanalysis ofnodes are observed;
oval nodes
are unknown
parameters.
p(x = 1|z = 0) = 0, so p(y|x = 1, z = 0) is undefined, hence these nodes
areIV
omitted.
the simple
model with no Always Takers or Defiers. Rectangular nodes
1·0·
0·
The parameters⇤CO, and parameters.
⇤NT
are identified
X, ⇤are observed;
oval nodesarep(x thus:= 1|z = 0) = 0, so
CO ,unknown
0·

1·
, ⇤CO
=hencepy1 |x1 ,zthese, ⇤nodespy1 ,x0 |z1 )/px1 |z1 ,p(y|x = 1, zCO==0) pisundefined,
are
x1 |z
y1 ,xomitted.
CO = (p110 |z0NT = px0 |z1 ,0·⇤NT= py1 |x0 ,z1 .

expense of multi-modality in the posterior, and greater prior sensitivity: notice thedifference between the posterior medians (indicated at the base of the plot).
On closer inspection the sensitivity should not be surprising, since the observed
data contain no information allowing us to learn about the ratio of (NT,HE) toTransparent Parametrizations of Models for Potential Outcomes

575

0·
The equation for ⇤COleads to the following restrictions on the distribution p(y, x|z):
0·
⇤CO
⌥1

py0 ,x0 |z1 ⌥ py0 ,x0 |z0 ,0·⇤CO

py1 ,x0 |z1 ⌥ py1 ,x0 |z0 .

0(3)

It is not hard to show that these inequalities define the set of distributions p(y, x|z)
arising from this potential outcome model. Consequently, we may parametrize theidentifiable portion of the model directly via the set of distributions p(y, x|z) thatobey the inequalities on the right of (3). Under a Dirichlet prior over the observed
distribution p(y, x|z), truncated so as to remove distributions violating (3), theposterior may easily be sampled from via conjugacy and Monte Carlo rejection
sampling.
As a by-product, we may also examine the posterior probability assigned to themodel defining restrictions (3) being violated under a uniform prior on the saturated
model. For the Lipid data, under this prior, the posterior probability of such a
violation is still 0.38, which is a consequence of the empirical distribution being
close to violating (3). (The prior probability of violating (3) is 0.5.) This might cast
doubt on the exclusion restrictions, Eq. (2). One possible explanation for a violationof Eq. (2), even in the context of a double-blind study, is the dichotomization of
the compliance measure; see Robins et al. (2009); Balke and Pearl (1997). Notethat although (2) implies (3), the converse does not hold. Similarly, if the posterior
probability of (3) holding is high, this does not imply that the posterior probabilityof (2) is high, unless there is high prior conditional probability that (2) is true giventhat (3) is true. This follows from the fact that the posterior probability that (2)is true given that (3) is true is equal to the conditional prior probability that (2) istrue given that (3) is true. The model that allows (2) to be violated is of the same
dimension as model (2).
In this example, where Equation (2) is assumed to hold, we could have used1·0·0·
( X , ⇤CO
, ⇤CO
, ⇤NT
) rather than p(y, x|z) to parametrize the identifiable part ofthe model. However, this approach does not generalize to more complex potential
outcome models such as those that include Defiers, or make fewer exclusion restrictions, since both X and ⇤ti· may themselves be partially identified; see Richardson
X
and Robins (2010).
3.2. Posterior Distributions for the ACE
1·
The ACE(X ↵ Y ) depends on the (wholly) unidentified parameter ⇤NT:
ACE(X ↵ Y ) =1·

CO (⇤CO

0·
⇤CO
)+

1·

NT (⇤NT

0·
⇤NT
).

1·
We elect to display the posterior for ACE(X ↵ Y ) as a function of ⇤NT; see Figure 5.
This permits readers to see clearly the dependence of the ACE on this parameter,1·
and to incorporate easily their priors regarding ⇤NT.

4. THE GENERAL FRAMEWORK
We now consider the general setting in which we do not assume Eq. (2), nor do werule out the possibility of Always Takers or Defiers. Thus, there are 4 ⇤ 16 possible
values for (tX , tY ).

576

T. S. Richardson, R. J. Evans and J. M. Robins

0.0
−1.0

−0.5

ACE(X ! Y)

0.5

1.0

ACE(X ! Y) as a function of "1#
NT

0.0

0.2

0.4

"1#
NT

0.6

0.8

1.0

Figure 5:

The posterior for the ACE(X ↵ Y ) for the Lipid data1·
displayedasafunction of the (completely)unidentified
parameter⇤NT
Figure 5: The posterior for the ACE(XY ) forthe Lipid data
displayedas:(blue) posterior median; (red) 2.5% and 97.5% quantiles; (green) si1·
multaneous
posterior region
obtainedfrom a 95%
HPD region fora function of95%
the (completely)unidentified
parameter
NT : (blue) posteriorp(y, x|z); horizontal lines are bounds on the ACE evaluated at the empirmedian;(red) 2.5%Aand
97.5% quantiles;
simultaneous
95%p(y,posteriorical
distribution.
uniform
prior was(green)used ondistributionsx | z)
that satisfy the inequalities (3).
region obtained from a 95% HPD region for p(y, x|z); horizontal lines arebounds on
the ACEevaluated
empiricalmodels
distribution.
uniform
Following
Hiranoet al.
(2000)atwetheconsider
under Awhich(1) priorholds, and(combinations of) the following three assumptions hold:
was used on distributions p(y, x | z) that satisfy the inequalities (3).

(MonX )

Monotonicity of compliance: X0 ⌥ X1 , or equivalently, there are no
Defiers.

(ExNT )

0100
Stochastic exclusion for NT under non-exposure: ⇤NT
= ⇤NT
, so
among Never Takers the distributions of Y00 and Y01 are the same.

(ExAT )

11
10
Stochastic exclusion for AT under exposure: ⇤AT= ⇤AT, so among
Always Takers the distributions of Y10 and Y11 are the same.

Note that assumption (2) implies stochastic exclusion for all compliance types⇧
under all exposures, i.e., ⇤tij = ⇤tij for all i, j, j ⇧ ⇣ {0, 1} and all tX ⇣ DX . Figure 6
X
X
and Table 4 list these eight models. Imposing other exclusion restrictions, besides
ExAT or ExNT , will correspond to merely relabeling a single node ⇤tij in Figure 6
Xwith ⇤ti· . Thus, although the causal interpretation of estimands may change, theX
implied set of compatible distributions p(y, x|z) will not.

The saturated model p(y, x|z) consists of the Cartesian product of two threedimensional simplices: 3 ⇤ 3 . The other seven models are all characterized bysimple inequality restrictions on this set.
4.1. Inequalities Defining Models with Defiers
Results of Balke and Pearl (1997) and Bonet (2001) imply that the set of distributions arising from a potential outcomes model satisfying (1), ExAT and ExNT may

Transparent Parametrizations of Models for Potential Outcomes40
p(y|x0 , z0 )00COp(x|z0 )p(y|x1 , z0 )

00
NT

10
AT

p(y|x0 , z0 )
10
DE

00
CO

11
AT

p(x|z1 )10DEp(x|z0 )

11
CO

01
DE

p(y|x1 , z1 )11COp(y|x0 , z1 )p(y|x1 , z0 )10ATp(y|x0 , z0 )
10
DE

01
DE

11
CO

00
CO

p(x|z0 )p(y|x1 , z1 )11COp(y|x0 , z1 )p(y|x1 , z0 )p(y|x0 , z0 )
00
CO

10
AT

01
NT

11
AT

p(x|z0 )11COp(y|x1 , z1 )p(y|x1 , z0 )

⇥X

1·
AT

p(x|z1 )p(y|x1 , z1 )p(y|x0 , z0 )p(x|z0 )p(y|x1 , z0 )

0·
NT

⇥X

1·
AT

00
CO

⇥X
11
AT

p(x|z1 )p(y|x1 , z0 )11COp(y|x0 , z1 )10ATp(y|x0 , z1 )p(x|z0 )MonX + ExAT

00
CO
0·
NT

p(y|x1 , z1 )

01
NT

MonX

p(y|x0 , z0 )p(x|z1 )

00
NT

⇥X

p(x|z1 )

1·
AT

⇥X

ExAT + ExNT00
NT

p(y|x0 , z1 )p(y|x1 , z0 )

01
DE

ExNT

p(y|x0 , z0 )p(x|z0 )

10
DE
0·
NT

11
AT

p(x|z1 )p(y|x1 , z1 )

00
CO

⇥X

p(y|x0 , z1 )p(x|z1 )

ExAT

00
CO
0·
NT

1·
AT

01
NT

Saturatedp(y|x0 , z0 )p(y|x1 , z0 )

⇥X

01
NT

p(y|x0 , z1 )p(x|z0 )

00
NT

⇥X
01
DE

577

T. S. Richardson, R. J. Evans & J. M. Robins

11
CO

11
CO

p(y|x1 , z1 )p(y|x0 , z1 )MonX + ExNT

p(x|z1 )p(y|x1 , z1 )MonX + ExAT + ExNT

ij ⇤Figure Figure
6: Functional
dependencies
the eight
eightmodels.
models.Terms
Terms
that do not
8: Functional
dependencies inin thetX dotX thatappear in the likelihood are not shown. See also Table 4.

ij

not appear in the likelihood are not shown. See also Table 4.

be characterized via the following inequalities:p(y0 , x0 | z0 ) + p(y1 , x0 | z1 ) ⌥ 1,

p(y1 , x0 | z0 ) + p(y0 , x0 | z1 ) ⌥ 1,

(4)

p(y0 , x1 | z0 ) + p(y1 , x1 | z1 ) ⌥ 1,

p(y0 , x1 | z1 ) + p(y1 , x1 | z0 ) ⌥ 1.

(5)

Note that any distribution p(y, x | z) can violate at most one of these four inequalities. In addition, they are invariant under relabeling of any variable. Cai et al.
(2008) give a simple interpretation of the inequalities in terms of bounds on averagecontrolled direct e⇤ects in the potential outcomes model that only assumes (1):p(y0 , xi | z0 ) + p(y1 , xi | z1 )1 ⌥ ACDE(xi ) ⌥ 1

p(y0 , xi | z1 )p(y1 , xi | z0 ) (6)578

T. S. Richardson, R. J. Evans and J. M. Robins

where ACDE(x) ⇧ E[Yx1 Yx0 ]. We may also obtain bounds on average controlled
direct e⇤ects for AT and NT:p(y0 , x0 |z10 ) + p(y1 , x0 |z10)p(y0 , x0 |z10) + p(y1 , x0 |z10 )⌥ ACDENT (x0 ) ⌥p(x0 |z0 ) p(x1 |z1 )p(x0 |z0 ) p(x1 |z1 )1

p(y0 , x1 |z1 ) + p(y1 , x1 |z0 )p(y0 , x1 |z0 ) + p(y1 , x1 |z1 )⌥ ACDEAT (x1 ) ⌥p(x1 |z1 ) p(x0 |z0 )p(x1 |z1 ) p(x0 |z0 )

1

1,
1,

where ACDEtX (x) ⇧ E[Yx1 Yx0 | tX ]. Causal contrasts such as ACDENT (x0 )and ACDEAT (x1 ) were introduced in Robins (1986, Section 12.2), in the context
of estimating treatment in the presence of censoring by competing causes of death.
Rubin (1998, 2004) and Frangakis and Rubin (2002) later coined the term “principal
stratum direct e⇤ect”. Bounds for ACDEAT and ACDENT have been derived byZhang and Rubin (2003); Hudgens et al. (2003) and Imai (2008).
ACDE(x0 ) may be bounded away from 0 i⇤ ACDENT (x0 ) may be bounded awayfrom 0 in the same direction (hence ExNT does not hold); see Kaufman et al. (2009)
and Cai et al. (2008). Likewise, with ACDE(x1 ), ACDEAT (x1 ) and ExAT . Notethat since any distribution p(y, x|z) may violate at most one of the four inequalities(4) and (5), in the absence of further assumptions (such as MonX ), every distribution
is either compatible with ExAT or ExNT (or both).
It may be shown that the model imposing ExNT alone is characterized by (4),while the model imposing ExAT is given by (5); see Richardson and Robins (2010).
4.2. Weaker Assumptions
It is worth noting that the inequalities (4) and (5) are implied by the larger counterfactual model which only assumes:p(Yx=i,z=0 = 1) = p(Yx=i,z=1 = 1),

for i = 0, 1,

(7)

and
Z◆
◆ Yx=0,z andZ◆◆ Yx=1 .,z

for z=0,1.

(8)(Note that (7) is implied by (2), though they are not equivalent.) This follows fromthe following simple argument. For i, j, k ⇣ {0, 1},p(Yx=i,z=k = j)==⌥p(Yx=i,z=k = j | Z = k)p(Yx=i,z=k = j, X = i | Z = k)
+ p(Yx=i,z=k = j, X = 1 i | Z = k)p(Y = j, X = i | Z = k) + p(X = 1 i | Z = k),(9)where the first equality follows from (8). It follows that:max p(Y = 1, X = i | Z = k) ⌥ p(Yx=i,z=k = 1) ⌥ min
1 p(Y = 0, X = i | Z = k⇤ ),⇤kkwhere the lower bound is obtained from (9) taking j = 0. The requirement that thelower bound be less than the upper bound, together with (7) then directly implies(4) with i = 0, and (5) with i = 1. Thus, (4) and (5) will hold in contexts whereZ⌘◆◆Xz , for example, where there is confounding between Z and X, or even whereTransparent Parametrizations of Models for Potential Outcomes579

Xz is not well-defined, provided that (7) and (8) hold. Further, we do not requirejoint independence:Z◆◆Yx0 , Yx1 .
(10)

However, we know of few realistic contexts where one would know that (8) holds,but (10) does not.
Robins (1989) considered the ACE(X ↵ Y ) under the model given by (7) and(8), deriving what Pearl (2000) calls the “natural bounds”; see also Manski (1990).
Although the natural bounds are sharp under the assumption of (7) and (8) alone,they are not sharp under the stronger assumption (1) and (7), for which Pearlderived the bounds. This is interesting given that, as we have seen, both of these
independence assumptions, combined with (7), lead to the same set of distributions
p(x, y | z) for the observables, characterized by (4) and (5). Finally, we note thatin fact the ACE bounds derived by Pearl assuming (7) and (1) are also implied by
the weaker assumption (10), i.e., without requiring Z◆◆Xz=0 , Xz=1 (Richardson andRobins, 2011). This appears to contradict a remark in Pearl (2009, p. 395).
4.3. Inequalities Defining Models without Defiers
The assumption MonX , that there are no Defiers, implies:p(x1 | z1 )p(x1 | z0 ),(11)

since the left and right sides are the proportions of (AT or CO) and AT respectively.
Thus (11) characterizes the observed distributions resulting from MonX alone.
Results of Balke and Pearl (1997) imply that the model assuming MonX + ExNT+ ExAT implies the following inequalities:p(y1 , x0 | z1 ) ⌥ p(y1 , x0 | z0 ),p(y0 , x0 | z1 ) ⌥ p(y0 , x0 | z0 ),(12)p(y1 , x1 | z1 )p(y0 , x1 | z1 )(13)

p(y1 , x1 | z0 ),p(y0 , x1 | z0 ).

The inequalities (12) and (13) imply (11), (4) and (5). A distribution p(y, x | z) may
violate all of the inequalities (12) and (13) simultaneously. However, if (11) holds,
then at most one inequality in each of the pairs (12) and (13) may be violated. The
inequalities (12) and (13) are invariant to relabeling Y , and to relabeling X and Z
simultaneously, but not individually; this is not surprising since relabeling X or Z
alone will turn Defiers into Compliers and viceversa.
It may be shown that (12) and (13) characterize the set of distributions p(y, x|z)
arising from the potential outcomes model MonX + ExNT + ExAT . Likewise, themodel imposing MonX + ExNT is characterized by (11) and (12), while MonX +ExAT is given by (11) and (13).
An interpretation of (12) and (13) is given by the following lower bound on DE
in the model that imposes ExNT + ExAT (but not MonX ):89p(x1 | z0 ) p(x1 | z1 ),
>
>
< 0,
=
p(y1 , x0 |z1 ) p(y1 , x0 |z0 ), p(y0 , x0 |z1 ) p(y0 , x0 |z0 ),; (14)
max
DE
>
>
:
;
p(y1 , x1 |z0 ) p(y1 , x1 |z1 ), p(y0 , x1 |z0 ) p(y0 , x1 |z1 )see Richardson and Robins (2010). Requiring that the lower bound be zero, as
required by MonX , leads directly to the inequalities (11), (12) and (13).

580

T. S. Richardson, R. J. Evans and J. M. Robins

Another interpretation of (12) and (13) arises in the model MonX that (solely)assumes that there are no Defiers. Under MonX , we may obtain tighter bounds on
the ACDE for AT and NT:p(y1 |x0 , z1 )max (0, 1

min (p(y1 , x0 |z0 )/p(x0 |z1 ), 1) ⌥ ACDENT (x0 ) ⌥p(y1 |x0 , z1 )(p(y0 , x1 |z1 )/p(x1 |z0 )))max (0, 1(p(y0 , x0 |z0 )/p(x0 |z1 ))) ,p(y1 |x1 , z0 ) ⌥ ACDEAT (x1 ) ⌥min (p(y1 , x1 |z1 )/p(x1 |z0 ), 1)p(y1 |x1 , z0 ).

(15)

(16)

However, the bounds (6) on the global ACDE(xi ) remain sharp, being unchangedby the assumption of monotonicity.
It is simple to show that ACDENT (x0 ) is bounded away from 0 by (15) i⇤ oneof the inequalities (12) is violated; likewise for ACDEAT (x1 ), (16) and (13). Thus,
if MonX , and hence (11) holds, then, as mentioned above, at most one inequality ineach of the pairs (12) and (13) may be violated. However, in contrast to the casewithout the monotonicity assumption, since it is possible for a distribution p(y, x|z)
to violate one inequality in each pair simultaneously, ACDENT and ACDEAT may
both be bounded away from zero. Thus, under the assumption of No Defiers both
ExNT and ExAT may be inconsistent with p(y, x|z).
Finally, we note that in the situation where (12) and (13) hold, the natural
bounds on ACE(X ↵ Y ) are sharp (regardless of whether MonX holds or Zx is
undefined).
Table 4 summarizes the constraints for the eight models we consider. For frequentist approaches to testing these constraints see Ramsahai (2008).
Table 4: Models and implied sets of distributions for p(y, x|z); (12) and (13)
imply (11).
Model

Assumptions

Constraints on p(y, x|z)

Saturated
ExNT
ExAT
ExAT + ExNT

Randomization (1)(1), Exclusion for NT(1), Exclusion for AT(1), Exclusion for AT and NT

None
(4)
(5)
(4), (5)MonX
MonX + ExNTMonX + ExATMonX
+ ExNT + ExAT

(1), No Defiers(1), No Defiers, Exclusion for NT(1), No Defiers, Exclusion for AT(1), No Defiers,
Exclusion for NT and AT(11)(11), (12)(11), (13)[(11)], (12), (13),ANALYSIS OF FLU VACCINE DATA
We consider the influenza vaccine data from McDonald et al. (1992), which waspreviously analyzed by Hirano et al. (2000); see Table 5. Here, the instrument Z
was whether a patient’s physician was sent a card asking them to remind patients
to obtain flu shots, or not; X is whether or not the patient did in fact get a flu shot.
Finally, Y = 1 indicates that a patient was not hospitalized.

should be X_z

Transparent Parametrizations of Models for Potential Outcomes

581

Table 5: Summary of Flu vaccine data; originally from McDonald e tal. (1992);
analyzed by Hirano et al. (2000).
z
0
0
0
0

x
0
0
1
1

y
0
1
0
1

count
99
1027
30
233
1389

p(y, x|z0 )
0.071
0.739
0.022
0.168

z
1
1
1
1

x
0
0
1
1

y
0
1
0
1

count
84
935
31
422
1472

p(y, x|z1 )
0.057
0.635
0.021
0.287

To examine the support for the restrictions on p(y, x|z), we fitted a saturated
model with uniform priors and then evaluated the posterior probability that theinequalities (4), (5), (11), (12) and (13) are violated. For a model without covariates,
these probabilities are shown in the first line of Table 7. The posterior probability
that at least one of the inequalities (13) fails to hold is greater than 0.5; a similar
conclusion may be arrived at by inspection of the row of Table 5 for (y = 0, x = 1). If(13) is violated, then, under the assumptions of no Defiers (which seems plausible)
and randomization, there is a direct e⇤ect for Always Takers.
Hirano et al. (2000) place priors over the (partially) identified parameters of thepotential outcome model and compute posteriors for the Intent-To-Treat e⇤ect:
ITTtX ⇧ E[YXz1 1

YXz0 0 | tX ]for NT, AT and CO under the models MonX , MonX +ExAT , MonX +ExNT and
00
MonX +ExAT +ExNT . Under additional exclusion assumptions for compliers, ⇤CO
=
01
10
11
⇤CO and ⇤CO = ⇤CO , ITTCO is equal to the Complier Average Causal E⇤ect of X
1·0·
on Y , ACECO (X ↵ Y ) ⇧ E[YX1 YX0 | tX ] = ⇤CO⇤CO.
In Figure 7, we display the joint posterior distributions over upper and lower
bounds on ITTCO under each of the eight models we consider. (Each scatterplot isbased on 2000 simulations.) The bounds were computed by applying the methods
described in Sections 2 and 3 of Richardson and Robins (2010).
5. INCORPORATING COVARIATES
In many situations, we wish to examine causal e⇤ects in sub-populations definedby baseline covariates V . In this situation, we assume that the randomization assumption (1), and (when we impose them) MonX , ExAT , and ExNT hold within
levels of V . With discrete covariates taking a small number of levels, we may simply repeat our analysis within each level of V . However, in order to incorporate
continuous baseline covariates, we require a parametrization of each of the sets of
distributions appearing in Table 4. For each model, we provide a smooth variation
independent parametrization of the relevant subset of 3 ⇤ 3 . This allows us toconstruct (multivariate) generalized linear models for p(y, x|z) as a function of V .
5.1. Parametrization of Models with Defiers
For each v, consider the set of distributions p(y, x|z, v) that result from models
assuming both ExAT and ExNT , and hence satisfy the inequalities (4) and (5) for
each v. In the following development, all models and probability statements areconditional on v, which we suppress in the notation.

for potential
582 Transparent parametrizations
T. of
S.models
Richardson,
R. J.outcomes
Evans and J. M.21Robins

0.6

1.0

0.6

1.0

0.2

0.6

0.6

1.0
−1.0 −0.6 −0.2

0.2

0.2

Upper Bound on ITTCO

0.6

1.0
0.2

−1.0 −0.6 −0.2

Upper Bound on ITTCO−1.0 −0.6 −0.2

1.0

−1.0 −0.6 −0.2

0.2

0.6

Lower Bound on ITTCOMonXMonX + EXATMonX + EXNTMonX + EXAT + EXNT

0.6

1.0

0.2

0.6

Lower Bound on ITTCO

1.0

5

1.0

Density

0

1

2

3

4

0.6
0.2

Upper Bound on ITTCO−1.0 −0.6 −0.2

−1.0 −0.6 −0.2

0.6
0.2

Upper Bound on ITTCO
0.2

Lower Bound on ITTCO−1.0 −0.6 −0.2

0.6
0.2
−1.0 −0.6 −0.2

1.0

Lower Bound on ITTCO

1.0

Lower Bound on ITTCO

1.0

Lower Bound on ITTCO−1.0 −0.6 −0.2

Upper Bound on ITTCO

0.6

1.0
0.2

0.2

Upper Bound on ITTCO−1.0 −0.6 −0.2

EXAT + EXNT

EXNT

−1.0 −0.6 −0.2

0.6
0.2
−1.0 −0.6 −0.2

Upper Bound on ITTCO

1.0

EXAT

−1.0 −0.6 −0.2

Randomization Only

−1.0 −0.6 −0.2

0.2

0.6

Lower Bound on ITTCO

1.0

−1.0

−0.6

−0.2

0.2

0.6

1.0

ITTCO

Figure 7:

Posterior distributions for upper and lower bounds on
ITTCO ; under MonX +ExAT +ExNT the parameter is identified.
Figure 6: Posterior distributions for upper and lower bounds on ITTCO ; underparameterp(y,is identified.
ItMon
is clearthatfor any
distribution
x|z0 ) there exists a distribution p(y, x|z1 )X +Ex
AT +ExNT thesuch that the pair satisfy (4) and (5). Thus, the set of distributions obeying (4) and(5) is:˛⌥
ffParametrization of Models with\
Defiers˛
˛p(y, x|z) ˛ p(y, x|z0 ) ⇣ 3 , p(y, x|z1 ) ⇣ 3
Hij (p(y1 i , xj |z0 ))(17)
For each v, consider the set of distributions p(y,i,j⌃{0,1}x|z, v) that result from models

whereassuming
Hij (p(yboth
⇧ Ex
{p(y,
x|z1hence) | p(y⌥ 1 p(y(4)
xj |z(5)Exandsatisfythefor i.e., a
1 i, xj |z
0 ))i , xj |z
1 ) inequalities
1 i ,and0 )},ATNT , andhalf-space. We parametrize the set (17) via the parameters: p(x1 |zi ), p(y1 |xi , z0 )v. In the following development all models and probability statements are
i = 0,each
1 and
two further parameters, ⇣0 , ⇣1 where„ we suppress in the notation.
«
conditional on v, whichp(y0 , xi |z1 )(1 p(y1 , xi |z1 ) {p(y0 , xi |z0 )})⇣i ⇧ log.
(18)p(y1distribution, xi |z1 )(1 p(y,p(y|z1 ) exists{p(ya1distribution, xi |z0 )}) p(y, x|z1 )0 ,0x
It is clear that for any
x|z) ithereThus,such⇣i thatreplacesthesatisfy
parameterp(y1Thus|xi , zthe(4) andobeying(5), p(ythe pair(4) and (5).
of distributions(4)1 |xand
1 ). setUnderi , z1 ) is
not variation independent of p(x1 |zi ) and p(y1 |xi , z0 ). In contrast, p(x1 |zi ) andis:p(y1 |x(5)i , z0 ), ⇣0 and ⇣1 are variation independent. The inverse map from the variation independent8
˛ parameters to p(y1 |xi , z1 ) is given by:
9
<

˛

\

=

˛
p(y1 |xi , zp(y,1) =x|z) ˛˛ p(y, x|z0 ) ⌅ 3 , p(y, x|z1 ) ⌅ 3 ⇧
Hij (p(y1 i , xj |z0 ))(17)„:«ffi “;q˛”i,j⇥{0,1}
bi + b2i + 4(e⌅i 1)p(xi |z1 )(1 p(y0 , xi |z0 ))2(e⌅i 1)p(xi | z1 ) ,where Hij (p(y1 i , xj |z0 )) ⇥ {p(y, x|z1 ) | p(yi , xj |z1 ) ⇤ 1

p(y1 i , xj |z0 )}, i.e. afor i = 0, 1, where bi = e⌅i (p(x1 i |z1 ) p(y1 , xi |z0 )) + p(xi |z1 ) + 1 p(y0 , xi |z0 ).
half-space. We parametrize the set (17) via the parameters: p(x1 |zi ), p(y1 |xi , z0 )If we let„«p(y0 , xi |z1 )(1 p(y1 , xi |z1 ))⇣˜i ⇧ log,(19)p(y1 , xi |z1 )(1 p(y0 , xi |z1 ))the parameter defined by removing the terms in braces from (18), then the model
imposing ExAT alone may be parametrized via (p(y, x|z0 ), p(x|z1 ), ⇣˜0 , ⇣1 ). Similarly,(p(y, x|z0 ), p(x|z1 ), ⇣0 , ⇣˜1 ) parametrizes the model imposing ExNT alone.
Inverse maps for these models are similar to those for ExAT + ExNT .

Transparent Parametrizations of Models for Potential Outcomes

583

5.2. Parametrization of Models without Defiers
The model with MonX alone may be parametrized via p(y, x|z0 ), ⌦x|z1 and p(y|x1 , z1 ),where⌦x|z1 ⇧ logit(p(x0 |z1 )/p(x0 |z0 )).
The model MonX + ExNT + ExAT may be parametrized via p(y, x|z0 ), ⌦x|z1 , ✏0and ⌘1 , where the latter are defined via:„«p(y0 , x0 |z1 )(1 p(y1 , x0 |z1 ) {1 p(y1 , x0 |z0 )})✏0 ⇧ log,p(y1 , x0 |z1 )(1 p(y0 , x0 |z1 ) {1 p(y0 , x0 |z0 )})
⇧

⌘1

log

„

p(y1 , x1 |z1 ))(p(y0 , x1 |z1 )p(y0 , x1 |z1 ))(p(y1 , x1 |z1 )(1(1{p(y0 , x1 |z0 )}){p(y1 , x1 |z0 )})«.

The inverse map from (p(y, x|z0 ), ⌦x|z1 , ✏0 , ⌘1 ) to p(y, x|z) is given by:p(x0 |z1 )=p(y1 , x0 |z1 )=p(x0 |z0 )expit(⌦x|z1 ),„q
c0 + c20 + 4(e⇤0p(y0 , x0 |z1 )=p(x0 |z1 )p(y0 , x1 |z1 )=1

p(y1 , x1 |z1 )=1«ffi “
1)p(x0 |z1 )p(y1 , x0 |z0 )2(e⇤0p(y1 , x0 |z1 ),p
c1 + c21 + 4(e⇧1p(x0 |z1 )1)(1 + p(x0 |z1 ))(12(e⇧1 1)”1) ,p(y0 , x1 |z0 ))!

,p(y0 , x1 |z1 ),

where
c0

=

e⇤0 (p(y0 , x0 |z0 )

c1

=

1

p(x0 |z1 )) + p(y1 , x0 |z0 ) + p(x0 |z1 ),e⇧1 (p(y1 , x1 |z0 ) + p(x0 |z1 )) + 1

p(y0 , x1 |z0 ) + p(x0 |z1 ).

Hirano et al. (2000) give an alternative variation independent parametrization
for the observed data distribution under this model. The fact that the Hirano
et al. model parametrizes the observed data distribution is a consequence of the factthat this model is nonparametrically identified; see also our rejoinder for further
discussion.
MonX + ExAT may be parametrized via p(y, x|z0 ), ⌦x|z1 , ✏˜0 and ⌘1 , where„«p(y0 , x0 |z1 )(1 p(y1 , x0 |z1 ))˜✏0 ⇧ logp(y1 , x0 |z1 )(1 p(y0 , x0 |z1 ))simply omits the terms in braces in ✏0 .
MonX + ExNT may be parametrized via p(y, x|z0 ), ⌦x|z1 , ✏0 and ⌘˜1 , where„«(1 p(y1 , x1 |z1 ))p(y0 , x1 |z1 )⌘˜1 ⇧ log(1 p(y0 , x1 |z1 ))p(y1 , x1 |z1 )again simply omits the terms in braces in ⌘1 .
Inverse maps for these models are similar to that for MonX + ExNT + ExAT .
Note that the parameters p(y, x|z0 ), ⌦x|z1 , ✏˜0 and ⌘˜1 provide an alternative
parametrization of the model MonX .

584

T. S. Richardson, R. J. Evans and J. M. Robins
Table 6: Parametrization of Models. Distributions appearing in the parameter
list are unrestricted.
Model

Parameters

Saturated
ExNT
ExAT
ExAT + ExNT

p(x, y|z)p(x, y|z0 ), p(x|z1 ), 0 , ˜1p(x, y|z0 ), p(x|z1 ), ˜0 , 1p(x, y|z0 ), p(x|z1 ), 0 , 1

MonX
MonX + ExNTMonX + ExATMonX
+ ExNT + ExAT

p(x, y|z0 ), ⇧x|z1 , p(y|x, z1 )p(x, y|z0 ), ⇧x|z1 , ⌥0 , ˜1p(x, y|z0 ), ⇧x|z1 , ⌥˜0 , 1p(x, y|z0 ), ⇧x|z1 , ⌥0 ,1

5.3. Flu Vaccine Data Revisited
Following the analysis of Hirano et al. (2000), we consider the baseline covariates
Age, and COPD (chronic obstructive pulmonary disease). Table 7 shows the posterior probability of violations of constraints under saturated models stratifying
on COPD, and under a model specified via 6 logistic regressions (for p(x|z) andp(y|x, z)) each with intercept, Age, COPD and COPD⇤Age.
Table 7: Posterior probabilities that inequalities are violated under models that
do not impose constraints. The two models without Age used a uniform prioron 3 ⇤ 3 ; the model with Age used logistic regressions with Normal priors.
Columns (4), (5), (12) and (13) give the probability that at least one inequality isviolated; (12)+(13) is the probability of at least one violation in both pairs; (12)bis the probability that both inequalities are violated; similarly for (13)b.
age
60
60
70
70
80
80

copd
N
Y
N
Y
N
Y
N
Y

(4)
0
0
0
0
0
0
0
0
0

(5)
0
0
0
0
0
0
0
0
0

(11)
0
0
0.001400.006400.00800.00020.0608(12)0.06030.0704
0.29690.07680.30160.04220.41540.09000.5338(13)0.5411
0.46350.58650.26000.62220.59580.55800.80640.5320(12)+(13)0.03430.03470.18290.03060.20740.02880.2626
0.07640.3214(12) b000.000300.001400.002600.0116(13) b000.000300.001600.003000.0128To illustrate our parametrization, we fitted the four models that include MonX .
Figure 8 shows posterior distributions on ITTCO under MonX + ExNT + ExAT inwhich this parameter is identified, and posterior distributions on bounds under theother three models. Each model was specified via logistic regressions for p(y|x0 , z0 ),p(y|x1 , z0 ), p(x|z0 ) and linear models for ⌦x|z1 , ✏0 (or ✏˜0 ) and ⌘1 (or ⌘˜1 ), again
each with intercept, Age, COPD and COPD⇤Age. Independent N (0, 3) priors wereused for all 6 ⇤ 4 coe⌃cients. Sampling was performed via a Metropolis algorithm.
The proposal for each of the six GLMs was multivariate normal, mean 0, covarianceTransparent Parametrizations of Models for Potential Outcomes
MonX












































MonX + ExNT



























































MonX + ExAT





















































MonX + ExNT + ExAT





























585




















7: Posteriors on lower (red) and upper (blue) bounds, and point esti-

Figure 8: Flu vaccine data. Posteriors on lower and upper bounds for ITTCO ,when not identified, and on ITTCO itself, when identified, as a function of Age and
COPD for the four models which preclude Defiers; medians are solid, pointwise
credible intervals are dashed. p(y, x|z0 , v) was parametrized via logits ⌃y|x0 ,z0 ,⌃y|x1 ,z0 and ⌃x|z0 . These logits and ⌦x|z1 , ✏0 (or ✏˜0 ) and ⌘1 (or ⌘˜1 ) aremodelled as linear functions of Age and COPD.

matrix ˆk2 VT V where V is the n ⇤ 4 model matrix, and ˆk2 (k = 1, . . . , 6) is an
estimate of the variance of the specific parameter, obtained via the delta method
at the empirical MLE for p(y, x|z). There were 2000 burn-in iterations followed
by 5000 main iterations. The Markov chain was initialized by setting all of thegeneralized linear model parameters to 0. Our results are generally consistent withthose obtained without including covariates; see the second row of Figure 8.

586

T. S. Richardson, R. J. Evans and J. M. Robins

6. PROTECTING THE CAUSAL NULL HYPOTHESIS FROM POSSIBLE
MODEL MIS-SPECIFICATIONIn this section, we consider a successfully randomized clinical trial (RCT) with data
available on a vector V of baseline covariates that includes continuous covariates,
such as height and weight, and with randomization probabilities p(Z = 1 | V ) that do
not depend on V . We further assume that Eqs. (1) and (2) hold for each v; hence,the causal model ExAT + ExNT holds within strata defined by V . The induced
model for the observed data is characterized by Eqs. (4) and (5) holding for each v.
Under this model, the sharp null hypothesis
Yx=1 = Yx=0 = Y(20)

of no causal e⇤ect of X on Y for each subject implies both the conditional andunconditional intention to treat (ITT) null hypothesesp(Y = 1 | Z = 1, V )p(Y = 1 | Z = 0, V ) = 0(21)p(Y = 1 | Z = 1)

p(Y = 1 | Z = 0) = 0.

(22)

and
Thus, a test of either of these ITT null hypotheses is a test of the sharp null (20).
Since the conditional ITT null (21) implies the unconditional null (22) but not viceversa, a test of the conditional null is preferable. Furthermore, tests that use data
on V may be more powerful than tests that ignore V .
Although the cost of RCTs is often an order of magnitude greater than the costof an observational study, the U.S. FDA will generally only license a new drug if
benefit has been demonstrated in such a randomized trial. The primary reason
behind this policy is that, in contrast to observational studies, the sharp null canbe empirically tested when Eq. (2) holds. Thus, it is critical to analyze these trials
with a robust methodology that guarantees that, under the ITT null, the estimatorof p(Y = 1 | Z = 1, V ) p(Y = 1 | Z = 0, V ) converges to zero in probability under
the true distribution of (Z, X, Y, V ) as the sample size n ↵ ✏, even under model
mis-specification.
Unfortunately, the procedure used to estimate the joint distribution of (X, Y )given (Z, V ) in the previous section does not fulfill this guarantee. Specifically, suppose we specify a variation independent parametric model for p(x|z1 , v), p(y, x | z0 , v),⇣0 (v), ⇣1 (v) and a smooth prior for its parameters. If, as will essentially always be
the case in practice, these parametric models are mis-specified, then the posterior
distribution of the function
ITT(v) = p(Y = 1 | Z = 1, V = v)

p(Y = 1 | Z = 0, V = v)

will generally concentrate on a non-zero function under the ITT null that ITT(v) =0 for almost all v. This follows from the fact that, in large trials, the posterior
distribution of ITT(v) will be centered on the MLE of ITT(v) and the MLE isgenerally inconsistent under mis-specification. Thus, in large trials, we will falsely
reject the sharp null hypothesis even when true. As a consequence, not only has
the large sum spent on the trial been wasted but, more importantly, a drug without
benefit may become licensed.

Transparent Parametrizations of Models for Potential Outcomes

587

Example: As a concrete illustration of the danger of mis-specification, we generated
5000 samples from the following data generating process, in which V is a covariate
taking three states:
ZV
Yx=0,z=0 = Yx=0,z=1 = Yx=1,z=0 = Yx=1,z=1

Ber(1/2),
Uniform({0, 1, 2}),
Ber(1/2),

and“5
Ber expit( 4 + y + z + v
2

Xz | Yx,z = y, V = v”
3 2v + y·(z + v) + v·z(1 + 2y)) .
2

Note that it follows from this scheme that Y ◆◆ {Z, V }, so that the conditional ITT
null hypothesis clearly holds. We fitted linear models (in V ) for the six parameters
logit(p(x1 | zi )), logit(p(y1 | xi , z0 )) and ⇣i , i ⇣ {0, 1} for ExAT + ExNT ; see
Eq. (18).
We performed inference via MCMC with a burn-in of 1000, retaining 5000 iterations. The posterior distributions for each of the ITT e⇤ects,p(y = 1|z = 1, v)

p(y = 1|z = 0, v)

are shown in Table 8. As can be seen, the posterior distribution indicates ITT e⇤ects
in two strata, even though none was present in the generating process.
Theorems 1 and 2 in the appendix provide necessary and su⌃cient conditions
for a parametric model to be robust to mis-specification under the ITT null. These
theorems include as special cases the results of Rosenblum and Van Der Laan (2009).
Table 8:

Summary of posterior for ITT(v) under ITT null with mis-specification
v
0
1
2

2.5%
0.02289
0.128200.02395

Mean
0.020410.09863
0.0691597.5%
0.061900.07088
0.11590

6.1. ITT-Null-Robust Parametrization of ExAT + ExNTThe key to constructing a parametric Bayes estimator robust to mis-specification
under the ITT null is to parametrize, for each v, the set of distributions (17) for theobserved data corresponding to a model constrained by (4) and (5) as follows: foreach V = v, we have the following six variation independent parameters:i ⇧ p(y1 | zi ),where:p(x1 | yi , z0 ),⇧i ,for i ⇣ {0, 1},

588

⇥0

T. S. Richardson, R. J. Evans and J. M. Robins

⌅

(1

p01
X)
p01
[1
X
(p10
X

p01
X )(1
(1
p10 )
X

⇤0 )

0

(p10
X

p01
X )(1
(1
p10 )
X

⇤1 )

⇥ @1 +⌅

p10X ⇤0p10 )⇤0X

0⇥ @1 +

⇥1

[1
(1

(1

p11
X)
11
p
[1
X

[1
(1

p00
X (1
p00 )(1
X

p01
⇤1 )]
X (1
(1
p01 )(1
⇤1 )]
X
1
A

10
p10
X /(pX

1p01 /(p10
X
X
A

⇤0 )
⇤0 )

00
p00
X /(pX

0

(p00
X
(1

1
p11
X )⇤0 A
p00 )
X

0

(p00
X
(1

1p11 /(p00
X
X
p11
X )⇤1 A
p00 )X

⇥ @1 +⇥ @1 +

0
p01
X)
@1
0
p01
X)
@1

(p10
X

(p10
X

p01
X )(1
p10
X
p01
X )(1
p10
X

⇤0 )

⇤1 )

1

(1

A

1(1
A

10
p10
X )/(pX

10
p01
X )/(pX

p01
X)

p01
X)
,

p11
X ⇤1 ](1p11 )⇤1 ]
X
0
p11
X)
@1
0
p11
X)
@1

(p00
X

(p00
X

1
p11
X )⇤0 A
p00
X

1(1
p11
X )⇤1 A
p00
X

(1

00
p00
X )/(pX00
p11
X )/(pX

p11
X)

p11
X)
,

and pijX ⇧ p(x1 | yi , zj ). Thus, we have replaced p(x1 | yi , z1 ) of the standard variation dependent parametrization of this model by ⇧i , i = 0, 1. Unlike our previous
parametrizations, the inverse map from the variation independent parameters top(x1 | yi , z1 ), i = 0, 1, is not available in closed form.
To model p(x, y, z, v) when V contains continuous covariates, we specify parametric models ⇧i (v; 0 ), p(x1 | yi , z0 , v; 1 ) i = 0, 1 and f (v; 2 ) as well as a logistic
regression model expit(Zm(V ; ) + q(V ; 3 )) for p(y1 |z, v) satisfying m(V ; ) = 0 if
and only if is the zero vector; the parameter vectors 0 , 1 , 2 , 3 , are variation
independent. Thus, under this model, = 0 if and only if the conditional ITT null
hypothesis Y ◆◆Z | V holds.
In the appendix, we show, following Rosenblum and Van Der Laan (2009), that
if we choose q(V ; 3 ) = 3T q ⇤ (V ) such that each component of ⌫m(V ; 0)/⌫ is in thelinear span of the components of q ⇤ (V ), then, under the conditional ITT null, theMLE of converges to its true value of 0 at rate n 1/2 even if the models ⇧i (v; 0 ),p(x1 |y, z0 , v; 1 ), f (v; 2 ) and p(y1 |z0 , v; 3 ) = expit( 3T q ⇤ (v)) are all mis-specified.
As a consequence, under a smooth prior p( , ), the posterior distribution of andof ITT(v) will concentrate on the zero function in large samples.
Note that we can always guarantee the linear span condition holds by choosingq ⇤ (V ) to include ⌫m(V ; 0)/⌫ as a subvector.
6.2. ITT-Null-Robust Parametrization of MonX + ExAT + ExNT
To obtain a parametric Bayes estimator with the aforementioned robustness property under the conditional ITT null for the model that excludes Defiers, we parametrize, for each v, the set of distributions for the observed data constrained by (12)and (13) as follows: for each V = v, we have the following six variation independentTransparent Parametrizations of Models for Potential Outcomes589

parameters:
i

⇧

↵i

⇧

⌥i

⇧

p(y1 | zi ),

h
pi1
X
0
(1 pi1
X)
[(1

pi0
X) 0

(1

i
pi1
)
1 ⇤
X

0

2
pi0
X 0
i1
(1 pi1X ) 1 ][pX 1

1/pi1
X

⇤

pi0
X 0]

i1
(1 pi1X )/pX
,
1

⇤

„

1
0

i1
i0
«(2pi1
X 1)/(pX pX ),for i ⇣ {0, 1}, where pij
X ⇧ p(x1 | yi , zj ).
In the appendix, we prove that any parametric submodel with p(y1 | zi ) modelled as in the previous subsection will enjoy the same robustness properties under
the conditional ITT null, even if the models for ↵i (v; 0 ), ⌥i (v; 1 ), f (v; 2 ) andp(y1 |z0 , v; 3 ) = expit( 3T q ⇤ (v)) are all mis-specified.
Though we do not do so here, our approach may be extended to the other six
potential outcome models in which fewer exclusion restrictions hold.
ACKNOWLEDGEMENTS
This research was supported by the U.S. National Science Foundation (CRI 0855230)
and U.S. National Institutes of Health (R01 AI032475). We thank McDonald, Hiu
and Tierney for giving us permission to use their flu vaccine data. Thanks also to
Susan Shortreed who implemented part of the Gibbs sampler used in Section 3, andto Sander Greenland and Adrian Raftery for helpful suggestions.
REFERENCES
Balke, A. and Pearl, J. (1997). Bounds on treatment e⇤ects from studies with imperfectcompliance. J. Amer. Statist. Assoc. 92, 1171–1176.
Bickel, P. J., Klaasen, C. A. J., Ritov, Y. and Wellner, J. A. (1993). Efficient and Adaptive
Estimation for Semiparametric Models. Baltimore: John Hopkins University Press.
Bonet, B. (2001). Instrumentality tests revisited. Proc. 17th Conf. on Uncertainty in
Artificial Intelligence, 48–55.
Cai, Z., Kuroki, M., Pearl, J. and Tian, J. (2008). Bounds on direct e⇤ects in the presenceof confounded intermediate variables. Biometrics 64, 695–701.
Chickering, D. and Pearl, J. (1996). A clinician’s tool for analyzing non-compliance.
AAAI-96 Proceedings, 1269–1276.
Efron, B. and Feldman, D. (1991). Compliance as an explanatory variable in clinical
trials. J. Amer. Statist. Assoc. 86, 9–26.
Frangakis, C. E. and Rubin, D.B. (2002). Principal stratification in causal inference.
Biometrics 58, 21–29.
Greenland, S. (2005). Multiple-bias modelling for analysis of observational data. J. Roy.
Statist. Soc. B 168, 267–306.
Gustafson, P. (2005). On model expansion, model contraction, identifiability and priorinformation: Two illustrative scenarios involving mismeasured variables. Statist.
Science 20, 111–140.
Heckerman, D. and Shachter, R. (1995). Decision-theoretic foundations for causal
reasoning. Journal of Artificial Intelligence Research 3, 405–430.
Hirano, K., Imbens, G. W., Rubin, D. B. and Zhou, X.-H. (2000). Assessing the e⇤ect of
an influenza vaccine in an encouragement design. Biometrics 1, 69–88.

590

T. S. Richardson, R. J. Evans and J. M. Robins

Hudgens, M. G., Hoering, A. and Self, S.G. (2003). On the analysis of viral load endpoints
in HIV vaccine trials. Statistics in Medicine 22(14), 2281–2298.
Imai, K. (2008). Sharp bounds on the causal e⇤ects in randomized experiments with“truncation-by-death”. Statist. Probab. Lett. 78, 144–149.
Imbens, G. and Rubin, D. (1997). Bayesian inference for causal e⇤ects in randomized
experiments with noncompliance. Ann. Statist. 25, 305–327.
Kaufman, S., Kaufman, J.S. and MacLehose, R. F. (2009). Analytic bounds on causal riskdi⇤erences in directed acyclic graphs involving three observed binary variables.
J. Statist. Planning and Inference 139, 3473–3487.
Leamer, E. (1978). Specification Searches. New York: Wiley.
Manski, C. (1990). Non-parametric bounds on treatment e⇤ects. American Economic
Review 80, 351–374.
McDonald, C., Hiu, S. and Tierney, W. (1992). E⇤ects of computer reminders for influenza
vaccination on morbidity during influenza epidemics. MD Computing 9, 304–312.
Neyman, J. (1923). Sur les applications de la théorie des probabilités aux experiences
agricoles: Essai des principes. Roczniki Nauk Rolniczych X, 1–51. In Polish, English
translation by D. Dabrowska and T. Speed in Statist. Science 5 463–472, 1990.
Pearl, J. (2000). Causality. Cambridge: Cambridge University Press.
Pearl, J. (2009). Causality (2nd edition). Cambridge: Cambridge University Press.
Ramsahai, R. (2008). Causal Inference with Instruments and Other Supplementary
Variables. Ph.D. Thesis, University of Oxford, Oxford, UK.
Richardson, T. S., Evans, R.J. and Robins, J. M. (2011). Variation independent
parameterizations for convex models. Tech. Rep., University of Washington, USA.
Richardson, T. S. and Robins, J. M. (2010). Analysis of the binary instrumental variable
model. Heuristics, Probability and Causality: A Tribute to Judea Pearl (R. Dechter,
H. Ge⇤ner, and J. Halpern, eds.) London: College Publications, 415–444.
Richardson, T. S. and Robins, J. M. (2011). Discrete instrumental variable models fordichotomous outcomes. Tech. Rep., University of Washington, USA.
Robins, J. (1986). A new approach to causal inference in mortality studies with sustained
exposure periods: Applications to control of the healthy worker survivor e⇤ect.
Mathematical Modeling 7, 1393–1512.
Robins, J. (1989). The analysis of randomized and non-randomized AIDS treatment trials
using a new approach to causal inference in longitudinal studies. Health Service
Research Methodology: A focus on AIDS (L. Sechrest, H. Freeman, and A. Mulley,eds.). Washington, D.C.: U.S. Public Health Service.
Robins, J. M., Richardson, T. S. and Rotnitzky, A. (2011). Robustness of parametricsubmodels. Tech. Rep., Harvard School of Public Health, USA.
Robins, J. M., Richardson, T. S. and Spirtes, P. (2009). Identification and inference fordirect e⇤ects. Tech. Rep., University of Washington, USA.
Rosenblum, M. and Van Der Laan, M. J. (2009). Using regression models to analyze
randomized trials: Asymptotically valid hypothesis tests despite incorrectly specified
models. Biometrics 65, 937–945.
Rubin, D. B. (1974). Estimating causal e⇤ects of treatments in randomized andnon-randomized studies. J. Educational Psychology 66, 688–701.
Rubin, D .B. (1998). More powerful randomization-based p-values in double-blind trials
with non-compliance. Statistics in Medicine 17, 371–385.
Rubin, D. B. (2004). Direct and indirect causal e⇤ects via potential outcomes.
Scandinavian J. Statist. 3, 161–170.
Zhang, J. L. and Rubin, D. B. (2003). Estimation of causal e⇤ects via principal
stratification when some outcomes are truncated by “death”. J. Educational and
Behavioral Statistics 28, 353–368.

Transparent Parametrizations of Models for Potential Outcomes

591

APPENDIX
To prove the claims in Section 6 we establish some general properties of parametric models under mis-specification. For this purpose, we require some additionalsemiparametric concepts; see Bickel et al. (1993).
Given a model M and smooth functional : F ✓↵ (F ) mapping F ⇣ M intodR , we wish to estimate (F ) based on n i.i.d. observations Oi ; i = 1, . . . , n, from an
unknown distribution F ⇣ M . In this appendix, our goal is to determine the highlevel conditions required for the MLE of based on a given parametric submodel of
M to remain consistent for at certain laws F ⇣ M that lie outside the submodel.
Formally, a parametric submodel of M is the range R(h) = {Fh, ; ⇣ A} ⌦ M
of a smooth map h : ✓↵ Fh, with domain an open set A ⇣ Rl (Bickel et al., 1993,
p. 13). Write fh {o; } for the density of Fh, with respect to a common dominating
measure µ. For convenience, we henceforth suppress the dependence on h and write
f {o; } and F . Let e( ) = (F ) be the value of at F .
Let M (b) = {F ⇣ M ; (F ) = b} be the submodel of M on which the functionaltakes the value b ⇣ Rd . Let R(h, b) = {F ; ⇣ A, e( ) = b} be thesubmodel of the parametric model R(h) = {F ; ⇣ A} contained in M (b). Let
A(b) = { ⇣ A; e( ) = b} be the pre-image of R(h, b). Let
b

(F )

=

arg max Pn [log {f [O; ]}] and=arg max EF [log {f [O; ]}]⌃A⌃A

be maximizers of the empirical and expected log-likelihood, where Pn denotes asample average. Similarly, letb(b)(b; F )=

arg max Pn [log {f [O; ]}] and=arg max EF [log {f [O; ]}]

⌃A(b)
⌃A(b)

be the maximizers of the empirical and expected “profile” log-likelihood given e( ) =b. Thus, (b; F ) is the maximizer of EF [log {f [O; ]}] over all ⇣ A subject tothe d constraints e( ) = b. Note {F ⇤ } = ⇤ , as the expected log likelihood is
maximized at the true density. Under regularity conditions b and b(b), respectively,
converge to (F ) and (b; F ) at rate n 1/2 .
Let S( ) = ⌫ log{f [O; ]}/⌫ be the score for
evaluated at F . Typically(F ) is the (assumed) unique solution to EF [S( )] = 0 for all F ⇣ M .

Since EF [S( )S( )T ] 1 S( ) is the influence function for , taking derivatives we obtain that IF⇥,par ( ) = {⌫e( )/⌫ }T EF [S( )S( )T ] 1 S( ) is the ddimensional “parametric” e⌃cient influence function for in the parametric model
R(h) = {F ; ⇣ A} at F . Further, varF {IF⇥,par ( )} is the parametric Cramér–
Rao variance bound for at F
For F ⇣ M (b), let ⇥( ) and ⇥(b; ) be the closed linear span in L2 (F ) of
the scores for all parametric submodels in M and M (b), respectively, that include
F . By definition, ⇥( ) and ⇥(b; ) are the tangent spaces for models M andM (b) at F , and ⇥(b; ) is the nuisance tangent space for in model M at F .
Further, the e⌃cient influence function IF⇥ (F ) forin model M at F is the

592

T. S. Richardson, R. J. Evans and J. M. Robins

unique element of ⇥( ) satisfying EF [IF⇥ (F )D] = ⌫ (F (t)) /⌫t|t=0 for all D ⇣⇥( ) and F (t) ⇣ M a parametric model with parameter t with F (0) = F andscore at F equal to D. In particular, EF [IF⇥ (F )D] = 0 if D ⇣ ⇥(b; ). ThenvarF {IF⇥ (F )} is the semiparametric Cramér–Rao variance bound for in model
M at F . Note varF {IF⇥,par ( )} ⌥ varF {IF⇥ (F )} with equality if and only ifIF⇥,par ( ) = IF⇥ (F ) w.p.1.
Robins et al. (2011) prove the following two Theorems.
Theorem 1 Given a fixed b, suppose for all F ⇣ M (b), EF [S( )] = 0 has a uniquesolution. If EF [S( (b; F ))] = 0 for all F ⇣ M (b), then, for all ⇣ A(b),IF⇥ (F ) = IF⇥,par ( ).
Note the conclusion of Theorem 1 can also be expressed as equality of the parametricand semiparametric (for model M ) Cramér–Rao variance bounds for at F , ⇣A(b).
Corollary 1.1 Suppose for all F ⇣ M (b), EF [S( )] = 0 has a unique solution.
Then the following hold:(i) If IF⇥ (F ) ⌘= IF⇥,par ( ) for some Fsuch that e { (F ⇤ )} ⌘= b.

⇣ M (b), then there exists F ⇤ ⇣ M (b)(ii) The MLE b ⇧ e(b) is not consistent for= b at F ⇤ .

Proof: (i) By Theorem 1 there exists F ⇤ ⇣ M (b) such that EF ⇤ [S( (b; F ⇤ ))] ⌘= 0.
Hence e { (F ⇤ )} cannot equal e { (b; F ⇤ )} ⇧ b.
(ii) Since e { (F ⇤ )} is the limit of the MLE e {b}, (ii) follows.
⇤
Remark: Robins et al. (2011) prove the following stronger result. Let
Q={ ;⇣ A (b) and IF⇥ (F ) ⌘= IF⇥,par ( )} .

Then there exists an injective map from Q ↵ M (b) such that e { (F )} =⌘ b for all F
in the range of the map.

Theorem 2 Suppose for all F ⇣ M (b), EF [S( )] = 0 has a unique solution. Thenif (i) for all ⇣ A(b), the parametric and semiparametric influence functions areequal, i.e., IF⇥ (F ) = IF⇥,par ( ) w.p. 1 for all ⇣ A(b), and (ii) for all F ⇣ M (b),(f (O)/f (O; (b; F ))) 1 is contained in the tangent space ⇥(b; (b; F )) of M (b) at(b; F ), thenEF [S( (b; F ))] = 0 for all F ⇣ M (b).
(23)
Remark: A su⌃cient condition for (ii) is that the model M (b) is convex: that is,
if F1 and F2 are in M (b) then so is the law F1 + (1)F2 for ⇣ [0, 1].
Corollary 2.1 Under the hypotheses of Theorem 2, for all F ⇣ M (b),(i) e ( (F )) = b;(ii) under regularity conditions, the parametric MLE b is a consistent, asymptotically normal (CAN) estimator of (F ) = b.

Proof: (i) The conclusion of Theorem 2 implies that e ( (b; F )) = e ( (F )) for
all F ⇣ M (b). But e ( (b; F )) ⇧ b and (F ) ⇧ b. (ii) Under standard regularity
conditions, the MLE is a CAN estimator of its limit.
⇤

Transparent Parametrizations of Models for Potential Outcomes

593

6.3. Proving the Robustness Claims of Section 6
We first consider the model ExAT + ExNT , which allows for Defiers.
Let M be a model for O = (V, Z, X, Y ) characterized by:(i) p(Z = 1|V ) = c is a known constant;(ii) logit[p(y1 | zi , v)] ⇣ {Zm(v; ) + q(v);m(·; ·) known satisfying m(·; ) = 0⇣ Rd , q : v ✓↵ q (v) unrestricted,= 0};(iii) ⇧i (v), logit[p(x1 | yi , z0 , v)], i = 0, 1 and f (v) are all unrestricted.
It follows from the construction of the parameters ⇧0 and ⇧1 that for each v, Eqs. (4)and (5) will hold with strict inequalities.
Let M (0) be the submodel of M with = 0, in which the conditional ITT Null(21) holds. Thus, M (0) is the model characterized by the constraints (4) and (5)and Y ◆
◆Z | V .
Henceforth, we use the
˘ semiparametric¯ notation introduced at the start of theappendix. Let R(h) = F ; ⇣ A ⌃ Rl be a parametric submodel Msub,par of
M characterized by known functions of= ( 0 , 1 , 2 , 3 , 4 ) ⇣ Rl : ⇧i (v; 0 ),T ⇤p(x1 |yi , z0 , v; 1 ), f (v; 2 ), q(v; 3 ) = 3 q (v), 4 = with the j variation independent and with each component of ⌫m(V ; 0)/⌫ in the linear span of the components of q ⇤ (V ).
Such a parametric submodel exists because our ITT-null-robust parametrization
is variation independent. We further assume that there is a unique solution to
EF [S( )] = 0, for all F ⇣ M (0).
Our goal is to prove the following:
Theorem 3 For F ⇣ M (0), (F ) solving EF [S( )] = 0 satisfies the constrainte( (F )) ⇧ 4 (F ) = 0, i.e., (F ) = (0; F ).

Theorem 3 implies that, under regularity conditions, the parametric MLE b is aconsistent, asymptotically normal (CAN) estimator of (F ) = 0, for all F ⇣ M (0),i.e., under mis-specification of the parametric (nuisance) submodels.
Remark: A natural approach to a proof is to establish that the premises of Theorem
2 hold and then appeal to its corollary. We shall see that, although premise (i) holds,
premise (ii) does not. However, a minor fix gets around this di⌃culty.
Before proving Theorem 3, we prove the following lemma that establishes premise(i) of Theorem 2.
Lemma 1 For all
are equal, i.e.,⇣ A(0), the parametric and semiparametric influence functionsIF⇥ (F ) = IF⇥,par ( ) w.p. 1 for all⇣ A(0).

(24)

Furthermore, they depend on
only through 3 and on the data only through(Z, Y, V ). (Recall that ⇣ A(0) iff 4 ⇧ = 0, so 4 is fixed.)
Proof: Write S ( ) = (S k ( ) ; k = 0, 1, 2, 3, 4). Consider a particular FIF⇥,par ( ) = IF⇥ (F ) if and only ifS⇥,e↵ ( ) ⌘ S 4⇥F [S 4 ( )| (0; )] = S⇥,e↵,par ( ) ⌘ S 4⇥⇢ M (0). Nowˆ˜
S 4 ( )|S \ 4 ( )594

T. S. Richardson, R. J. Evans and J. M. Robins

where S \ 4 = (S 0 , . . . , S 3 ) and ⇥F

is the projection operator in L2 (F ). This folˆ˜ 1lows from the fact that IF⇥,par ( ) = EF S⇥,e↵,par ( )⌅2S⇥,e↵,par ( ) and IF⇥ (F ) =ˆ
˜ 1⌅2EF S⇥,e↵ ( )S⇥,e↵ ( ).
Now, the likelihood for one observation isf (Y, X, V, Z)=nopy (Z, V ; 3 , 4 )Y {1 py (Z, V ; 3 , 4 )}1 YnnooI(Z=0)⇥ pz=0(Y, V ; 1 )X 1 pz=0
(Y, V ; 1 )1 XxxnnooI(Z=1)⇥ pz=1(Y, V ; \ 2 )X 1 pz=1
(Y, V ; \ 2 )1 Xxx⇥f (V ;

where py (z, v;
S 0( )=

S 1( )

=

3,

2 )c

4 ) = p(y1 | z, v;

˘I(Z = 1) X
˘
I(Z = 0) X=

S 3( )={log f (V ;

{Y

=

{Y

py (Z, V ;,z=0 (y, v;
4 ), px

pz=0(Y, V ;
x

1)

¯

pz=1
(Y, V ;x¯⇢1 ) = p(x1 | y, z0 , v;

1 ), etc. Thus

pz=1
(Y, V ; \ 2 )/ 0
x
pz=1
(Y,
V
;
\
pz=1
(Y, V ; \ 2 )}2 ) {1xx

pz=0(Y, V ; 1 )/ 1xz=0
px (Y, V ; 1 ) {1 pz=0
(Y, V ; 1 )}x
z=1¯
px (Y, V ; \ 2 )/ 1
\ 2 ) z=1
px (Y, V ; \ 2 ) {1 pz=1
(Y, V ;x,\ 2 )},

py (Z, V ; 3 , 4 )/ 3
py (Z, V ; 3 , 4 )}3 , 4 ) {1
⇢
ff
¯
pz=1
(Y, V ; \ 2 )/ 3
x
pz=1
(Y,
V
;
\
)
,
2
x
pz=1
(Y, V ; \ 2 ) {1 pz=1
(Y, V ; \ 2 )}
x
x

3,

4 )}

py (Z, V ;py (Z, V ; 3 , 4 )/ 4
py (Z, V ; 3 , 4 )}3 , 4 ) {1
⇢
ff
¯
pz=1
(Y, V ; \ 2 )/ 4x
z=1
px (Y, V ; \ 2 ).
pz=1(Y, V ; \ 2 ) {1 pz=1
(Y, V ; \ 2 )}
x
x

3,

4 )}

˘
+ I(Z = 1) X

py (Z, V ;Richardson et al. (2011) prove that under our ITT-null-robust parametrization, when
4 = 0,
pz=1(Y, V ; \ 2 )pz=1(Y, V ; \ 2 )
x
x
=
= 0.
4

3

Our parametrization was carefully constructed to ensure that these derivatives were zero
when 4 = 0. Consequently, under the ITT null ( 4 = 0) S 3 ( ) and S 4 ( ) are functions
only of 3 and the data (Y, Z, V ); crucially they are not functions of 0 , 1 , 2 and X.
Let E be shorthand for EF . At 4 = 0,
E

ff

2,

˘
+ I(Z = 1) X
S 4( )

3,

Z

\ 2)

2 )} /

py (Z, V ;

c)1

(1

pz=1
(Y, V ;x˘
+ I(Z = 1) X

S 2( )

Z

h
h
˘
¯T i
˘
¯T i
S 4 ( ) S \( 3 , 4 ) ( )= E S 3 ( ) S \( 3 , 4 ) ( )=0Transparent Parametrizations of Models for Potential Outcomes

595

since for i = 0, 1, and j = 3, 4,
˛
h
i
h
h
ii
˛
S j ( ) {S i ( )}T = E S j ( )E S i ( )T ˛ Y, V, Z = 0

E
and

E

˛ io
h
i
n
h
˛
S 2 ( )S j ( )T = E S 2 ( )E S j ( )T ˛ V
= 0.

ˆ
˜
Thus, ⇥F S 4 ( ) | S \ 4 ( ) = ⇥F [S 4 ( ) | S 3 ( )]. Now at
E 3 [Y |V ],py (Z, V ; 3 , 4 )/ 4
py (Z, V ; 3 , 4 ) {1 py (Z, V ; 3 ,
and

4 )}

py (Z, V ; 3 , 4 )/ 3
py (Z, V ; 3 , 4 ) {1 py (Z, V ; 3 ,

=

4 )}

4 = 0, py (Z, V ;Z m(V ; 0)/

3,

4) =

4

= q ⇤ (V ).

Hence,
S 3 ( ) = [YE 3 (Y |V )] q ⇤ (V ),S 4 ( ) = {YE 3 [Y |V ]} Z { m(V ; 0)/ ⌃ } .

The argument just given above applies to any parametric submodel contained in M (0)and containing F . Therefore, when 4 = 0, (0; ) = (0,1,2) (0; )3 (0; ) with
(0,1,2) (0; ) and 3 (0; ) orthogonal under F . Here3 (0;) = {(YE 3 (Y |V )) q (V ) ; q (·) unrestricted}

and (0,1,2) (0; ) is the linear span of scores corresponding to the set of unrestrictedfunctions and densities ⌅i (v) , p(x1 |yi , z0 , v), f (v). The argument also implies that thescore S 4 ( ) ⌘ S⇥ ( ) for ⌃ = 4 is orthogonal to (0,1,2) (0; ). Thus, when 4 = 0,S⇥,e↵ ( ) = S 4 ( ) ⇥F [S 4 ( ) | (0; )] = S⇥ ( ) ⇥F [S⇥ ( ) | 3 (0; )] . One cancheck that ⇥F [S⇥ ( ) | 3 (0; )] = {YE 3 (Y |V )} { m (V ; 0) / ⌃ } c with c = E [Z|V ].
Hence, S⇥,e↵ ( ) = S⇥,e↵,par ( ) if{Y

E 3 (Y |V )} { m (V ; 0) / ⌃ } {E [Z|V ]} = ⇥F [S 4 ( ) |S 3 ( )] .

But by m (V ; 0) / ⌃ in the span of the components of q ⇤ (V ), we know thatvarF (⇥F [S 4 ( ) |S 3 ( )])varF {⇥F [S⇥ ( ) |

3 (0;

)]} .

varF (⇥F [S 4 ( ) | S 3 ( )])  varF {⇥F [S⇥ ( ) |

3 (0;

)]} .

However, by S 3 ( ) ⇢

3 (0;

),

Thus, S⇥,e↵ ( ) = S⇥,e↵,par ( ).
HenceIF⇥ (F ) = IF⇥,par ( ) w.p. 1 for all
Further, from its formula, when
the data only through (Y, Z, V ).

4 = 0, IF⇥,par (2 A(0).

) depends on

only through(25)
3 and on⇤This lemma establishes premise (i) of Theorem 2. However, we cannot apply
convexity to establish premise (ii) of Theorem 2 because the model M (0) is not

596

T. S. Richardson, R. J. Evans and J. M. Robins

convex. This is because Y ◆◆Z | V is not preserved under convex combination.
For example the convex combination of laws with densities f1 (Y |V )f (Z)f1 (V ) andf2 (Y |V )f (Z)f2 (V ) does not in general satisfy Y ◆◆Z | V unless f1 (V ) = f2 (V ).
Based on this observation, we consider the submodel Mv of M and Mv,sub,par thatassumes f (v) equals a known density f0 (v) and that the model f (v; 2 ) satisfies
f (v; 2 )| 2 =0 = f0 (v). Note that this latter condition can always be arranged;
since f0 (v) is known, we simply choose any model f (v; 2 ) that satisfies the condition. The model Mv (0) = Mv M (0) is convex since f (z, v) is the same for all
F ⇣ Mv (0). Specifically, under convex combinations, (a) Y ◆◆Z | V is preservedand (b) the constraints Eqs. (4) and (5) are also preserved, as the constraints arelinear in p(y, x|z, v). Furthermore, by inspecting the proof of Lemma 1, we see thatIF⇥ (F ) and IF⇥,par ( ) under model Mv (0) and Mv,sub,par (0) and models M (0) andMsub,par (0) are identical. Thus both premises of Theorem 2 hold for models Mv andMv,sub,par . Hence, by Theorem 2, for F ⇣ Mv (0), (F ) solving EF [S( )] = 0 subject
to the constraint 2 = 0 required by model Mv,sub,par satisfies e( (F )) ⇧ 4 (F ) = 0.
However since S 2 ( ) only depends on through 2 and S \ 2 ( ) is not a functionof 2 , we conclude that for F ⇣ Mv (0), (F ) solving EF [S( )] = 0 without constraints also satisfies e( (F )) ⇧ 4 (F ) = 0. (The discussion in the last paragraphbecomes unnecessary if, as is often assumed, we treat the distribution of V as fixedat its empirical, i.e., we e⇤ectively condition on the observed values of V .)But each F ⇣ M (0) is an element of a model Mv (0); the model with f0 (v) equal
to the density of V under F . We conclude that for each F ⇣ M (0), (F ) solving
EF [S( )] = 0 satisfies e( (F )) ⇧ 4 (F ) = 0. This result holds therefore even whenf0 (v) is unknown and the chosen model f (v; 2 ) is mis-specified.
⇤Rosenblum and Van Der Laan (2009) give an alternate proof of the fact thatS⇥,e↵ ( ) = S⇥,e↵, par ( ) that does not use Theorem 2.
To prove the analogous result for our ITT-null-robust parametrization for MonX
+ ExAT + ExNT it su⌃ces to show that⌫pz=ix (Y, V ; \ 2 )/⌫z=i
4 = ⌫px (Y, V ;\ 2 )/⌫

3 = 0,

i = 0, 1 when

4 = 0,

under our parametrization. Again our ITT-null-robust parametrization was carefully constructed to ensure that these derivatives were zero. The remainder of theproof is analogous to that for the robust parametrization of ExAT + ExNT .
DISCUSSIONSTEPHEN E. FIENBERG (Carnegie Mellon University, USA)
Overview. Given the by now extensive literature on non-compliance in randomized experiments, one might have thought that there was little new to be said,
especially about the simplest such studies, when the idealized data come in theform of a 2 ⇤ 2 ⇤ 2 contingency table. Further, given the commentary about the roleof randomization for Bayesians that appears in at least part of the literature, one
might presume that there is nothing that is necessarily Bayesian about the problem
at all. Both of these presumptions are false.
Stephen E. Fienberg is Maurice Falk University Professor of Statistics and Social Sciencein the Department of Statistics, the Machine Learning Department, Cylab, and i-Lab atCarnegie Mellon University, Pittsburgh PA 15213-3890, USA

Transparent Parametrizations of Models for Potential Outcomes

597

Richardson, Evans, and Robins (henceforth RER) have written a stimulating
paper, one worthy of careful study by Bayesians and non-Bayesians of all stripes
and flavors, and not just by those interested in the non-compliance problem. RER
have a new and remarkably clear message about this topic and this is the central
idea in the paper, which I paraphrase:
In many randomized experimental contexts the causal estimands of interest
are not identified by the observed data. We need to re-parametrize partially
identified models to separate wholly-identified and wholly-non-identified parameters. Our goal as statisticians is to focus on what is identifiable. As
Bayesians, when we then look at such model structures, we need to recognizethat what we will get out of our posterior distribution for the non-identifiable
part will essentially be what we put into the prior, and una⇤ected by theexperimental data!

Like prior authors addressing the non-compliance problem, RER use latent structure
to resolve the identification of parameters and to make inference about causal e⇤ects
and adopt many of the conventions for these found in the earlier literature. Someof the specifications for the latent structure have heuristic appeal but in many
ways despite their continuing use are arbitrary. The impact of these choices, notsurprisingly, plays an important role in what is identifiable and estimable.
Whether this is a message for Bayesians or for all statisticians and those who use
statistical methods to analyze data from experiments subject to non-compliance isa topic to which I return below. In addition, in what follows I address the following
pair of questions:(i) How should we define causal e⇤ects?
(ii) Can we use algebraic geometry ideas to restructure the problem?
Latent structure and causal inference. The basic experimental structure RER
describe takes the simple form of a 2 ⇤ 2 ⇤ 2 contingency table with treatment
variable Z, exposure variable X, and response variable y each of which is binary.
Because they are dealing with a randomized experiment, the values of Z come inpre-specified form. These three random variables are linked via a graphical model.
Since Y has both X and Z as parents there are four potential outcomes:
Y (x = 0, z = 0), Y (x = 0, z = 1), Y (x = 1, z = 0), Y (x = 1, z = 1).
The latent structure on Y , tY , involves classes of individuals or types. These don’t
have any kind of agreed-upon name in the literature but they get simplified with laterassumptions. There is also a latent structure on exposure, X, which is unobservable
involving four types: {N T, CO, DE, AT }.
When Y has only X as a parent, Z has no direct e⇤ect on Y , and there
are just four Y -latent types, which by convention are typically called “Helped”= (Y (x = 0) = 0, Y (x = 1) = 1), “Hurt,” “Always Recover,” and “Never Recover.” Thus we have four Y -latent types and four X-latent types called “Never
Takers,” “Compliers,” “Defiers,” and “Always Takers” or 16 types for {tX , tY }:{N T, CO, DE, AT } ⇤ {HE, HU, AR, N R}, far more than the original counts in the
23 table about which we had hoped to make inferences. Something has to give way
or the infrastructure will crash in a heap.

598

T. S. Richardson, R. J. Evans and J. M. Robins

Finally, if no one in the placebo arm of the randomized trial can get the treatment (often not true), Z = 0 implies X = 0, and there are no “Defiers” or “Always Takers.” Thus there are only two X-latent types and four Y -latent types:{N T, CO} ⇤ {HE, HU, AR, N R.} as in Figure 9. As RER note, this is the simple
situation which arises for the Lipid data under Pearl’s analysis (which presumes
that Z does not have a direct e⇤ect on Y ).

Figure 9:

Graphical representation of the Pearl IV model for Lipid data.

At this point I’d like to o⇤er a major caution. As appealing as the heuristiclabels and categories for {tX , tY } are, we need to remember latent variables are
simply that and thus unobservable. They are basically fictions which we introduce
to help the modeling process. The true structure underlying compliance in a real
randomized trial is obviously more complex and possibly not captured by the nicelabels used in the compliance literature. We tend to forget this when we makegraphical pictures such as in Figure 9. Therefore, we must proceed with caution
when we move to the estimation of causal e⇤ects.
Once RER lay out this infrastructure for compliance, they spend most of theremainder of the paper working out plausible inequality restrictions that sharpen
focus on p(y, x|z). In particular they derive upper and lower bounds on causal
relationships and “e⇤ects” under di⇤erent assumptions about {tX , tY }, specificallywhat they refer to as(MonX ) Monotonicity of compliance;(ExNT ) Stochastic exclusion for NT under non-exposure;(ExAT ) Stochastic exclusion for AT under exposure.
These bounds allow RER to talk about identification and estimability. Alongthe way, they include a variety of insights to the modeling process including a a way
to represent the di⇤erent models that are not inherently graphical using directed
acyclic graphical ideas, albeit at the expense of some additional complexity. Thereis a richness of detail here worthy of study by those interested in the complianceproblem.
In praise of Bayes? Should we be surprised to see a paper like this at Valencia 9?
It does contain the naive Bayesian approaches to this problem, as I noted above, but
to make the point that these just disguise the identification problem and can easily
lead to nonsensical inferences. The only way to make inferences about essentiallynon-identifiable parts of the model is via strong information about them in the prior.
RER make this point convincingly by example.
Most of bounding arguments and results could easily be presented from a frequentist perspective. In fact, Richardson and Robins (2010), in a companion paperTransparent Parametrizations of Models for Potential Outcomes599

to the present one, actually describe this problem and the basic ideas from a likelihood perspective. The real action in the present paper is about the specificationof bounds on probabilities in the model and not so much on inference per se. There
are implications of argument for Bayesian analyses, but I find them more cautionary
than prescriptive and I would have liked to see the work culminate in a full-scale
informative Bayesian analysis.
Yet there may be a deeper reason for Bayesians to take note of what RER
have to say. After all, many of us have been long convinced by Rubin’s (1978)
argument about how randomization cuts the ties to covariates and allows for a
direct assessment of the causal e⇤ect. RER are also exploiting randomization for
the restrictions in the model and to garner identification, albeit in alimited way, but
Bayesians are barely better o⇤ than frequentists when it comes to dealing with thepart of the model that is not identifiable.
Alternative specifications for causal effects. RER follow prior authors and workwith what is usually called the “Average Causal E⇤ect”:»–»–
E[Yx=1 ](help) + (always recover)
ACE(X causes Y ) = log= log
E[Yx=0 ](hurt) + (always recover)as well as “Intent to Treat E⇤ect” (ITT) e⇤ects, principal stratification, and ACDEe⇤ects. It is as if these “causal” quantities were imbued with some objective statusindependent of the structure of the problem at hand. I think this is misleading.
Why not define causal e⇤ect based on log(E[Yx=1 ]/E[Yx=0 ])? If we used thisalternative specification for causal e⇤ect, we would be working with adjusted odds
ratios or ratios of odds ratios for the basic 2 ⇤ 2 table, and then the definition of thecausal e⇤ect would be rooted in a logit-like statistical model instead of the linear
model implicit in the definition of ACE, ITT, and ACDE. Sfer (2005) develops thiskind of argument for a simple randomized experiment involving a binary treatment
variable and a binary outcome. My guess is that much of the thinking in the present
paper would carry over to this model-based representation.
In the companion paper by Richardson and Robins, the authors derive much of
inequality results using geometry arguments. In many ways, I missed this elegant
geometric representation in the present paper. I’d be interested in exploring how
the restrictions derived in this pair of papers e⇤ect the characterization of p(x, y, z)
and thus p(x, y|z), through marginals, conditionals, and ratios of odds ratios for thealternative definition of causal e⇤ect based on odds ratios and log-linear parameters.
Then we might be able to exploit algebraic geometric arguments such as thosedescribed in Slavković and Fienberg (2010). This was a problem I tried to address,
with singular lack of success, when I first saw the bounds in Balke and Pearl (1997)
thirteen years ago. Perhaps it is time to return to it with RER’s work as a guide.
PAUL GUSTAFSON (University of British Columbia, Canada)
I congratulate Richardson, Evans and Robins (hereafter RER) on a very interesting
paper. I completely agree that given a partially identified problem, identifying atransparent parameterization is key to understanding the e⌃cacy of Bayesian inference. I also think that the paper breaks fruitful new ground in exploring Bayesianinference for the instrumental variables model based on potential outcomes. Given
the ease with which a Bayesian solution integrates uncertainty due to finite sampling

600

T. S. Richardson, R. J. Evans and J. M. Robins

and uncertainty due to a lack of identification, consideration of Bayesian inference
for potential outcome models seems important in general.
One curious point about partially identified models is that in the asymptotic
limit the Bayesian has more to convey than the frequentist. That is, both willagree on the set of possible values for the target parameter, but the Bayesian will
additionally weight the plausibility of di⇤erent values in this set with respect to one
another, i.e., the limit of the posterior distribution will have some shape. It seems
relevant to ask about the utility of this shape, and the extent to which it is driven
by the data versus the prior.
As a simple illustration, consider a slight extension of RER’s motivating example
in Section 3. Still making the “randomized trial” assumption Yx0 = Yx1 , consider
the three compliance types {N T, AT, CO}, i.e., always-takers, but not defiers, havebeen added to the mix. Following the RER notation, the situation can be understoodscientifically with reference to parameterization( NT ,

0·
1·
0·
1·
0·
1·
AT , ⇤CO , ⇤CO , ⇤N T , ⇤N T , ⇤AT , ⇤AT ),where implicitly CO = 1
NT
AT . These scientifically interpretable parameters
can simply be cleaved into a wholly identified component✏ = ( NT ,

0·
1·
0·
1·
AT , ⇤CO , ⇤CO , ⇤N T , ⇤AT ),1·0·
and a component ⇣ = (⇤N
T , ⇤AT ) which is not involved in the likelihood function.
Now, say the target of inference is the average causal e⇤ect,

ACE

=

(1

NT

1·
AT )(⇤CO

0·
⇤CO
)+

1·
N T (⇤N T

0·
⇤N
T) +

1·
AT (⇤AT0·⇤AT).

Thus, regardless of whether one pursues a frequentist or Bayesian analysis, in theasymptotic limit one learns the range of possible values for the target is a ± b, where
a

=

(1

NT

1·
AT )(⇤CO

0·
⇤CO
)+

N T (1/2

0·
⇤N
T) +

1·
AT (⇤AT

1/2),

and b = ( N T + AT )/2.
From a frequentist viewpoint, (a, b) are all that can be learned about the ACEfrom an infinite-sized dataset. The situation is di⇤erent, however, for a Bayesian.
The large-sample limit of the posterior distribution must have a ± b as its support,
but additionally the shape of the limiting distribution may depend on the identified
parameters.
For instance, say a uniform prior is applied in the scientific parameterization(more formally a Dirichlet(1, 1, 1) prior for ( N T , AT , 1 N T
AT ), and Unif(0, 1)
priors for each of ⇤’s, with independence throughout). Then it follows directlythat the large-sample limit of the posterior distribution for the ACE has a stochastic representation as a + N T (U11/2) + AT (U21/2), where U1 , U2 are iid
Unif(0, 1). It then follows that the limiting distribution is symmetric on a ± b, witha trapezoid-shaped density. Particularly, the top edge of the trapezoid extends alonga ± (max{ N T , AT } min{ N T , AT }) /2. Extreme cases are a uniform limiting
density (when 0 = min{ N T , AT } < max{ N T , AT }), and a triangular limiting
density (when N T = AT > 0). Thus, the peakedness of the limiting distribution depends on identified parameters, showing that the shape is not merely aTransparent Parametrizations of Models for Potential Outcomes

601

pre-ordained consequence of the shape of the prior. As a practical implication, thewidth of a central credible interval for the target, relative to the width of the set ofplausible values, varies according to the underlying true parameter values.
In fact, in the present problem the influence of the identified parameters on the
shape of the posterior on the target is fairly mild. Particularly, the limiting posterior
distribution is symmetric on a±b, no matter what. However, Gustafson (2010) gives
examples of partially identified models where the shape depends more strongly on
the identified parameters. This is more prone to occur when the identified parameter
vector ✏ is a complicated function of the original scientific parameters, rather thansimply a subset of them as in the example above.
FABRIZIA MEALLI (Università di Firenze, Italia) and
FAN LI (Duke University, USA)We appreciated the invited paper by Richardson, Evans and Robins (henceforth
RER), as the only one in Valencia 9 dealing with causal inference, an importantbranch of statistical inference for which Bayesian analysis is particularly and naturally suited (Rubin, 1978). However, in our view, the paper misses important
recent advances made in causal inference, and specifically, in Bayesian analysis of
broken randomized experiments. We would like to stress that a framework, namely
Principal Stratification (PS; Frangakis and Rubin, 2002), exists that allows one totransparently specify causal models, to separate structural behavioral assumptions
from model assumptions and priors on parameters, and to conduct model-based
Bayesian inference in a principled fashion; the area could certainly benefit fromcross-fertilization.
PS has been successfully applied to a wide range of more general and complicatedsettings, where the applicability of RER’s approach is not completely clear. The
aim of our discussion is three-fold: (i) to provide a brief account of the existing
literature on the subject of Bayesian causal inference with intermediate variables;(ii) to elucidate how Bayesian inference is conducted under the PS framework; and(iii) to discuss some inferential and practical restrictions embedded in RER.
The all-or-none noncompliance setting analyzed by RER is an example of causal
analysis with intermediate variables, that is, post-treatment variables potentiallya⇤ected by treatment and also a⇤ecting the response. Much of the notation andterminology in RER stems from the series of papers by Imbens, Rubin and coauthors in the 1990s (e.g., Imbens and Angrist, 1994; Angrist, Imbens and Rubin,
1996; Imbens and Rubin, 1997), which provided the terminology of NT, AT, CO,and DE. Further, Frangakis and Rubin (2002) proposed the general PS framework
for adjusting for intermediate variables, based on stratifying units by their joint
potential intermediate outcomes. Since then, advances have been achieved in causal
inference under PS, both from frequentist and Bayesian perspectives (the literature
listed below is limited to the Bayesian one), dealing with settings of binary, categorical, continuous, censored outcomes with and without covariates (e.g., Hirano, 2000;
Zhang et al., 2008); noncompliance coupled with missing data or/and censored data
(e.g., Barnard et al.; 2003; Mattei and Mealli, 2007); longitudinal treatments andintermediate variables (Frangakis et al., 2004); clustered treatments (e.g., Frangakis
et al., 2002); surrogate endpoints (Li et al., 2010); continuous intermediate variables,
including partial compliance (e.g., Jin and Rubin, 2008; Schwartz et al., 2010); justto name a few.

602

T. S. Richardson, R. J. Evans and J. M. Robins

While RER provide some insights on the information which can be drawn
from randomized experiments with noncompliance, their statement about “standard
Bayesian prior to posterior analysis” of “weakly identified” models may suggest to
those who are unfamiliar with the causal inference literature that the current stateof-the-art Bayesian analysis of such models is not done properly, or done in a rather“automatic” fashion, without posing attention on the nature of the di⇤erent causal
estimands, and on the information provided by the data on them. However, Imbens and Rubin (1997) already provided a complete recipe for model-based Bayesianinference and investigated the behavior of weakly identified models for the case of
all-or-none compliance. Their approach can be easily generalized to conduct analysis
with other intermediate variables, as briefly described below.
Let Zi be the binary variable indicating the treatment assignment of unit i.
Under SUTVA, the potential outcomes are a function of Zi rather than the entire
vector Z. Let Yi (z) and Xi (z) be the potential primary and intermediate outcomes
if unit i is assigned to treatment z for z = 0, 1. In Bayesian inference, the observable
quantities for a sample of N units, (Y (0), Y (1), X (0), X (1), Z, V ), are consideredas observed and unobserved realizations of random variables, with joint distribution
Pr(Y (0), Y (1), X (0), X (1), Z, V ) which may be written as
Pr(Y (0), Y (1), X (0), X (1)|Z, V ) Pr(Z|V ) Pr(V )= Pr(Y (0), Y (1), X (0), X (1)|V ) Pr(Z|V ) Pr(V ),where the equality follows from randomization, which allows one to separate thejoint distribution of the potential outcomes from the treatment assignment mechanism. Analysis is usually conditional on the observed distribution of covariates,
thus Pr(V ) is not modelled. The joint distribution of the potential outcomes,
Pr(Y (0), Y (1), X (0), X (1)|V ), can be rewritten as
Z Y
Pr(Yi (0), Yi (1)|Xi (0), Xi (1), Vi , ✓) Pr(Xi (0), Xi (1)|Vi , ✓) (✓) d✓(26)
ifor the global parameter ✓ with prior distribution (✓). The quantity (Xi (0), Xi (1))
is called a principal stratum Si and the cross-classification of units into the latentclasses of Si is called PS. Clearly the classification of units into NT, AT, CO, DE
is a special case of PS. The key insight is that Si is invariant under di⇤erent treatment assignments, thus the comparisons of {Yi (1) : Si = (x0 , x1 ), Vi = v} and{Yi (0) : Si = (x0 , x1 ), Vi = v} are well-defined causal e⇤ects (called principal causale⇤ects—PCEs). Factorization (26) suggests that model-based PS inference usually involves two sets of models: One for the distribution of potential outcomesY (0), Y (1) conditional on the principal strata and covariates and one for the distribution of principal strata conditional on the covariates. The definition of principle
strata does not involve response Y , unlike the approach in RER. We find it hardto envision how the approach in RER can be extended to, for example, the mostcommon case of continuous Y .
Another critical feature in the models adopted in the PS literature is that one
specifies models directly on potential outcomes instead of on observed quantities.
This, we think, is a more transparent way of modelling, and also, by doing so wecan “directly” check which restrictions are supported by the data.
Since causal e⇤ects are defined as (summaries of) comparisons between the potential outcomes of the same individuals, in our opinion directly modelling potential

Transparent Parametrizations of Models for Potential Outcomes603

outcomes enables analysts to transparently conduct causal inference. To conduct
Bayesian inference in PS, the complete-data likelihood of all units can be written
Pr(Y (0), Y (1), X (0), X (1)|V ; ✓)“YN=
Pr Yi (0), Yi (1)|Si , Vi ;
i=1

Y

”

“
Pr Si |Vi ;

S

”

,

where ✓ includes parameters Y ’s and S . The Bayesian model is completed by
specifying prior distributions for each set of parameters. Note that separating theparameters into an identifiable set and a non/weakly identifiable set, as RER do,
may lead to prior independence between parameters that are substantively thought
to be closely related.
DefineYiobs = Yi (Zi ),Yimis = Yi (1Zi ),

Xiobs = Xi (Zi ),Ximis = Xi (1Zi ).

All PCEs are functions of the model parameters ✓ and observed quantities, so full
Bayesian inference for PCEs is based on the posterior distribution of the parametersconditional on the observed data, which can be written as,Pr(✓|Y obs , X obs , Z, V )
Z Z Y
Pr(✓)
Pr(Yi (0), Yi (1), Xi (0), Xi (1)|Vi , ⌃)dYimis dDimis .
iHowever, direct inference from the above distribution is in general not available
due to the integrals over Dimis and Yimis . But both Pr(⌃|Y obs , X obs , X mis , V )and Pr(X mis |Y obs , X obs , V , ✓) are generally tractable, so the joint posterior distribution, Pr(✓, X mis |Y obs , X obs , Z, V ), can be obtained using a data augmentation
approach for X mis . Inference for the joint posterior distribution then provides inference for the marginal posterior distribution Pr(✓|Y obs , X obs , Z, V ).
Note that, by adopting PS, it is rather obvious that some parameters are “wholly”
nonidentified. They are usually those which depend on potential outcomes thatare never observed in a particular experiment for certain types of subjects (principal strata). For example Y11 is never observed for units (NT and DE) withXi (Zi = 1) = 0; it is thus an “a priori” counterfactual. As a consequence thee⇤ect of treatment receipt for NT does not appear in the likelihood, and no prior isput on it. Indeed, even after a Bayesian analysis, bounds can be derived on thesequantities, e.g., by letting the a priori counterfactual outcomes range from theirsmallest to their largest possible values (see, e.g., Imbens and Rubin, 1997, page319). More precise inference can be obtained, e.g., by “extrapolation”, assuming
the e⇤ect found for CO is the same as the e⇤ect that would have been observed forNT, had they been forced to take the treatment. Also, because the e⇤ect on CO iswell identified only under some restrictions, the Bayesian PS approach allows one
to directly and transparently check what restrictions are supported by the data, by
relaxing those restrictions and checking how much posterior support they receive.

604

T. S. Richardson, R. J. Evans and J. M. Robins

REPLY TO THE DISCUSSION
We thank the discussants for their thoughtful comments and criticisms. We comment on each contribution in turn. (Readers should note that Section 6 and theappendix concerning mis-specification were added subsequently, hence were not seenby the discussants.)
Different causal effect measures. Fienberg notes that all of the causal contrasts
in our paper are defined on the linear scale, and asks to what extent the methods canbe applied to the causal relative risk or causal odds ratio. Richardson and Robins(2010) characterize the set of possible values for( NT ,

AT ,

DE ,

0·
1·
0·
1·
0·
1·
0·
1·
CO , ⇤CO , ⇤CO , ⇤NT , ⇤NT , ⇤AT , ⇤AT , ⇤DE , ⇤DE ),compatible with a given observed population distribution p(y, x | z) under the model
EXAT + EXNT . Given this description it is straightforward to compute bounds on
any causal contrast for this model regardless of the chosen scale. The methods ofanalysis may be extended fairly easily to any of the other models we consider, as we
did to compute the bounds on ITTCO given in Figure 7.
More generally, we agree with Fienberg that algebraic geometry and, in particular, the theory of convex polytopes have an important role to play in understanding
identifiability in similar potential outcome models in which variables have more than
two states.
The role of re-parametrization. We thank Gustafson for pointing out that theuse of a transparent re-parametrization in no way precludes the specification of aprior on the non-identified parameters. From our perspective, re-parametrization
is purely a mathematical technique for clarifying the relationship between the data
and the posterior by separating the wholly identified from the wholly unidentified.
More formally, let ⌃ be the vector of parameters in the original (e.g., “Principal”
Stratum) formulation, and let (⇣, ) indicate the transparent re-parametrization
into identified (⇣) and unidentified () components, via some di⇤eomorphism g(·),
so ⌃ = g(⇣, ). A prior p(⌃) induces a prior p(⇣, ) = p(⇣)p( | ⇣). Then we have:p(⇣,  | y) = p( | ⇣)p(⇣ | y),(27)

where we have used the fact that  does not occur in the likelihood. Forward
sampling may be used to obtain samples from the posterior p(⌃ | y) by first sampling⇣ (i) from p(⇣ | y) and then sampling (i) from p( | ⇣ (i) ). The corresponding valueof ⌃(i) = g(⇣ (i) , (i) ).
In the illustrative analyses we present in the paper (that make use of re-parametrizations) we avoided placing prior distributions on parameters that were not
identified, instead opting to compute posterior distributions on bounds. This wasprimarily because we thought that in many circumstances useful subjective information relating to these specific unidentified quantities may be hard to come by.
We make a few further points in this regard below.
Transparent re-parametrization is, in principle, compatible with any Bayesiananalysis of a partially identified model. We say “in principle” because it may require
some technical work to be able to find such a re-parametrization: this is one of themain contributions of our paper for the unidentified models we consider.
However, it is therefore incorrect to suggest, as Mealli and Li do, that an analyst
must choose between a “PS approach” and our re-parametrization. Like co-ordinateTransparent Parametrizations of Models for Potential Outcomes

605

systems, an analyst is free to use more than one parametrization within a single
analysis: if background knowledge is more amenable to formulation via reference tocompliance types then the prior may be formulated in these terms. However, we,like Gustafson, Greenland and Leamer before us, believe that in order to assess theextent to which beliefs regarding unidentifiable quantities influence the posterior
distribution it is often necessary to use a transparent re-parametrization.
Relation to existing methods. Mealli and Li state that Imbens and Rubin (1997)
provide a complete recipe for model-based Bayesian inference of “broken” randomized experiments with non-compliance. We do not agree. We believe that themethod of Imbens and Rubin is incomplete in its treatment of two central issues:(i) the sensitivity of the posterior for a partially identified quantity to the prior; (ii)bias under model mis-specification in randomized experiments. We now consider
each in turn.
Prior sensitivity.
In order to eliminate extraneous issues, we assume baseline covariates are either
absent, or take only a few values.
Inference for partially identified quantities vs. inference for bounds. An important theme in our paper is that when faced with a partially identified parameter, it
is advisable to proceed by computing the bounds on this quantity implied by the
population distribution for the observables, and then to perform inference for thesebounds. Such bounds, being functionals of the observed distribution, are identified.
Though we did not stress this point in the paper, such an analysis need not preclude, and indeed may complement, a standard Bayesian analysis for the quantityof interest. Thus, contrary to Mealli and Li, it is a false dichotomy to suggest thatan analyst must choose one or the other.
Testing exclusion restrictions. Hirano et al. highlight the ability to relax individual exclusion restrictions as one of the strengths of their approach. In theirremarks Mealli and Li write:. . . the Bayesian PS approach allows one to directly and transparently check
what restrictions are supported by the data, by relaxing those restrictions andcheck[ing] how much posterior support they receive.

In our opinion, this remark indicates the danger of Bayesian analyses that fail todistinguish what is from what is not identifiable.
To see this point more clearly, consider the restriction ExAT , which is one of
the assumptions necessary to identify ITTCO . Note that the assumption ExAT isequivalent to ITTAT = 0. The quote of Mealli and Li above suggests that, had a
99.9% credible interval for ITTAT excluded zero, Mealli and Li would regard this as
overwhelming evidence that ExAT is false, even if they (following Hirano et al.) had
used “o⇤-the-shelf” priors. But such an inference would be erroneous. It is possiblethat the (identifiable) population lower and upper bounds, denoted by l ITTAT and
uITTAT , for ITTAT straddle zero, yet owing to the specific prior used, the posteriorcredible interval for ITTAT may exclude zero; compare to the right panel of Figure 3.
In contrast, an analysis based on credible intervals for bounds will (asymptotically)
not make such a mistake; see Table 9, with ExAT true.
Hirano et al. concluded, based primarily on subject matter considerations, that
there was reason to doubt the exclusion restriction for Always Takers in the McDonald et al. data. (However, this decision was not a consequence of their likelihood or606

T. S. Richardson, R. J. Evans and J. M. Robins“o⇤-the-shelf” prior.) Our Bayesian inference for bounds provides some empiricalsupport for this doubt; see Column (13) in Table 7.
Table 9: Contrasting large-sample inferences for ITTAT vs. large-sample inferences for upper and lower bounds on ITTAT . PD indicates that this is priordependent (even asymptotically); uITTAT and lITTAT are the (identifiable) upperand lower bounds. MonX is assumed to hold. LSCI is a credible interval in large
samples.
True StateExAT true
ExAT false
Result of Posterior Analysis(13) true(13) true(13) false

0 2 LSCI[ITTAT ]

PD

PD

False

02
/ LSCI[ITTAT ]

PD

PD

True

LSCI[uITTAT ] \ [0, 1) 6= ;and LSCI[l ITTAT ] \ ( 1, 0] 6= ;

True

True

False

LSCI[uITTAT ] \ [0, 1) = ;or LSCI[l ITTAT ] \ ( 1, 0] = ;

False

False

True

Table 10: Parameter values for simulation scenarios: all linear models (L);quadratic compliance model (QC); quadratic qesponse models (QR).

L
QC
QR

0

1

2

2
3.5
2

0.3
2.5
0.3

0
0.5
0

⇥0c
2
2
4.6

⇥1c
1
1
4.8

⇥2c
0
0
1

⇤0c
0
0
0

⇤1c
0
0
0

⇥0n
5
5
3

⇥1n
2
2
5

⇥2n
0
0
1.2

Examining prior sensitivity. The question of the extent to which prior specification influences the posterior may arise in any Bayesian analysis. However, we believethat ad hoc approaches, which may be appropriate in identified contexts, such as
finding “equivalent sample sizes” or comparing the prior and posterior standard
deviations for quantities of interest (see Hirano et al., 2000, p. 78), may be highly
misleading in the context of partially identified parameters and are not logicallyjustified.
As an example, suppose the population bounds on a partially identified parameter of interest were ( 2, 2). If a Bayesian analyst specified a di⇤use but proper
prior with a 99% credible interval of ( 50, 50) then in large samples, the posterior standard deviation will be at most 1/20 of the prior standard deviation (undersome assumptions on the shapes of the prior and posterior). Nevertheless, owing to
the sensitivity of the posterior to even a di⇤use prior, the Bayesian’s posterior 99%
credible interval could be, ( 1.8, 0.3) even though the true value of the partially
identified parameter was 0.5; see Figure 3.
“Principal” strata vs. strata defined by baseline covariates. One of the primary
motivations for the principal stratum (PS) approach is that, if the strata are based on

Transparent Parametrizations of Models for Potential Outcomes

●

●

●
●

●
●
●

●
●

ITTCO

0.05

●

0.10

●

0.00

ITTCO

Quadratic Compliance

0.00

0.10

Linear

607

●
●
●
●
●

2

3

1

3

4

Global ITT estimates

0.10

ITTCO

●
●
●

●
●

−0.10

2

3

Covariate value (V)

●
●
●
●
●
●
●
●

●

−0.04

●

0.04

0.08

Quadratic Response
0.20

Covariate value (V)

●
●

1

2

Covariate value (V)

0.00

ITTCO

4

●

●
●

0.00

−0.10

●
●

●

1

−0.10

●

●

4

●
●
●

L

●
●

QC

●

QR

Data Generating Process

Figure 10:

Boxplots showing the sampling distribution of the MLE for
ITTCO (v): linear (top left); quadratic compliance (top right); quadratic
response (bottom left). Sampling distributions of the MLE for the global
ITTCO (bottom right). The true values of all ITTCO parameters are
zero; see dotted line.
well-defined potential outcomes for the intermediate, contrasts between the treatedand untreated within such strata admit a causal interpretation. In this regard,
principal strata are analogous to a set of baseline covariates su⌃cient to control
confounding. However, in another important respect, principal strata are very different from baseline covariates in that, in general, we are never able to directly
observe such memberships. Consequently, prior information regarding di⇤erences in
response between compliance types are likely to be scarce and unreliable. This isa major concern in light of the extreme sensitivity of the posterior distribution forweakly identified parameters to the choice of prior.
Model mis-specification in randomized experiments
We now consider the consequences of model mis-specification for the method of
Hirano et al. in a randomized experiment with continuous baseline covariates V . Inclusion of baseline covariates in the analysis is useful because qualitative treatmentcovariate interactions can be detected and, as noted by Hirano et al., e⌃ciency may
be increased.

608

T. S. Richardson, R. J. Evans and J. M. Robins

We now consider the setting of a double blind (DB) placebo-controlled RCT in
which treatment is without side-e⇤ects and is not available to patients in the control
arm. In this setting Defiers and Always Takers are not present. Furthermore, the00
01exclusion restrictions for Never Takers (EXNT ) and for Compliers (⇤CO
= ⇤CO
,
10
11
⇤CO = ⇤CO ) can be assumed to hold within levels of V . Then the conditional intent
to treat e⇤ect ITTCO (V ) in the Compliers is identified by
ITTY (V )E [Y | Z = 1, V ]⇧ITTX (V )E [X | Z = 1, V ]E [Y | Z = 0, V ]E [X | Z = 0, V ]and equals the conditional Complier Average Causal E⇤ect ACECO (X ↵ Y | V ) of
X on Y . The unconditional ITT e⇤ect, ITTCO = E [ITTCO (V )] and unconditional
Complier Average Causal E⇤ect are also identified.
In Section 6, we described why it is critical to analyze randomized trials witha method that, under the null hypothesis that ITTY (V ) = 0, guarantees that theposterior distribution and MLE of ITTCO (V ) [and thus of ITTY (V )] concentrateon the zero function, even under model mis-specification. The following simulation
study demonstrates that the method of Hirano et al. does not o⇤er such a guarantee,
even when V is discrete.
We simulated the data under the model:2p(tX = CO | v, z)=

expit( 0 +=p(Y = 1 | tX = NT, v, z)=

expit(⇥0c + ⇥1c v + ⇥2c v 2 + z(⌅0c + ⌅1c v)),
expit(⇥0n + ⇥1n v + ⇥2n v 2 ),p(Y = 1 | tX = CO, v, z)

1v +

2v

),

with ⌅0c = ⌅1c = 0 hence ITTY (V ) = 0. The baseline covariate V is ordinal, distributed uniformly with sample space {1, 2, 3, 4}. Always Takers and Defiers were
excluded a priori. We considered data simulated under three di⇤erent parametersettings as shown in Table 10; in the first (L), there are no quadratic terms; in thesecond (QC), there is a quadratic term in the logistic regression model for the proportion of Compliers vs. Never Takers; in the third (QR), there is a quadratic termpresent in the logistic regression models for E[Y | X = 0, tX ] for tX ⇣ {CO, NT}.
For each scenario we simulated 500 datasets of size 5,000. We used the linear logistic model of Hirano et al. without Always Takers to analyze the data; we purposely
omitted the quadratic terms from the models fitted. Since we are interested primarily in large sample performance we used the MLE and standard asymptotic 95%
Wald confidence intervals as convenient approximations to the posterior mode and95% credible intervals. Table 11 gives the sampling distribution of the MLEs for⌅0c and ⌅1c , together with the actual coverage rate for nominal 95% and 90% asymptotic confidence intervals. Figure 10 shows sampling distributions of the MLEs
for ITTCO (V ) and ITTCO under each of these scenarios. As can be seen, misspecification of either the model for compliance types or for E[Y | X = 0, tX ] leads
to spurious inferences regarding the ITT e⇤ects even under the ITT null.
As discussed in our paper, the model (the last in Table 6), that we had proposed
in Section 5 to analyze such a trial also failed to satisfy the wished-for guarantee;
see the simulation study in §6. As noted earlier, Mealli et al. wrote their discussion
based on an earlier version of the paper that did not include Section 6; hence they
had no opportunity to express their thoughts on this issue.

Transparent Parametrizations of Models for Potential Outcomes

609

Table 11: Simulation results: distribution of MLE for ⌅0c and ⌅1c fitting potential
outcome models omitting quadratic terms. Coverage shows the actual coveragecorresponding to asymptotic confidence intervals based on the observed information.
Results based on 500 simulations; sample size was 5, 000. The ITT null holds so⌅0c = ⌅1c = 0.

L
QC
QR

⇤0c MLE
Mean Std. Err
0.001(0.010)0.331(0.005)0.362(0.008)Coverage
90%
95%
0.91
0.94
0.20
0.31
0.31
0.43

⇤1c MLE
Mean Std. Err
0.000(0.005)0.257(0.005)0.169(0.003)Coverage
90%
95%
0.91
0.94
0.22
0.35
0.25
0.34

As noted in the paper, we specifically developed the parametrization and parametric model described in the last subsection of §6 to provide robustness to model
mis-specification under the ITT null. Note, however, that this parametric model allows for the possibility of Always Takers. A robust model that assumes the absenceof Always Takers is obtained by simply setting the functions ⌥i (v; 1 ), i ⇣ {0, 1}, tozero in the aforementioned model.
Our simulation study demonstrated, by example, the non-robustness of the Hirano et al. approach in the simple setting of a single V with only four levels and asample size of 5,000; see Table 11 and Figure 10. As such, it is likely that various
goodness-of-fit statistics would reject the linear analysis model with high power.
However, when V is a vector with continuous components, it is more di⌃cult tospecify correct or nearly correct parametric models and the power of goodness-of-fit
statistics to reject even a quite mis-specified model is poor. Thus we suspect thatin high-dimensional settings the use of non-robust parametric models will typicallyresult in markedly incorrect inference under the conditional ITT null.
Summary
We certainly do not claim that all model-based analyses of partially-identified
quantities are equally misleading. Indeed, we found the Hirano et al. Bayesiananalysis to be interesting, thoughtful, and restrained in its conclusions.
In contrast, Zhang et al. (2009) analyze partially-identified direct e⇤ects in a jobtraining program. They entirely eschew a Bayesian approach, preferring instead to(i) specify a parsimonious parametric model whose functional form serves to pointidentify the direct e⇤ects, and (ii) estimate these e⇤ects by maximum likelihood(without associated standard errors). They summarize their inferences with boldpronouncements such as: “there is a group of individuals, about 8%, for whom
assignment to training is harmful in terms of employment,” without any measure of
uncertainty.
We can summarize our concerns by echoing David Freedman’s invocation of Will
Rogers, who famously said: “It’s not what you don’t know that hurts. It’s what
you know that ain’t so. . . ,” the model-based approach advocated by Mealli et al.
is likely to “increase the stock of things we know that you know for sure that justain’t so.”1
Finally, our paper and those referenced by Mealli and Li were concerned withmaking inferential statements about causal e⇤ects of scientific interest and not with
1 Sander Greenland notes that this quote is originally due to Mark Twain.

610

T. S. Richardson, R. J. Evans and J. M. Robins

decision-making under uncertainty. When a decision must be made and the optimal
choice depends upon an unknown partially-identified e⇤ect parameter, we, like all
Bayesians, would use our personal posterior distribution for the parameter of interest. In such a situation we would use our proper subjective prior; we would not use
either a default or “o⇤-the-shelf” prior.
ADDITIONAL REFERENCES IN THE DISCUSSION
Angrist, J. D., Imbens, G. W., Rubin, D. B. (1996). Identification of causal e⇤ects using
instrumental variables. J. Amer. Statist. Assoc. 91, 444–455.
Barnard, J., Frangakis, C. F., J., Hill, J. L., Rubin, D. B. (2003). Principal stratification
approach to broken randomized experiments: A case study of school choice vouchers in
New York City. J. Amer. Statist. Assoc. 98, 299–323 (with discussion).
Frangakis, C. E., Rubin, D. B., and Zhou, X. H. (2002). Clustered encouragement design
with individual noncompliance: Bayesian inference and application to advancedirective forms. Biostatistics 3, 147–164.
Frangakis, C. E., Brookmeyer, R. S., Varadhan, R., Safaeian, M., Vlahov, D., andStrathdee, S. A. (2004). Methodology for evaluating a partially controlled longitudinaltreatment using principal stratification, with application to a needle exchange
program. J. Amer. Statist. Assoc. 99, 239–249.
Gustafson, P. (2010). Bayesian inference for partially identified models. Internat. J.
Biostatistics 6, Art 17.
Imbens, G. W., Angrist, J. (1994). Identification and estimation of local averagetreatment e⇤ects. Econometrica 62, 467–476.
Jin, H., Rubin, D. B. (2008). Principal stratification for causal inference with extended
partial compliance. J. Amer. Statist. Assoc. 103, 101–111.
Li, Y., Taylor, J. M., Elliott M. R. (2009). A Bayesian approach to surrogacy assessment
using principal stratification in clinical trials. Biometrics 66, 523–531.
Mattei, A., Mealli, F. (2007). Application of the prinicipal stratification approach to the
Faenza randomized experiment on breast self-examination. Biometrics 63, 437–446.
Robins, J., Rotnitzky, A., and Vansteelandt, S. (2007) Discussion of Principal
stratification designs to estimate input data missing due to death by Frangakis, C.E.,
Rubin, D.B., An, M., MacKenzie, E. Biometrics 63, 650–653.
Rubin, D. B. (1978). Bayesian inference for causal e⇤ects. Ann. Statist. 6, 34–58.
Schwartz, S.L., Li, F., and Mealli, F. (2010). A Bayesian semiparametric approach tointermediate variables in causal inference. Tech. Rep., Duke University, USA.
Sfer, A. M. (2005). Randomization and Causality. Ph.D. Thesis, Universidad Nacional de
Tucumán, Argentina.
Slavković, A. B. and Fienberg, S. E. (2010). Algebraic geometry of 2 ⇥ 2 contingency
tables. Algebraic and Geometric Methods in Statistics (P. Gibilisco, et al., eds.).
Cambridge: Cambridge University Press, 67–85.
Zhang, J. L., Rubin, D. B., Mealli, F. (2008). Evaluating the e⇤ects of job training
programs on wages through principal stratification. Modelling and Evaluating
Treatment E↵ects in Econometrics (D. L. Millimet, J. A. Smith and E. J. Vytlacil,eds.). Amsterdam: Elsevier, 117–145.
Zhang, J. L., Rubin, D. B., Mealli, F. (2009). Likelihood-based analysis of causal e⇤ects ofjob-training programs using principal stratification. J. Amer. Statist. Assoc. 104,
166–176.