Assessing Heterogeneity of Treatment Effects∗
arXiv:2306.15048v1 [econ.EM] 26 Jun 2023

Tetsuya Kaji1 and Jianfei Cao2
1
2

University of Chicago

Northeastern University
June 28, 2023

Abstract
Treatment effect heterogeneity is of major interest in economics, but its
assessment is often hindered by the fundamental lack of identification of theindividual treatment effects. For example, we may want to assess the effect of
insurance on the health of otherwise unhealthy individuals, but it is infeasible
to insure only the unhealthy, and thus the causal effects for those are not
identified. Or, we may be interested in the shares of winners from a minimum
wage increase, while without observing the counterfactual, the winners are not
identified. Such heterogeneity is often assessed by quantile treatment effects,
which do not come with clear interpretation and the takeaway can sometimes
be equivocal. We show that, with the quantiles of the treated and control
outcomes, the ranges of these quantities are identified and can be informativeeven when the average treatment effects are not significant. Two applications
illustrate how these ranges can inform us about heterogeneity of the treatment
effects.
∗

This work is supported by the Liew Family Junior Faculty Fellowship and the Richard N. Rosett
Faculty Fellowship at the University of Chicago Booth School of Business and the support for visiting
faculty at the Department of Economics at the Massachusetts Institute of Technology. We thankYijing Zhang for excellent research assistance. We thank Rachael Meager, Toru Kitagawa, Kosuke
Imai, Alberto Abadie, Whitney Newey, Ismael Mourifié, Eric Auerbach, Ivan Canay, Federico Bugni,
Filip Obradovic, Muyang Ren, Josh Angrist, and seminar participants at various places for helpfulcomments.

1

1

Introduction

Heterogeneity of the treatment effects is a major concern in various places of economics. In causal inference with instrumental variables, heterogeneous treatment
effects play a key role in the correct interpretation of the estimates (Imbens and Angrist, 1994). In optimal policy targeting and ethical intervention design, assessmentof heterogeneity is vital in improving welfare and promoting equity (Kitagawa andTetenov, 2018, 2021). In the empirical examination of economic theory, it is essential to account for heterogeneous impacts that are consistent with the theoreticalprediction (Bitler et al., 2006).
The assessment of heterogeneity is often hindered by the lack of identification of
the individual treatment effects. Because of the fundamental impossibility of assigning
more than one treatment status to one subject, the joint distribution of the treatedoutcome and the control outcome is not identified. When the average treatment
effect (ATE) is of concern, this lack of identification is not an issue since the ATE
only depends on the marginal distributions of the potential outcomes. There aresituations, however, in which delicate assessment of heterogeneity is needed.
Consider an investigation of moral hazard in health care (e.g., Aron-Dine et al.,2013). Ideally, we want to know how health insurance affects medical spending acrossdifferent levels of untreated medical spending. However, since the joint distribution of
the treated outcome and the control outcome is not identified, such treatment effects
are not identified. In another example, consider the evaluation of a welfare program
for low-income and unemployed households (e.g., Bitler et al., 2006). The theoreticalprediction is often ambiguous even on who benefits and who loses from the program,so the average effects do not draw the complete picture. However, again, due to thelack of identification of the joint distribution, the shares of winners or losers are not
identified.
In these settings where heterogeneity is of primary interest, economists often use
the quantile treatment effects (QTEs) to grasp a sense of heterogeneity. While QTEs
can suggest the presence of heterogeneity, they can be hard to assess or interpret if
we are not willing to assume a strong condition of rank invariance.
In this paper, we focus on two quantities that summarize heterogeneity and complement QTEs, and show that—although they are not point-identified—we can calculate the ranges in which they reside. Using two empirical applications, we show that2

these ranges can be informative, even when the ATE estimates are not significant.
The first quantity is what we call the subgroup treatment effect, which is the ATE
for the subpopulation whose control outcome is in some range. For example, when thelower bound for the subgroup treatment effect is positive, we may conclude that the
subgroup of interest receives at least some positive treatment effects. We illustrate
the usefulness of this concept in Section 3 drawing an example from Tarozzi et al.
(2015). The second quantity is what we call the subgroup proportion of winners (orof losers), which is the share of those whose treated outcome exceeds (or falls under,
respectively) the control outcome in the subpopulation whose control outcome is insome range. For example, if the proportion of winners is above one fourth, we may
conclude that there are at least a quarter of subjects whose outcomes improved thanks
to the treatment. Section 4 illustrates the usefulness of this concept in an example
from Bitler et al. (2006).
There is enormous literature on (partial) identification of various heterogeneity
in program evaluation. Close to our paper are Heckman et al. (1997) and Tetenov(2012). Heckman et al. (1997) discussed bounds on the proportion of winners andthe distribution of treatment effects and investigated how the assumption of rational
choice by the participants helps sharpening the bounds. Tetenov (2012) derived
bounds on positive and absolute treatment effects and the proportion of winners on
the entire population. A key departure of our analysis is that we consider subgroups
that are defined by ranges of the control outcome, which gives a finer picture of who
receives how much treatment effects.
The rest of the paper is organized as follows. Section 2 lays down the notation
used in the paper and clarifies the extent to which the proposed methods can beapplied. Section 3 introduces the bounds on the subgroup treatment effects andillustrates their use drawing examples from Tarozzi et al. (2015). Section 4 presents
the bounds on the subgroup proportions of winners and applies them to revisit Bitler
et al. (2006). Section 5 concludes. Appendix A collects the proofs.

2

Notation and the Scope of Applicability

We employ the standard notation of the potential outcome framework; Y0 denotes
the potential outcome without the treatment and Y1 the potential outcome with thetreatment. The marginal cumulative distribution functions (cdfs) of the potential3

outcomes are denoted by F0 for the control and F1 for the treated. We define thequantile functions of the potential outcomes, Q0 and Q1 , by the left-continuous generalized inverses of F0 and F1 . We denote the normalized rank of Y0 by U ∈ [0, 1].
Mathematically, this is equivalent to having a uniformly distributed random variable
U ∼ U (0, 1) that satisfies Y0 = Q0 (U ). We define a winner (from the treatment) to
be a subject with Y1 > Y0 and a loser (from the treatment) a subject with Y1 < Y0 .
We use the notation (x)+ := max{x, 0} and (x)− := (−x)+ . We denote the left limitof a cdf by F (a−) := limx↗a F (x).
The key assumption in this paper is that the marginal distributions of the potential
outcomes are identified. This applies to settings beyond the standard randomized controlled trials (RCTs) with perfect compliance. In RCTs with monotone compliance,
Abadie (2002) discussed derivation of the marginal distributions of potential outcomesfor compliers. In standard difference-in-differences (change-in-changes) models, Athey
and Imbens (2006) provided the marginal distributions for the treated, which was extended to synthetic control settings by Gunsilius (2023). In models with selection
on observables, Firpo et al. (2009) developed unconditional quantile regression to obtain the marginal distributions, which was generalized to models with instrumental
variables by Frölich and Melly (2013) and to models with multiple and continuous
treatments by Powell (2020). Our method applies to all of these settings.
Moreover, the marginal distributions can be replaced with conditional distributions conditional on covariates, so our framework applies to cases with conditionalrandomization, for which conditional quantile regression estimates the conditional
distributions of the potential outcomes (Koenker and Bassett, 1978; Chernozhukov
and Hansen, 2005; Abadie et al., 2003).

3

Subgroup Treatment Effects

There is literature that considers access to microfinance as a way to reduce poverty indeveloping countries. Tarozzi et al. (2015) used data from an RCT conducted in rural
Ethiopia from 2003 to 2006 to assess the impact of microfinance on various outcomes.
The brief outline of the experiment is as follows. Randomization was carried out
at the Peasant Association (PA) level, which is a local administrative unit. Out of
133 PAs, 34 PAs were randomly assigned to microcredit and 33 PAs to the control.
The remaining 66 PAs were assigned to a different combination of treatments, which4we will not use. The compliance rate was 78%, and our analysis below focuses on theintent-to-treat (ITT) effects as did Tarozzi et al. (2015). We leave further details of
the experiment to Tarozzi et al. (2015).
Tarozzi et al. (2015) noted that “most loans were initiated to fund crop cultivationor animal husbandry, with 80 percent of the 1,388 loans used for working capital orinvestment in these sectors . . .” In light of this, we look at two outcome variables:(1) the net cash revenues from crops and (2) the total value of livestock owned.

3.1

Who Benefits from Microfinance?

For both of these outcomes, we are interested in knowing how much impact microfinance have had on those who were most in need, i.e., those who would have attained
low outcome in the absence of the treatment (Abadie et al., 2018). For example, we
may want to see whether a small amount of money helps increase the output of those
who are poor, which is approximated by low Y0 .
Precisely, we are interested in the ATE conditional on U < b for some level b, thatis,E[Y1 − Y0 | U < b].
Note here that when Y0 is continuously distributed, conditioning on U < b is equivalent to conditioning on Y0 < Q0 (b). We call this quantity the subgroup treatment
effect (STE). The STE is not point-identified since it depends on the joint distributionof Y0 and Y1 , but using their quantile functions, we can infer (Theorem 3)
1
b

Z b

1
[Q1 (u) − Q0 (u)]du ≤ E[Y1 − Y0 | U < b] ≤
b
0

Z b
0

[Q1 (1 − u) − Q0 (u)]du.

(1)

While QTEs in general cannot be interpreted as individual treatment effects without
the rank invariance assumption, (1) states that the integral of the QTEs can beinterpreted as the lower bound for the STE without any assumption. The lower boundis exact when rank invariance holds, but it is also expected to be close when rankinvariance approximately holds. This idea that individuals do not change ranks too
drastically by the treatment would be a reasonable presumption in many applications.
(Meanwhile the upper bound may not be very informative in many cases.) A possible
takeaway from (1) is that, if the lower bound is greater than zero, we know that thepoor individuals receive some positive treatment effects even though we do not know

5

Q1
Q0

2

Value of livestock
owned [k 2006 Birr]

Net cash revenues
from crops [k Birr]

3
Ȳ1
Ȳ0

1
0
−1

0

0.5
Rank

1

(a) Quantiles for the net cash revenues fromcrops. The KS p-value = 0.014.

8
Q1
Q0

6

Ȳ1
Ȳ0

4
2
0

0

0.5
Rank

1

(b) Quantiles for the total value of livestock
owned. The KS p-value = 0.0002.

Figure 1: Empirical quantile functions for the two outcomes in the treated and control
groups in Tarozzi et al. (2015). The difference of the green dashed line and the black
dashed line is the estimated ATE, which is insignificant in either case. However, theKS test indicates non-null effects in both outcomes.
exactly how much.
Now we turn to estimates from the data. We first note that the ATE estimatesfor both outcomes are not significant. Tarozzi et al. (2015) wrote “We documentthat despite substantial increases in borrowing in areas assigned to treatment thenull of no impact cannot be rejected for a large majority of outcomes.” However, thisdoes not mean that there is no evidence of heterogeneity in the data. In fact, theweighted Kolmogorov–Smirnov (KS) test of equal distributions (Monahan, 2011, p.
358) suggests that the marginal distributions of Y0 and Y1 are significantly different
from each other for both outcome variables (Figure 1).
Figure 2 shows the bounds on the STE given by (1). The black solid lines indicate
the theoretical lower bounds of E[Y1 − Y0 | U < b] across different values of b. The
black dashed lines are the upper bounds, but we do not scale the figures to cover thewhole of upper bounds since they are uninformatively large in most values of b. The
gray areas are the pointwise 95% confidence intervals for the STE at each point of b
constructed by the method of Imbens and Manski (2004). While we cannot exclude
the possibility that microfinance has no effect on the crop revenues across various
levels of crop revenues (Figure 2a), the effect on the value of livestock is significantly
positive for the mid-quantile levels of the livestock value (Figure 2b). The flat zeroregion on the left is likely those who do not own any livestock, so Figure 2b suggeststhat the effect of microcredit on those who has a modest to low level of livestock is

6

Value of livestock
owned [k 2006 Birr]

Net cash revenues
from crops [k Birr]

1
E[Y1 − Y0 | U < b]
0

−1

0

0.5
Rank b1(a) Bounds for the STEs on the net cash
revenues from crops.

1
E[Y1 − Y0 | U < b]
0

−1

0

0.5
Rank b1(b) Bounds for the STEs on the total valueof livestock owned.

Figure 2: Lower bounds for the STEs in Tarozzi et al. (2015). The black lines indicate
the estimated lower bounds for E[Y1 − Y0 | U < b] for each b. The gray areas are thepointwise 95% confidence intervals.
expected to be at least as big as a few hundred Birr.1

3.2

Who Should Be Treated?

These bounds can be used to determine who to assign treatment to for attaining higher
welfare. The literature on the welfare-maximizing assignment of the treatment was
initiated by Kitagawa and Tetenov (2018). They discussed a mechanism to decide
the treatment assignment based on covariates that maximizes the empirical welfare.
Meanwhile, policymakers may want to assign treatment to those who are mostin need, that is, those whose outcome are low without treatment. When the dataset
has a comonly observed baseline Y0 in the pre-treatment period, we may estimate thetreatment effect conditional on the predicted Y0 and consider optional assignment
based on it (Abadie et al., 2018). However, even when we only have a one-time crosssectional data, we can compute the bounds on the treatment effects conditional on
Y0 and discuss assignment mechanisms based on Y0 . In Tarozzi et al. (2015), there
were two waves of surveys, one for pre-treatment and the other for post-treatment,
but each wave surveyed different sets of households, so we cannot use the approach
by Abadie et al. (2018) to estimate the treatment effects conditional on predicted Y0 .
We apply our method to the post-treatment data to discuss optimal assignment oftreatment based on Y0 .
To illustrate the point, consider assigning microfinance to those whose Y0 is less
1

In January 2006, 100 Birr was worth about 11.4 USD.

7

Welfare by livestock
value [2006 Birr]

Welfare by crop
revenue [Birr]

400

200
Benefit00

0.5
Rank bCost

1(a) Welfare measured by the net cash revenues from crops in Birr. The welfare lower
bound is maximized at b = 0, i.e., by treating nobody.

400
Benefit
200
Cost
0

0

0.5
Rank b1(b) Welfare measured by the total value of
livestock owned in 2006 Birr. The dottedline (b = 0.96) maximizes the welfare lower
bound.

Figure 3: Welfare bounds when the treatment is assigned to individuals whose Y0
is below rank b. The solid blue lines give the lower bounds for the benefit, E[(Y1 −Y0 )1{U < b}], while the thin dashed blue lines give the upper bounds. The red lines
are the social cost, which is taken to be 100 Birr per treated individual, E[100 · 1{U <b}]. The blue areas are the pointwise 95% confidence intervals for the social benefit.
than some value for both outcomes. The social benefit is the aggregate treatment
effect, which equals E[(Y1 − Y0 )1{U < b}] in the per-capita basis. Since the averageamount of outstanding loans from the microfinance for the treated individuals was
299 Birr, suppose conservatively that about one third of the balance goes sour, that is,
the social cost is 100 Birr per capita. In sum, the social welfare from the assignment
mechanism U < b isE[(Y1 − Y0 )1{U < b}] − E[100 · 1{U < b}].
Figure 3 shows the graphs of the social benefits and costs for the two outcomes.
The solid blue lines are the lower bounds for the social benefit per capita, and thedashed blue lines are the upper bounds. The blue shaded areas indicate the pointwise
95% confidence intervals for the social benefits using Imbens and Manski (2004). For
the net cash revenues from crops, the lower bounds of the social benefits never exceed
the social costs, so it may not be justifiable to assign microfinance in order to maximize
the crop revenues (Figure 3a). On the other hand, the benefits measured by the valueof livestock exceed the costs in the middle range of b (Figure 3b). As the subgroupwelfare is not identified, we may go with the maximin approach: assign treatment
so as to maximize the worst-case welfare. The lower bound for the social welfare8Non-utilitarian welfare
from value of livestock

Non-utilitarian welfare
from crop revenues

400

200
Welfare00

0.5
Rank b1(a) Non-utilitarian welfare measured by thenet cash revenues from crops. The dotted line (b = 0.7081) maximizes the welfare
lower bound.

400

Welfare

200

0

0

0.5
Rank b1(b) Non-utilitarian welfare measured by the
total value of livestock owned. The dottedline (b = 0.928) maximizes the welfare lower
bound.

Figure 4: Non-utilitarian welfare bounds when the treatment is assigned to individuals
whose Y0 is below rank b. The welfare function weighs the loss 10% more than the gain.
The solid blue lines give the lower bounds for the welfare, E[h(Y1 − Y0 )1{U < b}],while the thin dashed blue lines give the upper bounds. The blue areas are thepointwise 95% confidence intervals for the social welfare.
(the difference between the benefits and the costs) is maximized at b = 0.96, whichcorresponds to providing microfinance to individuals with the value of livestock below
9,200 Birr.
Our framework also allows for non-utilitarian welfare. Let h be a welfare function
and suppose the policymaker wants to maximize the sum of h(Y1 − Y0 ). If h cannot
be written in the form of g(Y1 ) − g(Y0 ) for some g, h is called non-utilitarian welfare.
If h is nondecreasing and concave, Theorem 1 implies
1
b

Z b
0

h(Q1 (b − u) − Q0 (u))du ≤ E[h(Y1 − Y0 ) | U < b]Z
1 b≤h(Q1 (1 − b + u) − Q0 (u))du. (2)b 0As an example, consider a loss-averse non-utilitarian welfare functionxx ≥ 0,
h(x) =
1.1x x < 0,which overweighs the loss 10% more than the gain from the treatment. Such loss
aversion may arise naturally when the provider of the treatment can be held responsible for negative treatment effects, such as doctors (Bordley, 2009) or policymakers
9(Nicholson and Hellman, 2020).2 Figure 4 shows the range of social welfare in (2)when the treatment eligibility is capped by a variable threshold. The vertical dottedblack lines indicate the points where the lower bounds of the welfare are maximized.
Again, taking the maximin approach to partially-identified social welfare, the maximizer for the lower bound in Figure 4b is found to be b = 0.928, which is when theaccess to microfinance is granted to those with the baseline total value of livestock
owned below 7,299 Birr. The maximum of the lower bound in Figure 4a is attained
at b = 0.708, which corresponds to the baseline net cash revenues from crops below
520 Birr, and is statistically significant, but whether the welfare improvement in thiscase is economically significant is a judgment call.

4

Subgroup Proportions of Winners

In 1996, the welfare program for low-income women went over a reform in a waythat provided more generous benefits but with time limits on program participation.
Bitler et al. (2006) argued that labor supply theory predicted heterogeneous effectsof this reform on earnings, transfers, and income, and this called for more thanthe mean-impact analysis to evaluate the consequences of the reform. In particular,
Bitler et al. (2006) used QTEs on RCT data to demonstrate that the effects were
heterogeneous across different levels of outcome variables. In this section, we aim topush forward the boundary of quantitative knowledge we can infer from the data andprovide complementary evidence to their findings.
Figure 5 illustrates the stylized budget constraints faced by women supported by
Aid to Families with Dependent Children (AFDC) and by the Jobs First programin Connecticut. The horizontal axis is the time for leisure, which is a complementof working hours, and the vertical axis is the income. Assuming that leisure andconsumption are normal goods, each program participant enjoys the income andleisure pair on the budget constraint. Under AFDC, the budget constraint is givenby the two segments of black solid lines in Figure 5. The segment A corresponds
to receiving the benefits from AFDC. The Jobs First program replaces the AFDC
benefits with the green solid line, raising the budget constraint for individuals in A,
B, and C. The Jobs First program also comes with a time limit; after 21 months, the2On a related note, Heckman et al. (1997) questions utilitarian welfare in the context whereindividuals can choose the treatment.

10Income

IncomeJobs First

D
C
B

A

AFDC

Jobs First

AFDC

LeisureLeisure(a) Budget constraints before time limit.

(b) Budget constraints after time limit.

Figure 5: Income-leisure budget constraints under AFDC and under Jobs First (Bitler
et al., 2006, Figure 1). Food stamp benefits are not included in the figures.
budget constraint is pushed down to just the thin black solid and dotted line withno benefits.3 Before the time limit (Figure 5a), people in segments A, B, and C will
move to somewhere on the green line; people in segment D may or may not move toa point on the green line. After the time limit (Figure 5b), individuals receive zero
cash transfers and become worse off than AFDC (they may still receive food stamps).
As we will see below, this observation makes diverse predictions on how people
react. Our outcome variables are transfers, earnings, and income, and individuals in
different locations of the budget constraint may have responded in opposite directions.
Bitler et al. (2006) used QTEs to reveal this heterogeneity, while also facing limitations
that QTEs were not individual treatment effects. Our goal here is to provide more
specific, quantitative analysis on the proportion and location of winners and losers.
In this regard, Theorem 4 tells that the proportion of winners in an arbitrary interval
a < U < b is bounded by1
sup [x − a − F1 (Q0 (x))]+ ≤ P (Y0 < Y1 | a < U < b)b − a x∈(a,b)
1≤ P (Y0 ≤ Y1 | a < U < b) ≤ 1 −sup [b − x − 1 + F1 (Q0 (x)−)]+ . (3)b − a x∈(a,b)
The lower bound can be interpreted as the proportion of “identified” winners, andone minus the upper bound as the proportion of “identified” losers. The difference
between these bounds is the proportion of individuals for whom the data alone cannot
conclude the direction of the treatment effect without making further assumptions.
3

There was 6-month extension under some conditions.

11

1

6
4

Ȳ1
Ȳ0

1
Losers on left of b

Losers on right of a

Share

Q1
Q0

Share

Transfers Q4 [k$]8Winners on left of b

Winners on right of a

2
0

0

0.5
Rank

0

1

0

0.5
Rank b

0

1

0

0.5
Rank a

1

1
Q1
Q0

6
4

Ȳ1
Ȳ0

1
Losers on left of b
Winners on left of b

2
0

0

0.5
Rank

1

Share

8
Share

Transfers Q12 [k$](a) Quantile functions in Q4. (b) Winner bounds on the (c) Winner bounds on the
ATE = 230.1 (30.9).
left tail in Q4.
right tail in Q4.

0

0

0.5
Rank b1

Losers on right of aWinners on right of a
0

0

0.5
Rank a1(d) Quantile functions in (e) Winner bounds on the left (f) Winner bounds on theQ12. ATE = −133.5 (29.9). tail in Q12.
right tail in Q12.

Figure 6: Estimated bounds on the proportions of winners in terms of transfers in
Q4 and Q12. The proportion of winners is estimated to lie between the blue andred lines. The areas above the pointwise 95% confidence intervals (viz. significant
losers) are shaded in red; the areas below (viz. significant winners) shaded in blue.
Significant winners are found on the left tail in Q4 and significant losers on the righttail in Q12.

4.1

Heterogeneous Effects on Transfers

The first variable of interest is the transfers. In Figure 5, this corresponds to thedifference of the actual income and the corresponding income on the budget constraint
without benefits. The observed transfers also include the food stamps. Individuals
in segments A, B, and C as well as some of D will receive more transfers in Jobs
First before the time limit (Figure 5a), and individuals in A will receive less (maystill receive nonzero transfers from food stamps) after the time limit (Figure 5b).
Figure 6 shows the quantile functions and the bounds on the proportions of winners
for transfers in Quarter 4 (Q4) and Quarter 12 (Q12). Figures 6a and 6d give thequantile functions of Y1 and Y0 for transfers in Q4 and Q12. The mean effect in Q4 is230.1 with standard error 30.9, and the mean effect in Q12 is −133.5 with standard
error 29.9. Figures 6b and 6e plot the lower bounds for P (Y1 > Y0 | U < b) as the blue
12

lines and the upper bounds for P (Y1 ≥ Y0 | U < b) as the red lines. These bounds
are half-median-unbiased as developed in Chernozhukov et al. (2013). For example,
at b = 0.414, the lower bound gives 0.244, which means that among individuals who
receive transfers less than $1,000 under AFDC (that is, Y0 lower than the 41.4th
percentile), at least 24.4% of them receive transfers under Jobs First larger than theywould under AFDC in Q4. The blue shaded areas on the bottom and the red shaded
areas on the top indicate the region outside the pointwise 95% confindence intervals
for the proportions of winners constructed by the same method as Chernozhukov et al.
(2013). In other words, the blue shaded area indicates the “significant” proportionof winners and the red area the “significant” proportion of losers. Figures 6c and 6f
plot the same when the conditioning is replaced with the right tail, U > a. Amongindividuals who receive transfers greater than $1,350 (a = 0.701) under AFDC, 20.2%
or more of them receive less transfers under Jobs First (Figure 6f).

4.2

Heterogeneous Effects on Earnings

The second variable of interest is the earnings from work, which are a decreasing function of leisure. In Figure 5a, individuals in segment A might adjust the working hours
in either direction, and hence their earnings can go in either direction; individuals insegments B and C work less and receive less earnings; individuals in segment D might
as well work less for less earnings if they move to the green line, or otherwise do notchange working hours. After the time limit in Figure 5b, individuals in segment A
will increase the working hours if leisure is a normal good.
Figure 7 shows the corresponding quantile functions and bounds on the proportions of winners for earnings in Q4 and in Q12. The ATE of Jobs First on earnings
is not significant in either Q4 or Q12; in particular, it is estimated to be 86.0 withstandard error 53.9 in Q4 (Figure 7a) and 71.5 with standard error 75.5 in Q12 (Figure 7d). Figure 7b shows the proportions of winners and losers in the subgroups
defined by U < b. In the very left tail, there is neither significant proportion of winners or of losers. Unlike the theory predicts, however, as we include the median andabove, there appears a significant proportion of winners in earnings but not of losers.
For example, one possibility is that at low working hours, a small monetary support
from Jobs First can help individuals free their time from daily chores and they use
the free time to work more, that is, leisure may be an inferior good when much of

13

6
4

Ȳ1
Ȳ0

1
Losers on left of b

Losers on right of a

Share

Q1
Q0

Share

Earnings Q4 [k$]18Winners on left of b

Winners on right of a

2
0

0

0.5
Rank

0

1

0

0.5
Rank b

0

1

0

0.5
Rank a

1

1
Q1
Q0

6
4

Ȳ1
Ȳ0

1
Losers on left of b

Losers on right of a

Share

8
Share

Earnings Q12 [k$](a) Quantile functions in Q4. (b) Winner bounds on the (c) Winner bounds on the
ATE = 86.0 (53.9).
left tail in Q4.
right tail in Q4.

Winners on left of b

Winners on right of a

2
0

0

0.5
Rank

1

0

0

0.5
Rank b

1

0

0

0.5
Rank a1(d) Quantile functions in (e) Winner bounds on the left (f) Winner bounds on theQ12. ATE = 71.5 (75.5).
tail in Q12.
right tail in Q12.

Figure 7: Estimated bounds on the proportions of winners in earnings (proportional
to working hours) in Q4 and Q12. Significant winners are found on the left in Q4,
but no significant losers are found.
leisure time is spent for chores. Interestingly, this significant proportion of winners
does not seem to disappear after the time limit (Figure 7e). In the upper tail, there
is no significant proportion of winners or losers (Figures 7c and 7f).

5

Conclusion

In this paper, we proposed two new bounds related to the heterogeneity of the treatment effects that complement the commonly used QTE.
The first bounds were on the STE, which is the ATE in a subgroup whose Y0 resides
in an interval specified by the researcher. In Section 3, we went over an empiricalexample in which we are interested in assessing the effect of microfinance on poor
individuals, and found that individuals whose total value of livestock owned was not
too high received some positive ATE. We also applied our bounds to the problemof policy targeting in the context of social welfare. We considered two measuresof welfare: one was linear in the treatment effect with a per-capita fixed cost and14the other was non-utilitarian with a loss-aversive preference. We illustrated how ourbounds can suggest the treatment assignment mechanisms that maximize the lower
bounds of the social welfare.
The second bounds were on the proportions of winners and of losers in the same
subgroup. In Section 4, we investigated the heterogeneous effects of welfare reformon transfers and earnings predicted by labor supply theory. For transfers, our bounds
were able to confirm the presence of winners before the time limit and of losers after
the time limit as the stylized theory predicted. For earnings, our bounds detected asignificant proportion of winners in the mid- to left tail of the earnings distribution,
which was not entirely in line with the theoretical prediction.
In both examples, we saw cases where our bounds could detect significant STE or
proportion of winners even when the ATE estimates were not significant.

Appendix
A

Theoretical Results

In this section, we formally state and prove our bounds.
The following theorem yields the second-order stochastic dominance bounds on
the subgroup distribution of treatment effects. This extends the existing results on
the entire distribution of treatment effects (Fan and Park, 2010, Lemma 2.2).
Theorem 1 (Second-order stochastic dominance bounds on subgroup distribution oftreatment effects). Let 0 ≤ a < b ≤ 1. Conditional on a < U < b, we haveQ1 (b − U ) − Q0 (U ) ⪯2 Y1 − Y0 ⪯2 Q1 (1 − b + U ) − Q0 (U ),where A ⪯2 B means that the distribution of A is second-order stochastically dominated by that of B.
Proof. Let h : R → R be a nondecreasing concave function. Consider the optimaltransport from y ∈ (a, b) to x ∈ (0, b − a) with cost h(Q1 (x) − Q0 (y)). Then, Rachev
and Rüschendorf (1998, Theorem 3.1.2) imply
E[h(Q1 (b − V0 ) − Q0 (V0 ))] ≤ E[h(Q1 (V1 ) − Q0 (V0 ))]15where V0 ∼ U (a, b) and V1 ∼ U (0, b − a). Then, Beiglböck et al. (2009, Theorem
3) imply that there exist functions ϕ and ψ such that for x ∈ (0, b − a) and y ∈(a, b), ϕ(x) + ψ(y) ≤ h(Q1 (x) − Q0 (y)) with equality when x = b − y.4 Here, ϕ isnondecreasing since for every y ′ ≥ y,ϕ(b − y) + ψ(y) = h(Q1 (b − y) − Q0 (y)) ≥ h(Q1 (b − y ′ ) − Q0 (y)) ≥ ϕ(b − y ′ ) + ψ(y).
Thus, we see that for x ∈ (0, 1) and y ∈ (0, 1),ϕ(x∧(b−a))+ψ(y)1{y ∈ (a, b)}−ϕ(b−a)1{y ∈/ (a, b)} ≤ h(Q1 (x)−Q0 (y))1{y ∈ (a, b)}with equality when x = b − y and y ∈ (a, b) or x ≥ b − a and y ∈/ (a, b). Thus,
Beiglböck et al. (2009, Theorem 3) imply
E[h(Q1 (b − U ) − Q0 (U ))1{a < U < b}] ≤ E[h(Y1 − Y0 )1{a < U < b}],that is, Q1 (b − U ) − Q0 (U ) ⪯2 Y1 − Y0 conditional on a < U < b.
Similarly, Rachev and Rüschendorf (1998, Theorem 3.1.2) imply
E[h(Q1 (V1 ) − Q0 (V0 ))] ≤ E[h(Q1 (1 − b + V0 ) − Q0 (V0 ))]where V0 ∼ U (a, b) and V1 ∼ U (1 − b + a, 1). Beiglböck et al. (2009, Theorem 3)
imply that there exist functions ϕ and ψ such that for x ∈ (1 − b + a, 1) and y ∈ (a, b),h(Q1 (x) − Q0 (y)) ≤ ϕ(x) + ψ(y) with equality when x = 1 − b + y. Here, ϕ isnondecreasing since for every y ′ ≥ y,ϕ(1 − b + y) + ψ(y) = h(Q1 (1 − b + y) − Q0 (y))≤ h(Q1 (1 − b + y ′ ) − Q0 (y)) ≤ ϕ(1 − b + y ′ ) + ψ(y).

Thus, we see that for x ∈ (0, 1) and y ∈ (0, 1),h(Q1 (x) − Q0 (y))1{y ∈ (a, b)}≤ ϕ(x ∨ (1 − b + a)) + ψ(y)1{y ∈ (a, b)} − ϕ(1 − b + a)1{y ∈/ (a, b)}with equality when x = 1 − b + y and y ∈ (a, b) or x ≤ 1 − b + a and y ∈/ (a, b). Thus,
4

Beiglböck et al. (2009, Theorem 3) assume nonnegative costs, but the theorem trivially extends
to costs bounded from below. Then optimality follows for cost h(Q1 (·) − Q0 (·)) ∨ A for every A ∈ R,
so let A → −∞.

16

Beiglböck et al. (2009, Theorem 3) imply
E[h(Y1 − Y0 )1{a < U < b}] ≤ E[h(Q1 (1 − b + U ) − Q0 (U ))1{a < U < b}],that is, Y1 − Y0 ⪯2 Q1 (1 − b + U ) − Q0 (U ) conditional on a < U < b.

■

Theorem 1 is equivalent to the following lemma.
Lemma 2. Let f : R → R be a nondecreasing convex function and g : R → R anonincreasing convex function. For every 0 ≤ a < b ≤ 1,
Z b
Z b
f (Q1 (u−a)−Q0 (u))du ≤ E[f (Y1 −Y0 )1{a < U < b}] ≤ f (Q1 (1−u+a)−Q0 (u))du,
a
a
Z b
Z b
g(Q1 (b−u)−Q0 (u))du.
g(Q1 (1−b+u)−Q0 (u))du ≤ E[g(Y1 −Y0 )1{a < U < b}] ≤aaProof. Let f˜(x) = −f (−x), Ũ = 1 − U , and Ỹj = −Yj so the quantile function of Ỹj
is Q̃j (u) = −Qj (1 − u). Since f˜ is nondecreasing and concave, Theorem 1 implies
E[f˜(Q̃1 (1−a− Ũ )− Q̃0 (Ũ ))1{1−b < Ũ < 1−a}] ≤ E[f˜(Ỹ1 − Ỹ0 )1{1−b < Ũ < 1−a}]≤ E[f˜(Q̃1 (a + Ũ ) − Q̃0 (Ũ ))1{1 − b < Ũ < 1 − a}],which reduces to the first claim. Since −g is nondecreasing and concave, the second
claim follows straightforwardly from Theorem 1.
■
With this, we can prove the bounds on the STE.
Theorem 3 (Bounds on subgroup treatment effects). For every 0 ≤ a < b ≤ 1,
Z b
a

Z b

[Q1 (u − a) − Q0 (u)]du ≤ E[(Y1 − Y0 )1{a < U < b}] ≤[Q1 (u−a)−Q0 (u)]+ du ≤ E[(Y1 −Y0 )+ 1{a < U < b}] ≤

Z b
a
Z b

[Q1 (1 + a − u) − Q0 (u)]du,[Q1 (1+a−u)−Q0 (u)]+ du,
Z b
Z b[Q1 (1−b+u)−Q0 (u)]− du ≤ E[(Y1 −Y0 )− 1{a < U < b}] ≤[Q1 (b−u)−Q0 (u)]− du.
a

a

a

a

Proof. All results follow directly from Lemma 2.

■

Remark. We may use bootstrap to calculate the standard errors for the bounds in
Theorems 1 and 3 and Lemma 2 (Kaji, 2018). Then, we can construct confidence
intervals using Imbens and Manski (2004) or Stoye (2009).
17

The next theorem bounds the subgroup proportions of winners and losers.
Theorem 4 (Bounds on subgroup proportions of winners and losers). For every
0 ≤ a < b ≤ 1,
sup [x − a − F1 (Q0 (x))]+ ≤ P (a < U < b, Y0 < Y1 )x∈(a,b)≤ b − a − sup [b − x − 1 + F1 (Q0 (x))]+ ,x∈(a,b)sup [b − x − 1 + F1 (Q0 (x)−)]+ ≤ P (a < U < b, Y0 > Y1 )x∈(a,b)≤ b − a − sup [x − a − F1 (Q0 (x)−)]+ .
x∈(a,b)There exists a joint distribution of (Y0 , Y1 ) for which a lower bound is attained; there
exists a joint distribution of (Y0 , Y1 ) for which an upper bound is arbitrarily tight. Allbounds are nonincreasing and Lipschitz in a and nondecreasing and Lipschitz in b.
Proof. For the lower bound, observe that
P (a < U < b, Q0 (U ) < Y1 ) ≥ sup P (a < U < x, Q0 (x) < Y1 )x∈(a,b)≥ sup [P (a < U < x) + P (Q0 (x) < Y1 ) − 1]+ = sup [x − a − F1 (Q0 (x))]+ ,x∈(a,b)x∈(a,b)where the second inequality uses the lower Fréchet inequality.5 To show tightness, let∆ = supx∈(a,b) [x − F1 (Q0 (x))]+ and consider Y1 = Q1 (U1 ) such that U1 = U − a − ∆if a + ∆ < U < b. The joint distribution of (U1 , U ) when U ∈/ (a + ∆, b) is arbitrary.
Then U = u ∈ (a + ∆, b) are not winners sinceQ1 (u − a − ∆) − Q0 (u) ≤ Q1 (u − a − ∆) − Q1 (F1 (Q0 (u))) ≤ 0,
where the last inequality follows from the monotonicity of Q1 and(u − a − ∆) − F1 (Q0 (u)) = [u − a − F1 (Q0 (u))] − ∆ ≤ 0.
This means that the lower bound is binding for this joint distribution.
5

We thank Ismael Mourifié for suggesting the use of the Fréchet inequality.

18

To derive the upper bound, observe that
P (a < U < b, Q0 (U ) < Y1 ) = P (a < U < b) − P (a < U < b, Y1 ≤ Q0 (U ))≤ b − a − sup [b − x − 1 + F1 (Q0 (x))]+ ,x∈(a,b)where the inequality follows as the lower bound. For tightness, let ∆ = b − a −supx∈(a,b) [b − x − 1 + F1 (Q0 (x))]+ and, for arbitrary ε > 0, consider Y1 = Q1 (U1 ) suchthat U1 = U − a + 1 − ∆ + ε if a < U < a + ∆ − ε. The joint distribution of (U1 , U )for U ∈/ (a, a + ∆ − ε) is arbitrary. Then U = u ∈ (a, a + ∆ − ε) are winners since
F1 (Q1 (u − a + 1 − ∆ + ε)) − F1 (Q0 (u)) ≥ u − a + 1 − ∆ + ε − F1 (Q0 (u))= b − a − [b − u − 1 + F1 (Q0 (u))] − ∆ + ε ≥ ε > 0,
which implies Q1 (u − a + 1 − ∆ + ε) − Q0 (u) > 0. Since this holds for arbitrarily
small ε > 0, the upper bound can be arbitrarily tight.
The lower bound is nondecreasing and Lipschitz in b since for every ε > 0,
sup [x − a − F1 (Q0 (x))]+ ≤x∈(a,b)≤sup [x − a − F1 (Q0 (x))]+x∈(a,b+ε)

sup [x − a − F1 (Q0 (x ∧ b))]+ ≤ sup [x − a − F1 (Q0 (x))]+ + ε.

x∈(a,b+ε)x∈(a,b)The lower bound is nonincreasing in a since for every ε > 0,
sup [x−a+ε−F1 (Q0 (x))]+ ≥ sup [x−a−F1 (Q0 (x))]+ ≥ sup [x−a−F1 (Q0 (x))]+x∈(a−ε,b)x∈(a−ε,b)x∈(a,b)and is Lipschitz in a since for every ε > 0,
sup [x − a + ε − F1 (Q0 (x))]+ ≤x∈(a−ε,b)sup (x − a + ε)+ ∨ sup [x − a + ε − F1 (Q0 (x))]+x∈(a−ε,a]≤ε∨x∈(a,b)
sup [x − a − F1 (Q0 (x))]+ + ε

x∈(a,b)= sup [x − a − F1 (Q0 (x))]+ + ε.
x∈(a,b)The same properties follow analogously for the upper bound.
The loser bounds can be likewise derived.

■

Remark. Theorem 4 can be trivially extended to the bounds on P (a < U < b, Y1 −Y0 <c) by shifting the distribution of either Y0 or Y1 . Then, setting a = 0 and b = 1 makes

19them reduce to the classical Makarov bounds (Makarov, 1981).
Remark. Theorem 4 is in the form of intersection bounds for which Chernozhukov
et al. (2013) provided a method for estimation and inference.

References
Abadie, A. (2002): “Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models,” Journal of the American Statistical Association, 97,
284–292.
Abadie, A., J. Angrist, and G. Imbens (2003): “Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings,”
Econometrica, 70, 91–117.
Abadie, A., M. M. Chingos, and M. R. West (2018): “Endogenous Stratification in Randomized Experiments,” Review of Economics and Statistics, 100,567–580.
Aron-Dine, A., L. Einav, and A. Finkelstein (2013): “The RAND Health
Insurance Experiment, Three Decades Later,” Journal of Economic Perspectives,27, 197–222.
Athey, S. and G. W. Imbens (2006): “Identification and Inference in Nonlinear
Difference-In-Differences Models,” Econometrica, 74, 431–497.
Beiglböck, M., M. Goldstern, G. Maresch, and W. Schachermayer(2009): “Optimal and Better Transport Plans,” Journal of Functional Analysis,
256, 1907–1927.
Bitler, M. P., J. B. Gelbach, and H. W. Hoynes (2006): “What Mean
Impacts Miss: Distributional Effects of Welfare Reform Experiments,” American
Economic Review, 96, 988–1012.
Bordley, R. F. (2009): “The Hippocratic Oath, Effect Size, and Utility Theory,”
Medical Decision Making, 29, 377–379.
Chernozhukov, V. and C. Hansen (2005): “An IV Model of Quantile Treatment
Effects,” Econometrica, 73, 245–261.
20

Chernozhukov, V., S. Lee, and A. M. Rosen (2013): “Intersection Bounds:
Estimation and Inference,” Econometrica, 81, 667–737.
Fan, Y. and S. S. Park (2010): “Sharp Bounds on the Distribution of Treatment
Effects and Their Statistical Inference,” Econometric Theory, 26, 931–951.
Firpo, S., N. M. Fortin, and T. Lemieux (2009): “Unconditional Quantile
Regressions,” Econometrica, 77, 953–973.
Frölich, M. and B. Melly (2013): “Unconditional Quantile Treatment Effects
Under Endogeneity,” Journal of Business & Economic Statistics, 31, 346–357.
Gunsilius, F. F. (2023): “Distributional Synthetic Controls,” Econometrica, 91,
1105–1117.
Heckman, J. J., J. Smith, and N. Clements (1997): “Making The Most Out Of
Programme Evaluations and Social Experiments: Accounting For Heterogeneity in
Programme Impacts,” Review of Economic Studies, 64, 487–535.
Imbens, G. W. and J. D. Angrist (1994): “Identification and Estimation of Local
Average Treatment Effects,” Econometrica, 62, 467–475.
Imbens, G. W. and C. F. Manski (2004): “Confidence Intervals for PartiallyIdentified Parameters,” Econometrica, 72, 1845–1857.
Kaji, T. (2018): “Essays on Asymptotic Methods in Econometrics,” Ph.D. thesis,
Massachusetts Institute of Technology.
Kitagawa, T. and A. Tetenov (2018): “Who Should Be Treated? Empirical
Welfare Maximization Methods for Treatment Choice,” Econometrica, 86, 591–
616.
——— (2021): “Equality-Minded Treatment Choice,” Journal of Business & Economic Statistics, 39, 561–574.
Koenker, R. and G. Bassett, Jr (1978): “Regression Quantiles,” Econometrica,
46, 33–50.

21

Makarov, G. D. (1981): “Estimates for the Distribution Function of a Sum of
Two Random Variables When the Marginal Distributions are Fixed,” Theory ofProbability & Its Applications, 26, 803–806.
Monahan, J. F. (2011): Numerical Methods of Statistics, Cambridge: Cambridge
University Press, second ed.
Nicholson, K. M. and D. Hellman (2020): “Opioid Prescribing and the Ethical
Duty to Do No Harm,” American Journal of Law & Medicine, 46, 297–310.
Powell, D. (2020): “Quantile Treatment Effects in the Presence of Covariates,”
Review of Economics and Statistics, 102, 994–1005.
Rachev, S. T. and L. Rüschendorf (1998): Mass Transportation Problems, vol.
I: Theory, New York: Springer-Verlag.
Stoye, J. (2009): “More on Confidence Intervals for Partially Identified Parameters,” Econometrica, 77, 1299–1315.
Tarozzi, A., J. Desai, and K. Johnson (2015): “The Impacts of Microcredit:
Evidence from Ethiopia,” American Economic Journal: Applied Economics, 7, 54–
89.
Tetenov, A. (2012): “Identification of Positive Treatment Effects in Randomized
Experiments with Non-Compliance,” Working paper.

22