On Efficient Estimation of Distributional Treatment Effects under Covariate-Adaptive Randomization
Authors: Undral Byambadalai
Tomu Hirata
Tatsushi Oka
Shota Yasui
Venue: ICML 2025 poster (2025)
Keywords: treatment effects
field experiments
covariate-adaptive randomization
regression adjustment
causal inference
semiparametric estimation
Primary area: general_machine_learning->causality
TL;DR: Use ML to fold extra covariates into balanced trials and reveal the full outcome distribution with optimal precision—beyond averages.

Abstract: This paper focuses on the estimation of distributional treatment effects in randomized experiments that use covariate-adaptive randomization (CAR). These include designs such as Efron's biased-coin design and stratified block randomization, where participants are first grouped into strata based on baseline covariates and assigned treatments within each stratum to ensure balance across groups. In practice, datasets often contain additional covariates beyond the strata indicators. We propose a flexible distribution regression framework that leverages off-the-shelf machine learning methods to incorporate these additional covariates, enhancing the precision of distributional treatment effect estimates. We establish the asymptotic distribution of the proposed estimator and introduce a valid inference procedure. Furthermore, we derive the semiparametric efficiency bound for distributional treatment effects under CAR and demonstrate that our regression-adjusted estimator attains this bound. Simulation studies and analyses of real-world datasets highlight the practical advantages of our method.

On Efficient Estimation of Distributional Treatment Effects under Covariate-Adaptive Randomization
Undral Byambadalai Tomu Hirata
Tatsushi Oka Shota Yasui
Abstract
technology industry (Tang al., 2010; Bakshy et al., 2014; Kohavi et al., 2020).
This paper focuses the estimation of distri- butional treatment effects in randomized experi- ments that use covariate-adaptive randomization (CAR). These include designs such as Efron's s biased-coin design and stratified block random- ization, where participants are first grouped into strata based on baseline covariates and assigned treatments within each stratum ensure balance across groups. In practice, datasets often con- tain additional covariates beyond the strata indica- tors. We propose a flexible distribution regression framework that leverages off-the-shelf machine learning methods to incorporate these additional covariates, enhancing the precision of distribu- tional treatment effect estimates. We establish the asymptotic distribution of the proposed esti- mator and introduce valid inference procedure. Furthermore, we derive the semiparametric effi- ciency bound for distributional treatment effects under CAR and demonstrate that our regression- adjusted estimator attains this bound. Simulation studies and empirical analyses microcredit pro- grams highlight the practical advantages of our method.
In this paper, we examine the estimation of distributional treatment effects in randomized experiments that employ covariate- -daatiie randomization (CAR). Under CAR, in- dividuals are first partitioned into strata based on similar covariates, and treatments then assigned within each stratum to ensure balance across groups. The CAR frame- work encompasses various randomization schemes, from stratified block randomization to Efron's 's biased coin design (Imbens & Rubin, 2015), with simple random sampling as a special case.
We introduce a regression adjustment method for estimating distributional treatment effects under CAR when auxiliary data beyond stratum indicators are available. Incorporating this data enhances estimation precision. While experimental analyses often focus on the average treatment effect (ATE), relying solely on the ATE can overlook important insights. Examining distributional treatment effects provides more comprehensive understanding of the treatment impact by capturing changes in the entire outcome distribution.
Our approach employs distribution regression framework, leveraging the Neyman-orthogonal moment condition (Cher- nozhukov et al., 2018) to ensure first-order insensitivity to nuisance parameter estimation. These nuisance parame- ters- conditonnai outcome distributions given pre-treatment covariates are estimated using machine learning tech- niques such random forests, neural networks, and gradi- ent boosting, enabling flexibility in handling complex and high- soth-dimensional data. Incorporating cross- cross-fitting further strengthens robustness against estimation errors.
1. Introduction
Randomized experiments have been fundamental in uncover- ing the impact of interventions and shaping policy decisions since the seminal work by Fisher (1935). The of ran- domized experiments to estimate causal effects has been extensively adopted across diverse scientific fields (Rubin, 1974; Heckman et al., 1997; Imai, 2005; Imbens & Rubin, 2015) and has also become a widely accepted practice in the
Randomization schemes under CAR are extensively used across various disciplines. In clinical trials, stratified block randomization ensures balanced treatment allocation across key covariates such age, gender, and disease severity (Rosenberger & Lachin, 2015). In social experiments, re- searchers often stratify by geographical regions or socioeco- nomic characteristics (Duflo et al., 2007; Bruhn & McKen- zie, 2009), while the technology sector, covariate-adaptive designs enhance precision in A/B testing (Xie & Aurisset, 2016).
'CyberAgent, Inc., Tokyo, Japan Databricks Japan, Inc., Tokyo, Japan 'Department of Economies, Keio University, Tokyo, Japan. Correspondence to: Undral Byambadalai <un- www.yambbaaaiieebbenggn..ooo @ey.brageettt.o.p........
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
Our paper makes several key contributions. First, we extend the applicability of regression adjustment under CAR to estimate distributional treatment effects. While regression adjustment is widely used to reduce variance in ATE esti- mation under simple random sampling (Freedman, 2008a; Lin, 2013) and has recently been studied in the context of CAR (Rafi, 2023; Cytrynbaum, 2024; Wang et al., 2023), our work advances this framework to accommodate distri- butional treatment effects. Unlike Jiang et al. (2023), who focus regression adjustment for quantile treatment effects and assume continuous outcomes, our method is applicable to both discrete and mixed discrete-continuous outcomes.
ters depend on the target parameter itself, as seen in cases like the quantile treatment effect (QTH) and local QTE. In contrast, our estimation of the distributional treatment effect involves nuisance parameters that correspond to conditional means, which can be effectively estimated using machine learning algorithms.
Conditional Average Treatment Effects An alternative method for examining heterogeneity in treatment effects is to condition on observed variables and estimate the Condi- tional Average Treatment Effect (CATE) (Imai & Ratkovic, 2013; Athey & Imbens, 2016; Johansson et al., 2016; Shalit et al., 2017: Alaa & Van Der Schaar, 2017; Wager& & Athey,
Second, we establish the limit distribution of our estima- tor within the asymptotic framework for CAR, extending beyond the standard i.i.d. treatment assignment structure in causal inference. Third, we derive the semiparametric efficiency bound for distributional treatment effects under CAR and demonstrate that our estimator attains this bound. Finally, through simulations and an empirical analysis of microcredit programs, we illustrate the effectiveness of our approach in practical settings.
2018: Chernozhukov et al., 2018; Künzel et al., 2019; Shi
et al., 2019; Nie & Wager, 2021; Guo et al., 2023; Sverdrup & Cui, 2023; Van Der Laan et al., 2023). CATE quantifies the ATE within subgroups defined by observed attributes, such as gender, age, or prior platform use, thereby capturing the heterogeneity that can be explained by observable infor- mation. In contrast, our approach is designed to measure unobserved heterogeneity and can be extended to estimate distributional parameters conditional on observed data.
The rest of the paper is organized as follows. Section 2 reviews the literature, and Section 3 provides the setup. Section 4 introduces distributional treatment effect parame- ters, their identification, and estimation. Section 5 presents asymptotic results. Section 6 discusses findings from simu- lated and real data. Section 7 concludes. Appendix contains notations, proofs, and additional experimental results.
Regression adjustment under covariate-adaptive ran- domization The literature on utilizing pre- rre-treatment co- variates to reduce variance in estimating the ATE under simple random sampling is extensive, beginning with Fisher (1932) and followed by contributions from Cochran (1977); Yang & Tsiatis (2001); Rosenbaum (2002); Freedman (2008a;b); Tsiatis et al. (2008); Rosenblum & Van Der Laan (2010); Lin (2013); Berk et al. (2013); Ding et al. (2019), among others.
2. Related Literature
Distributional Treatment Effects Distributional and quantile treatment effects have long been recognized as im- portant parameters to estimate beyond the mean effects. The quantile treatment effect was first introduced by Doksum (1974) and Lehmann & D' Abrera (1975). Subsequently, estimation and inference methods for distributional and quantile treatment effects have been developed and applied in econometrics, statistics and machine learning commu- nity, including Heckman et al. (1997); Imbens & Rubin (1997); Abadie (2002); Abadie et al. (2002); Chernozhukov & Hansen (2005); Koenker (2005); Bitler et al. (2006); Athey & Imbens (2006); Firpo (2007); Chernozhukov et al. (2013); Koenker et al. (2017); Callaway et al. (2018); Call- away& & Li (2019); Chernozhukov et al. (2019); Ge et al. (2020); Zhou et al. (2022); Park et al. (2021); Kallus & Oprescu (2023); Oka & Yamada (2023); Näf & Susmann (2024); Xu et al. (2025), among others. Most of this work explore the conditional distributional and quantile treatment effects. Oka al. (2024) and Byambadalai et al. (2024) consider the estimation of unconditional distributional treat- ment effects but under simple random sampling. Kallus et al. (2024) address problems in which nuisance parame-
In the context of covariate- -adattiv randomization, some re- cent studies have explored regression adjustment for estimat- ing the ATE. Recent work by Cytrynbaum (2024) derives an asymptotically optimal linear covariate adjustment tailored to a given stratification. Similarly, Rafi (2023) investigates regression adjustment and establishes the semiparametric efficiency bound for estimating the ATE under covariate- adaptive randomization. Bai et al. (2024) examines covari- ate adjustment within a "matched pairs" design, where each stratum consists of two observations, with one randomly assigned to treatment. In biostatistics, Bannick al. (2023) and Tu al. (2023) analyze general approaches to covariate adjustment under covariate-adaptive randomization, while Wang et al. (2023) considers parameters defined by estimat- ing equations. Notably, most of these studies concentrate on ATE estimation, with the exception of Jiang et al. (2023), who investigates the estimation of quantile treatment effects in the same setting.
Semiparametric Estimation Our work builds on the semiparametric estimation literature, which addresses the challenge of estimating oww-dimensional parameters in the presence of high-dimensional nuisance parameters. This
2

---

Distributional Treatment Effeets under Covariate-Adaptive Randomization
Illustration: Comparison of Randomization Methods
Sample with Strata
Simple Random Sampling
Stratified Block Randomization
Stratum 1 30 subjects
Treatment
Control
Treatment
Control
10
20
15
15
Stratum 2
Treatment
Control
Treatment
Control
70 subjects
40
30
35
35
Figure 1. An illustration of treatment and control assignments under simple random sampling (SRS) and stratified block randomization (SBR). While both allocate 50 subjects per group, SRS may result imbalanced group compositions, whereas SBR preserves the proportional representation of strata in cach group, matching the overall sample distribution.
literature includes fundamental contributions from Robin- son (1988); Bickel et al. (1993); Newey (1994); Robins & Rotnitzky (1995), and more recent developments by Cher- nozhukov et al. (2018); Ichimura & Newey (2022). Our setup can be characterized as a semiparametric problem with syyyman-rrtoognaa moment conditions, as outlined in Neyman (1959); Chernozhukov al. (2022).
is over-represented in the treatment group under SRS. By contrast, SBR achieves balance within each stratum and maintains the same composition in treatment and control groups as in the full sample.
Let Y, E Y C denote the observed outcome of interest for the ith unit and W; E W == (l..... [W] denote the index of the treatment received by the ith unit for each unit [n] == {l....,,7
........73) n), where denotes the sample size. Also, let S; S {1.....S) denote the stratum variable and X; E C Rd denote the extra covariates besides S, We allow X; and S; to be dependent. To rule out empty stratum, we assume the probability of individuals being assigned to each stratum is positive, i.e.. p(s) := P(S, = s) 0 for every S. We adapt the potential outcome framework (Rubin, 1974; Imbens & Rubin, 2015), and let Y, (w) denote the potential outcome under treatment E W for the ith unit. The observed outcome and potential outcomes are related to treatment assignment by the relationship Y; (WW)).
3. Setup and Notation
We consider randomized controlled trials (RCT) with covariate-adaptive randomization when there are multiple treatment arms. In covariate-ddaptive randomization, indi- viduals are first grouped into strata based on similar values of their baseline covariates. Within each stratum, treatments assigned to achieve "balance" across groups, typically using complete randomization within each stratum. This approach ensures that the treatment allocation accounts for covariate distributions within strata, thereby improving com- parability between treatment groups
In order describe the treatment assignment mechanism, let The = P(W,, w w| s) E (0, be the target assignment probability for treatment w in stratum which may vary across strata. We define sub-sample as n2,, Σ i= L1W,; = w, S; s} and n(() := Σi= 11(S; = s}. where I1(.} denotes the indicator function. The empirical analogues of p(s) and (s) are given by p(+) = n*))
Figure provides a simple illustration. Consider an exper- iment involving 100 subjects, where 50% assigned to treatment group and 50% to control group. Suppose the subjects are divided into two strata based on baseline covariates (e.g., Stratum I represents younger individuals, and Stratum represents older individuals). Stratum com- prises 30 subjects, while Stratum comprises 70. Under simple random sampling (SRS), subjects are randomly as- signed treatment or control groups without regard to strata. In contrast, stratified block randomization (SBR) in- dependently assigns subjects within each stratum, ensuring proportional representation. While both methods allocate exactly 50 subjects to each group, SRS does not guaran- tee that the composition of treatment groups reflects the overall sample distribution. For example, Figure depicts one possible outcome, where Stratum 2 (older individuals)
and n. (s) == Th (s)/n(s). respectively. line with Bugni et al. (2019), we impose the following assumptions on the treatment assignment mechanism.
Assumption 3.1. We have
(i (Y,(1),.... ((()),,..,,,(()))) S,, X. are i.idd.
(ii) (Y((1).... (((W).
(iii) T. (w) (1( for every (w, 8) WW S.
3

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
Assumption 3.1 (i) allows for cross-sectional dependence among treatment statuses ( (W:) thereby accomodating many covariate-adaptive randomization schemes. Assump- tion 3.1 (ii) states that the assignment is independent of potential outcomes and pre-treatment covariates conditional on strata. Assumption 3.1 (iii) states the assignment prob- abilities converge to the target assignment probabilities as sample size increases.
these outcomes, (w) = s). can be identified. This is because, under Assumption 3.1, Fy s) coincides with (- function of the observed outcome given treatment w within stratum 8. By the law of total probability, the distribution function (w (v) is then expressed as, for any E V.
w.S = s), the conditional distribution
8
Fr(w)()) = (o)
(1)
Common randomization schemes satisfying Assumption 3.1 include simple random sampling, stratified block ran- domization, biased-coin design (Efron, 19 and adaptive biased-coin design (Wei, 1978). In Section 6.2, we rean- alyze a field experiment on microcredit programs, where stratification occurs at the provincial level, and treatments are randomly assigned within each province based target probabilities.
Hence, (r) (y) is identifiable under Assumption 3.1.
4.2. Empirical estimator
Under the RCT setting, the distribution function can be calculated without conditioning on pre-treatment covariates, unlike observational studies. Under CAR, we define an empirical estimator of the distribution function (y) for E y and treatment w E W as:
4. Distributional Treatment Effects
The parameters of interest are based on the distribution functions of potential outcomes, denoted by
H.W, = w/ D1,, S y}
FY()) (v)
(2)
To (S.)
(w( (v) = P(Y S s)
This estimator aggregates the empirical distribution func- tions across strata, and takes the form of inverse- propensity weighting (IPW) estimator Rosenbaum & Rubin (1984), The empirical estimator for the DTE is then formed as
for y E y and E W.
First, we define the distributional between treatments w, w' E W as
treatment effect (DTE)
A DTE
(v) (w) (y(
A DTE Demp (y) == ((w) (1) Y (v). W.w!
for y E V. The DTE measures the difference between the distribution functions of the potential outcomes.
While the estimator is unbiased and consistent, its efficiency can be enhanced by leveraging pre- treatmemeett covariates.
We also define the probability treatment effect (PTE) be- tween treatments w, w' E W as
4.3. Regression-adjusted estimator
APTE (yz) == (Uj))(Uj) - (y;w)
To incorporate pre-treatment covariates X, we adopt the distribution regression framework, treating the conditional distribution function Pau (y, a.) == (w) = 8, X x) as the mean regression for a binary outcome 11 (w) <
for each j == L.....J, J. where, given a set of points y, == {y1," ... ,y6} C y with yo = -o, the probability mass function [Y(w) (-) is defined as
v}. Specifically, for each y E y and w E W, we can write
(r())(u))= : = P(ys-1 < Y(w) S y1) = (w)) (3)
Mie (y, S, X) E (w) S v]]
Each probability fr( (3/)), which refer to the bin probability, can be obtained as difference between the values of the distribution function at yj and 1/-1- The PTE quantifies the difference in probabilities across bins, effec- tively capturing differences in "histograms" of the potential outcomes.
The conditional mean function can be estimated at each location y E y using supervised learning algorithms, such as LASSO, random forests, boosted trees, or deep neural networks. Let (-) be an estimator for Mw (-).
The regression-adjuster estimator of Fr (3) for each w W and y then defined
4.1. Identification
1 n
Although the potential outcomes (Y(w): : E W} are un- observed variables, the conditional distribution functions of
foods (2) Y
V V,()) 11
(3)

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
where
uniformly bounded functions mapping arbitrary index set y to the real line.
v,(y) = 1W =] (1(2; S y) (y X.))
(S:)
Assumption 5.1. (i) For w E W, Ow, To (V, X,) Mw 88, we have
+ (y St, X:).
(s) (, Σier (0) bw s, Xi)
The regression-adjusted estimator for DTE can then be formed as
sup VEVi
(w)
(w
0p((n 1/2)/ (1/2))
A DTE, mn (y) T (1)
(w) Tradj (( (1))
(4)
where (s) = ii [m] W; W. s}.
The estimator takes the form of the well-known augmented inverse-propee propensity weighting (AIPW) estimator, relying on doubly-robust moment condition (Robins et al., 1994; Robins & Rotnitzky, 1995). Specifically, the moment con- dition exhibits the Neyman orthogonality property (Cher- nozhukov et al., 2018; 8; 2022), making our estimator first- order insensitive to estimation errors in the nuisance pa- rameters (-). To further improve robustness, we apply cross-fitting as outlined in Chernozhukov et al. (2018). The estimation procedure outlined in Algorithm
(ii) For w E W, let Fu = {nw (v.
} with an
envelope (s, Then, max,es E[[ (S,, | = s] < co for q > and there exist fixed constants (a, u) > 0
such that
sup N Fol l2(1)) & a2 ( ve E (0, 1],
where (-)) denotes the covering number and the supremum is taken over all finitely discrete probability measures Q.
The empirical and adjusted estimators for the PTE can be defined in similar fashion. See Appendix B for the details.
Assumption 5.1 provides a high-level condition on the estimation of The (y, 8, X;)- Assumption 5ii imposes mild regularity conditions on (w 8, X,). Specifically, Assumption 5.1 holds automatically when y a finite set.
Algorithm ML regression-adjusted DTE estimator with cross-fitting
Input: Data {(Y, W,, X,, S split randomly into I, roughly equal-sized folds (I. 1); MM supervised learning algorithm for level y E y do for (treatment wEW, stratum sES, fold f=1 toT.) do Train M on data excluding fold é, using observa- tions in treatment group w within stratum Use M to obtain predictions (w, S,, for all observations in stratum for fold l.
We now establish the weak convergence of our proposed estimator in the following theorem, which serves as the theoretical foundation for statistical inference. To that end, we define the following terms. Letting pw (w s) = Elpmoy St, X.) = s), we define
((()) = m (y. St)) pww (y. Si), (,(y.w) w) I1(Yi(w) S n} Mho (W)
(s))
end for
Compute Fradj (y) according to Eq. (3).
1 ((((s) (Hho 8, 9x) (y s))
Compute A F,,dy (y) using Eq. (4) for w, w/ e W.
w,w!
-Morr 8, X,) )) paw (y. s))
end for Output: DTE estimator (A DDT (u)bey.
w.w"
Theorem 5.2 (Asymptotic Distribution). Suppose Assump- tion 3.1 and 5.1 hold. Then, for every w, w' W, in I)"(),, uniformly over y E Y, the regression-udjjused estimator defined in Algorithm I satisfies
5. Asymptotic Distribution of the DTE estimator
Vu(ADTE.at (v) -ADFF(U) ~ G(v).
In this section, we derive the asymptotic distribution of the proposed estimator, which enables statistical inference and the construction of confidence intervals. Additionally, we establish the semiparametric efficiency bound for the DTE and demonstrate that our estimator achieves this bound under the specified assumptions. We begin by introducing some additional notation to formalize our results. Let || lrr. denote the T.g P) norm, and (J) be the space of
where ((v) is Gaussian process with covariance kernel 2(y, 1/), which is given by
Myy v/) = a1(v. 3/ w) (2. w/) R2(y.3')
with 2 (V.V// = Bl (S.)). w).(1/ w)] and D2(y. (EE(()))
5

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
We next derive the semiparametric efficiency bound and show estimator achieves this bound in the following theorem. This implies that the asymptotic variance of any regular, consistent, and asymptotically normal estimator of DTE cannot be smaller than this variance.
where b(X)) is the scaled Friedman (1991) function given
- -0.5) X + a.. the treatment effect function is c(X,) 0.1(X. + log(1 + exp(Xx2))) and 0.1. The error term is u, /V 1). This data generating process introduces a complex, highly nonlinear relationship between covariates and the outcome, while including many covariates that do not affect the out- come. The heterogeneous treatment effects are correlated with specific covariates. This setup is modified version of the setups in Nie & Wager (2021) and Guo et al. (2021).
by b( (X²) sin( (t xx x +2(X,3
Theorem 5.3 (Semiparametric Efficiency Bound).
(a) Under Assumption 3.1, the semiparametric efficiency bound for A E (y) given E y is S2yy), which is defined by
(((y)= = (y,yy, + DI ) + +2(( u),
We draw a sample of size 5,000 from the data gata-generating process and estimate the DTE at quantiles (0.1...., ,0.9 using empirical and pnaressioonaajjsstee estimators. To ap- proximate the ground truth, a separate sample of size 10 is drawn, and the DTE is computed at the same locations. Linear and machine learning (ML) regression adjustments are implemented using linear regression and gradient boost- ing, with 2-fold cross-fitting. Linear regression is included as a baseline for comparison, as it has traditionally been widely used for regression adjustment in estimating average treatment effects.
where l1() and (2)(.) are defined in Theorem 5.2. (b) Under Assumptions 3.1 and 5.1, the regression- adjusted estimator A w0
add (y) attains the semipara-
metric efficiency bound.
As corollary to this theorem, it follows that the variance of the eggresionnadjjssee estimator with known nuisance functions is smaller than that of the empirical estimator, since the latter can be regarded as a special case in which the adjustment terms Pw (-) are set to zero.
Corollary 5.4 (Variance reduction). Under Assumption 3.1, for each y V
Figure 2 presents the results from 1,000 simulation runs with sample size of squared error (RMSE) and the length of the confidence in- tervals are smaller for the linear adjustment method, and even further reduced with MI. adjustment, compared to the empirical estimator. The confidence intervals are cal- culated using sample estimates of the asymptotic variance. The empirical estimator achieves a 95% confidence interval coverage close to the nominal level of 0.95. In contrast, regression-adjusted estimators show slight over-coverage, with coverage rates ranging from approximately 0.96 to 0.97 for linear adjustment and 0.97 to 0.99 for MI. adjustment. Additional results on RMSE reductions across quantiles for sample sizes n E {1,000, 5,000, 10,000} are provided in Appendix Section D.
1,000. Both the root mean
Var (A add (v)) < Var (A A
DDEE,mpp(y (y)),
where A DDT o,0d) wmww (y) is the regression regression-adjusted DTE estima- tor that incorporates known adjustment terms.
Theorem 5.3 and Corollary 5.4 indicate that, asymptotically, regression adjustment enhances the precision of the DTT estimates compared to the unadjusted empirical estimator. In the following section, we evaluate the finite sample per- formance of our estimators using both simulated and real datasets.
6. Experiments
6.2. Real Data: The Impacts of Microfinance
6.1. Simulation Study
We reexamine the randomized field experiment conducted by Attanasio et al. (2015) in 2008 to evaluate the impact of joint-liability microcredit program targeting women. The study took place in northern Mongolia, encompassing 40 villages across five provinces. Randomization occurred at the village level, with three treatment groups: jiabillia (group) lending, individual lending, and a control group. Stratification was carried out at the provincial level to en- sure balance (S, E (1..... 5)), as complete randomization could have led to some provinces containing only some particular treatment or control villages. This type of geo- graphic stratification is commonly employed in social sci- ences, medicine and other disciplines to facilitate robust comparisons between treatment groups.
In this section, examine the finite sample performance of the estimators through simulation study. The design consists of four strata (S = 4) constructed by partition- ing the support of Z; ~ U0,, 1) into S equal-Length in- tervals, where S, indicates the interval containing Z, For each unit i, we draw a 20-dimensional covariate vector from a multivariate normal distri- bution (00 T20x20). The treatment indicator W, follows Bernoulli distribution with probability 0.5 within each stratum, maintaining constant target proportion of treated units (W,- = 1) across strata with (s) = 0.5 for all s E S. We generate the outcome variable Y, as follows:
X, = [x= (X1,,,..,,220.
Y = b(X)) W( + 72, + ui,
6

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
RMSE
Average CI Length
Coverage Probability
0030
0.99
0.99
Value
0.02
0.10
0.020
0.09
0.90
0.055
0.94
0.1
02
0.3
0.5
0.7
0.8
0.9
0.1
02
0.3
0.5
0.7
0.8
0.9
0.1
02
03
0.6
0.
0.5
0.8
0.9
Quantile
Quantile
Quantile
Empirical
Linear adjustment
ML adjustment
Figure RMSE, average length and coverage probability 95% confidence intervals (CI) on simulated data 1,000). Linear adjustment uses linear regression, and machine learning (ML) adjustment uses gradient boosting, both with 2-fold cross-fitting, Number of simulations 1,000.
Empirical DTE
Adjusted DTE
0.2-
0.2
Probability
Probability
-02-
20
40
00
80
120
190
180
200
120
140
160
180
200
Enterprise revenue
Enterprise revenue
Empirical PTE
Adjusted PTE
Probability
0.0
Probability
0.0
40
80
120
160
180
200
20
60
100
140
180
Enterprise revenue
Enterprise revenue
Figure 3. The Impacts of Microfinance: Distributional Treatment Effeet (DTE) and Probability Treatment Effect (PTE) of joint liability lending on enterprise revenue (in thousand Mongolian Tugriks). The left panels depict empirical estimates, while the right panels present regression-adjustes estimates obtained using gradient boosting with 10-fold cross- fitting Shaded regions and emor bars represent 95% confidence intervals. Sample size: n = 611.

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
The experiment is one of many studies to evaluate the ef- fectiveness of microcredit as tool for alleviating global poverty. Since the launch of microcredit programs by the Grameen Bank in Bangladesh, which earned its founder Muhammad Yunus the Nobel Peace Prize in 2006, skepti- cism about their impact has grown in later years, fueled by the publication of findings on the subject. A primary goal of microcredit programs is to promote investment in and expansion of small-scale enterprises. However, Attanasio et al. (2015) found no significant average impact of these programs on enterprise revenue, profit, or other income sources.
Overall, the evidence points to joint-liability lending mainly helping individuals move out of the "zero-revenue" trap, with little discernible effect elsewhere in the distribution. Economically, this pattern implies that the program acts more as a hedge against downside risk than as a catalyst for broad productivity gains, so welfare improvements are likely to flow from reduced business failure and smoother con- sumption rather than from large increases in aggregate out- put. Because the confidence bands remain wide beyond that lowest bin bin-even after flexible covariate adjustment- larger samples or richer, more predictive covariates will be needed to sharpen estimates across the rest of the revenue range.
In this paper, we revisit their analysis to estimate the distribu- tional treatment effects of the lending program on enterprise revenue (Yt), aiming to uncover potential heterogeneity be- yond the average effects across the distribution. Our focus is on the comparison between joint-liability lending (W; 2) and the control group (W; = 1), as this was the central concern of the original study. Notably, group lending was pionecred by the Grameen Bank in Bangladesh during the 1970s.
7. Conclusion
We introduce a novel regression adjustment method de- signed to efficiently estimate distributional treatment effects under covariate-adaptive randomization. Our framework supports high-dimensional settings with many pre-treatment covariates and enables flexible modeling through the in- tegration of off-the-shelf machine learning techniques for regression adjusstiae adjustment.
Figure depicts the distributional and probability treatment effects of joint-liability lending on enterprise revenue. The outcome is measured in thousands of Mongolian Tugriks (MNT), with an exchange rate of 1 USD= = 1150 MNT at the time of the study. We compute the DTE and PTE for y E 10,10..... 200} accounting for the stratified design. For regression adjustment, we use gradient boosting with 10-fold cross-fitting, with pre- treatment covariates (X2) in- cluding enterprise revenue prior the experiment, house- hold size, education level, age, etc. The full list of covariates can be found in Table 3 in the Appendix.
Despite its strengths, our method has certain limitations. First, it assumes experimental data with perfect compliance and no interference. While this setup is appropriate for some applications, it may limit applicability in contexts where these assumptions do not hold. Second, the effectiveness of our approach relies on the availability of pre-treatment co- variates that are highly predictive of the outcome. Although we leverage flexible machine learning techniques to enhance prediction quality and achieve greater variance reduction compared to linear regression, the potential for variance reduction diminishes when covariates provide limited pre- dictive power. Third, in scenarios with a large number of strata, improving precision using pre-treatment covariates becomes increasingly challenging, particularly when the sample size is modest. These limitations point to several promising directions for future research, such as designing methods to handle imperfect compliance and network ef- fects, as well as enhancing estimation efficiency in settings with a large number of strata and locations. Additionally, extending recent advances in kernel mean embeddings to characterize outcome distributions (e-g.. Park et al. (2021); Naf & Susmann (2024)) to the CAR framework represents promising direction for future research.
The top-left panel of Figure 3 presents the empirical DTE, while the top-right panel shows the regression-adjusted DTE. The shaded areas represent the 95% pointwise confidence band computed using multiplier bootstrap (Giné & Zinn, 1984; Belloni et al., 2017) with 1000 repetitions. Although the sample size in the experiment is modest (n = 611), the regression adjustment reduces the standard errors by 1 1% to 13% across revenue levels, with an average reduc- tion of 7%. The bottom-left panel of Figure 3 presents the empirical PTE,, and the bottom-right panel depicts the regression-adjusted PTE on enterprise revenue. For the PTE, the effectiveness of regression adjustment is limited by the small sample size, and it does not consistently reduce standard errors compared to the empirical estimator.
A Acknowledgements
The analysis of DTE and PTE indicates that regression ad- justment reduces the standard error by approximately 10% when estimating the probability of revenue being zero, lead- ing statistically significant negative effect. Specifically, the probability of revenue being zero decreases by 10 per- centage points (pp), with a standard error of 4.6 pp.
We extend our gratitude to the four anonymous reviewers and the program chairs for their insightful comments and discussions, which significantly enhanced the quality of this paper. Additionally, Oka acknowledges the financial support from JSPS KAKENHI Grant Number 24K04821.
8

---

Distributional Treatment Effeets under Covariate-Adaptive Randomization
Impact Statement
Bickel, P. J., Klaassen, C. A., Ritov, Y., and Wellner, J. A. Efficcint and adaptive estimation for semiparametric mod- els, volume 4. Springer, 1993.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Bitler, M. P., Gelbach, J. B., and Hoynes, H. W. What mean impacts miss: Distributional effects of welfare reform experiments. American Economic Review, 96(4):988- 1012, 2006.
References
Abadie, A. Bootstrap tests for distributional treatment ef- fects in instrumental variable models. Journal of the American statistical Association, 97(457):284-292, 2002.
Bruhn, M. and McKenzie, D. In pursuit of balance: Ran- domization practice in development field experiments. American economic journal: applied economics, 1(4): 200-232, 2009.
Abadie, A., Angrist, J., and Imbens, G. Instrumental vari- ables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica, 70(1): 91-117, 2002.
Bugni, F. A., Canay, I. A., and Shaikh, A. M. Inference under covariate-adaptive randomization. Journal of the American Statistical Association, 113(524):174-1173) 2018.
Alaa, A. M. and Van Der Schaar, M. Bayesian inference of individualized treatment effects using multi-task gaussian processes. Advances in neural information processing systems, 30, 2017.
Bugni, F. A., Canay, I. A., and Shaikh, A. M. Inference un- der covariate-adaptive randomization with multiple treat- ments. Quantitative Economics, 10(4):1747-17855 2019.
Athey, S. and Imbens, G. Recursive partitioning for het- erogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27)7353-7366, 2016.
Byambadalai, U., Oka, T., and Yasui, Estimating dis- tributional treatment effects in randomized experiments: Machine learning for variance reduction. In Interna- tional Conference on Machine Learning, pp. 5082-5113. PMLR, 2024.
Athey, S. and Imbens, G. W. Identification and inference in nonlinear rica, 74(2):431-499 2006.
models. Economet-
Callaway, B. and Li, T. Quantile treatment effects in differ- ence in differences models with panel data. Quantitative Economics, 10(4):1579-1618. 2019.
Attanasio, O., Augsburg. B., De Haas, R., Fitzsimons, E., and Harmgart, H. The impacts of microfinance: Evidence from joint-liability lending in mongolia. American Eco- nomic Journal: Applied Economics, 7(1):90- 1-.90-12,,20 2015.
Callaway, B., Li, T., and Oka, T. Quantile treatment effects in difference differences models under dependence restrictions and with only two time periods. Journal of Econometrics, 206(2):395 2018
Bai, Y., Jiang, L., Romano, J. P., Shaikh, A. M., and Zhang, Y. Covariate adjustment in experiments with matched pairs. Journal of Econometrics, 241(():105740) 2024.
Chernozhukov, and Hansen, C. An iv model of quantile treatment effects. Econometrica, 73(1):245-261, 2005.
Bakshy, E., Eckles, D., and Bernstein, M. S. Designing and deploying online field experiments. In Proceedings of the 23rd international conference World wide web, pp- 283-292, 2014.
Chernozhukov, V., Fernández-Val, I,, and Melly, B. Infer- ence counterfactual distributions. Econometrica, 81 (6):2205-22) 2268, 2013.
Bannick, M. S., Shao, J., Liu, J., Du, Y., Yi, Y., and Ye, T. A general form of covariate adjustment in randomized clinical trials. arXiv preprint 1rXiv:2306.10215 2023.
Chernozhukov, V., Chetverikov, D., and Kato, K. Gaussian approximation of suprema of empirical processes. The Annals of Statistics, 42(4): 1564, 2014.
Belloni, A., Chernozhukov, V., Vernandez-Vall I., and Hansen, C. Program evaluation and causal inference with high-dimensional data. Econometrica, 85(1):233-298 2017.
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1-C68 01 2018.
Berk, R., Pitkin, E., Brown, L., Buja, A., George, E., and Zhao, L.. Covariance adjustments for the analysis of randomized field experiments. Evaluation review, 37 1304):170-116, 2013.
Chernozhukov, V., Fernandez- Val, I., Melly, B., and Wüthrich, K. Generic inference on quantile and quantile effect functions for discrete outcomes. Journal of the American Statistical Association, 2019.

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
Chernozhukov, V., Escanciano, J. C., Ichimura, H., Newey, W. K., and Robins, J. M. Locally robust semiparametric estimation. Econometrica, 90(4): 2022.
Guo, Y., Coey, D., Konutgan, M., Li, W., Schoener, C., and Goldman, M. Machine learning for variance reduction in online experiments. Advances in Neural Information Processing Systems, 34:8637-8649 2021.
Cochran, W. G. Sampling techniques. john wiley & sons, 1977.
Hahn, J. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica, pp. 3 315-331, 1998.
Cytrynbaum, M. Covariate adjustment in stratified experi- ments. Quantitative Economics, 15(4):971-998, 2024.
Heckman, J. Smith, J., and Clements, N. Making the most out of programme evaluations and social experiments: Accounting for heterogeneity in programme impacts. The Review of Economic Studies, 64(4):487-535, 1997.
Ding, P., Feller, A., and Miratrix, L. Decomposing treat- ment effect variation. Journal of the American Statistical Association, 1144525):304-37, 2019. 2019.
Ichimura, H. and Newey, W. K. The influence function of semiparametric estimators. Quantitative Economics, 13 (1):29 61, 2022.
Doksum, K. Empirical probability plots and statistical infer- ence nonlinear models in the two-sample case. The annals of statistics, pp. 267-277, 1974.
Imai, Do get-out-the-vote calls reduce turnout? the importance of statistical methods for field experiments. American Political Science Review, 99(2):233-300, 2005.
Duflo, E., Glennerster, R., and Kremer, M. Using randomiza- tion in development economics research: A toolkit. Hand- book of development economics, 4:3895-3962, 2007.
Imai, K. and Ratkovic, M. Estimating treatment effect heterogeneity in randomized program evaluation. Annals of Applied Statistics, 2013.
Efron, B. Forcing sequential experiment to be balanced. Biometrika, 58(3):403-417, 1971.
Firpo, S. Efficient semiparametric estimation of quantile treatment effects. Econometrica, 75(1):259-276, 2007.
Imbens, G. and Rubin,
Estimating outcome distri- butions for compliers in instrumental variables models. The Review of Economic Studies, 64(4):555-57) 574, 1997.
Fisher, R. A. Statistical methods for research workers. Oliver and Boyd, 1932.
Imbens, G. W. and Rubin, D. B. Causal inference in statis- tics, social, and biomedical sciences. Cambridge Univer- sity Press, 2015.
Fisher, R. A. The Design of Experiments. Oliver and Boyd, 1935.
Jiang, L., Phillips, P. C., Tao, Y., and Zhang, Y. Regression- adjusted estimation of quantile treatment effects under covariate-adaptive randomizations. Journal of Economet- rics, 234(2):758-776, 2023.
Freedman, D. A. On regression adjustments to experimental data. Advances in Applied Mathematics, 40(2):180-192 2008a.
Freedman, D. A. On regression adjustments experiments with several treatments. Annals of Applied Statistics, 2: 176 776-96,77 2008b.
Johansson, F., Shalit, U., and Sontag, D. Learning repre- sentations for counterfactual inference. In International conference on machine learning, pp. 3020-3029. PMLR, 2016.
Friedman, J. H. Multivariate adaptive regression splines. The annals of statistics, 19(1):1-67, 1991.
Kallus, N. and Oprescu, M. Robust and agnostic learning of conditional distributional treatment effects. Interna- tional Conference on Artificial Intelligence and Statistics, pp. 6037-6060. PMLR, 2023.
Ge, Q. Huang, X., Fang, S., Guo, S., Liu, Y., Lin, W. and Xiong, M. Conditional generative adversarial networks for individualized treatment effect estimation and treat- ment selection. Frontiers in genetics, 11:585804, 2020.
Kallus, N., Mao, X., and Uehara, M. Localized debiased machine learning: Efficient inference on quantile treat- ment effects and beyond. Journal of Machine Learning Research, 25(16):1-58 2024.
Giné, E. and Zinn, J. Some limit theorems for empirical processes. The Annals of Probability, pp. 929-989, 1984.
Koenker, R. Quantile regression, volume 38, Cambridge university press, 2005.
Guo, X., Zhang, Y., Wang. J., and Long, M. Estimating het- erogeneous treatment effects: mutual information bounds and learning algorithms. International Conference on Machine Learning, pp. 12108- 12121. PMLR, 2023.
Koenker, R., Chermozhukov, V., He, X., and Peng. L. Hand- book of quantile regression. CRC press, 2017.
10

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
Kohavi, R., Tang, D., and Xu, Y. Trustworthy online con- trolled experiments: A practical guide to a/b testing. Cam- bridge University Press, 2020.
Robinson, P. M. Root-n-consistent semiparametric regres- sion. Econometrica: Journal of the Econometric Society, pp. 931-954, 1988.
Künzel, S. R., Sekhon, J. S., Bickel, J., and Yu, B. Met- alearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences, 116(10)4156-4166 2019.
Rosenbaum, P. R. Covariance adjustment in randomized experiments and observational studies. Statistical Science, 17(3):286-327, 2002.
Rosenbaum, P. R. and Rubin, D. B. Reducing bias in obser- vational studies using subclassification on the propensity score. Journal of the American statistical Association, 79 5387):516-524, 1984.
Lehmann, E. L. and D' Abrera, H. J. Nonparametrics: sta- tistical methods based on ranks. Holden-day, 1975.
Lin, W. Agnostic notes on regression adjustments to ex- perimental data: Reexamining freedman's critique. The Annals of Applied Statistics, 7(1):295-318, 2013.
Rosenberger, W. F. and Lachin, J. M. Randomization in clinical trials: theory and practice. John Wiley & Sons, 2015.
Näf, and Susmann, H. Causal-drf: Conditional kemel treatment effect estimation using distributional random forest. arXiv preprint arXiv:2411.08777 2024.
Rosenblum, M. and Van Der Laan, M. J. Simple, efficient estimators of treatment effects in randomized trials using generalized linear models to leverage baseline variables. The international journal of biostatistics, 6(1), 2010.
Newey, W. K. The asymptotic variance of semiparametric estimators. Econometrica: Journal of the Econometric Society, pp. 1382, 1994.
Rubin, D. B. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Edu- cational Psychology, 66(5):688, 1974.
Neyman, J. Optimal asymptotic tests of composite hypothe- ses. Probability and statsitics, pp. 213-234, 1959.
Nie, X. and Wager, S. Quasi-oracle estimation of hetero- geneous treatment effects. Biometrika, 108(2):299-319 2021.
Shalit, U., Johansson, F. D., and Sontag, D. Estimating individual treatment effect: generalization bounds and algorithms. International conference on machine learn- ing. pp. 3076-3082 PMLR, 2017.
Oka, T. and Yamada, K. Heterogeneous impact of the min- imum wage: implications for changes in between-and within-group inequality. Journal of Human Resources, 58(1):335-365 2023.
Shi, C., Blei, D., and Veitch, V. Adapting neural networks for the estimation of treatment effects. Advances neural information processing systems, 32, 2019.
Oka, T., Yasui, S., Hayakawa, Y., and Byambadalai, U. Re- gression adjustment for estimating distributional treat- ment effects in randomized controlled trials. arXiv preprint arXiv:2407.1407. 2024.
Sverdrup, E. and Cui, Y. Proximal causal learning of con- ditional average treatment effects. In International Con- ference on Machine Learning, pp. 33285-33298. PMLR, 2023.
Park, J., Shalit, U., Schölkopf, B., and Muandet, K. Con- ditional distributional treatment effect with kernel con- ditional mean embeddings and u-statistic regression. In International Conference on Machine Learning, pp. 8401- 8412. PMLR. 2021.
Tang, D., Agarwal, A., O'Brien, D., and Meyer, M. Over- lapping experiment infrastructure: More, better, faster ex- perimentation. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 17-26, 2010
Rafi, Efficient semiparametric estimation of average treatment effects under covariate adaptive randomization. arXiv preprint 000500000000.000 2023.
Tsiatis, A. A., Davidian, M., Zhang, M., and Lu, X. Co- variate adjustment for two-sample treatment comparisons in randomized clinical trials: a principled yet flexible ap- proach. Statistics in medicine, (7(2))4465-444 46788 2008.
Robins, J. M. and Rotnitzky, A. Semiparametric efficiency in multivariate regression models with missing data. Jour- nal of the American Statistical Association, 90(429):122- 129, 1995.
Tu, F., Ma, W., and Liu, H. A unified framework for co- variate adjustment under stratified randomization. arXiv preprint arXiv:2312.01226. 2023.
Robins, J. M., Rotnitzky, A., and Zhao, L. P. Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89442)):846-88 866, 1994.
Van Der Laan, L., Ulloa-Pérez, E., Carone, M., and Luedtke, A. Causal isotonic calibration for heterogeneous treat- ment effects. In International Conference on Machine Learning, pp. 34831-34854 PMLR, 2023.

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
der Vaart, A. and Wellner, J. Weak Convergence and Empirical Processes: With Applications to Statistics.
Springer Science & Business Media, 1996.
Wager, and Athey, S. Estimation and inference of hetero- geneous treatment effects using random forests. Journal of the American Statisfical Association, 113(523):122) 1242, 2018.
Wang, B., Susukida, R., Mojtabai, R., Amin-Esmaeili, M., and Rosenblum, M. Model-robust inference for clinical trials that improve precision by stratified randomization and covariate adjustment. Journal of the American Statis- tical Association, 1 118(542):1 152-1163, 2023.
Wei, L.J.. The adaptive biased coin design for sequential experiments. The Annals Statistics, 601)922-10,, 1978.
Xie, H. and Aurisset, J. Improving the sensitivity of on- line controlled experiments: Case studies netflix. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 645 654, 2016.
Xu, R., Gao, J., Oka, T., and Whang, Y.J. Quantile random- coefficient regression with interactive fixed effects: Het- erogencous group-level policy evaluation. Econometric Reviews, 44(5):630-644 2025.
Yang, L. and Tsiatis, A. A. Efficiency study of estimators for treatment effect in a pretest pretest-posttest trial. The American Statistician, 55(4):3 14-321, 2001.
Zhou, T., Carson IV, W. E., and Carlson, D. Estimating potential outcome distributions with collaborating causal networks. Transactions on machine learning research, 2022, 2022.
12

---

Distributional Treatment Effeets under Covariate-Adaptive Randomization
Appendix
Appendix is organized as follows. Section A provides a table summarizing the notation. Section B elaborates the Probability Treatment Effect. Section C presents all proofs. Finally, Section D offers additional details on the simulation study and real-data analysis.
A. Summary of Notation
Table
Summary of Notation
X;
-r-ttteameet covariates
S:
stratum indicator
W;
treatment variable
Y
outcome variable
(w)
potential outcome for treatment group w
p(s)
proportion of stratum 8 treatment assignment probability for treatment group w in stratum sample size
(s)
(s)
number of observations in treatment group in stratum
n(s)
number of observations in stratum
p(s)
n(s)/n, /n, proportion of stratum 8 in the sample
(s)
nw (n)(((s),, estimated treatment assignment probability for treatment group w in stratum s [[ (w) S y}], potential outcome distribution function
(V(
[K]
{1...., K} for a positive integer
|all
Va a, Euclidean norm of a vector a (a1..... ap) E R
-
(P) norm
(()))
space of uniformly bounded functions mapping an arbitrary index set y the real line convergence in distribution law
equality in distribution
X, = (a()a)
lim -co lim,- P(| > Ka,) () for sequence of positive constants On
X, = op( (ax)
supk>o lim,- p([ > Ka,) = O for sequence of positive constants On for sequences Vn, and in R, Tn < Ay,, for a constant that does not depend on n max(k E Z & < b). greatest integer less than or equal to
In S yn
13

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
B. Probability Treatment Effect
Let Pwo (ylj, 5, x) == P(yj-1 < (w) S =, - S = xj and its estimator be denoted by Pe (Mg. 2.).. The empirical estimator for the bin probability for yj y is as follows:
s,
8 ((s)
(Uy) = (b(w)(b)) [r(w))
Σ
U| < Y, - m/}
nw
1=1
;:W;
=WSS,
=
My
< YY S
(B.1)
m (S:)
Also, similarly, the regression-adjusted estimator for the bin probability for yj E V is:
padj (u3)
I(W =w Tw (S:)
(11 < Y, S 5ッ) - Pur Si, X.)) + Par Pun St, Xi)
(B.2)
Using these, we can define empirical and regression-adjustel PTE estimators as follows:
A PTEp (y5) = (r(u)) (U3) (r( (Ui),
(B.3)
w.w
(yj) = (x(w)(U)) Pr(ior)
Fodj
[ (U1).
(B.4)
w,w
Then, the results in the paper also apply the PTE, the indicator functions (1(Y Uj} used in the analysis of DTE can be replaced with 1k < Y (w) yı for the analysis of the PTE.
C. Proofs for Section 5
C.1. Some definitions
We first introduce some definitions from empirical process theory that will be used in the proofs. See also van der Vaart Wellner (1996) and Chernozhukov et al. (2014) for more details.
Definition C.1 (Covering numbers). The covering number N(s, FF. || II) is the minimal number of balls {g: - fll < #} of radius needed to cover the set F. The centers of the balls need not belong to F, but they should have finite norms.
Definition C.2 (Envelope function). An envelope function a class F is any function x +> F(x) such that [f(z)) < F(x) for every x and f.
Definition C.3 (VC-type class). We say F is of VC-type with coefficients (a, v) and envelope F if the uniform covering numbers satisfy the following:
sup Nup N (ellFFla...
( Ve (0,
where the supremum is taken over all finitely discrete probability measures.
C.2. Asymptotic Distribution
Proof of Theorem 5.2. Recall the notation Pow (w, 8) El St, X.) = s). Additionally, denote 1) (v, = (1))( 5 5 phe (y, s) and (s) == Σ (II(W, w) Toe (s)) 11S; = ss. Note that we have f. Tun (5)
Dia(e) We start with the linear expansion of (w)) for treatment w and decompose it into two terms (3) and 12(2) as n(s)
14

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
follows:
Vn(Fy(m)) (v) Fr(w)(())
1
[[W -) (1 S 5)
St, + Pw (v. S xi) - -vnFF(w))() 5 (S,)
(()))()))))
vn
1
1(W, w)
1[Y: S y)
Tw (S:)
(1())
(IW w
Ae (S.))
io (y, Si Xi)
To (S:)
=12(0)
Then, for the first term, we have
(1)) = 1
1WW,
IW = w)
I1S: = 8) S 5n}
Vn 1= SES
(s)
1(W, = [[S = () -T)(s) 1[,, S y) (n)r((n)
Vn
Tw (s)
1=1 SES
H.W = w)
(5) 11 = sl S y}
sES
11W, = w 11S: = (s)
Σ Du(s) Dwl()-
Dun (s) s)
ne (V,
n(s) n (S)
n(8) ni (8)Tro())
Vntir (8)
SES
RES
.ES
1(W: = w) 11 =}]
(s)
M Mnl Si))
(U,
VT Mo (V, 8) +
(s) (s)
vn
SES
SES
1=1
n
1W w) [1s 8) (w)
(io(s) 8)
Dw (s
ni (y,
n(s) V ne (T)Tw( (w)(s)
Σ
Tz /
(S) (s)
Vntw (s)
1= SES
KES
SES
LWW; = w]f $}
n
(w S.)
Th (10, 8)
Vn + 11,1
(8)
SES
where
n
sw(()) n(s) (s) (T)
D (s)())) (s)
(,((y)
Σ
nh (y. -
-
n(s) / /u (3)
SES
SES
(mn())
Vn
(s)
(s)
#ES
11(W, = =w 11s = Dw((s)
(y.
nls) (w)
i= SES
Note that the class [1[(((w)= < v) - uw nw S.) S:): y} is of the VC-type with fixed coefficients (a, v) and bounded envelope, and ([] & y) - (w. S.) = s) = 0. Therefore,
sup SSS
= u = (1. = Op()).
By Assumption 3.1, for all w e W, we have max,es (w.(s)/((s) = (1),,, maxses (s) (())))((s))
(w)l = =p()), and
minses Tw (s) > e > 0, which imply supyey [,,] = (1).
15

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
Next, we analyze the second term Iz()).
1WW = w (s)
1
(2(1= =
8, =}
Me (U, X,)
Vn 1=1
#ES
1=1
(IsW. = w/ - (s)) (1, 8, x.) Hw 8, Xi)) I15: = =}
SES
1m1
11W, w
(1,0 - -u, =} vn
(1h St, Xi) - (y,
(s)
#ES
(IsW, = w/ - (s)) (1. s, - Hw 8, X;)) I15: =s}
(s)
SES
11W, w
n
Two (y. X.) - uw (v. = = vn
(w. Se, Xi) - Mw S.))
SES
(s) (5(
l(W, wH - Hw $)) =s}
Anw ( (z)
SES
12,1
ΣΣ
Zanw. w) Am (s)) (1, 8, X.) 8, DIS: - 8)
BES
im
= 22.2())
where the second equality holds because
11W, = w
Mao (y, 8,,, =3 Σ
())))u((U,s)) Σ
(w (y, Si).
(s)
#ES 1=1
#ES
1=1
For the first term 12,1 (v), we have
(w)(s) (x)
1(W; = () X.) - pmo ()) = =a)
sup yEy
Aio (5) (s)
#ES
(s)
1
I[W llfs = (H. - Mro (v,, ())ss))
sup VEV.SES
nhw (8) T (3)
Vn im1
#ES
Assumption 5.1 implies that the class (ww (y, X.) M (y. y E y} is of the VC-type with fixed coefficients (a, v) and an envelope F; such that [[ = s) < 0o for d > 2. Therefore,
1
sup VEYEES
1(W, = w) = (um,,, Xi) - (1, = (1))).
is also assumed that Pm (s)/n(s) = (1) and (s) F, 1/TT (8) < co. Therefore, we have
sup
12,,1 (v)) (1) weY
As for the second term 22), using the notation 6. (y, 8, X,) = Tuo 8, Xx) Hwl 8, Xx), by Assumption 5i), we
16

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
have
sup BEy
a(W: = w) (M
s, X.) = =s}
vn
Ter (8)
SES
Eie Sw, 8, Xx)
(s) (1, X;) (s)
n(s)
sup y
o,((1).
(s)
$ES
Therefore, we have
sup (v) + 12,2(1)1 (1). vEy
Combining the two terms, we have
VnllEgtes (v) (v) (r())
Σ
n
1W, = w =s}
(V, s) (1
1 11
(Ah s, the mw. s))
Tw
(s)
SES
n
k[W w))l({s = =s} (Mro (1/ 8, Xi) H Mw ())
Me (v. St)
+ Ry)
vn
where supyey (v), op()).
Define
(y, Yi(w), X;) == n. (v.s) + +11--/ 1 (o)) (ww (ul(y,, - u (y, 8)) (Mw (1, (v, (w. (C.5)
(s)
(vv) = (y, St) H St).
Taking the difference between treatments w and w', we obtain
TTTE (1) A ATE (v))
Σ Σ u(W; w) s,
n
I(W; = S, Y:(w'), Xt)
St,
.ES Vn
Σ
+
vn
where supyey [R(y)] (1). Denote
(v) = Σ
HW, w, S s,,
Σ
-w.=w'S. =:y,::
(v,
SES
i=1
SES 1=1
and
pn.(())= =
Vn Σ (o(y).
1
Then, uniformly over y V, we first show that
(Pn, (n)))))))))) ~n (9) (z).
17

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
where (v))) independent Gaussian processes with covariance kernels 9 (1,, v') and R2 y/) respectively, such that
Q1 3/) = =E[ S,, Xi)o S, X(() + w (S:) (y. St. Yi(w"), Xi) Ss ((w'),
and
D2(y.8') ((G(v)))())))
The following argument follows the argument provided in the proofs of Bugni et al. (2018, Lemma B.2) and Bugni et al. (2019, Lemma C.1). We first argue that, uniformly over E V
(1),, (n.((1) (19h, (3). pn)))
where 1 (v) and (1) ~ (1(3) uniformly over y E V
Under Assumption 3.1, the distribution of 1 the same as the distribution of the same quantity with units ordered by W within strata. In order exploit this observation, we introduce
strata and then ordered by treatment assignment the following notations. Define ((s) := Ei= 11S,, < Nw Σx= 1WW, w. S, =s) F(s) := P((S <s), and F. (s) := PIW, < W, = s) for all (w, s) E W S. Furthermore, let {(X; (?(1),.... Ye([W)) :1 i < n) be sequence of i.i.d. random variables with marginal distributions equal to the distribution of (X,, X((()))).....)))))))))) and
Pn, (W., [
where
Nw+1()))
(V)
Σ
1
E
S, YF(W), x;)
Σ #ES
Σ
Sw 8, (?("),, X;).
#ES Vn
Vn
As 2n))(() is only a function of (S: we have
(p), pv)) (v), Pn.()))
Next, define
(1) = (n(F())+
Fwr44)))
Σ
&uly, 8,
Σ
Σ
(u, (,, (ww)),xx;) X;))
dES (aF(s))+P
EES
FF
Note 1 (v) is a function of (V;(1), ((()),,.,,,(((W)))>>>)))........ X (E(m)ees which is independent of (W;, S: by construction. Therefore,
(1)
Note that
N(s)
(s) P, Fs) and n(s) p p(s). P, F(s),
n
n
We shall show that
sup
(v) (x))(() op (1) and ( ~ Gz(v)
vEy
Since S and W have finite cardinality, we fix (s, w) E S x W in the remainder of the proof. We define
Tn (w) = llfa < |tt (u. Y.ww),
18

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
for H E (0, 1]. The function &n (y. 8, (w)), X defined in (C.5) can be decomposed weighted of bounded random functions indexed by y E y' with bounded weight functions. More precisely, the class J == { ( (?)), x;) y } consists of functions from the following function classes: Fi == { Ti. s)}. F2, := {y - Mw (v, X,)} and Fs,u = 13 + Mw s)}. We can show that the class F1 is Donsker, for instance, by using the bounded, monotone property as established in Theorem 2.7.5 of van der Vaart & Wellner (1996). Also, under Assumption (ii), Theorem 2.5.2 of van der Vaart & Wellner (1996) yields that 2, and F3,. are Donsker. Since all the random weights are uniformly bounded, Corollary 10.13 of van der Vaart& Wellner (1996) shows that Donsker. Also, the class {l + I[ 2 [nd]] is VC class and hence Donsker. Since Theorem 2.1 of van der Vaart Wellner (1996) shows that products of uniformly bounded Donsker classes are Donsker, conclude that the indexed process (t, ow) t E (0, 1), ou E F) is Donsker. Hence, the result follows.
Next, for given y, by the triangular array central limit theorem,
( ~ (10,
where
(Ln(F(s) + Fw+1 (+)] (m(((s)) + (+)) Bi (. (?(w),
(y. = lim
n-+00
n
sES
([n(F(s) + Fwr+1 [m(F(s) - Far (5)) Blo (y. s, (,
+ lim
10000
n
Σ ()) S, Y((w), X:) + T.d (s) (v. St Vi(w"), x,)[S; = s]
= E Tw (S.) -h (y. S. Y.(w x,) + Tw (S.) (v. S, ())))))).))) X.)]
Using the Cramér-Wold device to verify finite-dimensional convergence, we find that the covariance kernel can be written as
Pi 21 v/) -E [ S.. Y((),, (V) St, Y((),, X.)] El (S:) (y. Sr, Yi(w"), Xi) (1/ S. Y((w),, X.)]
Finally, since [Mr S:) E y is of the VC-type with fixed coefficients (a, v) and a constant envelope function, {nw (y, S:) - Mw (y, St) y E y} is a Donsker class and we have
pn.2(1) ~ G2(v),
where (y) is a Gaussian process with covariance kernel
O2(v. 3²) [[ (y. St) How (v, S:)) (1 , St) - pw (V.S)) ((G())))()))
Therefore, combining the results, we have, uniformly over y E V,
addi (v) A DTT (1)) ~: G(v),
w.w!
where G(y) is a Gaussian process with covariance kernel
2(y.v') =E [T (Si) Sr, Yi(),, X.)ow (V. St, (.(w,,
+ [ (S:) (y. Sr, Yi(w'), ) Xi)ow (v', S,, Ye(w"), X.)
+ (.((),
This concludes the proof.
19

---

Distributional Treatment Effeets under Covariate-Adaptive Randomization
C.3. Semiparametric Efficiency Bound
Proof of Theorem 5.3. Part (a). We follow the approach used in Hahn (1998) and calculate the semiparametric efficiency bound of the DTE: A A (a) for given y. Throughout the proof of part (a), to indicate this evaluation location to keep the notation clear. First, we characterize the tangent space. To that end, the joint density of the observed variables (Y, W. X, S) can be written as:
where (ul2,, = P(Y w/W w. x and nm = P(W w| =x for all X.
Consider regular parametric submodel indexed by 0:
f(y. w. 2;0, (1) s;
which equals f'y, w. 8) when Oo-
The corresponding score of fy, w, x;; 0) is given by
8 (V, wi, 0) Iln f [V, w. 8; 0)
00
n(W = w ful H(W w))k((:0)) + +(,,,,,))
where f denotes a derivative of the log, i.c. f(;; @ in f (sr:0)
300
At the true value, the expectation of the score equals zero. The tangent space of the model is the set of functions that are mean zero and satisfy the additive structure of the score:
T L(W (ykw,, + H&W wJa (s)) + (2) a
(C.6)
where (yl2,, s), ax (s), az and (s) are mean-zero functions.
The semiparametric variance bound of A D (u) is the variance of the projection on T of function (), W. X, (with mean zero and finite second order moment) that satisfies for all regular parametric submodels
0 DTE w. (u; Fo)
Elb. (Y, WX,
(C.7)
00
10=60
8=8p
If th itself already lies in the tangent space, the variance bound is given by E(va).
Now, the DTE is
Fg) = IIʃ Mf s ())
A DTE
-11l ull ollaa 0ddydods.
20

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
We thus have
DTE AA
W:,00 (u; Fe)
10=00
09
// Mo bu
ff Mal
Letting Mmnwo wo S, = (u, S) Ma (u, S, X), we choose vu (Y, W, S) as
11W = w
(1 S u} Mw (U, S,)) 11W w') Tw
W,
(1fY u) pw (u,
Tu (S)
oo (u, X) A (w).
Notice that th satisfies equation (C.7) and that vh,, lies in the tangent space T given in equation (C.6). Since th lies in the tangent space, the variance bound is given by the expected square of tha
((u)= E MM (Y, W, X, S)
11W w} (IlYY S un Pm pm S,,
MWW w'} (I(Y u) -w (u, S, X))
E
Teg (S)
Tua (S)
(w) )
H/e (u, S.X) mi
DTE
= B| (5) (u, S, Y (w), X)] E/Tw
(w)]
This concludes the proof of part (a).
Next, for part (b), under Assumption 5.1, the regression-adjuster estimator defined in Algorithm satisfies the following asymptotic distribution for any given E y:
DTE, (y) - A unan!
(v)) ~9 (0, 2(3)).
where (yy) is the semiparametric efficiency bound derived in part (a). This completes the proof of part (b).
(y) is a regular, consistent and asymptotically normal estimator for the DTE. Moreover, the variance of Á A (y) coincides with the semiparametric efficiency bound R(y) established in Theorem 5.3. Consequently, obtain
Proof of Corollary 5.4. The result follows directly from the fact that, for given y E V, A DTE.emp
w.ww
DTF
V Var( A
DTE, (y)) < Var D.pp (y)). w.w!
w.w!
which concludes the proof.
21

---

Distributional Treatment Effeets under Covariate-Adaptive Randomization
D. Additional experimental details
All experiments were carried out on a MacBook Pro equipped with an Apple M3 Pro chip and 36GB memory. The replication code is publicly available using the Python library e adj (htps://pppi.orrpppoiccttttt
car, and the method can be implemented
D.1. Simulation Study
Figure illustrates the percentage reduction in RMSE of pigression-adjussee estimators relative to the empirical estimator for different sample sizes: {1,000, 5,000, 10,0003. Across all outcome levels, ML adjustment consistently outperforms linear regression, achieving reductions in RMSE of up to 50%. The RMSE reduction increases notably the sample size grows from 1, 1,000 to 5,000, but the improvement is less pronounced when the sample size increases from 5,000 to 10,000.
n=1000
n=5000
n=10000
RMSE Reduction
Quantiles
Linear adjustment
ML. adjustment
Figure 4. RMSE reduction (%) of regression-adiee adjusted estimators vs. empirical estimator across quantiles for continuous outcomes, with {1,000, 5,000, 10,000}. Linear adjustment uses linear regression; ML adjusment gradient boosting. Both 2-fold cross-fitting. Number of simulations is 1,000.
D.1.1. ADDITIONAL SIMULATION RESULTS
We consider discrete outcome Y; that follows Poisson distribution with the conditional expected value given by:
X| W,, Z.] 0.2|b(X,) + e(X,)W, +ZZ1,
where 6(X,) and c(X,) are given in Section 6. All other variables and treatment assignment mechanism consistent with those described in Section 6.1. Figure 5 presents the RMSE of the DTE estimators, along with the average length and coverage probabilities of their 95% confidence intervals, for the empirical estimator and the prreession-aduustte estimators using both linear and machine learning (ML) adjustments, with a sample size of n = 1,000. Figure 6 shows the percentage reduction in RMSE of the eggression-adjustee estimators relative to the empirical estimator across varying sample sizes: E {1,000, 5,000, 10.000). Similar to the results observed in the continuous outcome simulation, ML adjustment consistently outperforms linear regression. The benefit of regression adjustment increases with sample size: while neither method achieves RMSE reduction for large values of Y when 1,000, both show significant RMSE reductions across all outcome levels when 10,000.
We compare our method with the quantile treatment effect (QTE) estimator by Jiang et al. (2023) in the simulation setting described in Section 6.1, using linear regression. Additionally, we examine a scenario with discrete covariates, which leads to discrete outcome variable. In this setting, we retain the same data-generating process but sample each covariate X; independently from Uniform 5) distribution and round the values to the nearest integer. Table 2 presents the results, showing that our proposed method with linear adjustment achieves reduction in RMSE ranging from 6.4% 12.5% for the continuous outcome and from 0.8% to 6.1% for the discrete outcome. In contrast, the method by Jiang et al. (2023) achieves up to 6.5% reduction in RMSE for the continuous outcome but does not yield improvements for the discrete outcome. Furthermore, the DTE estimator demonstrates substantially better computational efficiency. It requires only 0.008 seconds for the continuous outcome and 0.015 seconds for the discrete outcome, while the QTE estimators take 148 and 0.145 seconds, respectively. This corresponds to our method being approximately 18 times faster for the continuous outcome and times faster for the discrete outcome.
22

---

Distributional Treatment Effects under Covariate-Adaptive Randomization
RMSE
Average Length
Coverage Probability
0.007
0.99
0.005
0.03
Value
0.003
0.004
0.96
0.001
Y
Y
Empirical
Linear adjustment
adjustment
Figure 5. RMSE, average length and coverage probability of 95% confidence intervals (CI) on simulated data with discrete outcomes (n 1,000). Linear adjustment uses linear regression, and ML adjustment uses gradient boosting, both with 2-fold cross-fitting. Number of simulations is 1,000.
n=1000
n=5000
n= 10000
RMSE Reduction
Linear adjustment
adjustment
Figure 6. RMSE reduction (%) of regression-adjusted estimators vs. empirical estimator across quantiles for discrete outcomes, with v E {1,000, 5,000, 10,000). Linear adjustment uses linear regression; ML adjusment uses gradient boosting. Both use 2-fold cross-fitting. Number of simulations is 1,000.
Table Comparison of RMSE reduction (%) between the proposed method and the estimator by Jiang al. (2023), using linear adjustment on simulated data (n 1, 000)
Method
Quantiles
Execution Time (SD)
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Continuous Outcome
Jiang et al. (2023)
-0.55
0,44
2.82
3,90
6.51
5.66
2.96
2.89
4.45
0.1480 (0.0200)
Proposed Method
6.42
9.26
11.67
11.51
10.79
11.41
12.48
11.58
9.91
0.00829 (0.0168)
Discrete Outcome
Jiang et al. (2023)
-1.77
-2.20
-2.59
-4.93
-2.69
-4.95
-5.19
0.08
0.38
0. 0.1446 (0,01112)
Proposed Method
5.62
5.36
2.10
0.86
2.67
2.41
5.67
3.75
6.10
0.0153 (0.00665)
23

---

Distributional Treatment Effeets under Covariate-Adaptive Randomization
D.2. The Impacts of Microfinance
Dataset The dataset from the field experiment conducted by Attanasio al. (2015) is available for download at OpenICPSR (Project 113597, Version V1).
Figure 7 highlights the five provinces in Mongolia where the experiment was conducted. The original map was sourced from htips://en. seapecii..rg/iiik/Prroooo Mongolli and subsequently modified by the authors to display the provinces, their associated stratum indicators, and the location of Ulaanbaatar, the capital city.
Sc1
Sc2 Khowsgol
Livs
Sm4 Budgen
S-3 Arshangai
UlamBaatar
Sud Khentil
Figure 7. Five northern provinces in Mongolia where the experiment conducted. Randomization was stratified the provincial level to ensure the inclusion of all treatment groups within each province.
Table Pre-treatment covariates included in regression adjustment
Variable name
Description
Age
Age in years of respondent
Age squared
Age in years of respondent squared
Buddhist
Respondents is of the Buddhist religion
Children 16
Number children the household younger than years
Education high
Dummy variable that is if the respondent completed grade VIII or higher
Education vocational
Dummy variable that is if the respondent completed vocational training Number of female houschold members aged 16 older
Female adults
Halh
Respondent ethnicity is Halh Number children
Household size
adults in household
Loans at baseline
Dummy variable that is if the household had at least one loan outstanding at the time of the baseline interview Number of male houschold members aged 16 older
Male adults
Married
Dummy variable that is if the respondent is married living together with partner Total annual household income prior to the experiment Total annual household income from wage employment prior to the experiment Annual enterprise revenue prior to the experiment
Household income
Wage income
Enterprise revenue
Enterprise profit
Annual enterprise profit prior the experiment
Table Estimated treatment assignment probabilities within each stratum (joint-liabilit lending control). The stratum indicator S, corresponds to provinces in Mongolia, and #(S.) represents the estimated probability of assignment to the joint-liability lending.
S
3
5
(S.))
0.53
0.61
0.55
0.55
0.69
24


Reviews:
[{"review_id": "gNDQ0JofOG", "reviewer": "Reviewer_qJAh", "summary": "The paper develops a regression‐adjusted estimator for distributional treatment effects (DTEs) under covariate‐adaptive randomization (CAR). It presents (i) a derivation of the estimator’s limit distribution under CA, (ii) a semiparametric efficiency bound result, and (iii) empirical/simulation demonstrations (including a microcredit dataset). The main claim is that incorporating extra covariates (beyond basic stratum indicators) can reduce variance in estimating how an intervention shifts the entire outcome distribution.\n\n## update after rebuttal\n\nThe authors have addressed some of my concerns regarding discrete outcomes and finite sample performance. As a result, I have raised my score from **2** (Weak Reject) to **3** (Weak Accept). I still find the contribution somewhat incremental, and believe the work could be further strengthened with clearer theoretical development and a discussion of computational limitations. Thus, I maintain a borderline positive score to be considered alongside the evaluations of the other reviewers.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful to your detailed review of our manuscript with constructive criticism, which has helped us identify important areas for improvement.\n\n**Discrete Outcomes and Assumption 5.1**\n\nYou raised an important point regarding our claim about handling discrete outcomes versus Assumption 5.1 requiring continuity. This was imprecisely stated in our manuscript. To clarify:\n\nOur method allows for discrete outcomes. We will revise Assumption 5.1 to explicitly accommodate mass points in the distribution while maintaining asymptotic guarantees. The continuity assumption was unnecessarily restrictive in our presentation. More precisely, in order to prove stochastic equicontinuity for continuous or finite discrete outcomes, we need \n\n$\\sup_{f \\in \\mathcal{F}}\\mathbb{E}f^2 \\leq c\\epsilon$\nfor some constant $c>0$, where $\\mathcal{F} = \\\\{ \\phi_w(y_2,s,Y_i^s(w),X_i^s)- \\phi_w(y_1,s,Y_i^s(w),X_i^s): y_1,y_2 \\in \\mathcal Y, y_1 < y_2 < y_1+\\epsilon \\\\}$ for some sufficiently small $\\epsilon >0$.\n\nIf discrete outcome has infinitely many points in its support, one can impose the following assumption: there exists constants $C_1, C_2>0$ such that \n\n$\\sup_{f \\in \\mathcal{F}}\\mathbb{E}f^2 \\leq   c_1  \\epsilon+ C_2e^{-\\alpha/\\epsilon}$.\n\nAlso, we will add a discrete outcome simulation to demonstrate this capability, which is an advantage over some existing methods. See response to Reviewer G1Ux for simulation results with discrete outcome. \n\n**Computational Considerations**\n\nWe agree that computational complexity deserves more attention. \n\n1. Since the outcome variable of interest is one-dimensional, our algorithm can handle most problems within a reasonable timeframe. Indeed, our algorithm is more straightforward and faster than the ones in the papers you kindly raise. \n2. Our approach offers computational advantages over quantile-based methods, because our method does not require minimizing a non-differentiable objective function through multi-step optimization and also do not rely on sorting algorithm.\n3. We review the algorithm of Kallus et al. (2019) as you suggested and we will add some discussion on the algorithmic difference as follow: \n    \n    Kallus et al. (2019) address problems in which nuisance parameters depend on the target parameter itself, as seen in cases like the quantile treatment effect (QTE) and local QTE. In contrast, our estimation of the distributional treatment effect (DTE) involves nuisance parameters that correspond to conditional means, which can be effectively estimated using machine learning algorithms.\n    \n\n**Contributions**\n\nWhile our approach extends doubly-robust methodology to DTEs under CAR, this extension provides several valuable contributions:\n\n1. It addresses the specific challenge of CAR designs that are increasingly common in practice but require distinct statistical treatment.\n2. Our framework unifies treatment effect estimation across outcome types under a single methodology with less restrictive assumptions.\n\nIn our revision, we will further elaborate on these contributions and highlight their practical significance.\n\n**Finite Sample Performance**\n\nWe acknowledge your concern about finite sample performance:\n\n1. We will add theoretical results on finite sample performance in our revision. As shown in our paper, the proposed estimator is unbiased in finite samples. Additionally, we can demonstrate that the variance of our estimator in finite samples is smaller than that of the standard estimator up to a deviation of order O(n^{-1}). If you are instead asking about non-asymptotic bounds, we would appreciate if you could kindly share particular papers in your mind. We are not aware of such papers and will consider this direction for future work.\n2. The real-world gains in the microcredit example will be better contextualized by benchmarking against expected improvements based on sample size. This is usually the case where the regression-adjustment method is needed in practice. Researchers want to reduce the variance without any additional cost and our method highlights that our method with proper experimental design can significantly reduce the variance of the treatment effect estimator.\n\n**Additional Comparisons**\n\nFollowing your recommendations, we will:\n\n1. Add Jiang et al. (2023) as a benchmark in our synthetic experiments — Please see responses to Reviewer sqwn and Reviewer G1Ux\n2. Include discussion of Kallus et al. (2019)\n3. Provide clearer comparative analysis of computational versus statistical efficiency\n\nWe believe addressing these points will substantially strengthen our paper. We appreciate your thorough review and feedback.", "claims_and_evidence": "* Claims: \n\t* The proposed estimator is unbiased and consistent for the DTE.\n\t* Adjusting for covariates improves precision under CAR.\n\t* The proposed estimator attains the semiparametric efficiency bound.\n* Evidence: \n\t* They provide formal asymptotic proofs for consistency and efficiency.\n\t* Simulation and one real-data example give some empirical support.\n\t* The real-data example has a small sample, so results there are less conclusive.\n\nOverall, the claims seem well supported. However, the paper should have a more detailed discussion (and theoretical results) of performance in finite samples that is common with doubly robust/Neyman‐orthogonal style estimators. In addition, although the introduction asserts that the method can handle discrete outcomes, Assumption 5.1 requires $\\mu_w(y, S, X)$ to be continuous in $y$, which does not apply for discrete outcomes. Thus, there appears to be a mismatch between the paper’s stated applicability and the continuity assumption in its theoretical results.", "methods_and_evaluation_criteria": "* The proposed method logically follows from doubly robust/Neyman‐orthogonal style estimators, adapted to CAR for DTEs.\n* Using standard approaches (like cross‐fitting and machine‐learning regression) is sensible, although carrying it out “for every $y\\in \\mathcal{Y}$” can be computationally expensive for continuous $y$ (Algorithm 1 in the paper). The authors should consider [1] or similar since their method may be more computationally tractable for handling continuous $y$ (although [1] only applied to quantile effects)\n\n[1] Kallus, N., Mao, X., & Uehara, M. (2019). Localized debiased machine learning: Efficient inference on quantile treatment effects and beyond. arXiv preprint arXiv:1912.12945.", "theoretical_claims": "* The authors prove asymptotic normality and derive a semiparametric efficiency bound. I skimmed the proofs and they appear sound, but Assumption 5.1 (especially (i)) might not be standard in the semi-parametric efficiency literature and require more discussion.\n* They do not provide much discussion about finite‐sample guarantees besides simulations.", "experimental_designs_or_analyses": "* The simulation design is reasonably standard for demonstrating coverage and bias. However, the gains in RMSE are moderate, so real‐world improvement may be small.\n* Since the synthetic experiments are effectively looking at quantiles, the authors should compare with  Jiang et al. 2023 as a baseline.\n* The real microcredit experiment is interesting but limited by sample size, reducing the credibility of the precision‐gain claims.", "relation_to_broader_scientific_literature": "* This work extends prior analyses of regression adjustment under CAR from average to distributional effects (cf. Rafi 2023, Jiang et al. 2023).\n* The closest relevant approach is  Jiang et al. (2023) which studies quantile treatment effects under CAR\n* A mention of specialized methods for quantile treatment effect estimation (e.g., Kallus et al., 2019) would be important—especially since their approach may be more computationally tractable than doing a separate regression for each $y$.", "essential_references_not_discussed": "The paper might profit from a deeper discussion of Kallus, Mao, & Uehara (2019) regarding more efficient approaches for quantile (and possibly distributional) estimation approaches.", "other_strengths_and_weaknesses": "To summarize from the scattered sections above:\n\n* Strengths\n\t* The paper applies doubly‐robust methodology to distributional treatment effect (DTE) estimation under covariate‐adaptive randomization (CAR).\n\t* It gives a rigorous semiparametric efficiency analysis, aligning with modern causal‐inference techniques.\n\t* The method is illustrated both in simulations and a real‐world microcredit dataset.\n* Weaknesses\n\t* Some incremental feel: the main novelty is “standard DR logic” extended from average effects to distributional effects under CAR.\n\t* Computational impracticalities for “every $y$” are not fully addressed.\n\t* Gains in finite samples (especially in the real dataset) are small, so the practicality is unclear. Also, should add Jiang et al. (2023) as benchmark for the synthetic experiments.\n\t* The paper claims the ability to handle discrete outcomes; however, Assumption 5.1 requires $\\mu_w(y, S, X)$ to be continuous in $y$, which does not align with discrete $Y$ distributions. This mismatch undermines the stated applicability and advantage over, for instance, Jiang et al. (2023).", "other_comments_or_suggestions": "**General comment:** The paper is neither here nor there. It needs more theoretical results for finite samples to justify the robustness/orthogonality results, it needs an alternative method for estimating with continuous $Y$ (otherwise, one can just use the quantile paper of  Jiang et al. (2023)), and it feels incremental in the sense of “just adding DR/Neyman-orthogonal logic” to existing estimators. Furthermore, I don't think the current approach  properly handles discrete $Y$. Thus, I lean towards rejection, but I am open to reconsidering my rating.", "questions_for_authors": "See concerns listed in the \"Weaknesses\" section above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d2azIuoD9Z", "reviewer": "Reviewer_G1Ux", "summary": "The paper proposes a method to estimate distributional treatment effects in randomized experiments that leverages additional covariates, beyond stratum indicators, to improve precision. The authors posit a regression adjustment based on Neyman-orthogonal moment conditions to flexibly estimate the nuissance parameters using off the shelf machine learning techniques. Theoretically the paper provides asymptotic guarantees for their estimator and show that it achieves the semiparametric efficiency bound. Empirically, the paper shows through simulations that the ML adjustment reduces the variance of the estimated quantile treatment effects and the practicality of the method is shown in an application to the impacts of Microfinance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your thoughtful and constructive feedback on our paper. Your positive assessment regarding clarity and theoretical development is encouraging.\n\nFollowing your comments, we will highlight three key distinctions from Jiang et al. (2023) in the revision:\n\n1. **Discrete Random Variables**: Our method is applicable for any type of outcome variables, including discrete or mixed-type outcome variables that are common in real-world applications. In the revision, we will add a simulation design featuring discrete outcomes to demonstrate this advantage.\n2. **Computational Benefits**: Our approach offers significant computational advantages that were not emphasized enough in the current version. Jiang et al.'s algorithm minimizes a non-differentiable objective function through multi-step optimization, and their last step potentially involves a grid search within the estimated interval. Our approach is much more straightforward in practice. Our simulation study suggests that this leads to substantial efficiency gains without sacrificing accuracy.\n3. **Relaxed Assumptions**: Our framework operates under less restrictive assumptions. In particular, our approach relaxes several continuity requirements that are present in Jiang et al. We will elaborate on these differences in Assumption 5(ii) and other relevant theoretical sections to clarify how our method maintains asymptotic guarantees under broader conditions.\n\n**Addressing Specific Recommendations**\n\n- We will revise Section 2 to carefully explain distinctions from Jiang et al. (2023).\n- We will add a new simulation with discrete outcomes and discrete covariates and direct comparison to Jiang et al. where applicable.\n    \n    First, we consider the same DGP setup as before but with a discrete outcome variable following a Poisson distribution. The conditional mean is given by\n    \n    $E[Y_i | X_i, W_i, Z_i] = 0.2 \\left| b(X_i) + c(X_i)W_i + \\gamma Z_i + u_i \\right|.$\n    \n    The table below presents the RMSE reduction (%) of our proposed DTE estimators relative to unadjusted estimators with n=1000. Both linear and logistic regression achieve reductions ranging from -1% to 6%. Under this DGP, computing QTE is not meaningful, as most quantiles equal zero due to a large mass point at zero.\n    \n    | Location | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | Execution time |\n    | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n    | Proposed Linear (DTE) | 1.29 | 2.10 | 1.89 | 0.08 | -1.33 | -0.28 | -0.48 | -0.61 | 0.28 | -0.22 | 0.0120s (SD 0.00632) |\n    | Proposed Logistic (DTE) | 2.49 | 5.81 | 4.59 | 2.38 | 0.71 | -0.45 | -0.44 | -0.13 | 0.25 | -0.28 | 0.0246s (SD 0.0144) |\n    \n    Next, we consider a setting with discrete covariates. Under the same DGP setup, we sample X independently from Uniform(−5,5)and round each value to the nearest integer. In this case, the DTE estimators achieve RMSE reductions ranging from 0.8% to 6.1% for linear regression and from 0.9% to 12.6% for logistic regression. However, the QTE estimator fails to yield reductions across most quantiles due to the discrete nature of the outcome variable.\n    \n    In terms of computational efficiency, the DTE estimators require 0.01s for linear regression and 0.04s for logistic regression, whereas the QTE estimators take 0.14s and 0.22s, respectively. This implies that our method is approximately 10 times faster for linear regression and 6 times faster for logistic regression.\n    \n    | Quantiles | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | Execution time |\n    | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n    | Jiang Linear (QTE) | -1.97 | -2.20 | -2.59 | -4.93 | -2.69 | -4.95 | -5.19 | 0.08 | 0.38 | 0.1446s (SD 0.0111) |\n    | Jiang Logistic (QTE) | -123.03 | -313.59 | -381.79 | -251.43 | -22.60 | -206.06 | -454.47 | -171.93 | -115.41 | 0.2206s (SD 0.0164) |\n    | Proposed Linear (DTE) | 5.62 | 5.36 | 2.10 | 0.86 | 2.67 | 2.41 | 5.67 | 3.75 | 6.10 | 0.0153s (SD 0.0067) |\n    | Proposed Logistic (DTE) | 2.38 | 3.12 | 0.97 | 0.51 | 2.19 | 3.28 | 7.05 | 8.97 | 12.62 | 0.0386s (SD 0.0106) |\n- Figure D.1 will be moved to the main body to better illustrate the quantile treatment effects, while emphasizing that our main focus is on DTE or PTE rather than QTE.\n- We will enhance our simulation discussion with expected RMSE reduction context given the known ground truth, , when possible. In most cases, however, we have chosen relatively complex processes where theoretical expectations may not be available.\n\nThese revisions will clarify the novelty and practical advantages of our approach while maintaining the strong theoretical foundation you recognized in your review.\n\nWe appreciate your comments and would be happy to elaborate further on any questions or concerns you might have.", "claims_and_evidence": "The paper makes three main claims: \n\n1. That it provides a new method to use auxiliary covariates in CAR to estimate quantile/distributional treatment effects that is also applicable for discrete random variables.\n\n2. That it provides an asymptotic regime under which the limit distribution of their estimator can be derived and the estimator achieves the semi-parametric efficiency bound.\n\n3. That it demonstrates the effectiveness of the estimator in real settings.\n\nOverall, the paper is very well written and the theory and method are clear and convincing. However, the theory and method are very similar to Jiang et al. 2023 (JOE). The authors recognize this and suggest that their advantage lies in that their method is applicable to discrete random variables. However, the paper does not highlight this either in the theoretical section nor in the simulations. It would be beneficial to flesh out the differences with Jiang et al. 2023 and provide an example of when their method works but not Jiang's and a comparison in the simulation section.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria are sensible.", "theoretical_claims": "The theoretical derivations and statements are polished and appear correct, but both the setting, proofs and theoretical statement seem to rely on Jiang et al. 2023. It would be beneficial to discuss the differences more and where the assumptions of each paper differ. For example, Assumption 5 (ii) etc.", "experimental_designs_or_analyses": "It would be good to have a simulation design in which the covariates are discrete and comparison to other methods (ie. Jiang et al. 2023 if applicable, and if not, say why). Also, it would be good to comment in the simulation design on whether we expect a large or small RMSE reduction, given that the ground truth is known. It might be more compelling to add Figure D.1. in the main body given that seeing the QTE is the goal of the paper.", "relation_to_broader_scientific_literature": "The key contribution is to provide a new method for QTE/DTE for CAR with regression adjustments to improve the variance that also works with discrete random variables. This appears to be a useful addition to the literature but the novelty relative to Jiang et al. 2023 should be better framed.", "essential_references_not_discussed": "The literature review is good. More comparison to Jiang et al. 2023 would be beneficial.", "other_strengths_and_weaknesses": "See above.", "other_comments_or_suggestions": "The paper is very well written and a pleasure to read! I think it is very polished and convincing, my only caveat is its novelty with respect to Jiang et al. 2023.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y2WvZYSZxW", "reviewer": "Reviewer_CJRK", "summary": "The authors propose an estimator and inference method based on asymptotic normality for distributional treatment effects under a covariate-adaptive randomization. The primary estimand considered in this work is the difference between cumulative distribution functions  for a fixed value $y$ between treatments. The authors propose an estimation approach based on inverse probability weighing and regression adjustment, and show that it satisfies asymptotic normality, enabling inference. Furthermore, they provide results showing semi parametric efficiency of their estimator, and provide experiments using their approach.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your detailed review of our manuscript. Your insightful feedback has greatly helped us identify key areas to strengthen in our work.\n\n**Distinguishing Our Work from Existing Literature**\n\nYou kindly raised a question about positioning our work uniquely beyond existing approaches. We acknowledge that our current manuscript does not sufficiently differentiate our contributions, and we will address this limitation in revision as follows: \n\n1. While doubly-robust approaches for distributional treatment effects exist in the general causal inference literature including Kallus & Oprescu (2023), our work addresses the unique challenges of covariate-adaptive randomization (CAR) designs. CAR introduces specific dependence structures that require different theoretical handling than standard randomized experiments. Also, the focus of Kallus & Oprescu (2023) is the conditional distributional treatment effect rather than the unconditional distributional treatment effect that we consider.\n2. A key distinction is that our approach directly estimates distributional effects rather than quantile treatment effects (QTE). Existing literature, including Kallus et al. (2024) and Jiang et al. (2023), focuses primarily on QTE, which requires much more stringent conditions such as smooth density around the quantiles of interest. Our approach does not require these requirements, making it applicable to a broader range of outcome distributions, including discrete and mixed distributions.\n3. We will also add simulation results to compare our method with Jiang et al. (2023). Please see responses to Reviewer sqwn and Reviewer G1Ux.\n\n**Contributions of Our Work** \n\n1. Our efficiency bound results are derived specifically for the CAR setting, which has not been previously established for distributional treatment effects. This distinction is important as it provides experimenters with guarantees on the maximum precision achievable in these increasingly common experimental designs.\n2. The efficiency bound and asymptotic results under CAR differs from that under complete randomization, particularly in how the stratum indicators interact with additional covariates. Our theoretical results characterize this relationship precisely and provide a valid inferential procedure.\n3. We think that the linear form of our estimator is a key strength compared to quantile treatment effects. This simplicity should be taken as advantageous rather than a limitation. The CAR setting introduces specific challenges that our method effectively addresses, resulting in superior practical performance compared to quantile treatment effects, which require non-differentiable optimization methods. Our approach is theoretically valid and practically useful. \n\n**Empirical Validation**\n\nFollowing your advice, You correctly point out limitations in our empirical validation:\n\n1. We will add error bars to Table 1 to determine statistical significance of the improvements.\n2. We will conduct additional experiments to validate the Type I Error control using Wald-style confidence intervals, demonstrating the practical implications of our asymptotic normality results.\n    \n    We report the average length and coverage probabilities of the 95% confidence intervals based on analytic standard errors derived from our asymptotic variance, for our original DGP with n=5000. Both the linear and ML adjustment methods (using XGBoost) result in shorter confidence intervals. The coverage probabilities remain close to the nominal level of 0.95 for both the simple and adjusted estimators, with a slight over-coverage observed for certain quantiles, reaching approximately 0.97.\n    \n\n    | Quantiles | Simple (CI Length) | Linear (CI Length) | XGBoost (CI Length) | Simple (Coverage) | Linear (Coverage) | XGBoost (Coverage) |\n    | --- | --- | --- | --- | --- | --- | --- |\n    | 0.1 | 0.0337 | 0.0332 | 0.0328 | 0.945 | 0.964 | 0.966 |\n    | 0.2 | 0.0456 | 0.0445 | 0.0438 | 0.964 | 0.967 | 0.970 |\n    | 0.3 | 0.0534 | 0.0517 | 0.0506 | 0.948 | 0.946 | 0.947 |\n    | 0.4 | 0.0586 | 0.0564 | 0.0548 | 0.954 | 0.955 | 0.956 |\n    | 0.5 | 0.0619 | 0.0591 | 0.0572 | 0.951 | 0.960 | 0.965 |\n    | 0.6 | 0.0637 | 0.0603 | 0.0581 | 0.963 | 0.963 | 0.964 |\n    | 0.7 | 0.0639 | 0.0598 | 0.0576 | 0.946 | 0.962 | 0.972 |\n    | 0.8 | 0.0627 | 0.0578 | 0.0558 | 0.949 | 0.958 | 0.975 |\n    | 0.9 | 0.0600 | 0.0541 | 0.0528 | 0.935 | 0.952 | 0.978 |\n\nWe believe these revisions will address your concerns about the novelty and positioning of our work in relation to existing literature. Thank you again for your thoughtful feedback.", "claims_and_evidence": "The claims made by the author in their theorems are well justified, and appear to follow from standard techniques in the literature.\n\nThe authors make the following claim about regression adjustment in their Introduction (l. 61): \"our work advances this framework to accommodate distributional treatment effects.\" However, regression adjusted / doubly robust approaches to estimation of distributional causal parameters have been done extensively in previous works (including for discrete outcomes) by references the authors themselves cite (e.g. Kallus et. al., 2024, Kallus & Oprescu, 2023). These papers even extend their estimand to general distribution dissimilarity measures (such as f-divergences) between the potential outcomes of two distributions. \n\nCould the authors elaborate a bit more on how their work positions itself uniquely beyond existing approaches?", "methods_and_evaluation_criteria": "The proposed methods are tested on both empirical and real-world data. It would be helpful to have error bars in Table 1 in order to determine whether the changes are significant or not. Additionally, given that this paper provides asymptotic normality results, it would be nice to see additional experiment results that show approximate Type I Error control if using Wald-style confidence intervals for inference. .", "theoretical_claims": "The proofs seem reasonable - I have not checked carefully.", "experimental_designs_or_analyses": "N/A", "relation_to_broader_scientific_literature": "This paper provides an application of regression adjustment / doubly-robust approaches for covariate-adaptive randomization. The main contribution of this paper is its choice of estimand, which can be used to determine the difference in probability masses between two treatments, given upper/lower bounds on the outcome value. \n\nThe estimand and its efficiency results seem to follow directly from many existing works.", "essential_references_not_discussed": "The paper cites all related work in the literature, but fails to distinguish its results from the previous works. It could be helpful to contrast their results (both in terms of estimator and efficiency bound) in order to make their contributions more clear.", "other_strengths_and_weaknesses": "Strengths:\n* The writing and explanation of this work is clear. For example, Assumption 3.1 is well described by the authors, with each component explained clearly.\n* The authors incorporate covariate-adaptive randomization (which differs from standard context-dependent propensities slightly by assuming it only depends on the stratum). \n\nWeaknesses:\n* Unclear if these results provide additional benefits from existing work on doubly-robust estimation of distributional treatment effects\n* Does not empirically validate important results in the paper, such as asymptotic normality and Type I error of its Wald confidence interval.\n* Does not clearly separate its results from existing, well-established frameworks from related literature.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "It would be nice for the authors to clarify their contributions beyond defining an estimator for the difference in probability for outcomes under some threshold $y$ between treatments, and using the well-known doubly-robust framework for estimating this quantity. This quantity is easier to estimate than other distributional quantities, such as differences between quantiles of outcomes under each treatment, due to having no nuisances and a nice linear form. The proposed estimator appears to be a simple extension of the doubly-robust framework for causal estimates, and I'm unsure where the covariate-adaptive randomization plays a novel role in either the estimator or its analysis. \n\nCould the authors please clarify this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ltwBuoghJs", "reviewer": "Reviewer_sqwn", "summary": "This paper proposes a method of estimating the distributional treatment effect, in a setting that uses randomised experiments using covariate-adaptive randomisation. The distribution is captured through the cumulative distribution function, and estimation is done via regression-adjustment.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are deeply grateful for your detailed and thoughtful review and appreciate your positive assessment of our paper.\n\n**Theoretical Presentation**\n\nThank you for the valuable suggestions to improve our mathematical presentation:\n\n1. We will provide definitions of VC-type function classes in the Appendix as you suggested.\n2. We will move the quantitative statements of Theorems 5.2 and 5.3 to the main body.\n3. We will correct the notational inconsistencies you kindly pointed out, including:\n    - Moving the definition of the indicator function to its first use\n    - Using different indexing in Algorithm 1 to avoid confusion\n    - Using L∞(Y) instead of l∞(Y) for the space of bounded functions\n    - Correcting the statement \"for uniformly over y∈Y\" by removing \"for\"\n    - Fixing the bracket placement in μw(y,s) and the repeated notation of Ix(y)\n    - Correcting the Bickel et al. reference\n\n**Novelty and Contributions** \n\nRegarding your comment about incrementality compared to Jiang et al. (2023):\n\n1. While Jiang et al. focus on quantile treatment effects under CAR, our distributional approach offers important advantages. Quantile estimation requires much stronger conditions (continuity and positive density around quantiles of interest), while our approach is applicable to non-continuous outcomes. \n    \n    We add the following comparison of our method with Jiang et al’s below. We use the same DGP (our original DGP) with n = 5000 to compute the QTE. The table below presents the RMSE reduction (%) of regression-adjusted QTE estimators relative to the unadjusted QTE, following Jiang et al.'s method. Linear adjustment achieves a comparable magnitude of RMSE reduction to the DTE estimator. In contrast, logistic regression fails to reduce RMSE due to bias. The DGP in this setting is highly nonlinear and complex, with many irrelevant covariates. Consequently, simple linear adjustment outperforms ML-based adjustment in this specific case for the QTE.\n    \n    | Quantiles | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n    | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n    | Jiang et al Linear (QTE) | -0.55 | 0.44 | 2.82 | 3.90 | 6.51 | 5.66 | 2.96 | 2.89 | 4.45 |\n    | Jiang et al Logistic (QTE) | -322.58 | -352.95 | -260.41 | -114.72 | -11.48 | -120.13 | -250.86 | -289.59 | -194.00 |\n\n1. The formulation of our approach leads to a more straightforward algorithm for estimating distributional or quantile treatment effects. This offers more than mere computational advantages and delivers substantial practical improvements compared to existing methods. Please refer to the response to Reviewer G1Ux for detailed comparisons.\n\nWe will clarify these distinctions more explicitly in the revised manuscript.\n\n**Regarding Your Specific Question**\n\nIn terms of Assumption 5.1(iii), we realize that this assumption is more restrictive than necessary. We will revise the condition to hold locally. To prove our main results, we need the following condition: \n$\\sup_{f \\in \\mathcal{F}}\\mathbb{E}f^2 \\leq c\\epsilon \\equiv \\sigma^2$\nfor some constant $c>0$, where $\\mathcal{F} = \\\\{ \\phi_w(y_2,s,Y_i^s(w),X_i^s)- \\phi_w(y_1,s,Y_i^s(w),X_i^s): y_1,y_2 \\in \\mathcal Y, y_1 < y_2 < y_1+\\epsilon \\\\}$ for some sufficiently small $\\epsilon >0$.\n\n**Alternative Distributional Representations**\n\nThank you for raising an inspiring question about the applicability of Kernel Mean Embedding (KME) approaches. Our simple answer regarding whether KME is applicable for generic outcome distributions is positive. For multivariate outcomes or non-standard outcomes, estimating the conditional distribution function would be challenging, while working in the Reproducing Kernel Hilbert Space (RKHS) would be more straightforward. One caveat is that if the kernel function in use is continuous, the approach requires the continuity of underlying distributions. Since your question is both challenging and important, we will include it in the discussion section as a promising avenue for future research.\n\nThank you again for your careful reading and constructive suggestions, which will help us improve the clarity and presentation of our work.", "claims_and_evidence": "Theoretically, it is claimed that the proposed estimator for the distributional treatment effect is asymptotically a Gaussian process (Theorem 5.2), and that the estimator attains the semiparametric efficiency bound. I went through some of the proofs and could not find any obvious errors, and these results are very believable. Practical advantages are demonstrated through experiments.", "methods_and_evaluation_criteria": "The method given in algorithm 1 makes sense for the problem at hand. \n\nBut I personally do not believe that the cumulative distribution function is the best way of capturing the features of a distribution, because one needs to regress for many $y$ values to get a sense of the distributional treatment effect. Quantiles, kernel mean embeddings, or even specific distributional quantities like the variance would be more of interest in my opinion. Please correct me if I'm mistaken about this, but I think the authors should at least give a discussion of this issue in the paper. See also \"Questions\".", "theoretical_claims": "I checked some parts of the proofs of Theorems 5.2 and 5.3, I did not go through every line, but I could not spot any obvious errors. The proofs are essentially lifted from previous works and I can fully believe that they are true. The mathematical presentation is in general very nice, and I appreciate the effort that the authors put into writing the proofs. \n\nHowever, I think the presentation of the proofs can be improved. The authors use many concepts that are taken directly from the proofs of previous papers, such as VC-type function classes or Lemma N.2 of (Jiang et al., 2023). VC-type function classes were introduced by (Chernozhukov et al., 2014) but I would not say the definition and the ensuing results are standard enough in the literature for most readers to be familiar, and it would certainly be worth giving the definitions, at least in the Appendix before the proofs start. \n\nAlso, the quantitative statements of both Theorems 5.2 and 5.3 are deferred to the Appendix and it's quite hard to find it hidden in the proofs, and I think the precise statements should be given in the theorem statements in the main body.", "experimental_designs_or_analyses": "No issues I could find.", "relation_to_broader_scientific_literature": "The authors do a great job of reviewing the related literature in Section 2, going much further back than what is usual, with papers cited from the 70s. The review of recent literature is extensive and thorough.", "essential_references_not_discussed": "Nothing essential missing I would say.", "other_strengths_and_weaknesses": "The paper is well-written, well-structured, and delivers its messages in a clear way. It was a pleasure to read. \n\nHowever, the mathematical notations and presentation could be improved somewhat, especially in Section 5. See \"Other Comments Or Suggestions\".\n\nIn general, I would say that this is a solid paper, but not groundbreaking. The main contribution is the proposal of the estimator itself, and the algorithm to compute it, but distributional treatment effects in the case of CAR seems already done in the form of quantile treatment effects by (Jiang et al., 2023), and to tackle the same problem with quantiles replaced by cumulative probability distributions, seems somewhat incremental. However, I don't like to see papers based on reviews of the form \"incremental\" or \"lack of novelty\", and the submission does do quite a solid job of the problem at hand. \n\nThe theory is nice to have and it is well written (with some caveats - see other sections) but there are no novel insights that arise from it - the results and proofs are very similar to those of existing papers. I think it would have been better to provide slightly novel forms of results, perhaps non-asymptotic high-probability bounds or lower bounds, but then that would go towards making this paper a theory paper, which of course this submission is not. The experiments also seem to be carried out thoroughly, but I'm not convinced that they demonstrate a clear need or advantage for this method.", "other_comments_or_suggestions": "216L: The indicator function is defined here, but it is actually used first on page 3. If you want to explicitly introduce the indicator function, I think it should be done when it is first used. But I think it is not strictly necessary, the vast majority of readers should be familiar with the indicator function. Indeed, I thought nothing of it when it was used on page 3 without being explicitly introduced. \n\nAlgorithm 1: This is being pedantic, but you use $w$ for both a particular treatment and as indexing through the two treatments $\\{w,w'\\}$. I think it would be better to use another indexing letter, what do you think?\n\n270L: The notation $l^\\infty(\\mathcal{Y})$ (or $l_\\infty(\\mathcal{Y})$) is usually reserved for the space of bounded sequences, i.e. bounded functions on the domain $\\mathbb{N}$. The notation $L_\\infty(\\mathcal{Y})$ would be better for the space of bounded functions with the supremum norm. \n\n255R: In Theorem 5.2, \"for uniformly over $y\\in\\mathcal{Y}$\" sounds strange. I think \"for\" can be removed. \n\n260R: I think it would be much better to be specific about what $\\mathcal{G}(y)$ is here, rather than referring the reader to the appendix, especially given that this is one of the main results of the paper. The same applies for Theorem 5.3. \n\nThe reference [Bickel et al., 1993] is strange - some authors are repeated. \n\n673: In the definition of $\\mu_w(y,s)$, the closing round bracket is in the wrong place. \n\n675: The notation $I_x(y)$ was already used on 226R, for a certain set of indices. It would be better to use another notation.", "questions_for_authors": "You use the cumulative distribution function to represent a distribution, and of course this is one good way, as it completely characterises a real distribution. However, when the outcome variable is not real-valued, or when you want to capture more specific distributional aspects, there are other ways to characterise a distribution, for example, kernel mean embeddings or kernel witness functions, as done in [Park et al., 2021], or the concurrent paper by Näf and Sussman (https://arxiv.org/abs/2411.08778, which would be relevant to cite). Do you think it would be possible to apply the kernel representations of distributions to CAR, or equivalently, to use your approach for CAR using kernel representations of distributions?\n\nIn Assumption 5.1(iii), does the constant depend on $w$ and $y_1,y_2$?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper develops a regression-adjusted estimator for distributional treatment effects under covariate-adaptive randomization. The reviewers found the paper to be theoretically sound, and their concerns were satisfactorily addressed in the rebuttal. However, the authors are encouraged to incorporate the reviewers’ suggestions into the final version.
Decision: Accept (poster)