Treatment Effects Estimation By Uniform Transformer
Authors: Ruoqi Yu
Shulei Wang
Venue: ICLR 2024 poster (2024)
Keywords: Causal inference
covariate balance
nonparametric estimation
observational studies
weighting method
Primary area: causal reasoning

Abstract: In observational studies, balancing covariates in different treatment groups is essential to estimate treatment effects. One of the most commonly used methods for such purposes is weighting. The performance of this class of methods usually depends on strong regularity conditions for the underlying model, which might not hold in practice. In this paper, we investigate weighting methods from a functional estimation perspective and argue that the weights needed for covariate balancing could differ from those needed for treatment effects estimation under low regularity conditions. Motivated by this observation, we introduce a new framework of weighting that directly targets the treatment effects estimation. Unlike existing methods, the resulting estimator for a treatment effect under this new framework is a simple kernel-based $U$-statistic after applying a data-driven transformation to the observed covariates. We characterize the theoretical properties of the new estimators of treatment effects under a nonparametric setting and show that they are able to work robustly under low regularity conditions. The new framework is also applied to several numerical examples to demonstrate its practical merits.

Published as a conference paper at ICLR 2024
TREATMENT EFFECTS ESTIMATION BY UNIFORM TRANSFORMER
Ruoqi Yu
Shulei Wang
University of Illinois Urbana-Chani -hampaaan Champaign, IL 61820, USA ruoqi. ry www.yygg@m...ommommmmmmmm.....ma com
University of Illinois Urbana- Champppin Champaign, IL 61820, USA shuleiw@ www.ewiiiiiniiss...
ABSTRACT
In observational studies, balancing covariates in different treatment groups is es- sential estimate treatment effects. One of the most commonly used methods for such purposes is weighting. The performance of this class of methods usually de- pends on strong regularity conditions for the underlying model, which might not hold in practice. In this paper, investigate weighting methods from functional estimation perspective and argue that the weights needed for covariate balancing could differ from those needed for treatment effects estimation under low regu- larity conditions. Motivated by this observation, we introduce a new framework of weighting that directly targets the treatment effects estimation. Unlike existing methods, the resulting estimator for a treatment effect under this new framework is a simple kernel- U-statistic after applying data-driven transformation the observed covariates. We characterize the theoretical properties of the new es- timators of treatment effects under a nonparametric setting and show that they are able to work robustly under low regularity conditions. The new framework is also applied several numerical examples demonstrate its practical merits.
INTRODUCTION
In order to infer causal relations in an observational study, a major difficulty is reduce the bias brought by the confounding covariates related to both the treatment assignment and the outcome of interest (Imbens & Rubin 2015). This task can be accomplished by balancing the empirical distributions of observed covariates in different treatment groups. One common strategy to adjust
imbalances of confounders is weighting (Rosenbaum) 1987 Robins et al. 1994, 2000 Hirano & Imbens 2001 Hirano et al. 2003), which seeks weight for each sample so that covariates distri- butions are similar between the weighted groups.
A conventional approach widely employed in the literature for estimating weights is the inverse- probability weighting (IPW) method, where the weight of each sample is the corresponding inverse probability of receiving the treatment (Horvitz & T'hompson 1952 Hahn 1998 Robins al. 2000 Hirano & Imbens] 2001 Hirano et al., 2003). It has been shown that this method can entirely remove the bias the estimation of treatment effects when the true propensity score, defined as the condi- tional probability of receiving treatment given the covariates (Rosenbaum & Rubin 1983), is used, Practical applications of IPW usually require estimating the propensity score based on presumed model, as the true propensity score is typically unknown in advance. However, misspecification of the propensity score model can induce large biases in estimating treatment effects using IPW (Kang & Schafer] 2007). This observation motivates recent works to develop more robust weighting methods, aiming to estimate the propensity score or the weight itself by directly comparing some prespecified moments/basis functions of covariates between the treatment groups (Graham et al.) 2012] Hainmueller 2012) [mai & Ratkovic 2014| Zubizarreta 2015 Chan et al.] 2016 Zhao & porcival|2010 Wong & Chan 2018 [hao 2019] Wang & Zubizarreta, 2020 Kallus 2020 Hirsh- berg & Wager 2021 Singh] [2011]Bruns
et al. [2023]
| Fan et al.] 2023). These direct balancing
weighting methods have been shown to work more robustly than the IPW method in practice.
The good performance of the inverse- probaabity weighting or direct balancing weighting methods usually relies strong regularity conditions for either the propensity score or the response func-

---

Published as a conference paper at ICLR 2024
tions. Specifically, the smoothness levels of the propensity score and the response functions must be at least as large as half of the covariate dimension in these methods (Chan et al. 2016; Fan et al, 2023 Wong & Chan 2018 Wang & Zubizarreta 2020). However, it is not immediately clear how to construct the weights when these criteria are not met and to what extent the treatment effects could be estimated by the weighting method in such non-smooth cases. Therefore, this paper aims to address these issues and develop a new framework of weighting methods to fill these needs.
We first investigate the weighting methods from a functional estimation perspective effect is essentially a functional of the response and density functions. Through this perspective, we argue that the best way to estimate the ideal weights does not necessarily lead to the most efficient weighting estimator for treatment effects in the non non-smooth case. In other words, the weights needed for covariate balaneing could be different from the weights needed to estimate treatment effects when the response and density functions are non-smooth. Then, a natural question arises: how can we design a weighting method that directly targets the treatment effects estimation?
treatment
To address this question, we introduce a new weighting framework called Weighting by Uniform Transformer (WUNT). Note that the term "transformer" here differs from its conventional usage as neural network architecture (Vaswani et al. 2017). Our motivation for this new framework stems from a noteworthy insight into the uniform transformer, defined as a transformation mapping the covariate distribution in the control group to uniform distribution. The uniform transformer gives us a clean form of the weighting method, allowing us to directly make an accurate trade-off between the bias and variance for the treatment effects estimation. In addition, we show how to construct a data-driven uniform transformer from the covariates of control samples in a computationally effi- cient way. With the data-driven uniform transformer, the weights in WUNT are customized for the treatment effects estimation, and the resulting weighting estimator is a kernel-based U Usstatistic.
To demonstrate the merits of the newly proposed framework WUNT and the corresponding estima- tors, we study the theoretical properties under nonparametric setting, especially when the response and density functions are non-smooth. Specifically, we show that the proposed estimator is consis- tent under very mild conditions. In addition, if the covariate density in the control group is known or can be estimated accurately, the minimax optimal converge rate for the mean square error of estimating the average treatment effect on the treated group is
n |
+ +
where n is the sample size, is the dimension of covariates, and a and 3 are the smoothness levels of the response surfaces and density functions of the covariates, respectively. This result suggests that estimation of the treatment effects becomes more difficult when the response and density functions are less smooth. The converge rate presented here also appears in Robins et al. (2008) 2009 2017), where an accurate trade-off between bias and variance is achieved by semi-parametrie methods with additional bias reduction techniques. Our result shows that the newly proposed weighting method is also able to do so because of the uniform transformer. To our best knowledge, this is the first minimax rate-optimal weighting method when the response and density functions are non-smooth. Practical merits of WUNT are further demonstrated through simulation experiments in Appendix B
2 PROBLEM SETTING AND WEIGHTING METHODS
2.1 PROBLEM SETTING AND NOTATIONS
Suppose that the observed data (X,, Z4, Yi), 1, 1,...,,7 are independent and identically distributed observations of (X, Z, Y), where X E Idd are the observed covariates, Z binary indicator variable for the treatment and Y is the outcome of interest. Under the potential outcome framework for causal inference (Rubin| 1974 Imbens Rubin 2015). y and Y'1 are the potential outcomes if the individual is assigned to the treated (Z- 1) or control group (Z 0). Then, the observed outcome can be written as Y = (1 Z) + ZY Throughout this paper, we always assume the strong ignorability of the treatment assignment (Rosenbaum & Rubin 1983)
(1) It is of interest to estimate the average treatment effect (ATE), TATE, the average treatment effect on the treated group (ATT), TATT. Let (X() = EEY (X) and (x) |I we have
1 2 X and
D<PP - 1|x) < 1.
TATE = [x) (x)
and
TXTT Exur(X) =c(x) 2 1).
2

---

Published as a conference paper at ICLR 2024
For the sake of concreteness, focus primarily the average treatment effeet the treated group TATT in this puper. The techniques are also applicable to more generalized cases, e.g., Section C.1 of the Appendix discusses the robust estimation for the average treatment effect TATE under the new framework. Let fr(X) P( z 1) and fc(X) p( 0), then TATT be written
TATT PTT PCT = / pT (T( f -
natural to estimate the first term PTT by Ei= / Σm= Z, The second term PCT is the major challenge in estimating TATT since the data with both response function fic and sampling distribution (X) are inaccessible. Therefore, the main parameter of interest in this paper pcr.
2.2 WEIGHITNGG METHODS: A FUNCTIONAL ESTIMATION PERSPECTIVE
In general, a weighting method (Rosenbaum, 1987; Hirano et al. 2003) seeks weights for each control sample so that covariates of the weighted control samples are more similar to those of the treated samples. Given the weights w; for each sample, por is estimated by the weighted mean
PCT Σ (0 - Z.)Y,
(2)
In this section, we discuss the weighting methods from functional estimation perspective as PCT pc(Xi (X) is essentially bilinear functional of the response and density functions. The main intuition behind weighting is that the functional PCT can be rewritten as
fx(
fo(X)dX - (x(8))6(X))xXX4 fo(x)
PCT =
(3)
where the weighting function w( w(X) is
fr fc(X)
((x) P 2 0) 1 - π (X) (z =
w wxX)=
Here, the weighting function at each X; can be treated as ideal weights for the weighting methods since w(X) can make the distributions in the treated and control groups perfectly balanced. In practice, the weighting methods aim estimate the weighting function each X,, by some estimator vi (xx)) and then replace wo( (X) by ((X) in equation Despite the difference between IPW and direct balancing weighting, the common goal to estimate the ideal weights wxX).
On the other hand, the ultimate goal in treatment effects estimation is estimate por rather than w(X), the weighting methods focusing estimating the ideal weights can be seen plug-in estimator for μCт since w( replaced by (xX) in equation However, such plug-in in strategy does not necessarily lead to an efficient estimator for the functional PCT (Lepski et al. 1999 Newey
al. 2004 Cai & Low 2011 Robins al. 2017 because the best estimator for w w(X) might not be the most suitable choice for estimating PCT. An explicit example is given to illustrate this point design the weights in equation @ aiming at estimating pcr directly? In this paper, we demonstrate that this is indeed possible; fact, the weights needed for estimating PCT are over-debiased and variance-inflated estimators for w(X) when the response function in the control group is not smooth.
in Section 2.3 Therefore, natural question arises:
2.3 A WARM-UP EXAMPLE
To illustrate the idea, we start with special case where the covariate X one-dimensional (d 1) and the distribution of X in the control group (x) is the uniform distribution on [0, 1]. For simplicity, we assume fx(
()) = O X = 0 and 1 to avoid boundary bias. In this case, the weighting function w( X) becomes fr X), estimating the weights is equivalent estimating the density of
3

---

Published as a conference paper at ICLR 2024
covariates in the treated group. To estimate the density fo without assuming any parametric form, consider using one of the most commonly used density estimators - -the kernel density estimator
fr(2)
Σ K
2
KA xx X.) Z::
Th
where K()) is one-dimensional kernel function and h is the bandwidth. Standard analysis for kernel density estimators suggests that
((fr(x) fr(x)) S 20 + 1/(nh)
Bias
Varionce
we assume fr(()) belongs to Hölder class H ([O, the formal definition of Hölder class is introduced in Section See Tsybakov (2008) for a detailed proof. Therefore, if we aim to estimate the weighting function w (X), the bandwidth h should be chosen
n 1/11(1)) i.e., h is of the same order as 11(++22)) Is this bandwidth also the most suitable one for estimating PCT?
It seems reasonable to expect that the best way to estimate the weighting function w(X) leads naturally to the best weighting estimator for pCT and hence the ATT. However, we now show that the bias and variance trade-off for estimating the weighting function (X)) can be very different from that for estimating PCT. If the weights in equation are replaced by the above kernel density estimator, the resulting weighting estimator for por is then
En Yn (1 Kn
(x.,
-Xig)
Zio
PCT
E E (1 Kn X X1o
Ziz
Since jAr is a U-statistics, our analysis in Section shows that
E(per HCT)² z 11((a00)) 1/n 1/(
Bins
Varsance
if we further assume the response surface in the control group (c(x)) belongs to Hölder class ((0, 1)). The proof is omitted here since the result is a special case of Theorem 1 Unlike es- timating the weighting function wxX). the bias in estimating Per also relies the smoothness of the response function. The new bias and variance trade-off suggests that the optimal choice of the bandwidth h for estimating per is
21(1+((++)))
a + 3 < 1
[n . 1 1/2((+1))))))))
a a 3
where / x [a, 6] means x e for arbitrary c [a, When the response function in the control group (c(x) is smooth enough, i.e., 2 1/2, the optimal bandwidth for estimating the weighting function is also optimal for estimating (CCT- This explains why we can estimate the ATT efficiently through targeting the covariate balancing in this case. On the other hand, if the response function (o(r) is non-smooth, i.e., < 1/2, the optimal bandwidth for estimating PCr is much smaller than the optimal choice for estimating w(X). In particular, response function into account when we choose the optimal bandwidth for estimating μCCT. Putting differently, the optimal choice of / for estimating μCr can result in a suboptimal estimator for w( w(X), which is over-debiased and variance-inflated. The similar phenomenon is also observed in estimation of integrated squared density (Bickel & Ritov| 1988) Laurent 1996 Giné & Nickl] 2008).
need to take the smoothness of the
This example not only illustrates that the optimal estimator for (xX) does not necessarily lead to an efficient estimator for Per but also suggests a potential strategy design the weights tailored for treatment effects estimation.
3
WEIGHTING BY UNIFORM TRANSFORMER
3.1 WEIGHTING FRAMEWORK FOR TREATMENT EFFECTS ESTIMATION
The warm-up example above suggests that the weights targeting the weighting function w w(X) may not lead to an efficient estimator for per and hence the ATT. In this section, we build upon these insights and introduce weighting framework tailored for treatment effects estimation.
4

---

Published as a conference paper at ICLR 2024
A unique feature of the warm-up example in Section 2.3 is that the covariate follows uniform dis- tribution in the control group. Because of this property, the weights estimation problem is reduced to density estimation problem. To apply this technique to covariates with any distribution, can map the covariate distribution in the control group to a uniform distribution. Any such transforma- "uniform transformer" in this paper. We assume the uniform transformer is
tion i is referred known in the current section and leave the discussion on the construction of uniform transformer
the next section. Let U = (xx) denote the data after transformation. Since f& is uniform distribution density, we can rewrite the propensity score-based weight w;
x(x)) (X.)
fT(X.) (x(X.)
(P)))) 18(0.) X (T.))
W/ X
X
O
That is, to estimate the weights w;, we only need to estimate the density f (Ū) at each U. However, as demonstrated in the warm-up example, the density estimation should be done carefully since the optimal tuning parameters targeting the density estimation itself can differ from the optimal choices for estimating the treatment effects. In other words, the key difference between our framework and the classical density estimation problems is the choice of tuning parameters. Under this new framework, the weights in equation
2 can be constructed in two steps: (i) transform the covariates X, by uniform transformer 4, (ii) estimate the weights by any density estimator with tuning parameters targeting the treatment effects estimation. The framework is summarized in Algorithm and we call it "weighting by uniform transformer" (WUNT).
Data: {(x,, Zi, YO
Result: Weights w, and an estimator of PCT-
Construct the uniform transformer by {X:)az=0 and apply transformation for data (;=4(Xi); Estimate (T)) from (U.)aZ,=1:
(U.) Evaluate the weights by w; (0,))) (t( for Z; = 0;
Assign the weights = 1/nn for Z, = 1;
Estimate PCT by equation?? 2
Algorithm 1. Weighting by Uniform Transformer (WUNT)
For the density estimation in the second step, we introduce two of the most popular nonparametric density estimators in the literature. The first density estimator the kernel density estimator, which has been widely used in many applications. The kernel density estimator is defined
Σ del(H)) T1 eZ-=1
K ((-((U-0)))
1 Σ
KH (ū (U-Ú.).
Th1
where K--) is a kernel function and H = diag(h11... ha) is diagonal matrix of the bandwidths which controls the amount of smoothing. Let KT denote the scaled kernel with bandwidth matrix H and write KH as K₁, when h = =... = ha = h. In particular, we assume ((X) = G(X())) x X ((X())), where G(-) is a univariate kernel f G(x)ddr = 1. We call kernel an a order kernel if f o (x)(x = 0 for any integer 1 Σ a and f Ix: (())())) de < oc. With the kernel density estimator, the final estimator of PCT in Algorithm can be written
PCT Ei Y,, - Z,, Kr (I( - Φ 1) E Zz, )H (5 (Xx,, 1 (xx,,,,, (),,,)
(4)
Another popular nonparametric density estimator is the projection density estimator. Given a se- ries of orthonormal basis functions v)x()), 1,..., x.. f (T) can be decomposed as f (Ū) = n (U), where the coefficients are defined as ri f (U))) The projection method seeks to estimate (T) with the first basis functions, i.e.,
AB( Σ Fnbr((), where A: 11 :7,=1
1
Σ vi(U.).
5

---

Published as a conference paper at ICLR 2024
The projection density estimator then lead to the final estimator of PCT
pCT Ei Vi (1 Zz, K. (Xx;) 2i Emib (1 (,),,)(((X,))), ((x,,) 2)
(5)
where (Z, (vz(-) :1=))/=11, 1..... L} Giné & Nickl 2016).
Σト the (1) denotes projection kernel defined by the orthonormal basis
Although Algorithm seems to suggest that f is designed to estimate f at first glance, we would like to emphasize again that our ultimate goal is to estimate PCr instead of ff. so the choice of tuning parameters, H in equation and I, in equation[] shall rely on the bias and variance trade-off in the final estimator ÅCT. The straightforward form of the estimators for PCT after applying the uniform transformer allows more accurate trade-off between the bias and variance in estimating per. We leave the detailed discussion of the tuning parameters H and L to Section
3.2 ROSENBLATT'S UNIFORM TRANSFORMER WITH EMPIRICAL DENSITIES
There are various options transform distribution into uniform distribution. In this section, we focus on a uniform transformer proposed by [Rosenblatt (1922). More concretely, given the density
of covariates in the control group, (xX) we consider the following transformation $ Ω - [0, 1jd
(())(1) (X)(X(1) I =(1)),
(())(2) =Pc(X(2) I 412) X(1) = 2(1)):
(6)
((F) (4) =Po S ((a))(a-11 = ().-1),..... X(1) = =(()),
Rd is vector, X (X())...., ((a)) random vector with density fx( and the corresponding probability Pc- When X is continuous random vector, ((X) follows a uniform distribution on [0, 1] It is worth noting that the uniform transformer relies on the condition that X is continuous. some component of is discrete, a random perturbation can be added to the observed covariates in the preprocessing step, discussed in Brockwell (2007). In the following discussion, on discrete random variables, see Section C.2 in the Appendix. Clearly, the Rosenblatt's uniform transformer relies on the choice of the order X ((.):-..; ((a)- In practical applications, one could randomly select permutations of the orders and take the average of results.
where (Z)
assume X is continuous random vector; for more discussions
In practice, we usually do not have much knowledge about the density fx)X), so we have no access to the uniform transformer defined in equation 66 and need to construct the uniform transformer from the data. One natural way to construct the uniform transformer is to first estimate the density of covariates in the control group fc(X) by some estimator fx) and then define the uniform transformer based on fx( following the transformation in equation 6
Furthermore, the construction of uniform transformers becomes easier if the covariates have spe- cial correlation structure. For instance, equation 6 suggests that the uniform transformer & only relies on each marginal distribution of fx'(X) if components of X are mutually independent. In other words, suppose fc(X) (X) can be decomposed as fc(X)= (x,)(())) x x... x (X()()(a)). then it is sufficient to construct the uniform transformer by estimating each marginal distribution. this special case, we refer to the uniform transformer as a marginal uniform transformer. This idea can also be extended to densities with group-wise wise mutually independent structure.
Moreover, the construction of the uniform transformer in WUNT only relies on the covariates of control samples. This feature enables the integration of substantial amount of extra 'unlabeled' control samples (outcome of interest is not observed) which is available in lot of applications, such as analysis of electronic health record data (Gronsbell Cail 2018, Chakrabortty & Cail 2018). In such scenarios, large amount of unlabeled data can be used to accurately estimate the density (x) using any suitable density estimator. Subsequently, the uniform transformer can be con- structed based on this estimate and applied to the 'labeled' data set to compute the weights designed for treatment effects estimation.

---

Published as a conference paper at ICLR 2024
3.3 ADAPTIVE UNIFORM TRANSFORMER
Section 3.2 mainly focuses on Rosenblatt's uniform transformer and its empirical version with a density estimator. We provide different angle construct an empirical version of Rosenblatt's uniform transformer in this section. To illustrate the idea, we start with one-dimensional case. When d' = I, Rosenblatt's 's transformation is defined by the cumulative distribution function of X in the control group. The cumulative distribution function can naturally be estimated by its empirical distribution, i.e., Pc.n (T) nu Zi T(X; S F), where f(.) is an indicator function. we plug Pc.. in Rosenblatt's 's transformation 4, the resulting transformation maps {X, Z, = 0} to {1/mo..... 1). In other words, Pc., help transform (X, Z; 0} to the grid points between 0 and 1. The benefit of this transformation is that it does not rely on estimating fx(),, it is computationally simple. Can we construct a uniform transformer in a similar fashion for a multi- dimensional case?
To apply this idea for a d-dimensional covariate X, we need to partition the data points evenly to d-dimensional grids. Without loss of generality, assume the support of density is Ω = [0, 1] and no = No6 for some positive integer No in this section. Based on ) we define the following data-driven partition of 2, Ω = -
fa- in x x x... x Li jj...r-.- Here, each Qj..z......jaa is a cube and each T Je....... for le S d is an interval. We construct the data- driven interval [,, Je. in a hierarchical way. We first construct { 1
.a...aa = No
1j.=1, which is partition of [0, 1] such that there are exactly No-11 points in each Lj, x [0, 1j After construction of {1 > are ready to construct j2 J2 For each jn. (2) is partition of [0. 1] such that there are exactly No Ad-2 points in each 1 x Tiiv x [0, 1|d-2. The rest of Ja...... can be defined in similar way. In doing so, each cube Ja..... contains exactly one point. The idea is illustrated with example of 9 data points on [0, 1]² in Figure iin Appendix A
Then, we can easily approximate the cumulative distribution function and conditional cumulative distribution functions based on the partition to form empirical Rosenblatt's uniform transformer. With this construction, we can put exactly mass 1/ng for each cube Qj..i........ The formal result is summarized in the following proposition. The proof is included in Appendix
Proposition 1. Let LD be the uniform transformer defined in equation 6 by replacing fo(X) with x... x..x5 S X(a) M(I, i.maa))) ). for any X E Qjs.ja---aa
f(X) Tii....ig S (( (xn-M(l
1521
Vii...iial
where X (X(1)...., ((a))))) 1-1
represent the volume of cube the length of an interval and (()) is the middle point of an interval. Here, S(.) is smooth kernel function defined on [-0.5,0.5] such that S(-0.5) S(0.5)
S(x) > if x E (-0/5, 0.5) and _ S(x)dr = 1. Then & D satisfies the following properties: For each cube 52...r.... S J1,...,Ja S No. we have j1-1 51 No No x...x No No
d D (Q
j--1 ia
2) & is smooth map.
This proposition suggests that ΦD is able to map the covariates in the control group to an approxi- mately uniform distribution on [0, 1]4, and we call it an adaptive uniform transformer. In particular, when is on the real line (d = 1). &D can been seen as a smoothed version of the empirical cu- mulative distribution function of {X, )x2,)=20=0 When there is no integer N, such that vig = =Nd. we can choose Ng as the largest integer such that No < no and follow a similar procedure as above to distribute the data points evenly in the grids.
THEORETICAL PROPERTIES
We now turn to analyze the theoretical properties of our newly proposed framework. In this section, we consider uniform transformers introduced in Section 3 (either uniform transformer constructed with separate data set or an adaptive uniform transformer) and study the performance of both kernel and projection density estimators. In particular, investigation focuses on Hölder class (van der
Vaart & Wellner 1996). H (2) = f Ω [1llllo S M where the norm || 1 is
7

---

Published as a conference paper at ICLR 2024
defined as || m Sla] supzen | [(z) + max k|=[a] suPa, [27) s(
[ -211 [a]
Here, k (ki...., .a.....,.. ka) with |k] b + +.... ka and the differential operator defined as D
@l We assume the basis functions {&/ : 1 1.....oo) oo) in the projection density estimator
form an orthonormal basis and satisfy
Iral S M,
and
for any I, sup [y((a) S M2 I
where M2 and M2 are some constants and ri is the coefficient of some given function g E H 11) i... -(]) Ei (T). For example, the wavelet basis satisfies this property GGnn
& Nickl 2016 Liang, 2019). When Zi = O, we write Y, = (X.) + eis where E(e:]X,) 0.
Through this section, we assume
:- < S
for some constant o².
(8)
We first investigate the performance of the proposed estimator when the uniform transformer in equation 6 is defined by some fixed density fx(X), which might be different from fc(X). The following theorem characterizes the convergence rate of the estimator.
Theorem 1. Let üCT be the estimator defined in equation 4 with an a order kernel or the one
defined in equation 5 with basis function satisfying equation 7 Suppose the uniform transformer is
defined in equation 6 with some density fc Assume Fee E (1o, for E H. 11)) and
fC H²' (10, 1) with 0 <, a, < <% We further assume conditions equation [ and equation & hold
and (X) = 0 when X is at boundary of [0, 1] if we use estimator defined in equation If we
choose h1 = =... = ha = h = n -2/(dd2((a- in equation 4 oor n 2dd/(d+2(a in equation 5
then there exists constant Co such that
E(jer PCT)² S Co (n
(a++8) 24310+0] +n + +1²). +n=
where A is the difference between fo(X) and fx(X) ) in La norm, i.e., = (1
Theorem Suggests that the performance of new estimators depends on the sum of smoothness levels of the response and density functions. Notably, they can still work well even when the response function is non-smooth (a < d/2). In addition, it is worth noting that the optimal choice of tuning parameter h or relies on the levels of smoothness of both the response and density functions. The n n/((d+2)) can lead to suboptimal
optimal choice for estimating ff (h = 1/(d+28) or convergence rate in estimating /Cr. In other words, the best way to estimate fo (hence the weights w:) may not necessarily lead to the best estimator for PCT.
An immediate result of Theorem I characterizes the performance of the proposed estimators when the density of covariates in the control group fx) known. Specifically, with known fc(X), the convergence rate of the estimator in equation
4 or equation 5 is
1(+88
n
+ nn
If the density of covariates in the control group fx) is not known, it can be estimated by some density estimator fc (X) with separate data set, discussed in Section 3.2 In this case, the uniform transformer is constructed with this separate data set. The following corollary can further characterize the performance of the new estimators.
Corollary 1. Let fx( in Theorem 1 be density estimated by N = cn' control samples for some t 2 and constant > If (X) fc(x)]l2 I N==/(d+22) for some constant K, we have the
following results. When a + 3 < d/2 and > 2(a + 3)d/(td + (2t - 4)( + BB)) then
E(ACT- (CTT)² < On 243(aT0)
When a + + d/2 and x > d/2(t - 11). then
Vī(peT- - HCT) + N0,,
VV where (O, 1) is standard normal distribution, P- P(Z 1) and the variance V is V (.). (T)) dF (o dž 4 (ʃ (E)))
(1)))
8

---

Published as a conference paper at ICLR 2024
We can conclude from this corollary that the converge rate in equation Dis still achievable as long fc(X) ) be estimated accurately. Similar many other popular weighting methods (Hahn 1998
Hirano et al. 2003 Chan et al. 2016 Fan et al. 2023), PCT is a v/n-consistent estimator when a + 3 d/2. Now, show that the converge rate equation 9 is actually sharp in terms of minimax
optimality. More specifically, we consider the following family of data distribution (X, Z) F in Fa,3 = {F: HC H.
H ((0, 1 and equation equation hold].
Theorem 2. Assume fc(X) is known in advance that 4 is defined based fc(X). Consider estimating poT on Fa. with B >> O. Then there exists constant Co such that
inf sup B(Acr -MCT)2 2 co (n ACT FEFo.
4/ 19) aT DDj + +n=1)
Theorem 2 characterizes the difficulty of estimating the treatment effects in the ideal case when fc(X) () is known. In general, if (X) unknown, the problem becomes harder, so the convergence rate will be lower bounded by the result in Theorem 2 Notably, Theorems [ and together show that the converge rate in equation 9 the minimax optimal for estimating pCr with known (x)) That is, if we know the density fc( (X) beforehand, the minimax optimal rate of estimating TATT is
inf sup EffATT - TATT)² x 72 + 71 FATT FEFo.
We now show that the new estimator is still reliable without assumptions on fc(X) if the uniform transformer constructed as Proposition] [
Theorem 3. Let PoT be the estimator defined in equation with an a 3 order kernel or the one
defined in equation 5 with basis function satisfying equation Assume the uniform transformer is defined in Proposition
1. Suppose 3 HD 1] JT E H ([o, 114) with arbitrary a., 3 >> 0 and conditions equation equation ( &hold. For the kernel estimator, we choose bandwidth hı ...
ha = h satisfying n + oo and h + 0. For the projection estimator, we choose the number of basis L satisfying n² I.- oc and L 0o. Then,
PCT 7p PCT,
Theorem 3 shows that this new uniform transformer can help build a consistent treatment effect estimator with no assumption on (X) and very mild conditions on ff and p Proofs can be found in Appendix[D] In this section, we focus mainly on the estimation of PCT. All these results can naturally lead to the conclusion for the average treatment effect on the treated group, TATT- Besides the theoretical results, we also conduct the numerical experiments in Section Bl the Appendix.
5 CONCLUDING REMARKS
In this paper, we propose novel framework of weighting methods for treatment effects estima- tion, named Weighting by Uniform Transformer (WUNT). Unlike the existing weighting methods, WUNT employs a data-driven uniform transformer the observed covariates, effectively transform- ing the control group's covariate distribution to a uniform distribution. This transformation allows us design the weights in WUNT specifically tailored for treatment effects estimation, resulting in a straightforward kernel- based U-statistic as the final estimator. We delve into the theoretical prop- erties of the newly proposed framework under a nonparametric setting. Our investigation shows that, with weights chosen by WUNT, the weighting method can achieve the minimax optimal rate of estimating the average treatment effect on the treated group, even under low regularity conditions. Notably, the tuning parameter in WUNT needs to be chosen based on the smoothness levels of both the response and density functions to achieve an accurate trade-off between bias and variance un- der low regularity conditions. In cases where the smoothness levels unknown, Lepski method (Lepski) 199T; 1992) can help WUNT select the tuming parameter in a data-driven way. Additional implementation guidelines provided in Section C.3 of the Appendix.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
The authors are grateful for the helpful comments from Paul R. Rosenbaum, Dylan S. Small, Eric J. Tchetgen Tchetgen and Jose R. Zubizarreta. S. Wang would like to acknowledge support for this project from the National Science Foundation (NSF grant DMS-2113458).
REFERENCES
Bickel and Y. Ritov. Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhya: The Indian Journal of Statistics, Series A, pp. 381-393, 1988.
A. E. Brockwell. Universal residuals: A multivariate transformation. Statistics & Probability Let- ters, 177344:1473-147. 2007.
D. Bruns-Smith, O. Dukes, A. Feller, and E. L. Ogburn. Augmented balancing weights as linear regression, arXiv preprint arXiv:2304. /4545, 2023.
T. T. Cai and M. G. Low. Testing composite hypotheses, hermite polynomials and optimal estimation of a nonsmooth functional. Annals of statistics, 39(2):0112-141,, 2011.
A. Chakrabortty and T. Cai. Efficient and adaptive linear regression in semi-supervised settings. The Annals of Statistics, 46(4):1541-1572, 2018.
K. C. G. Chan, S. P. Yam, and Zhang. Globally efficient non non-parametric inference of average treatment effects by empirical balancing calibration weighting. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(3)x673-700, 2016.
J. Fan, K. Imai, H. Liu, Y. Ning, and X. Yang. Optimal covariate balancing conditions in propensity score estimation. Journal of Business & Economic Statistics, 41(1):97-110, 2023.
E. Giné and R. Nickl. A simple adaptive estimator of the integrated square of density. Bernoulli, 14(1):47-61, 2008.
E. Giné and R. Nickl. Mathematical foundations of infinite- statisticul models, vol- ume 40. Cambridge University Press, 2016.
E. Giné, R. Latala, and J. Zinn. Exponential and moment inequalities for u-statistics. High Dimensional Probability 11, pp. 13-38. Springer, 2000.
B. S. Graham, C. C. de Xavier P., and D. Egel. Inverse probability tilting for moment condition models with missing data. The Review of Economic Studies, 7933):0553-1079, 2012.
J. L. Gronsbell and T. Cai. Semi-supervised approaches to efficient evaluation of model prediction performance. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80(3): 579-594, 2018.
J. Hahn. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica, pp. 315-331, 1998.
J. Hainmueller. Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political Analysis, pp. 25-46, 2012.
K. Hirano and G. W. Imbens. Estimation of causal effects using propensity score weighting: An ap- plication to data on right heart catheterization. Health Services and Oulcomes Research Method- ology, 2(3-4):259 278, 2001.
K. Hirano, G. W. Imbens, and G. Ridder. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4): 11111161111999 2003.
D. A. Hirshberg and S. Wager. Augmented minimax linear estimation. The Annals of Statistics, 49 (6):3206-3227, 2021.
D. G. Horvitz and D. J. Thompson. A generalization of sampling without replacement from a finite universe. Journal the American statistical Association, 47(260):663-633 1952.
10

---

Published as a conference paper at ICLR 2024
K. Imai and M. Ratkovic. Covariate balancing propensity score. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(1):243-22) 263, 2014.
G. W. Imbens and D. B. Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
N. Kallus. Generalized optimal matching methods for causal inference. The Journal of Machine Learning Research, 21(1):2300-23) 2020.
Kang and L.. Schafer. Demystifying double robustness: comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22(4):523- 2007.
B. Laurent. Efficient estimation of integral functionals of density. The Annals of Statistics, 24(2): 659-681, 1996.
O. Lepski. On problem of adaptive estimation in gaussian white noise. Theory of Probability & Its Applications, 35(3):454446 1991.
O. Lepski. Asymptotically minimax adaptive estimation. i: Upper bounds. optimally adaptive esti- mates. Theory of Probability & Its Applications, 36(4):682 662-6777,77 1992.
O. Lepski, A. Nemirovski, and V. Spokoiny. On estimation of the norm regression function. Probability Theory and Related Fields, 113(2):221-253, 1999.
Liang. Estimating certain integral probability metries (IPMs) is as hard estimating under the
IPMs. ERN: Statistical Decision Theory; Operations Research (Topic), 2019. URL https:
7/api. semanticscoola. wwtphhoolaarrrrooooooooeeo
W. K. Newey, Hsieh, and M. Robins. Twicing kernels and small bias property semipara- metric estimators. Econometrica, 72(3):947 2004.
M. Robins, Rotnitzky, and Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal the American statistical Association, (9(427):446886)) 1666 1994.
J. M. Robins, M. A. Herman, and B. Brumback. Marginal structural models and causal inference in epidemiology. Epidemiology, 11(5):550-560, 2000.
J. M. Robins, I.. Li, E. T. Tchetgen, and A. van der Vaart. Higher order influence functions and minimax estimation of nonlinear functionals. In Probability and statistics: essays honor of David A. Freedman, pp. 335 421. Institute of Mathematical Statistics, 2008.
J. M. Robins, F. T. Tchetgen, I.. Li, and A. van der Vaart. Semiparametric minimax rates. Electronic Journal of Statistics, 3:1305, 2009.
J. M. Robins, L. Li, R. Mukherjee, E. T. Tchetgen, and A. van der Vaart. Minimax estimation of a functional on structured high-dimensional model. The Annals of Statistics, 4555):195--1187, 2017.
R. Rosenbaum. Model-based direct adjustment. Journal of the American Statistical Association, 823393) 1987.
P. R. Rosenbaum and D. B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.
M. Rosenblatt. Remarks on multivariate transformation. The Annals of Mathematical Statistics, 23(3):470 472, 1952.
D. B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5):688, 1974.
Y. Shen, C. Gao, D. Witten, and F. Han. Optimal estimation of variance in nonparametric regression with random design. The Annals of Statistics, 48(6):3589-3618 2020.
R. Singh. Debiased kemel methods. arXiv preprint arXiv:2 2021.

---

Published as a conference paper at ICLR 2024
A. B. Tsybakov. Introduction nonparametric estimation. Springer Science & Business Media, 2008.
A. W. van der Vaart and J. A. Wellner. Weak convergence and empirical processes with application to statistics. Springer Verlage, 1996.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polo- sukhin. Attention is all you need. In Advances in Neural Information Processing Systems, vol- ume 30, 2017.
Y. Wang and J. R. Zubizarreta. Minimal dispersion approximately balancing weights: asymptotic properties and practical considerations. Biometrika, (07(1))93-105) 2020.
R. K. Wong and K. G. Chan. Kernel-based covariate functional balancing for observational studies. Biometrika, 105(1):199-211 2018.
Q Zhao. Covariate balancing propensity score by tailored loss functions. The Annals of Statistics, 47(2)9955-993, 2019.
Q Zhao and Percival. Entropy balancing is doubly robust. Journal of Causal Inference, 5(1), 2016.
R. Zubizarreta. Stable weights that balance covariates for estimation with incomplete outcome data. Journal of the American Statistical Association, 1106111;910-922, 2015.
A APPENDIX EXAMPLE FOR THE CONSTRUCTION OF ADAPTIVE UNIFORM TRANSFORMER
In this section, we illustrate the construction of an adaptive uniform transform using a toy example with 9 data points on [0, 1 in Figure
(
X (2)
Figure An illustrative example for the construction of an adaptive uniform transformer when d= d= 2.
B APPENDIX NUMERICAL EXPERIMENTS
In this section, we study the numerical performance of our proposed framework by carrying out several simulation studies to estimate the average treatment effect on the treated group (ATT).
B. COMPARISON OF UNIFORM TRANSFORMERS
In the first set of simulation studies, compare four ways to construct the uniform transformer from the control samples. More specifically, the adaptive uniform transformers are constructed in four different ways according to the following: if extra 'unlabeled' control samples used
12

---

Published as a conference paper at ICLR 2024
not, and if the uniform transformers are based joint marginal distribution. To simu- late the observed data, we draw (W..... W.. ~ [ 0.5), Σ) in the treated group and (Wi,.., Ws) ~ [N. 0), Σ) in the control group, where N represents the normal distribution and each entry of the covariance matrix E is defined Σij = pli-al, We vary p from O, 0.1, 0.2 and 0.3. The observed covariates of each sample X (X1, X5) that X, exp(W)) + W We consider two models for the outcome of interest: Y1 = W?W? w3 Eim. W; and Y2 10 10 W; + 100 II. sin(2x + 100 II cos (TT( c2. where e; (0) follows independent standard normal distribution. We consider two density estimators the kernel density estimator and the projection density estimator, and denote their corresponding estimators of the ATT by RK ATT and + AATT- respectively. The sample size is 500 for the treated group and 1000 for the control group. The extra 'unlabeled" control samples are drawn in the same way with sample size 10000. The performances of different uniform transformers are evaluated by bias and root mean squared error (RMSE), calculated from 500 replications of simulation experiments.
The results are summarized in Table These results show that when the covariate distribution of control samples is independent, the marginal uniform transformer works slightly better than the joint uniform transformer. This observation makes sense because when fc is an independent distribution, is sufficient construct the uniform transformer for each marginal distribution, as discussed in Section 3.2 On the other hand, the joint uniform transformer is more robust when the observed covariates are correlated. addition, the uniform transformer works in a better way when extra data available.
Table 1: Comparison of uniform transformers under different covariance matrices.
No extra data
With extra data
Joint
Marginal
Joint
Marginal
p
Bias
RMSE
Bias
RMSE
Bias
RMSE
Bias
RMSE
-0.09
0.49
-0.11
0.47
0.06
0.48
-0.03
0.45
0.1
-0.16
0.56
-0.26
0.59
0.00
0.53
-0.18
0.54
AK
ATT
0.2
-0.22
0.64
-0.36
0.74
-0.06
0.60
-0.3
0.67
0.3
-0.30
0.71
-0.36
0.88
-0.13
0.68
-0.37
0.80
Yi
0
0.18
0.56
-0.06
0.5T
0.03
0.53
-0.06
0.52
AP
0.1
0.13
0.61
-0.11
0.58
0.00
0.58
-0.11
0.58
TATT
0.2
0.11
0.68
-0.14
0.66
-0.01
0.66
-0.14
0.66
0.3
0.09
0.75
-0.16
0.75
-0.02
0.74
-0.16
0.75
0
1.15
4.31
1.46
4.16
1.74
4.36
0.82
3.96
0.1
1.24
4.22
0.47
4.08
1.84
4.43
0.28
3.98
ATT
0.2
1.13
4.14
-0.82
4.17
1.77
4.32
-0.55
4.02
0.3
0.88
4.05
-2.31
4.61
1.46
4.10
-1.59
4.19
Y22
0
2.19
4.61
1.98
4.70
1.83
4.48
1.96
4.69
PP
0.1
2.24
4.57
1.86
4.57
1.74
4.42
1.84
4.56
TNTT
0.2
2.23
4.57
1.71
4.44
1.65
4.31
1.69
4.43
0.3
2.13
4.43
1.51
4.19
1.55
4.16
1.49
4.18
B.2 COMPARISON OF ATT ESTIMATORS
The second of simulation studies compares the newly proposed estimators with other existing methods under the above model. The four new estimators we consider here are: uniform transformer on joint distribution kernel density estimator, uniform transformer on marginal distribution + ker- nel density estimator, uniform transformer on joint distribution + projection density estimator, and uniform transformer on marginal distribution + projection density estimator. We compare them with the inverse probability weighting estimator (IPW) with the propensity score estimated by random forests with R package randomF orest. covariate balancing propensity score (CBPS) proposed by Imai & Ratkovic (2014) with package CBPS, empirical balancing calibration weighting (CAL) by
Chan et al. (2016) with R package ATE, and stable weights (SBW) proposed by Zubizarreta (2015) with R package sbw. We still adopt bias and RMSE, caleulated from 500 replications of simulation experiments again, as our measure of performance of these estimators for ATT. The data is generated in the same way the first set of simulation studies with p 0. Results summarized in Table
13

---

Published as a conference paper at ICLR 2024
suggest that new proposed estimators perform better than the other methods in terms of bias and RMSE..
Table 2: Comparison different ATT estimators on model and Y2-
Y
Y
Bias
RMSE
Bias
RMSE
Kernel+Joint
-0.10
0.51
0.94
4.38
Kernel+-Margina
-0.11
0.47
1.29
4.21
Projection+Joint
0.20
0.58
2.01
4.57
Trojeition+MMarinaall
-0.04
0.53
1.85
4.65
IPW
0.71
0.85
6.18
7.15
CBPS
1.11
1.96
3.49
6.00
CAL
1.07
1.64
3.93
5.99
SBW
0.45
0.84
3.4
5,41
B.3 COMPARISON OF ATT ESTIMATORS WITH DIFFERENT SAMPLE SIZES
In the third set simulation studies, we further compare the eight estimators ATT in the second set of simulation studies and assess their performance with different sample sizes. In this set of simulation experiments, the data simulated based on the example in Kang & Schafer (2007). More concretely, draw W (Wi, , W2, Wa. W4) from ((00, 0, 0, 0), /). where
4 identity
matrix and consider the following two models of the outcome of interest: Y,- = 210 - 27.4W, + 13.7W, + 13.7W3 2W2) (cxp(()) + ( + +2W + W4 c4. Here, G; also follows independent standard normal distribution. Each sample is assigned to the treated group with probability (i.e., true propensity score) 1/(1 + e 0.5W', 2 + 0.252; + 0. O.1. Instead of observing the covariates W, we are able to observe only the transformed data X1 exp(W_/2) X₂ W2/(1 + exp(W1)) 10, X3 = (W1 25 + 0.6) and X, = (W₂ + Wa + 20) In order to assess the effect of sample size. we vary it from 1000, 2000, and 5000. Similar to the previous two simulation studies, bias and RMSE based on 500 replications of simulation experiments are summarized in Table 3 The results show that
13 + E3 (the same with Kang Schafer (2007) ) and Y, (4W1 +
estimators generally outperform other existing methods. The only exception is the kernel density estimator equipped with a uniform transformer based on marginal distribution. The reason is that the observed covariates are highly dependent and the kernel density estimator seems to be sensitive to the correlation among covariates. Table 3 also suggests that the new estimators based on the kernel density estimator can constantly reduce the bias as the sample size increases.
B.4 COMPARISON OF COMPUTATION COMPLEXITY
We compare the computation time of these eight methods in the last set of simulation studies. In particular, we consider two optimization solvers for SBW: quadpros (the default choice) and mosek (a commercial solver available from ht :ttttp::/wwrr mosek. com/). We record the average time from 10 replications of the model Y3 with sample sizes 1000, 2000, 5000, and 10000 and summa- evaluated with the same laptop (Intel Core i5 @2.3 GHz/8GB). From Table 4 we can conclude that the new estimators based on the projection density estimator be computed efficiently. The main computation obstacle of the new estimators based on the kernel density estimator is the kernel U statistics which have O(n²) computation complexity. It is also interesting to note that the uniform transformer (<c 0.66s even when n 10000) and thus can be applied on a larger scale dataset.
rize them in Table 4 All these algorithms
be constructed in very short time
C APPENDIX DISCUSSIONS
C.I AVERAGE TREATMENT EFFECT
In the paper, we mainly focus on the estimation of average treatment effect on the treated group. All the methodology and theoretical properties be readily generalized to average treatment effect.
14

---

Published as a conference paper at ICLR 2024
Table 3: Comparison of different ATT estimators model Y3 and V4-
1000
2000
5000
Y
Bias
RMSE
Bias
RMSE
Bias
RMSE
Kernel+Joint
-7.90
8.21
-6.32
6.51
-4.50
4.62
MergiI-Marginal
-9.88
10.16
-9.46
9.62
-8.88
8.96
Projection++Joint
-4.26
4.48
-4.39
4.49
-4.27
4.30
Projection-Marginall
-4.00
4.15
-4.01
4.09
-3.98
4.00
IPW
-10.13
10.26
-9.96
10.02
-9.61
9.64
CBPS
-5.35
5.56
-5.40
5.51
-5.31
5.37
CAL
-4.36
4.49
-4.43
4.49
-4.37
4,40
SBW
-7.22
7.32
-7.40
7.44
-7.4
7.42
Y4
n
1000
2000
5000
Bias
RMSE
Bias
RMSE
Bias
RMSE
Kernel+Joint
-0.40
0.44
-0.31
0.34
-0.22
0.24
Acrnel+Margina
-0.67
0.71
-0.66
0.68
-0.65
0.66
Projection+Joiin
-0.38
0.41
-0.38
0.39
-0.37
0.38
Projection+Marginnn
-0.38
0.4
-0.38
0.39
-0.39
0.39
IPW
-0.63
0.64
-0.61
0.62
-0.58
0.59
CBPS
-0.56
0.59
-0.57
0.59
-0.57
0.58
CAL
-0.46
0.49
-0.47
0.49
-0.48
0.48
SBW
-0.64
0.66
-0.66
0.67
-0.66
0.67
Table 4: Comparison of different ATT estimators in terms of the computation time, which is shown in seconds.
n
1000
2000
5000
n
10000
Kernel+Joiin 0.40
1.36
7.98
31.08
Margin+aarginal
0.36
1.31
8.01
32.19
Projection+Joint
0.06
0.12
0.30
0.66
Projection+Maaginnl
0.04
0.11
0.49
1.85
IPW
0.33
0.73
2.24
4.61
CBPS
0.31
0.64
1.67
4.23
CAL
0.14
0.27
0.67
1.41
SBW(quadpros) SBW(mosek)
0.14
1.19
20.60
152.18
0.03
0.04
0.08
0.19
Recall the average treatment effect is
TATE = PTA - PCA / (r(
order to estimate μTA and PCA separately, we can construct two uniform transformers based on treated samples (i,,= and control samples [X,h:z.=o respectively, which are named 4y and Pc. After applying transformation oy and & all data, can pick suitable density estimators for for (X) and (X), which are density of (X) and (x))). The corresponding algorithm is For example, if we adopt the projection density estimator, the resulting
summarized in Algorithm 2
estimator is
ÍXTE Enjtio Ye, Zi, Kr (T
L (Xx)))
En Z,, K2 ( (X,, Pr (X,,,)
En (1 Z,, KL (x
C.2 UNIFORM TRANSFORMER FOR DISCRETE VARIABLES
Most of this paper focuses when the observed covariates continuous random variables. How- ever, we also observe some discrete or categorical covariates in practice. Motivated by Brockwell 20077, first transform the discrete random variable continuous random variable. For the
15

---

Published as a conference paper at ICLR 2024
Data: ((x,, 24 Y)J=
Result: Weights wy and an estimator of TATE-
Construct the uniform transformer Dr and &e by (Xih:2,11 and {Xiliz,00
Estimate f (X) and (*e(() from (fr(X:)) and (x,((;));
Evaluate the weights by
j (x)(()))) (0c
Z, =
W1
for (OT((x))) (Xi))/ Ex:2,=1
(x))//E:,,=>) l f (TTT(x)))
Z,
Estimate TATE by
ÍATE =
w;Y;
1=1
Algorithm 2. Weighting by Uniform Transformer for Average Treatment Effect
sake of concreteness, we assume X is a binary variable in this section, i.e., X E {0, In order to construct a continuous variant of X, we add a random perturbation to X
x
x= aR x=1'
R
x
where R is an independent uniform random variable on [0. 1] and is some positive number. It clear that X is continuous random variable and satisfies strong ignorability of the treatment
assignment in equation ! if X satisfies it. we choose a P(X 1)/P(X = 0), the resulting distribution of X also smooth one. Therefore, it is sufficient to consider uniform transformer constructed by X. It is clear that the resulting uniform transformer from data X and the resulting weight and estimator highly rely on particular realization of R. In order to reduce the effect of particular realization of R, also suggest considering multiple independent copies of R (i.e., independent copies of X given X), evaluating the weight on each copy of X and taking the average of these weights as the final weight.
Based on the above discussion, we consider very simple preprocessing step for discrete variable in practice. Specifically, we transform the discrete variable to its rank by breaking ties randomly. In
this way, we obtain similar variant of X with the number of elements in set. Then, we can regard these ranks as continuous random variable and evaluate uniform transformer and weights on them directly. Again, in order to reduce the effect of randomly breaking ties, we can repeat the process multiple times and take the average of the results.
# #(X = #) = 0}, where # represents
C.3 IMPLEMENTATION SUGGESTIONS
We present several uniform transformers and density estimators in Section and now would like to give some practical suggestions on them. In terms of uniform transformer, we recommend the newly proposed adaptive uniform transformer defined in Proposition[1 [ because a) its computational complexity is On)); b) there is no need for sample splitting and c)the resulting estimator does not rely strong assumptions of (X). We can apply the adaptive uniform transformer in two differ- ent ways based on the knowledge of (x). More concretely, if we know fx(XX is approximately mutually independent, then construction of the adaptive uniform transformer based on marginal dis- tributions suggested. On the other hand, if the distribution not mutually independent, we suggest constructing the adaptive uniform transformer based on the joint distribution directly.
We can choose different density estimators according to our knowledge of (X) and f (X). If the specific form of response function and density is not known, we suggest the kernel density estimator. The reason is that a) its implementation is easy; b) there is no need to choose basis functions and it performs relatively robustly. If we know p (X) f (x) can be represented by few simple
16

---

Published as a conference paper at ICLR 2024
basis functions in advance, suggest choosing the projection estimator for density. Based few basis functions, the resulting projection estimator simple and accurate, and can be rapidly computed. We summarize the choice of uniform transformers and density estimators in Table
Independence of fc
Yes
No
Yes
Projecil
Joint Projection Joint + Kernel
Knowledge of u ft
No
Marginal + Kernel
Table 5: Summary of implementation suggestion.
When smoothness levels are unknown, we introduce a data-driven way to select tuning parameter by following Lepski method (Lepski 1991 1992). See also Giné Nickl (2008). Write por(h) as the estimator defined in equation when the bandwidth h. ah,, n) L/(nhd/2) and d(h) v /log(hi/h).. Define a grid of bandwidths
log r n2//d 1/a
logn
log n
H
hn
l/d hx-1 h2/p, -1,2... 1.2.....
where p > 1. The bandwidth estimator
= max h H
|ACT (h) (cT(g)) S ())n))(()), <g< h, E म}
Similarly, also define data-driven way to select the number of basis function in the estimator but omit the details.
defined in equation
5
D APPENDIX - PROOFS
To distinguish from the constants that appeared in the previous sections, we shall use the capital letter C' and lower case letter to denote generic positive constants that may take different values at each appearance.
D.1 PROOF OF PROPOSITION 1
To study the properties of & first calculate the marginal distribution of f(X). For any X Lns the marginal distribution is
Nod-1
X M(G
((X)= = f(x)xx ... ..
S
110
115al
Similarly, for any X (X(1)..... X(n)) E Qjimnnn = Lij Lintz X... X XLi Limman where & S d, the marginal distribution can be written
8(8) /
F(x))X
... .X.)
Nd-k k
X(o) M M
II S
Qirenii 1100
is clear that
(X
1
No
Then, we can define conditional cumulative distributional function
X (X)dx
{(X1))
and Xx(Xx X(1-1))=
X(m) (i(X X(b-1), X) dX. (x).(X
17

---

Published as a conference paper at ICLR 2024
The transformation D is then defined ( (X)),, ..... (a(X(a) ((1) (,
We are now ready to verify the first property. Suppose X E Qiim.. Since ((1) E Ti
ji
No - h-1
P(x)dxsA(Xx))) (X))XX S (((())) <
h Σ F(X)dX= j1 1=1
No
Similarly, because (() E Tmmndd
X(0)
(X()),..... X(k-1),
No Nd-k
k-1
X(s) - -M((inmnns))
IIs
S
Qii jt no
s=1
No
Jk
S
X(s)
Pirm.ee- 120
Similarly, we have
X(k)
No Nd-k
(jk 1)
k-1
X(() - M M(Jnnnnne)
>
S
Qu.n. 1100 s=1
Since
No Nd-k+1
R-1
X(s)
II S
1Q J....
Nu
s=1
we have
fu - No (0))
Jk
No
Thus, the first property is satisfied. function.
easy to check the second property as f(X) is a smooth
D.2 PROOF OF THROREM 1
Proof for kernel density estimator
Recall estimator be written
PCT En Y1 - Z,, Ki, (x(,,) -Xx,,,,
En (1 (o(X,) ()
In this proof, we write U, = ((X)) Then, we can define P P(Z = 1),
Bu P(I-
and
8r P(1 P) (8(0)
We can also define
To nnn 1) 61.12=1
Y1 (,,,, -U,,)
and
Tr. n(n 1 f1-12=1
1
Σ
(1 Zz, Zn)K((ū -..))
18

---

Published as a conference paper at ICLR 2024
The proof is divided into three steps.
Step In this step, we show that
(c(10
B(TU 001 sc
11//d + + h2(0+3)
and
Tv-0u-
1
Σ
)00)+0²
+0, (11
nhd/2 + -1
TV is actually a U-statistic, so we decompose Ty as
Tu- 8V
1
Σ
- U.g) 2x)
Ku,,, Y1
0v
n(n - 1)
$1
1
(1- -2,)K, - -U,)
(1 - (10-1//000000000000..
n(n
1) 11712
- P f Y1, (1 Kn,,, - ())+( + +Bu)
1 Σ
(1 (1- P)) / (((0))8(0)) Zz -U - -11 (1 + +00
71 12=1
+
1
Σ
a
P Y1, Z,),,,,, Ü)f+ Bu PY, (1 Z,, (U,) +00
n
1
Σ
(1 Bu)
+ 1=
n
PY, (1 ((,,)(((n)))))))))))))))) -
n 11=1
+ OU Ou
=A + A2 + Ag + As + As - As,
where Or = P(1 P) -000000000000000000000000
ai', We now bound each term. For
A1, we have
C E(Y, (U, 0,,))² < n(n 1)hed
C
E(A <
n 1)
In terms of A2, we have
B(AB) -C - - -C(())))((0..)
C
- E
(HESC @ -(
2h
19

---

Published as a conference paper at ICLR 2024
Here, use the fact that y and @ represents convolution. With similar techniques, have
C
-(x: -Y,
EE(A3)
11
C
E YT, s km(U,)) 17 -3((0)-
E(?)) & -4
n
Ch
Applying the central limit theorem to As and Aş yields
1 (1-P (() -20u
Vn(A1 + As)
Vn
1=1
where
(Ivé
[ 10
[[v 22 S(02 +
x2
No&
x
(fnerer8))
"V P2 P)2
1-P
For the Ac. we apply the similar technique in Giné & Nickl
2008
2016
/ -
| - - -1((0)dUaa
/ 1 -v) -fa(U)aūaV
K(VIFF @ ( 3 @ (0)oV.
Here. (+) (-z) and represents convolution. Since (Z) H 11d) and
(T) H 1 we can know fi & && E H ([0, 1Ja). Then,
A < 1048
Putting A,, 1...., together yields the conclusion. Step 2. In this step, we work on Tz to obtain
E 02)2 S
n2 + n
1
n2(
and
log log n
F
Tr. -0. > Cn (V
nhd/2 10
20

---

Published as a conference paper at ICLR 2024
where 82 P(1 decompose Ty as
and is arbitrary large constant. We
Tt. -0 0z n 1)
E(
(1 7r,,) Kn(u,, - -Uig) - 11
f1
1
(1 (1 (1 (1- P) Kn(0 - (0)(0-0)f Zin
-
n
1)
$1712
/ (1 Zi, Kn(ū, - + +
-P
"
(r/( -1- -0
-
Σ
(1-P) P) (I-P)_KATTEB))&((d
(1) - -11-P)2,,88(U,,
12=
+ Σ (1-P)Z.8&(U.) + P(1 (0.r(U,)) -28)
-20))
+ 0r. Or.
where 0r P(1 P) f8(0)) - -'') (( dū' we apply similar proof in the last step, we can get
1 + 1,20+)))) n2 n
BTT, - 0,) S
Since Tr is U-statistics.
apply concentration inequality in Giné al. (2000) (also
Lemma 9 in [hen et al. 2020). By following the notation of Lemma 9 in Shen et al. (020)) we have
CISC, B₂ I Cynh 4/2, B3 S Ch d of SCC, oz I Ch
which leads
logn n
log nhd/2
1
F [r, 0z On
n1"
where 77 is an arbitrary large constant.
Step In this step, we put the results of steps and together. Define the event
A = IT' Öz > On cu((
(v logn +
log n2 nhd/2
Recall 0y and 02 are
0U = P(- f 1 and
02 P(1 P(1-P) 1
By variable transformation, rewrite them as
fo(x) fo(() (o()))()
fo(X)
OU P(1 PPI-P) P)
fo(x) Fo(x)
and
0 0, = P(1 P(1-P)
(x))x
fo(X)
we define 0g and 04 as
0g P(1 11l-P)on
and
06 = P(1 PP),
then we can know that
(0) (0) - 0y)² 5 (l) (o)11
This immediately suggests that
B(Ty - 0 < C
1
+
- +
1
1(( + - - 113)
nd
n
21

---

Published as a conference paper at ICLR 2024
and
E(T2 - -%)) 5C
1
+ -
1
+ +11fo(X)- -c(xx13
n2/pd
n
Then, we have
Tu Tí.
2
((CT) E(pot - ( =E
PCT
Tu
(TH Tr.
2 PCT
Iae)
=E
HOT
IA
+
E
T₁
We work above two terms separately. For the first one, when is large enough, we have
0%
Ty 2
on event A.
2
Thus,
(7z)
Tu T
2
2
μCT
LA
S
B(Ty - -TEMCT) )²
00"
2
2
$
(B(Tu -02) + E(TT -0)))
00
<
C
+ - + 2((+3)) + [|((()) - sc(x)11
n2hd
< C
2421++2]
1(a-8)
+n (1f fo(x8)13) -(((811))
The last step is due to h = n 2+20+1)
For the second term, we have
To - MCT)
2
E
IA-
S 2E( 24
Tı
2E max 1²( + P(A
:22=00
I 4E max م + 12/LTT 4 2 (X)))
=
x
< (4E( max + + 2/2 + 4suu (
:222=0
8
S -n
Here, we use the fact that
2 = /
P max o > t)dt S n /
E max i7.-
PP > t)dt nE((E)
:
If we choose 1) >>> 2, we can conclude that
E(per 1CT)2 S SC
<((n"
4(a-1) 442(a+m))
+n + + -11fc(()-
Proof for projection density estimator
Similar with the proof for kernel density estimator, we define
To = n(n 1)
Σ
Y1 2,,) KL,,,, Zi.
in.b==
and
T3 n(n - 1)
1
Σ
(1 ZU,) U.)
22

---

Published as a conference paper at ICLR 2024
Similarly, the same notation of P. By and oz. Step In this step, we show that
I.
1
2ay
B(Tu 00) SC
+ レ
112
and
Tu-8u
1
(1-2P)8CCh-((0222PPXX--)}}}CC +02
L
+ L
2iu
3
n
n2
Tr actually a U-statisties, so we decompose Ty as
Tu 8V
1
Σ
(r (U,, -0
n(n - 1)
$1
1
(1- (U) -1 21 1.
n(n 1)) 11712
- P// f Y1, (1 Z, (U,,, ())+(()) +00)
- Σ
0u -1 + 0u
7 1==1
1 a
+
Σ
p (,) Ou PY,, (1,))(((), Au
n 1=
1
Σ
(a1 (A Bu)
n
1=
PY, (1 (U,,) -
n1 11=1
+ OU 0u
=A + A2 + Ag + As + As As,
where Or P(1 P) :0
We now bound each term. For
A1, we have
E(A < n(n 11 B(Y, Ky (Ū,, 0w)) < a(n 1) / f U,,,,, dū, - n(n 1)
CL
Since th is the basis function, write
((1 M (1) and 57 = Σ (1),
1=1
where
lay| < (a/d4///2
byl (3/d+1/2))
23

---

Published as a conference paper at ICLR 2024
Here, use the fact that In terms of Ag, we have
C E
(ʃ
B
n
C
E
Σ
- -2d0
n
C
achi(ta)
E
n
V-L+1
CI.-2a/d
Similarly, we have
B(A3)
<-E
C
Σ brbr((,))) -nvn((0))-A000 -AA0))
E
Y
n
E(Y?,)
)
-28/
11
CL-2811
n
With the same argument in the proof of kernel density estimator, we have
Vn(Ai As) = Vn ==1
Σ
(1- - 200y
For the Ag, we have
/
-J Jf (
1
abbi
Σ
abb.
1=1
8
atbi-
1=++1
Because |a:b:| < 1- (x++8)//
| < [a(a+)))/
Putting A; - 1,..., together yields the conclusion.
Step 2. In this step, work on Tr. obtain
L 1 n
B(Tr - 00)) 02)2 SC
+ 1,-2(8+7)/d
and
logn +
log nV
P
[Tr. - -0rl > On
17"
24

---

Published as a conference paper at ICLR 2024
where 82 = P) (7 and is an arbitrary large constant. We
decompose Tz as
Tτ -0-1 8
Σ (1 Z₁,) (2,,,, - 81
( 1)
1
(U,,, (1 P) -0-P
nnn
(1
/ (1 Z,,
P
(p (r (1 (i,, PPL (11)())((Un))
Σ (1
-11-))000-----------1111 P(1-
Σ (1 (1- ())((()))) 201)
+ 0 1g
where 0r P(1 P)
If apply a similar proof in the last
step, we can get
1/2
L n
E(T, 0,)² C
Since Ty is U-statistics, we can apply concentration inequality in Giné et al. (2000) (also see Lemma 9 in [Shen et al. 2020p. By following the notation of Lemma 9 in Shen et (2020), we have
Bi C, B2 s 12 B3 S 12 of S. uz s CL.
which leads to
logn +
log nVT.
m - n
P [T 8zl > On
n
where 7 is an arbitrary large constant.
Step 3. In this step, we put the results of steps and 2 together. Recall Orr and 0,, are
Ou = P)I ʃ
fo(X)
Fx(X) uo (X (X)
and
Or P P(L P(L-P) p\
fo(X)
(x)))
fx(
If we define 8g and 09, as
og. = P(1 - P)сeT
and
8h = P(1 - -P),
then know that
B(Tu-001 00) <
+
L
2aa (1fc((8) - fx)33)11
n2
and
E 88)22 < C
L
1 +
2/
n²
n
As L 7 2d
application of the same argument in the proof of kernel estimator yields
- < C
(+ + + (1)e(X) - (c(
25

---

Published as a conference paper at ICLR 2024
D.3 PROOF OF COROLLARY
We follow the notation in the proof of Theorem Theorem
Now, we prove the first part of the corollary. By
D
we have
(PeT- (ooer-)))(((((3)) sc(n
212 +n + +11fo(8) - fo(1))1))
Due to + B S d/2, we have
E(per = d42(++0)
<0(1)
+ N 24
If we apply 2(a + B)d/(td (2t 4)( 3)), get
4l++ 242(0-95,
E(per PCT)² < On
We turn to the second part of the corollary now. By the Step in the proof of Theorem
1
To Tv-0u = Σ (1=2188Uhhh(((222=PXX--))))f
+0p
nhh/z2 + hat8
11 i=1
By a + B d/2, the choice of h suggests that
Ty - 0g = - Σ () PY,(1 - (.))(((.) 201 + +0p (1-1
n i=1
Because of d/2(t 1). we can know that
0f - By = 0p 12 111))
Thus, we can conclude that
Vn(Te -0:)
N(O,1),
VV
where
[
22 r JT
[ +
He N
22
2
=((erert))
p)²
P
1-P P
The variable transformation suggests that
fP) (2 PP2
1
(T
1-P / di
1
(o²(3) + + de
P
Po(
F2
( di
(
fo(
1
1 /
(o² ( ( dr ((no(E)s(())))))
P (Z) 1
fc(
=V
This leads to
Vn(Te 69)
1(0
VVP(1 P)
The proof of Theorem
1 also suggests that
Tr. -p 04 P(1 - P).
Therefore, we can conclude that
Vn(PAT - HCT)
+ 10.
VV
26

---

Published as a conference paper at ICLR 2024
D.4 PROOF OF THEOREM 2
Without loss of generality, we can assume (x(X) is uniform distribution and & is identity transfor- mation. We prove the two parts of the lower bound by considering the two cases. Case I: In order to construct the worst case, we consider the following model where Y only take values from {0, 1). i... Y a binary variable. The distribution this model can be written as
2,Y
(1- L (r)))()))) ? (1IP))cC8)c(() ()) 1-2
= -(Pfr(X))r(
= where / = P( = 1). Let L be the integer closest to n (24((4+((++9) and H be a smooth function
defined on (0. 11 such that f (ž)dž
and j H²) 1. The space [0, 1] can be divided into non-overlap cubes of size L-1/d On each cube Q1, we can insert scaled and shifted version of H, i.e. define the following function
L-11/d We name these small cubes Q1,.... Qr.
cτ( =
(e - ((Q
where .M) is the bottom left point of the cube. Clearly, we know th is supported on Qi and have I b/ = 0 and f v? = L/L. Now, we construct the least favorable hypothesis separately when a < B and a 2 3.
< 3 The hypothesis we consider are
Ho(n): PP -P 1,
11 =1 Σ nir 1/2, uT = 1/2
111):=P=1/ffc 1+ Σ mT 1/2 + Σ nie Iir,,rr,H HT 1/2.
M. Here, 1) (..............> (1) is sequence taking value from {-1, 1)4, r- T.-3/d and ri L- L-a/d, By construction, we can know that fr E H² 17d) and μ.e ([0, 1] Let (P)()) be distribution of (X, Y.Y under (n) and (1)) be distribution of (X, Z, Y) under H. (n). The null and alternative hypothesis consider here mixture distributions Po 2 E Po(n)
and P1 2- Σn (v). Simple calculation suggests that
2(a+8) 2126-23
PCT 1/2 under (n) and μCT 1/2 +
under HI, (7))
In order to calculate Hellinger distance between IPo and Pl, we adopt Theorem 2.1 in Robins et al. 2009 to obtain
H (PP. PT) Cn² (r + , SC
I,
By applying Theorem 2.15 in Tsybakov 2008
have
4(a+s)
inf sup por FEFu.e
EBAer -CT)² cn オイ.(.+1]
3 Now consider the hypothesis
1:po= Ho(r) P= = 1/2; 1, 1:po 1/2 Σ Mr nTT.ii HT 1/2
- = = Σ nor no 1/2 Σ n1 nr 1/2.
We use the same notation as when < 8. We can know that
PcT 1/2 under (l))) and Per 1/2 + n 2T under H1).
and
1
H(P.. PPt) < Cn
L (r) + +,)r)) SC
27

---

Published as a conference paper at ICLR 2024
So conclude that
4fa
inf sup E(jer (CT)² on 2+3(++7), peT FEFo,e
Case II: this case, we consider the same model the previous case and the following hypothesis
Hy : = 1/2: = 1, = P: = 1/2, PT = 1/2
H₃ :P 1/2: 1. fr 1:pc = 1/2 + r. T. 1/2.
Here, r= 11/2, Po is distribution of (X, Z, Y) under Hil and P1 distribution of (X, under
H.. Calculation suggests that
PCT = 1/2 under Hi and PCT = 1/2 1/2 under H1.
and
(dPPT)2 dTP /
(1 + x2)"
This implies
(PT, PB) (1 (1 12) 15 exp (nr2) ( - -15C.
By applying Theorem 2.2 in Tsybakov (2008), we have
inf sup Per FEFa.
B(jer HOT)² 2 en cn
D.5 PROOF OF THEOREM 3
Proof for kernel density estimator Through the proof, we define A = (2,)x(1-Z)))-a....
and conduct the proof conditioned on A. Conditioned on A, the map &D can be seen as fixed. Recall
Σn 2n (1- (0), 6 (Xx,))) pCT
En (1 (on)),( - 6n -x.
We write Gy = PCT, 11
n Σ
Tu
(1 -1 -x
7g i1,i==1
and
Tr Rg
(1-22,
Σ
(o {8.1111)
11,4==1
The proof is divided into three steps. Step 1. In this step, we show that
B( - (1) S
1
+ 12al++3))
nonghd
110
11
Tig
2n0/d
where og
1. Denote U; (D(..) for = =1,,... 1,...,
We decompose the To into two parts
To 7g i1.igml
(1 Ze, Ki (X) - (
Σ
(U,))Kn(O, -U.,), e Kn, -Ui)) + notli in: 20,
Σ
Pe t
6n:2,, =0,b2:2.,=1
28

---

Published as a conference paper at ICLR 2024
where 6; Y; - MCc (U.) when Z4 We work these two terms separately. The first term can be
written as
Σ
E KU,, - -Ura)
niT in:
20,
Σ
Kn - -Uig)= -ei / KnlUi, - ())1o((0)))) ti
non i =1
Σ
/
Kn(Un, P (Ū)dū
i1: =0
A= A2
In terms of A1, we have
B(A E
Σ
4, (U 022) )
nin E, -0,
Ro
For the A2, note that 6, are independent given A, which yields
E(A
A)
E
2
E
A
S
iz:Zin
0
710
The second term in Ty can then be decomposed as
E
MC (ū,)))),, - -U,))
nolli in in =1,==:21,=11
Σ
MC (L, -(,)) - (a -0) -()))((0)a))
no
1
Σ
(frE () - (0) Ū')
(ū')
+ no i1:771
+ /
PC (ū)Kn(ū م (ū")dūaū' - (0)) (T)
= AA + A1 + As
Define
R(U,,) 1
Σ
[C AD (U,,) - -(x)) / d (T,,
- - (i)aū).
TO Sp:Za
Simple calculation yields
B(R(U.,)) = and E(R((U,,)) SC
Since Ag Eig.2,,-1 R(U..)/ //1,, we can have
C
E(A
n1
Because of property 1 in Proposition
we have
C
|Ag| <
au/d
no
29

---

Published as a conference paper at ICLR 2024
In other words, we have
C
B(AiIA) S
200od'
Th
For the As, we apply the similar technique in the proof of Theorem
|Asl - h
Putting 4;, 1...., together suggests that
B( - 0 c
1
2exo/d / + 11 (2
Monghd
Step In this step, we show
1
B((TL - ((Tz--2)91A)) SC
2/d
111
no
We write Tr.
Tt - 11
1
Σ
Kn 0./z) - 1L
Ti iy:2,, =1,
1
Σ
Kn - -Un)- - |Kulū,, -015 - (ū)aū
no721
i1:7,, =0,2g
1
/ Kn -Ū) (ū)) -1 Kn - aū
/ Kn -0) (ū)aū Or.
With the similar techniques in Step 1, we can conclude that
B((T, - 0z) < C
2/d
Step the last step, we put the result of Steps and together, leading to the final conclusion. As h is chosen as h -> 0 and idd + O. we can know that
To > Ou
and
Tr. -p Or.
Therefore, we can conclude that
per App PCT-
Proof for projection density estimator We omit the proof here, as we can apply the same parallel argument the proof of Theorem !
30


Reviews:
[{"review_id": "vrWKg6KL6L", "reviewer": "Reviewer_oSbf", "summary": "This study proposes a novel framework for average treatment effects (ATEs) estimation by using uniform transformation. The authors develop an estimator that does not employ the exact estimation of the propensity score. Then, they show the finite-sample nonparametric minimax optimality for the estimator (but I think the estimator is not nonparametric in a usual sense...?) and asymptotic distribution.", "questions": "See above.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Firstly, I note that I could not grasp the contributions of this study, especially the novelty of the proposal of the uniform transformer. Therefore, I could not assess this study well. If possible, I want to deepen my understanding and clarify the contributions of this study through an interaction with the authors.\n\nThe reasons why I could not understand the contributions are based on the following three points.\n\n1. Necessity of the exact estimation of the propensity score.\nIn the 2000s, there were enormous arguments about what kind of weighting functions are effective in ATE estimation. As a result of long arguments, researchers found that using the **true** propensity score does not achieve the asymptotic lower bound proposed by Hahn (1998). That is, some finite-sample bias improves the asymptotic efficiency of estimators, as shown by Hirano et al. (2003). Does this study lie in those literature or completely discuss different topics?\n\n2. Significance of the use of a kerne-based U-statistics.\nTo the best of my knowledge, existing studies such as Hirano et al. (2003) employ kernel-based U-statistics, at least in theoretical analysis. What is the difference from the existing approaches?\n\n3. Minimax rate.\nI could not understand why the authors use the minimax optimality for nonparametric estimators. The estimator used by the authors is typically referred to as a semiparametric estimator, which is characterized by nonparametric and parametric estimators. Then, the authors are interested in the estimator error of the parametric part of the semiparametric estimator. Therefore, I believe that the nonparametric minimax optimal rate is a bit meaningless in this context (in other words, I think that although the authors say that the proposed estimator is nonparametric, the estimator is (semi)parametric). Furthermore, the asymptotic variance in Corollary 1 is not efficient from the viewpoint of the semiparametric efficiency bound proposed by Hahn (1998). Is the estimator really efficient or minimax optimal?\n\nAdditionally, I could not understand connections to existing estimators whose asymptotic variance aligns with the semiparametric efficiency bound; that is, the authors' estimator cannot be more efficient than the existing efficient estimators. Overall, I could not understand the author's intent in this study.\n\nMy assessment may be biased my knowledge of existing literature of this topic, and I am not confident on my assessment. I hope that I can deepen the understanding via communication with the authors.", "weaknesses": "See above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a03LaQARiC", "reviewer": "Reviewer_424z", "summary": "The paper introduces a method for estimating treatment effects using balancing weights, involving two key steps:\n- a uniform transformation ensuring a uniform distribution density on the weight's denominator\n- a density estimation of the transformed numerator\n\nUnder some regularity conditions and given that the density on the denominator is known, the authors establish an upper bound on the estimator's error rate and show that this rate of convergence is minimax. Additionally, they propose an adaptive uniform transformer that, when combined with the estimator, yields consistent treatment effect estimation.", "questions": "- What is the difference between estimating both the densities of $f_T(X)$ and $f_C(X)$, as compared to the proposed strategy? In particular, under the conditions on $f_C(X)$ estimation posed in Corollary 1, is it possible for an estimator without the transformation to achieve the same rate?\n\n- In general, what is the rate that can be achieved using the adaptive transformer? When adding strong enough regularity conditions, can it in general achieve the rate specified in Corollary 1? And what would those conditions be like?\n\n- What is the advantage of using the proposed method, as compared to a conventional doubly robust estimator?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors introduce a novel effect estimation strategy utilizing balancing weights through uniform transformation, together with a new adaptive uniform transformer.\n\n- The paper provides a sharp characterization (in terms of minimax optimality) of effect estimation through uniform transformers, which is new to the literature. \n\n- The paper is clear and well-written.", "weaknesses": "- The use of a uniform transformer is not well-motivated. It is not clear to me, from the paper, why a uniform transformation is preferred over an estimator with densities $f_T(X)$ and $f_C(X)$ estimated separately without any transformation. Is it because it is harder to study such an estimator and thus harder to obtain the optimal bandwidth?\n\n- The upper bound in Theorem 1 still relies largely on having accurate knowledge of $f_C$ (and also its smoothness level due to the $\\alpha,\\beta<\\gamma$ constraint), and it doesn’t seem to me that the proposed method with the adaptive uniform transformer is able to achieve the minimax rate in general.\n\n- The minimax optimality results are only for a very restricted class of problems.\n\nAlso see questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YmDRgBNY09", "reviewer": "Reviewer_Ybv2", "summary": "The authors study the average treatment on the treated (ATT) in the Holder smooth paradigm of Robins et al. (2008), where minimax rates are known. Previous work characterizes the lower bound and shows that a higher order influence function estimator attains it. This paper proposes an alternative estimator based on kernel smoothing to also achieve the minimax lower bound. In the “hard cases” where the nuisances are non-smooth, relatively few estimators have been shown to have good properties; for the “easy cases” where the nuisances are collectively smooth enough, this is a very mature literature.", "questions": "What are the advantages of this approach versus the higher order influence function approach, which seems to have the same kinds of guarantees?\n\nHow do we choose the bandwidth in practice?\n\nIs Theorem 2 a straightforward extension of the Robins et al. (2008) lower bound result? What are the aspects of it that are new? Close comparisons would help here.\n\nI will raise my score if these are addressed.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: To my knowledge, this proposed estimator is new.\n\nQuality: The results are cohesive. I recommend the authors mention Appendix B in the main text; I did not notice it at first.\n\nClarity: The writing is clear.\n\nSignificance: Proposing alternative estimators that achieve the minimax lower bound in hard cases is theoretically interesting, though perhaps not practically significant.", "weaknesses": "The introduction is too general for a paper that is ultimately about ATT only. \n\nThe kernel balancing weight literature is rich with many recent installments that should be at least mentioned in the introduction: Kallus (2020), Hirshberg et al. (2019), Singh (2021), and Bruns-Smith et al. (2023).\n\nIn Corollary 1, the convergence in distribution result for “easy” cases ends up being similar to several works that are unreferenced. \n\nI will raise my score if these are addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)