Optimal Nuisance Function Tuning for Estimating a Doubly Robust Functional under Proportional Asymptotics
Authors: Sean McGrath
Debarghya Mukherjee
Rajarshi Mukherjee
Zixiao Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: Debiased Estimation
Sample Splitting
Nuisance Parameter Selection
Proportional Asymptotics
Semiparametric Inference
Primary area: probabilistic_methods

Abstract: In this paper, we explore the asymptotically optimal tuning parameter choice in ridge regression for estimating nuisance functions of a statistical functional that has recently gained prominence in conditional independence testing and causal inference. 
  Given a sample of size $n$, we study estimators of the Expected Conditional Covariance (ECC) between variables $Y$ and $A$ given a high-dimensional covariate $X \in \mathbb{R}^p$. Under linear regression models for $Y$ and $A$ on $X$ and the proportional asymptotic regime $p/n \to c \in (0, \infty)$, we evaluate three existing ECC estimators and two sample splitting strategies for estimating the required nuisance functions.
  Since no consistent estimator of the nuisance functions exists in the proportional asymptotic regime without imposing further structure on the problem, we first derive debiased versions of the ECC estimators that utilize the ridge regression nuisance function estimators. 
  We show that our bias correction strategy yields $\sqrt{n}$-consistent estimators of the ECC across different sample splitting strategies and estimator choices. We then derive the asymptotic variances of these debiased estimators to illustrate the nuanced interplay between the sample splitting strategy, estimator choice, and tuning parameters of the nuisance function estimators for optimally estimating the ECC.
  Our analysis reveals that prediction-optimal tuning parameters (i.e., those that optimally estimate the nuisance functions) may not lead to the lowest asymptotic variance of the ECC estimator --  thereby demonstrating the need to be careful in selecting tuning parameters based on the final goal of inference.
  Finally, we verify our theoretical results through extensive numerical experiments.

Optimal Nuisance Function Tuning for Estimating a Doubly Robust Functional under Proportional Asymptotics
Sean McGrath Yale University sean. mann.mcrrath@yale. edu
Debarghya Mukherjee Boston University mdeb@bu, edu
Rajarshi Mukherjee Harvard University 1mm5210mail. www.aaar.ddee............... edu
Zixiao Jolene Wang
Harvard University
zixi zixiaowang@fas. harvard.edu
Abstract
In this paper, we explore the asymptotically optimal tuning parameter choice ridge regression for estimating nuisance functions of statistical functional that has recently gained prominence conditional independence testing and causal in- ference. Given a sample of size Covariance (ECC) between variables Y and A given nigh-dimensional covariate X E R2, Under linear regression models for Y and A on X and the proportional asymptotic regime p/n - c E (0, oo), we evaluate three existing ECC estimators and two sample splitting strategies for estimating the required nuisance functions. Since no consistent estimator of the nuisance functions exists in the proportional first
study estimators of the Expected Conditional
asymptotic regime without imposing further structure the problem, derive debiased versions of the ECC estimators that utilize the ridge regression nuisance function estimators. We show that our bias correction strategy yields Vii-consistent estimators of the ECC across different sample splitting strategies and estimator choices. We then derive the asymptotic variances of these debiased estimators to illustrate the nuanced interplay between the sample splitting strategy, estimator choice. and tuning parameters of the nuisance function estimators for optimally estimating the ECC. Our analysis reveals that prediction-optima. tuning parameters (i.e., those that optimally estimate the nuisanee functions) may not lead to the lowest asymptotic variance of the ECC estimator - thereby demonstrating the need be careful in selecting tuning parameters based on the final goal inference. Finally, we verify our theoretical results through extensive numerical experiments.
Introduction
Over the past decade, powerful prediction tools from the machine learning arsenal have been used to construct robust estimators of statistical functionals such as treatment effects and conditional dependence measures. Specifically, the seminal work of Double Machine Learning (DML) develops methods for estimating functionals that require fitting two nuisance functions (e.g., outcome regression and propensity scores for causal effect estimation), which use novel debiasing techniques that impart robustness to the
the nuisance functions !. DML allows researchers to perform sample splitting where one sample is used to optimally estimate the nuisance functions and the other sample is used to estimate the statistical functional efficiently based on the fitted nuisance functions. Importantly, one can be agnostic of the statistical functional that is the target of inference while fitting
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

nuisance functions. Hence, the tuning of learning algorithms for such function estimation can be operationalized from a prediction optimality perspective.
The validity of DML relies on suitable rates estimation of the estimated nuisance functions - which need to be verified on case-by-case basis, depending the choice of learning method employed for fitting. However, modern complex observational studies can result in situations in which one might not be able estimate nuisance functions consistently, let alone at sufficiently fast rate. One regime where this occurs, termed as the inconsistency regime by Celentano and Wainwright [2], is the proportional asymptotic setting, i.e., when the number of variables in the study proportional to the sample size. In this regime, the DML theory does not apply owing to the inconsistency of any nuisance function estimator. Therefore, an alternative estimation strategy is required.
number of works have explored the use of certain sample splitting and nuisance function tuning (e.g., undersmoothing) strategies construct optimal estimators of the functional of interest 1115) Indeed, recent literature with nonparametric nuisance function models has shown that even a naive plug-in principle-based approach can be asymptotically minimax optimal when the nuisance functions are tuned carefully [14 [5]. Moreover, a fascinating interplay has been observed between the choice of the estimation technique for the functional of interest and whether different splits are used to estimate the nuisance functions [4.. This phenomenon has been studied in nonparametric problems under Hölder type models for nuisance functions that allow for consistent estimation.
This paper aims to understand optimal tuning functions in the inconsistency regime through a doubly robust functional lens. Specifically, consider the Expected Conditional Covariance (ECC) between a continuous outcome and treatment variable given baseline confounders an object that has been 18 115]. learning weighted causal effect estimation to deal with positivity violations 119 20 conditional independence testing [21], and understanding genetic correlations (22). Under linear models for the nuisance functions and proportional asymptotics, the overarching goal of this paper is to perform sharp asymptotic analyses and thereby paint complete picture of the interplay of sample splitting, the type of estimator used, and the choice of optimal tuning of the nuisance functions estimated through regeerrggulrrzedd regression.
central in the development of model agnostic inference [16 17 12
1.1 Our Contributions
We take a first step towards a comprehensive understanding of how the tuning of nuisance function estimators and the choice of data-splitting strategy affect the asymptotic variance of estimators of the ECC. We present a rigorous theoretical analysis of the asymptotic behavior of three different estimators of the ECC: a plug-in based estimator, the doubly robust, i.e., DML estimator, and variant of the DML estimator introduced in [7]. We focus on the regulreggl estimators for the nuisance functions and leave the investigation of other estimators to future work. Since the type of data-splitting plays pivotal role in the proportional asymptotic regime, we consider two distinct data-splitting strategies: (i) two-split approach: where the data is divided into two parts- one used to estimate both nuisance functions and the other to estimate the ECC; and (ii) three-split approach: where the nuisance functions are estimated using two separate parts, and the third part is used to estimate the ECC. Below, we summarize our key contributions:
Since no consistent estimators of the nuisance functions exist in our setting with proportional asymptotics, we show that each of the ECC estimators is asymptotically biased in each data- splitting strategy. We then derive bias-corrected versions of these estimators by inverting the asymptotic bias.
We show that each of the bias-corrected estimators achieve the optimal vie-rate for estimating the ECC and derive expressions for their asymptotic variances. Unlike settings amenable to DML, the asymptotic variance of these estimators is not immune to the effects of nuisance function estimation. Therefore, the tuning parameters can be selected to minimize this asymptotic variance of the ECC. Surprisingly, the optimal tuning parameters for inference differ from those that are optimal for prediction, that is, the values that optimally estimate the nuisance functions. This highlights key difference from the standard DML framework: one should not estimate the nuisance functions by minimizing prediction error (which is the gold standard in fiee-diieesiooal settings), but instead by minimizing the asymptotic variance of the functional of interest.
2

---

We perform extensive numerical experiments that verify and complement our theoretical results. We identify the optimal choices of tuning parameters for minimizing the asymptotic variance of the ECC and compare those to prediction-optimal tuning parameters. We then provide practical guidance on which approach practitioner should prefer under different cireumstances.
Problem Setup
To set notation, consider tuple of random variables (A, Y, X), where A Y E R and X TR.. We aim to estimate the ECC of A and Y given X, i.e,, our parameter of interest is:
0o = Blcov(Y, A X)] = [[AY] E X]E[A X]].
As mentioned in the Introduction, the estimation of ECC has a long-standing presence in the causal inference literature among other applications. ECC also serves as a foundational stepping stone toward understanding a broader class of doubly robust functionals [9 is easy to show that 8o is doubly robust with respect to two nuisance functions, E[A X and E X], i.e., one can consistently estimate Bg as long at least one of these two nuisance functions is consistently estimated. Despite its simple structure, the ECC functional encapsulates many of the key features necessary for thorough understanding of doubly robust estimators. (e.g.. minimal smoothness or convergence rate required for the nuisance estimators to achieve /ii-consistnncy for 0g. feasibility of adaptive inference, etc.). While there a relatively clear understanding of the statistical properties of 0, when X is finite dittesiiinnsionall (or in ultra nigh-dimensional regimes under suitable sparsity assumptions), our knowledge of its behavior in the proportional asymptotics setting remains limited 23. This motivates us to take first step toward analyzing the ECC functional under proportional asymptotics, which we believe will serve as a foundation for understanding the broader class of doubly robust functionals in this asymptotic regime.
2.1 Estimation Strategies
In this section, we present three different estimation strategies of 0o- To formalize our analysis and investigate the behavior of 0o under the proportional asymptotic regime, now describe the data-gent geteraennn process in detail. We assume to observe n i.i.d. observations {(x,, Yi, (x),)ssss)) Ai) We assume that the dimensionality of X grows proportionally with the sample size, i.e., p/n (0, oo). In this paper, we assume (Y, A) follows bivariate normal distribution conditional on X, i.e.,
A = X og +
(2.1)
Y x Bo + 14,
We assume that coordinates of X E R2 are i.i.d subgaussian random variables, with mean 0, variance 1, and uniformly bounded (over p) subgaussian norm. It is immediate from our data-generating process that the nuisance functions
be expressed as E[AA X =xTa aag and E XX== X Bo- The key challenge in estimating Og under the proportional asymptotic regime lies accurately estimating the nuisance parameters Og and fu. In the proportional asymptotic regime, particularly when > n,, a common strategy is to use convex regularized estimators. For simplicity and clarity of exposition, we focus in this paper on ridge regression. The analysis of other estimators an important direction for future work. Given tuning parameters λ1, A2 > O, the ridge regression estimators of oo and 9o are defined as:
Xal 2 + A [|]]] = E + X1 X a(A1) = arg min, {
(2.2)
3(22) = arg ming { X,3 + A2 [ = S + \2 ) xx
where a = A,) (Yi...., YT)T and X E R the matrix of (X1,... Xn) and
Σ = (X x)/n is the sample covariance matrix X.
To construct our first estimator, observe that we can express Og as
0u = E[AAY - E[E[Y| X|E[A| Xi] = E[AY] - a T[ = E[[AY] - aj So
As both A and y are sealar random variables, one may estimate B[[AY] by the sample av- erage (E, A,Y./nn which is Vr-consistent and asymptotically normal (CAN), provided that
3

---

(AY( < oo (which can be ensured if I|ool22 and || are assumed to uniformly (over p) bounded, see Appendix for more details). However, the key difficulty lies in estimating 0gg 9o under the proportional asymptotic regime. The integral-based plug-in estimator of 8g involves estimating ag So by d(A1) (X2)., i.e.
JINT - A.Y - a(2))
(2.3)
Our second estimator is based on the work of Newey and Robins [7]. Owing to the representation 8o E (A [[ B| (A - x a()i))], we can consider the following Vewey-Robins plug-in in estimator of 0o
@NR = - - -TT(Ai))
(2.4)
Observe that one may construct a similar estimator of 6, based on exchanging the roles of A and Y. i.e., a plug-in estimator based on On = EEAAAE--ABBB [E[]]X]]]] it straightforward to see that both estimators share essentially the same asymptotic properties. Therefore, we focus on the former.
Our third estimation strategy is inspired by the one-step influence function-based correction strategy to construct a semi cemi-parametrically efficient estimator. In our case, this estimator is given by
@DR 2/2
- ((A))) -xT
(2.5)
Typically, in a Exee-diieesioonl setup (or in ultra-high hew--hhhhhhiia digh-ddimesiooall setup), it is possible to estimate the nuisance functions B[y X] and BE[A X at rate faster than n 1/4 under some appropriate smoothness/sparsityy assumption. This leads to the traditional DML wisdom: one should first split the data into two parts: use the first part to estimate the nuisance components, namely E|Y X] and E[A X). and then plug these estimates to construct @DR using the second part of the data. Finally, cross-fitting should be used to reduce asymptotic variance. Following this wisdom, is not hard to show that the estimator is Co-CCssstttet and Asymptotically Normal (CAN) well as a sempparametriaalyy efficient estimator of Og. However, this traditional wisdom is not readily applicable to the proportional asymptotic regime for two main reasons. First, although the mean functions in our setting are infinitely smooth (since they are linear), we typically do not assume any form of sparsity. As result, it is not possible to estimate E|Y X] E[A X] at rate faster asymptotically negligible; in
than n -114 1/4 In other words, the product of the estimation errors fact, we will show that this product is itself Vri-CAN and contributes non-trivially to the asymptotic variance. Second, demonstrated in Jiang al. [23] cross-fitting is not straightforward in this regime: estimates obtained by rotating the data are generally not asymptotically independent, and the resulting correlations between folds introduce additional contributions to the asymptotic variance. In fact, the precise characterization of the minimum achievable variance when c p/n > 1 remains an open problem.
2.2 Sample Splitting
The three estimators introduced in the previous subsection all involve the estimation of two nuisance parameters, (oo, 80). As result, one can consider two distinct sample splitting strategies. The first the two-split approach, where both ag and 30 are estimated using one half of the sample (Di), and the resulting estimates are then combined with the other half of the sample (D2) to construct ğINT, @NR, and @DR, The second is the three-split approach, where the data is divided into three parts: one (Dt) used to estimate ogp another (D2) to estimate fo, and the third (Dx) to construct the final estimator. To make this distinction explicit, adopt notational convention: for instance, iINT and 3ep denote the versions of (INT constructed using the two-split and three-split strategies, respectively. Same convention are also followed for @N and @DR However, these two sample splitting strategies result in different effective sample sizes being used for constructing the final estimator. For notational convenience, we consistently use n to denote the number of samples involved in the construction of the final estimator. This means we implicitly assume access to 2n total samples when employing the two-split approach, and 3n samples for the three-split approach.

---

3 Main results
3.1 Bias Correction
In the previous section, we introduced three distinct strategies for estimating the target parameter do. and all of them rely estimating the nuisance components oxp and 8 (directly indirectly). which can be learned either from a common auxiliary sample (two-split approach) or from two separate subsamples (three-split approach). However, all six resulting estimators suffer from non-negligible asymptotic bias, which must be carefully removed to even achieve consistency for On- In the following sections, we will motivate and develop the bias correction strategy for each of the six estimators.
3.1.1 Integral-Bane Baagral-Based Estimator
To illustrate the origin of this bias, and to motivate our debiasing strategy, let us consider the three- split version of ②INT (Equation (2.3). ). i.e., we estimate and S0 from two separate subsample, then use third subsample to estimate IINT Since E[AY] = 0o + ag Po. it follows immediately that:
E[ 0 + ag So E (e(2)))((2)))
(3.1)
As a(A1) and ((2)) are estimated from a separate sampse,ple, they are independent, and X is independent of (e, pd. Using this, it is easy
that:
((A)) B(22)) = E क (L + AAI, + 121p) -
If p/n + E (0, co), details):
show using tools from random matrix theory that (see Appendix for
E Li + + A2Ip) E2 ax 3o (s =dF ) xt11
(s=dfrn))) + o(1)
P ag 9o g5ep (A1, 22) + o(1).
where FXP is the famous Marchenko-Pasee distribution (Section 3 of (24). which arises as the limiting spectral distribution (LSD) of the eigenvalues of the sample covariance matrix (Σ, or Σz). The above sketch indicates that (A()(A)) ((A2)] = (a (o) (o)gsssp (x1, Az), i.e. the ridge-regression based plug-in estimates approximate (a, (B) up scalar function INT (A1, A2). This scalar function, however, is deterministic, as it does not depend on any unknown parameters. Therefore, naturally motivates simple debiasing procedure by inverting the asymptotic bias:
[ E ((A)) B(22)] zu (oo Bo) 93ep
INT
A2) = E ((((A))B((22)))
INT (A1, da)] s ag 9o 9eep
Consequently, we can re-define debiased version of QINT (using three-split) as:
@INT,db db EiED, A.Y; 0sp
&(A)) ((Az)) ghop (A1,Aa)
(3.2)
In Theorem 3.1 we establish that the above estimator is not only asymptotically unbiased but also achieves the optimal Vπr-rate for estimating 9g. However, in the case of the two-split approach, where og and 3 are estimated from the same subsample, the bias correction step becomes slightly more involved, due to the correlation between the associated estimation errors. As evident from Equation (.22), the error estimating ((x)) depends on e, while that in 8(A2) depends on μ. which are themselves correlated. As a result, an additional bias term arises, given by:
B[ā(A1)T 3 = [[ ( A1 ( A2Ip) E13
(3.3)
+ E/c X I1I,)) - 11]
Again, some tools from random matrix theory yield (details are provided in the Appendix):
[[((Ar)T 3(AA2) = (al 3o) f
x dFup(z) + 0o dF + o(1) [a+Ar)(a+2o)
(++A))
A (aT 30) g I X2) + Og 9222pp
INT (A1,A2).
Interestingly, now the bias of a(>:) T(AI))BA B(A2) not only involves a. 3o. but also involves 0o- Therefore, we cannot simply divide â á(A1) (()2) by some function of the tuning parameters to
5

---

remove the bias, as we also have to take care of the second part. Following the same inversion trick as in the three-split case, we now have:
E[ā(A1) (22)) INT (A1, Az)l = ag Bo + Bu INNT
(A1. /2)/ 91.l//1.200 INT
(A1, A2))
And consequently:
E LiED, A,Y, 4(A)) B(A2)
0o (1
INT (A: .22)
g. (A1))2)
91.2ep (A1
Now, the above equation suggests using the inversion trick one more time (as both Sl, are deterministic) to obtain the following debiased version of ÕINT:
INT
and INT 92,2sp
@NTT,db b (1 2sp
ob (A1.A2) 2rp (A1,Az)
E A.Y, ((A))
(3.4)
Sep (A1,22)
INT
We next present one of our main results, establishing that these debiased versions of the integral-based estimator are indeed Vn-consistent.
Theorem 3.1. Consider the debiased version of the integral-based estimator @INT, either obtained by a two-split approach (Equation (3.4) or three-split approach (Equation (3.2) Assume X are iid subgaussian random variables with mean 0, variance 1, with uniformly (over p) bounded subgaussian norm. Furthermore, assume that (e, v) v) X and follows a bivariate normal distribution
with mean variance 1, and correlation p. and there exists Ca Cg >> 0 such that ||ool2 Ca and ||:oll2 S Ca for all p E N. Then, we have:
Vū(QINT,d 00) Op(1) both for two and three split versions.
3.1.2 Newey-Robins (NR) Estimator
In this subsection, we extend our bias-correction technique the estimator HNR (Equation (2.4) To understand the nature of the bias, we once again consider the three-split version of @NR The expected value of this estimator is given by:
O 3ap
= 0o + ag 30 E[ā(A1) [o] = 0o + ag Bo E (Σ) + -111))=1Sann 15
0u + az Bu 1 œ+h1 dFMr(1)) + o(1)
0, + ag 3o 93ep (2)) o(1).
where, before, the last line follows using some techniques from random matrix theory, which can be found in the Appendix. To eliminate this additional bias (i.e., the second term in the equation above), we adopt the same inversion technique introduced in the previous subsection. As noted earlier, when â(A1) and B(A2) are estimated from two independent subsamples, we have a. So. We leverage this observation to correct the bias as E B(22)) INT (A1;
follows:
@NR.db
1 1 EiED, YAAA, xTT(())) ((Az)T((Az))
giap (AI.2)) IST (A.
(3.5)
For the two-split approach, the bias correction is bit more involved as of (INT due to the correlation between the estimation error of ag and So. However, similar approach before yields the following debiased estimator:
INT (A)o (2)
12 E.EE, YIAA xT a(A.) ((A2)
B 22.2ap INT (N) (A)
2ap
(f, (2)
INT (A) T.
(3.6)
with A = (A1, The following theorem establishes that these bias-corrected estimators are also /n-consistent:
Theorem 3.2. Consider the debiased version of the Newey-Robing Sobwey-Robins estimator (NIdb either obtained by a two-split approach (Equation (3.6) or three-split approach (Equation 3.5 Under the same assumptions as in Theorem 3.7 we have:
V dNN,dd 00) (L) both for two and three split versions.

---

3.1.3 Doubly Robust Estimator
Last but not least, we turn to the bias correction procedure for the doubly robust estimator of 86- denoted by @DR and defined in Equation (25 As discussed in Section 2 this estimator has
well-established history in the literature, particularly from the perspective of double machine learning. A straightforward computation of the expectation yields:
E|ODR| = 0o + B[(as - (5) - 3(22))
(3.7)
Therefore, the primary source of the bias is the second term of the above equation. Under the threc-split setup, one can show that:
Ei(ao -(()) (% -(A2)) at Po (s क+ं dFur(z)) (s a dFua(())) + o(1)
A (ag B0) g3pp (A1, A2) + o(1).
The calculation proceeds similarly to that for @INT or @ which skip for brevity; the key step is to derive the asymptotic bias of ((A1) ((22). This characterization of bias leads to the following bias-corrected estimator:
@DR,db 3ap
@DR - ((A1) ((22)
S ( A2)
(3.8)
93ep (Az,
However, as explained for the previous two estimators, the bias takes more complicated form for the two-sample split. Similar calculation using the tools from deterministic equivalence in the random matrix theory yields:
B|OPR|
0o [an &(Ai)) (Go 3(A2))]
0o (ao (o)
didy dFup(z)
ps
crdFur(() + +o(1
(atA))(+td))
(o+A)](=+Aa)
0u
(x)
(ag Bo) f
dndadFup(e) + o(1)
(atAi)ls+22))
A 0o (1 + 92,2xp (A1, (ad 3n) 92,2sp (A1, +ol).
The details can be found the Appendix, which we do not provide here for the sake of space. Based on this expression, we propose the following bias-corrected estimator:
db 2sp
(1
INT 92,2a
(A)
22.2e (A)
@DR &(A1) 8(A2)s (A)
(3.9)
9 (A)
The following theorem establishes that these bias-corrected estimators are also ci-consistent:
Theorem 3.3. Consider the debiased version of the doubly robust estimator @DR.d either obtained by a two-split approach (Equation (3.9)) or a three-split approach (Equation (.8)). Under the same assumptions in Theorem 3.1 have:
(n 0o) = Op(1) both foth.for two and three split versions.
3.2 Limiting Variance
In the previous subsection, established that the bias-corrected versions of all three estimators, namely (INT are Vri-consistent for any choice of A2 O. This naturally raises a question for practitioners: which estimator should one use in practice? The answer, of course, depends on the downstream objective. Since our goal is to perform inference on fg. the ideal choice would be the estimator that achieves the minimum asymptotic variance. Therefore, it is imperative to develop precise understanding of the asymptotic variance associated with each estimator, how this variance is influenced by the choice of estimator, and the sample splitting strategy employed. However, quantifying the asymptotic variance of this debiased estimator the proportional asymptotic regime remains a challenging task and requires much finer analysis. To the best of our knowledge, only few papers (e.g., [2 25 23 have addressed related questions; although, they are limited to the setting where p/n E (0, 1). In this paper, we take first step toward characterizing the asymptotic variance of our debiased estimators the regime where e = p/n E (0, oo), i.e., we allow p > n. To facilitate this analysis, we assume that the covariates Xij are independent and follow \ 1).
7

---

Gaussianity enables the use of powerful tools, particularly those involving the analysis of Haar- distributed eigenvectors and the independence between eigenvalues and eigenvectors. both of which play a crucial role in simplifying our analysis. While we believe that our results extend beyond the Gaussian setting, we leave such generalizations as interesting avenues for future research. The following theorem provides the exact asymptotic variance of the six debiased estimators introduced previously:
Theorem 3.4. Consider the same setup as in Theorem 3.1 along with the assumption that Xiş i..d.
(O,,)) 1). Furthermore, assume that there exists (u, v, 2) such that:
lim 1100122 u.
1im |1 V, limatoc 0g Bo 2
Then there exists six functions (VINT VINT VNN V 2spp> 3ap such that:
oo n x var (0A.10) = -V (A1
for A E {INT, NR, DR) and u E (2sp,3sp), The expression of VINT and VINT Equation (3.1T) and Equation (3.12 respectively, and the expression for the rest of the variances can
3Ep, are provided in
be found in the Appendix.
The above theorem provides us with the precise limiting variance of six estimators. Ideally, one should choose the estimation which has minimum variance over all possible limiting variances, i.e. goPt should be g ,db where A and v* should be chosen as:
(A',0*) arg min AE (IN, DR), A Vi((A,, A2).
(3.10)
"E (2sp.3ep)
The reason we do not include the exact expressions for all the asymptotic variances is primarily due to space constraints; the formulas are quite involved and lengthy. Below, we present the expressions for VINT and V/INT to illustrate the complexity and structure of these quantities: 2sp 'ssp
IINT = 2sp
22p 91 21.2pp (X)
-2
var + a 2² +20tc(++0)
x² dFsp(a)
(01,200 VT
(A)32
(1+t)))
(3.11)
a 22²++0
X²
TKT e ( (0)1).pp(A)) Var
(X (X+x:)(X+aa)
INT 3ep
= var(AA)++ + TINTT (111)2))²
Cosup
varx=Fxp
+ u² x+>2
x
edFxp(()
[a+t
(s
(
(P)pp(())))
z+31
+ varxx xAPI
e²
Edfhaplo) x=
u² u²
x² (d
(x+
c
zd'app(z) (x+
+ +122 (J
2 dfssp(e)) (x+22)"
+
z (x) (or+22)2
dffsp(e) (x+
(3.12)
) differs significantly from the prediction optimal choice of A2) (derived in the Appendix), as illustrated by our simulation in Section 4 This departs from the traditional DML framework, where the nuisance parameters are typically estimated using prediction-optinn proportional asymptotic regime, such rrdictiion-ppimaal choices do not necessarily yield the best performance for inference. As a result, it is more appropriate to directly minimize the asymptotic variance rather than predict optimal choices to achieve a more accurate inferenceenarrower confidence interval.
worth noting that the optimal choice of (A1, A2) (which is the minimizer of Equation (3.10)
rediitioo-oopimall
methods. However, in the
In practice, the asymptotic variances can be estimated by using resampling techniques. We outline a parametric bootstrap approach to estimate the asymptotic variances in the Appendix, which we evaluate in our simulations.
4 Simulations
We assessed the performance of the ECC estimators through simulations under the different sample splitting strategies. We considered the size of the splits to be 500, making the total sample size N 1000 for the two sample splitting case and N 1500 for the three sample splitting case. For
8

---

each sample splitting case, we set 250 (i.e., c= 0.25) and p 1000 (i.e., c= 2). In each case, we generated 10, 000 independent data sets, each consisting of iid copies of (A, X, Y). Specifically, we consider random vector X of dimension generated as X (O, /p). Then, we generated A and Y with / = 0.5 by Equation (2.T). We applied the integral-based estimator, the Newey-Robins estimator, and the doubly robust estimator, where we set t₁ X2 = À and set the bias correction constants by using Monte Carlo with 10, 000 iterations. Each estimator was applied across 100 distinct values of X, spanning grid from 0.05 to 10.
We investigated the impact the tuning parameter À on the asymptotic variance of the debiased estimators. Results for 1000 (i.e., 2) are presented in Figure [] while results for the 250 setting (i.e., 0.5) are provided in the Appendix. Interestingly. we demonstrate that prediction- optimal choices of the tuning parameters fail to yield statistically optimal estimates of ECC, and one should rather aim optimizing the variance the ECC directly and carefully to account for the type of sample splitting and estimator chosen. We note that in the two-split scenario, the variance blows up at values around 1.48 due to the bias-correction constant diverging. However, this does not affect our primary conclusions, as our goal is to select the value of À that minimizes the asymptotic variance.
Due to space constraints, we present additional details of the simulations and our other analyses in the Appendix. In brief, we found that the debiased estimators had negligible bias (unlike the original estimators) and that the parametric bootstrap approach often performed well for estimating the asymptotic variance.
Integral-besed Estimator (Two Splits
Newey and Robins Estimator (Two Splits)
Doubly Robust Estimator (Two Splits
Variance
06
Verience
15.6
Varfance
0.6
04
044
02
0.2
00
0.0
0.0
10
10
Integral-besed Estimator (Three Splits)
Newey and Robins Estimator (Three Splits)
Doubly Robust Estimator (Three Splits)
0.05
0.5
0.01
0.01
003
Varianee
Vartance
003
0.2
002
0.0
0.21
0.00 6a
0.00
A
Variances of debiased estimators as functions tuning parameter X. Top row: Two-split (N 1000 total, split into two subsamples of 500 each). Bottom row: Three-split (N 1500 total, split into three subsamples of 500 each). Red line indicates the optimal for estimator variance; Blue line is for inddictton-optiman X. The figures shown here zoomed-in in views: complete versions included the Appendix.
Figure 1:
Conclusion
In this paper, we have explored the intricate interplay among sample splitting, estimator choice, and optimal tuning of nuisance functions for estimating the expected conditional covariance (ECC), popular doubly robust functional in the literature causal inference and missing data. The

---

philosophy we espouse in this regard can be best understood within the DML framework, where typically prediction-optimal nuisance functions are used for downstream efficient estimation of a low-dimensional functional that depends on them. In this paper, we explore the subtleties of this discourse when no consistent nuisance functions exist. In particular, for ECC, this challenging regime arises when the dimension of the problem is proportional to the sample size, and our results uncover a precise asymptotic picture of the interplay between sample splitting, choice of estimator, and optimal tuning of the regularized nuisance function in this regard under a stylized linear model for the nuisance functions. Our analysis reveals an interesting phenomenon: prediction-optima tuning parameters are not necessarily the best choices for estimating ECC, in terms of its asymptotic variance (and consequently the length the confidence interval), which differs from standard DML wisdom.
While this is the first work to analyze ECC in the proportional over peer-parameteizeed regime, our framework invites several natural extensions. For example, our analysis assumes the Gaussianity of either the errors or the covariates for variance calculation an assumption that remains to be explored through the lens of universality principles. It is, however, worth noting that, unlike the operational domain of the DML proposal, consider regime where asymptotic optimality for ECC has not yet been established, and the literature has been active in constructing estimators that navigate specific contours of the problem difficulty, such as debiasing regularized estimators of the nuisance functions. Indeed, the estimators studied here are not the only ones that achieve /n-connistency in this regime; one completely bypass nuisance function estimation using method of-moments approach (26). One can also envision appealing to more sophisticated debiasing techniques when generic convex regularization, such as Lasso, is employed to estimate the nuisance functions [25] and derive results analogous to those presented here. In this regime, where an efficiency theory is unavailable, the choice of estimator that minimizes asymptotic variance is priori unclear. We believe that the template of exploration presented here might serve as a framework for such future endeavors. Specifically, the availability of large class of sample-split-sensttie optimally tuned estimators of double-robust functionals might shed light on eventual efficiency bounds for estimating the functional itself. A case study of ECC presented here hopefully serves compass for future conjectures in this journey.
Acknowledgments and Disclosure of Funding
Rajarshi Mukherjee was supported by NSF CAREER 2338760. Debarghya Mukherjee was supported by NSF DMS-2515787. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NSF.
References
[1] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1 -C68 2018.
[2] Michael Celentano and Martin J Wainwright. Challenges of the inconsistency regime: Novel debiasing methods for missing data models. arXiv preprint arXiv:2330.0 2023.
[3] Peter Hall. Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. The Annals of Statistics, 20(2):675-694, 1992.
[4] Whitney Newey, Fushing Hsieh, and James Robins. Undersmoothing and bias corrected functional estimation. Working Paper 98-17, 17, Massachusetts Institute of Technology (MIT), Department of Economics, 1998.
[5] Evarist Giné and Richard Nickl. A simple adaptive estimator of the integrated square of a density. Bernoulli, 14(1):47-61, 2008
[6] Liam Paninski and Masanao Yajima. Undersmoothed kernel entropy estimators. IEEE Transac- tions on Information Theory, 54(9):4384 2008.
[7] Whitney K Newey and James R Robins. Cross-fitting and fast remainder rates for semiparametric estimation. arXiv preprint 18010:1801..9118. 2018.
10

---

[8] Mark J van der Laan, David Benkeser, and Weixin Cai. Efficient estimation of pathwise differentiable target parameters with the undersmoothed highly adaptive lasso. The international journal of biostatistics, 19(1):2112289 2023.
[9] Mark J van der Laan, David Benkeser, and Weixin Cai. Causal inference based on undersmooth- ing the highly adaptive lasso. In Association for the Advancement of Artificial Intelligence Spring Symposia, Palo Alto, California, 2019.
[10] Aaron Fisher and Virginia Fisher. Three-way cross-fitting and pseudo-outcome regression for estimation of conditional effects and other linear functionals. arXiv preprint (xXi::2306.072300 2023.
[11] Edward H Kennedy, Sivaraman Balakrishnan, James M Robins, and Larry Wasserman. Minimax rates for heterogeneous causal effect estimation. The Annals of Statistics, 52(2):793-816, 2024.
[12] Sivaraman Balakrishnan, Edward H Kennedy, and Larry Wasserman. The fundamental limits of structure-agnostic functional estimation. arXiv preprint arXiv:2305.041 16,00,1116666 2023.
[13] David Bruns-Smith, Oliver Dukes, Avi Feller, and Elizabeth Ogburn. Augmented balancing weights as linear regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkaf019, 2025.
[14] Sean McGrath and Rajarshi Mukherjee. Nuisance function tuning and sample splitting for optimal doubly robust estimation. arXiv preprint arXiv:2212.14857 2022.
[15] Alec McClean, Sivaraman Balakrishnan, Edward H Kennedy, and Larry Wasserman. Double cross-fit doubly robust estimators: Beyond series regression. arXiv preprint 151755, 2024.
[16] Lin Liu, Rajarshi Mukherjee, and James M Robins. On nearly assumption ffree tests of nominal confidence interval coverage for causal parameters estimated by machine learning. Statistical Science, 35(3):518-553) 2020.
[17) Lin Liu, Rajarshi Mukherjee, and James M Robins. Rejoinder: On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning. Statistical Science, 35(3):545 554, 2020.
[18] Lin Liu, Rajarshi Mukherjee, and James M Robins. Assumption-lean falsification tests of rate double-robustness of wuuble-machin--learing estimators. Journal of Econometrics, 240(2):
105500, 2024.
[19] James Robins, Lingling Li, Eric Tchetgen, Aad van der Vaart, et al. Higher order influence functions and minimax estimation of nonlinear functionals. Probability and statistics: essays in honor of David A. Freedman, volume 2. pages 235-422. Institute of Mathematical Statistics, 2008.
[20] Richard K Crump, V Joseph Hotz, Guido W Imbens, and Oscar A Mitnik. Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1):18-119, 2009.
[21] Rajen D Shah and Jonas Peters. The hardness of conditional independence testing and the generalised covariance measure, Annals of statistics, 48(3):1514-1588, 2020.
[22] Yiliang Zhang, Youshu Cheng, Wei Jiang, Yixuan Ye, Qiongshi Lu, and Hongyu Zhao. Compar- ison of methods for estimating genetic correlation between complex traits using gwas summary statisties. Briefings in bioinformatics, 22 2021.
[23] Kuanhao Jiang, Rajarshi Mukherjee, Subhabrata Sen, and Pragya Sur. A new central limit theorem for the augmented ipw estimator: Variance inflation, cross-fit fit covariance and beyond. The Annals of Statistics, 53(2):647-675 2025.
[24] Zhidong Bai and Jack W Silverstein. Spectral analysis of large dimensional random matrices, volume 20. Springer, 2010.
[25] Pierre C Bellec and Cun-Hui Zhang. Debiasing convex regularized estimators and interval estimation in linear models. The Annals of Statistics, (1(2):3914436 2023.
11

---

[26] Xingyu Chen, Lin Liu, and Rajarshi Mukherjee. Method--ofmmometts inference for glms and doubly robust functionals under proportional asymptotics. arXiv preprint 040000200000603, 2024.
[27] Walid Hachem, Philippe Loubaton, Jamal Najim, and Pascal Vallet. On bilinear forms based on the resolvent of large random matrices. In Annales de I'THP Probabilités et statistiques, volume 49, pages 36 63,63,222 2013.
[28] Zhidong D Bai and Jack W Silverstein. Clt for linear spectral statistics of large-dimensional sample covariance matrices. In Advances In Statistics, pages 281 281-333. World Scientific, 2008.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The claims made in the abstract and introduction match the theoretical and simulation results. In the discussion and conclusion section, we comment on how results can be generalized to other settings.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Section (Conclusion).
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer: [Yes]
Justification: Proofs of all theoretical results are given in the supplemental material. Some proof sketches provided in the main text well.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The Simulations section and details supplemental sections provide a detailed description of the simulation architectures.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be nECCssary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
14

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The supplemental materials contain all relevant codes and information.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.co. ccippi.cc guwdes/ddeeeee CooeeSumeeeeeeeeeeeeee ssi onyllicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) nECCssary to understand the results?
Question:
Answer: [Yes]
Justification: The paper's main message is about the relationship between sample splitting and optimal hyperparameter tuning: therefore, all the results are aimed at elaborating on this aspect.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is nECCssary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Numerical results provided in the simulations section detail precise descriptions of the bias and variance of the methods considered in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources are detailed in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
The research conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification:
The paper concerns foundational research in semiparametric theory.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
16

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not release data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with nFCCssary safeguards allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
The paper does not use existing assets.
Guidelines:
The answer NA means that the paper does use existing assets. The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
17

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
No new assets are released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The research does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research does involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
18

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy for what should or should not be described.
19

---

A Proof of Theorem 3.1
A.1 Proof of Three-Split
Recall that in the three-split strategy, we assume to have 3n observations: We use the first n observations (Di) to estimate á(Ai), ), second set of n observations (D2) to estimate (A2). and the last set of n observations (D3) construct the final estimator. For the three-sample splitting, the debiased version of the integral-basee estimator takes the following form:
A Jap
- Y:A;
((A)))A(A2)) Bay (A1.Aa)
(A.1)
To establish the vii-consistency of idb,INT, we start with the definition of the ridge-regulatized estimators:
a(\1) arg min, [1 Xıali + x11|a111] = (É) +11p)) X a1
+ (81 + \1Ip) X
=Ei +11p) 'S
n
= - - + X111) 0p + + = -111p) X 61
((22) arg ming { 7 lly2 X2313 + +
X,
(22 + )211))
n
X2 142
=(E2 + A211)) 32 + (E2 + \2In))
n1
8o + (L2 + >21p)) X 42
= 8o ~2 + -22
Hence, we have:
3. + ag (Σg + Xx1) X,2 j42
a(Ar)TB(X>)) = aT - an 22 + 121)
-drao ($i, + 11 30
212 ($) 1 + 22
A1
X 12
111o (5, 1
1
EΣ3 + X2I)
n
X 61
A2 T (22 + A21) -1 1(S + 111) X 61
+ 00 (2) + A11)
X1
X M2
(Σ) A1 1 (22 + A21)
+ n
n
9 ΣΣ1
T
Combining the parameter aij 3o. and the terms that contribute to its estimation bias, namely Tú, T2, T4, T3 (observe that all the other terms have mean 0), let define an aggregated term Bn as:
= ag (I- + 11)) (I-12(20 + (21) Bo
Using this new notation, we have:
1 Σ A.Y, B[AM] - a(A1) (
gINT,db.
Ou
-ad Go
n1
93ep (2))
iEDs
Ba
Σ [14(1.2.455) Tj
Σ
A&X, - B[AY]
Okg 3u
n
I.NT (A1, A2)
93pp (A1, A22)
IED
As consequence,
O Sop
0%)
20

---

1 Σ
( A.Y, A.Y. B[Ar))
Bn
-n -v Eje(1,2,45) T3
Vn
at 30
(A.2)
INTT Je<p
(A1,
ggep (A1, A2)
INT
VEDs
That the first term is Op(1) (in fact, asymptotically normal) follows from CLT var( < c0. We now show that:
Dn (org (o) 93sp
INT (AI, O₂,
(A.3)
Towards that end, we apply Theorem 1.1 of 27
Conditioning on D2, and defining
(I- 1222(E2 + x21)-1) So. we have:
C
[[n (1 XpmMr(-Ai))ay w/ [D]] S ||0oll2 | ({1))*
(A.4)
for some function (-) and constant C > 0 (see the proof of Theorem 1.1 of [27 for details). That ||oo|| S Ca by our modelling assumption (see Theorem 3.1 For the other term, observe that:
|| = || 20 + X21 221)-11 8o
| = 10 (I
X2( + A21) 1221) 1)
2
8o
P
A2
2
(0g 80)2
(hy +22)
2
Σ
[0) B0)
\3 X2
$ Σ (0) = 12 - C3.
=113011
j=1
Therefore, using this bound and taking expectation on both sides of Equation A.4 we have:
CCaCs
E n
Bnn (1 )1mmM((Ai))a S
(A.5)
<b(11)
Now, another application of Theorem 1.1 of 27 yields the following for some constant C² > 0:
C'C,CB
E Vn 0g
AP (-)2))ag 8o]]
(A.6)
(222)
Combining Equation A.5) and (A.6), we conclude Equation (A.3). Therefore, it only remains to establish the tightness of the last term of Equation (.2 As starter, consider vnTīg:
X P2
Volz = aj (Eg + A21)
Vn
A 100,) is of X, we have:
Ta Ta X2 ~ N (0 NO
(Lx + A21) - - -221)
Furthermore, observe that ad (Ee + +221) - + X21) a0 =
P
d
_ 20)2
(A, + 2222
j=1
I C(22)
(1] ox)²
= C(22))lao11 < C(A1. 22)
Here, C()22) is the maximum value of f(x) = x/(2 A2) This implies T3 (n Now, all that remains to show is that /n T7 is tight. From the expression of Tg. we have:
Vn Tg = E
X
X_ P2
[ + X1
(S2 + -211
n
21

---

Now conditional of D2 and X1, vn T3 follows a normal distribution:
Vn V (D2, ~ ~ N (o. 12 X (E2 + 22) (Σi + A - - 'E, - X²I)
X pi
n
Therefore, all we need to show is that the variance has uniform upper bound with probability going to 1. Towards that end, we use the fact ||Ellop 2(1 + ve)² with probability going to (24)) First of all, we have:
11 X2 (E2 + +21)
X P22
'É, - E + ( +21))
n2
n
2(1 + J0)²
n2
From the independence of μ and X and the fact that NOO, 1), we have:
X2X
=
[r([in)] cE [ d e f dfMpp()
E
(d) + O(n
Therefore, the variance is uniformly upper bounded with probability approaching 1, which, in turn, implies that Tg = Op(n-1/2)) This concludes the proof.
A.2 Proof of Two-Split
dINT,ab, as defined in We first consider the case when A1 # A2. We start with the definition of 22p
Equation 3.4
INT (A1, A2)
a( ((22) A&Y,
gINT,ab 2ap
80-
1
2sp
INT (A1, x22) 2ap
I1,2ap (A1, A2)
iED2
22,2sp (A1;A2)
T( INT 912ssp (A1, A22)
Σ RED2
A - E[AY] + E[AY]
-p
INT 212ssp (A1, A2)
That (E, A,Y))/n [[AY] is Op -1/2,, follows from CLT. Furthermore, we have [[AY]] 0g 3o + Oy. Therefore, all we need to show
(1- INT (A1,
(((x))T((>2)) 91,2sp (A1, A2)
1
T 060
30 80
0 = Op
INT 91,2ap (A1, A2)
INT
which is equivalent to show the following:
a(A1) (ag (o) gl.2sp (A1; + 0 INT
92,2sp (A1, A2) + 1122)
For the rest of the proof, we denote by R(A) (É) + -A) for notational simplicity. We use the following identity our proof:
A2 A1 (R(A1) R(22)) R(A))
(A.7)
(B - A)B- We start with the definition of which follows from the fact A-1 - B-1 = A ((A1)TB(A22))
a()1)T((22) (at (6) DI (A1, 22) 0 92.2pp (A1, A2)
INT o0 (I- R( - R2 (a. Bo) g1 (A1,
==
X F4
X + + (((A))))(
+ af T R(A))((())))
n
n
22

---

X1 n
X 11 do 92p (A1, X2)
R(A2)
n
Our goal is to show each T; is Op(n Op(n=1/3) 1123). We start with T2: from the normality of μ and its independence with X, we have:
vn T2 X1 N S
Now to show that variance is uniformly upper bounded with probability going to 1, denote by Σ VÀVT, the eigendecompossiion of ΣT Then we have:
23
ag ELHAAWRRAAAEERRARRR =
Σ (0) a0) j=1
(î₂ +
)1 *
+ x2)²
< C(A1, 12)
(0) (0Jani) an)²
= C(Ar, (2)A112))los 12) 112 S C(11, 22)
Here C(a, b) is the maximum of the function f(x) = x²/ /(xx//(++)))) a) b)) over (0, oc). This shows that T- (p(nn113) The proof of T3 = Opp(n 1/2) is similar, hence skipped. For Tú, first observe that
(x n
X F R(A))
(1 (1+ (1+0 tr (RIA)RRW)
X var
X1
n
n
= + (1+)0)) / (1) + 22)
x2
dān. (x)
(1+ 2 (21,)222
where C²(a, b) is the maximum of the function f(x) = x// /(x + 2)2(2 + 6)3) over [0, oo). As the upper bound does not depend on X1, we have T3 - E[T]] = Op(n" (1/2). Now, let us consider E[T,]:
E E[Ta]=
tr (R(A)) E )]
INT Oo
92,2sp (d1, A2)
n
I dinn (I)
x dl MP (x)
cao E
A1) + A2)
(I (z) Az)
= =O [Theorem 1.1 of [28]
Therefore, T2 = Op 1/2). Finally, we need to show Ti = (n-11112)
a (I- X₁ (T - >₂ (R() (a) Bo) of
INT 22)
28p
=a, (0 g1,2pp (A1, A2)) - A R(A1 So X2ay R()2) Bu + A1 2
= =a (o D1.25p (11, xxz) - ao
( (2 - u 1(22)
\1 ai (R(Ai) R((A2))0
+ A2 - A1
A1d2
A1d2
=a 01 g1.2sp (1,)22))
ag T R(A)33
122
ag R()2)83
A2 - A1
d - N1
X2
X3
-g 3o 91,2ap (A1, A2))
+
Ogj R((A))3
ag R()2)33
d A22
À - A2
From Theorem 1.1 of 27
we have:
00 R()))0 a 30
/
x. dFFMp())
00p(n
x x
I. xx
T. R(22) = aT 30
+ Op
I x X2
23

---

This implies:
00 (I - (an Ba) gi,2ap (A1,
x (F)
XB
d #dFFMP(2
a0 30
INT (1,2)((11 + \1 X2 (1-
+Op(n 1/2).
x + A
A1
X2
x A2
where the last equality follows from the definition of INT
91,2sp (A1, A2). This completes the proof.
Now, consider the case A = X2 = x. The entire proof almost remains identical to that of A * X except that cannot use Equation (A.7) to establish that
ag (A)30 ag Bo /
dFMp(1)
(x X)2 Op(n
However, observing that = a =I) =1) le=x, one can follow the exact same argument as used to prove Theorem 1.1 of [27 and arrive at the conclusion. We skip it here for brevity.
B Proof of Theorem 3.2
B.1 Proof of Three-Split
The proof shares certain similarity with that of the proof of Theorem 3.1 Recall the definition of 0
{ -
(11,
@NR,db 3ap
T(A1)T
93ep
93sp
INT
(A1,
iEDs
Expanding the definition Y, and A, in terms of X; and (Es; p;), we have:
db On (an T(
Eggh a(A1)"
((22)
93ep
NR.
A1 A2)
Σ
ex,,800 3o
93sp
INT
(A1: A2)
:=
72
+
,xX; (00 a(Ai))
+ Σ
144 60
kEDg
AEDs
It is immediate T, = (1122 by CLT. For T2, observe that:
var(e x (8) = 1|Bul12 < C3..
Now, for T3, observe that,
1 Σ U.X. (ao a(\1)) Dj )
n var (T3)
var
7
iEDs
E vaa (μ X (as a(Az)) Di)]
=E ||000 á(>1)l13
112 ((())) XTEE
E
|| R( R(Ai)ao
n
S C + 1 E tr (1(()1)):]
C + cE /
(x+ 21)2 x d ddn)(()) S 50 + +c((((
x
24

---

Here C(A1) is the maximum value of the function f(z) = x/ x1)2. This, along with the fact that [[[ 0, implies T3 = Op( We now focus on 7i. Towards that end, first observe that,
conditional on Dj:
1 Σ (o) (x, (0o á(X1))
(0o - ((A))
iEDs
= E [ 30) (x? (00 - -((Ai))) [Di] + 0p(n-1/3)
(0o a())) ( 80 On(n11/2)
(B.1)
Here, the remainder in the second equation is Op(n 1/2) because
var ((X, Po) (x, (og a(Ai))) Pi) - -
which uniformly upper bounded with probability going to Therefore, to conclude Ti Op(n (112),,) all need to show is:
(ao a(Ai)) 8o a(AI) 8(22)
93ep
(A1, A2)
= =0p(n=1/2)
(B.2)
INT
(71, A2)
We start with the following decomposition:
Deep (^1 A2)
(oo a(A))) Bu -
g3pp (^1, A2))
a(\1) 3(x2)
$ (a0 30 - (aa Bo)
NR 93sp
(A1, 22) +
NR 9iep
(A1, 22) (A1, (ap B0)
INT (11, Az)
That the second summand is Op(nn 11/2) is already established the proof of Theorem we need to show:
3.1 Therefore,
(000 a(A:)) 3o (aig (o) M3ep
NR (Ai, A2)l On 11/2.
Towards that end we use the definition ((>1) and Theorem 1.1 of [27] From the definition of ((A)). we have:
(00 ((Ai)) 3o {AICE + +X11)= +(E, + A11)
-1 x Ei n
3o
Hence:
(on a(A1)) Bo - (ar 30) NR (X1, X2)
-1 (Σ1 + -1 (a Bo) gāep (A1 (A1, A2) + ( + +111 X
NR
Now, from the normality of we have:
X N CO,N00,,80000 ( - + -11)-
% ($) + A1
Vn
Now, using similar calculation as before,
TT + A - + +111) 5 c)C(1)
where, as before, C(A1) is the maximum value of the function f(x) = x/(x + > For that final part, we have from Theorem 1.1 of (27]
CCaCs
A (E: + A11) 30 (ai Ba) F3ep
NR (Ag, X2)
(b(1))
This completes the proof.
25

---

B.2 Proof of Two-Split
In this subsection, we establish the vii-consistency of 2sp
(N,4b From the definition (Equation (36)
we have:
00
2sp
9
INT (A) NR (2)
V - x7T((A) a(AI)T B(A2)
93ep
NR
A
00
91.2sp
INT (2)
Ji.2pp
SED2
22p2pp ())SEE NR (A) (A)
INT
Vi(A, -x7x(x(())) - (A - (AV7A-X((AA1)) xTa((i))]++
INT 91.2sp
(2)
n IED2
gäsp (A)
+A (A)((AXXT((A) - a(A))
9i,2sp (A)
INT 92,2sp (A) (A) (A)
1
Yi(A. xT)(Ai)) ELY X xTa(ii)
INT 91,25p (2)
n SED2
93ep (A)
+0p + (ag a(At)) 3o - (ag Bop (A) + (ap Su)g (A) a(A1) B(A2)
11.2ap (A)
INT
Furthermore, in the proof of the vnn-consistency of 22pp db
we have already established that:
a(Ai) 3(x2) = (an 90) gi. (A) + On 222
(A) + Op(n11/2).
which implies
INT
NR(A (A)
a(A:) 3(22)
3ep
(aT 30)g9apppppp NR (A) + 0o
92,2sp
+ +Op(n
91,2sp (A)
INT
2s, (A)
INT
This implies that all we need show is:
(ao a(A))) 8o - (ax B0)
N (A) = (p
Towards that end, first note that:
(ao a(\1)) 80 (ag 80) 3a)gasp NR (A)
XTE1
= A1 R(A1)ao - (ao Bo) 936p (A) - -8 R(x1)
=-
T=Ta
That T, = On follows from an application of Theorem of (27. For T2, we again use normality of c and its independence with X.
Vn T2 X1 N (0 ((N)),)))))))
That the variance is upper bounded follows by a similar argument used to show that T2 is Op 1/2) in the proof of the Vn-consistency of 22p @lNT,abb This completes the proof for Àι + A2. The argument for X1 A2 is similar to that used in the proof of Theorem (.] (two-split case) and hence skipped.
C Proof of Theorem 3.3
C.1 Proof of Three-Split
We start with the following decomposition of (DR,db;
0 db 00
26

---

1 Σ (Y -Y--X, - (A X, a(A)) a(A1)* B(22)
DR (AI, A22)
800
935p (21; A2)
iED3
D
(11, 22)
(9u - [(x22)TEs((ar) Es(aw a(h1)) - a(A1) 3(22)
+ +(5 - -B(22)
Σ X1a
93ep (A1: A2)
iEDa
T2
1
+ (oo â(A1)) Σ Xit4 +
Σ xED3
P4G - 80
iEDs
:=1g
That T2, T3, T4 are Op(n follows from the exact same argument as in the proof of Theorem 3.2 (three-split case). Therefore, we focus on T1. Similar argument as in Equation (B.1) yields:
(Bu 3(22)) Es(as - á(\1)) T (3o {(22)) T (au - á(x1) + Op(n 1/2).
(C.1)
Furthermore, from the definition of ((Ai) and 3(22), we have:
(Bo - B(22)) &(A1)) = A12 (Σ) + X11 22
A21)
X. 122
-1a (Σ)
+ A21)
X
A200 (E2 A21) (Σ: + A11)
n1
6 [
X p22
+
{ + 11 (50z + A21)
n
n
= Tn + T12 + Tı3 + Tu4
(C.2)
First, we show that T12, Tis and Ti4 are Op(n (./2). -1/2). Towards that, first observe that:
vn T12 X2 ~ AW (0, + I 1 - + - + x21)-1($1
+ -1
To show that the variance is uniformly upper bounded, first observe that:
C2
(52 + a (E) A11) 1 + A21) -
-
lo S
Σ llppp
Furthermore, we know 1/2p < 2(1 + 222² with probability going to probability going to 1:
(24
Therefore, with
2C2 an ( + 15++ (Ex + A21) - Ex Eg + -2 + +111 an S
(1 Vo)2.
which establishes that Tia On -2/2). The proof of Tia is the same and hence skipped for brevity. Observing that T11 is exactly same as Tg defined in the proof of Theorem 3.1
3 (three-spli part), we
conclude that Ti4 is Op(nn11/) Therefore, combining Equation (..l and C.3 all we need to show
DR (A1, 22)
(() (E) +
+ 121) T ((22)
1/22)
(C.3)
Jösp (^1, A2)
INT (A1, A2) = (n (Theorem 3.1b we similar calculation, as used to show Bn ag 3o 93sp
have:
Aoag (É1 + A1 111) ((E2 + 121) 3o - (a. 30) A (emmar(->2aalaaappm (21)) = Op(nn1122)
Recall that, by our definition of DR (A1 A2) (Section 3). have:
DR 93ap
(1 x2)
27

---

Therefore, we have established:
Aιdz0y (E1 + 111) +221 - (ag Bo) 93sp (12, X2)
DR
= =Op(n =pp(n-1/2)
Furthermore, we have already established in Theorem 3.1 that:
((A)))B(A2))
on 30
Op(n
TNT 3ap
(A1 A2)
Combining this with the above equation concludes Equation
C.3
which concludes the proof.
C.2 Proof of Two-Split
We start with the definition, as mentioned in Equation 3.9
DR,
Oo
2sp
1
Σ [Y, AA xTa(A))) a(\i) T( ((22)g2,22pp DR (A)
H(A)
00
LNT gi.2ep (A)
SED2
where
H(A) = 1 INT (A) (1-
22.2up (1)
( (A) (x))
))
g
First, observe that:
(Y x X X xTa(A))) Pi (ao a(A,)) -B0-)) + +80
Using this, we can expand the expression as: @DR,db
2xp
H())
=
1
- CXxxX)B(N))44 xTa(A)) - E (Y
[2]]
1ED2
((Ai))(A))) (Bo--(Aa)) + +00 a(\1) (122) DR (A)
-00
9122.p (A)
INT
The first term inside the square bracket is Op(2=112) by CLT. Therefore, all we need to show is:
H()) {co0- (coo-a(Ai)) ( - ((A2)) + 0n) a(\1) ((22)) (2)
0. Op 112))
91 g1,2sp (A)
INT
The term insider the curly bracket can expanded as follows:
(ao (o-(((2))) (80-3(22)) = 22 R(2)) - Do R(A)))(2) XIMI
((22)
X E _ R()))((2) d2 R
X
n
n
That the second and third term are Op(n 1/2) follows from the same argument used to show that Th and T33 Op(n=1/2 in the proof of /n-consistency of @db Furthermore, using same technique used to proved that T4 (n (1/2) in the said proof, we can conclude:
T R(A))((2))
X f1
INT 2
(A) + Op(n-1/2
and the same technique used to prove that Ti On 1/2) in that proof can be used to show:
ADaarr
(ag Bo)
g2.2pp
DR (A) On
28

---

Therefore, we conclude:
((oo))(Ar)) (B0 - ((2)) + =00] (an (o) 9222ap (2) + 00 + 92,2ap (A)) + Op(n -1/2).
As a consequence, all we need to show is:
H()) (as Pb) 22.2ap (2) 0o(1 + 92.2sp (A)) a(\1) ( ((22) DR (A)
-0 = 0n 112)).
91,2ep (X)
Furthermore, it is also proved the proof of Vīi-consistency of @INT, that:
a(A1) (og Bo)
INT 9i,2sp
(A) + 8g 92,2sp
INT (2) + Op (n 1/2).
Hence,
((A1) ((22) DR (A)
INT 92,2sp (A)g2.25p (A)
DR (ag 30)92.2sp (A) + 0o
+ Op(n 1/2))
91,2sp (A)
9 2sp
(A)
This, along with the definition of (A).. concludes the proof. The argument for A = A2 is similar to that used in the proof of Theorem 3.1
(two-split case) and hence skipped.
D Proof of Theorem 3.4
D.1 Limiting Variance for aN
3ep
We start with the definition of dN
3sp
A,Y, ((A))T((A))))
db 3ep
Σ
93ep (21;
REDs
From the independence of D1, D2 and D3, we have:
n X var (OLNr.ab) var(AY)+ +
(ggap (A1, 22))2 X var (e(x1)T((22))
1
INNT (11,
As consequence, our goal is to derive the limit of n x var (Q(A))T from different subsamples. Let's start with another expansion bit simpler version) of &(A1) ((x2):
((22))
when they are estimated
a(\1) a(
E100 + x T
}
X, 12
{E, + x11) (Ie + (E2+>el)
Eg00 +
=f En X11)
(In + +211
Egs0 a
I
En
ES =xx1) -1
{Sg +
+221 121
X p2 n
+ 20 Eo
Ei ( + 22l)
-1
[ ( + \1l)
-1 X {
\ + +1+1
-1
(So + 121
-1 X.J M2
n
n
n
A T + T2 + T3 + T4
(D.1)
We will compute the variance from the first principle, i.e.
var (a(A))T 3(x2)) E (a(A)) (A -B [(xa)))
E
First, let us take care of the easy term, i.e., the expectation. As (M, 17) ILX and and μm are also independent (as they are from different subsamples), we have:
((A)) ((22)) = E ag T (Σn + 211 (Z2+ + 221) E26
29

---

To compute the expectation of the right- side. let us first compute the conditional expectation given D2. For notational simplicity, define w2 {Sig Az2
En
E ((A))T((>)))) D2
-1 a En Σ1 + 111) W2 D2 ag wylE
[
x
x A1
where the last equality follows from the same calculation as in the proof of Claim 1 of Lemma E. and the independence of the eigenvectors and eigenvalues of Σj. Here, nn,1 is the ESD of E. Now taking expectation with respect to D2, we have:
/ x + A2
E a W2
E ao
Σ + A21)
Σ260 ag ao BolE
dā dām2(a))
where nn is ESD of Eg. Therefore, we conclude:
E ([(())T((25))
=00 30 E
I
d 1
E
x
di 2
+ A1
+ d2
=00
T 3o E
+ A1 d (x)
x
x
dar 2(x)
(D.2)
x
x2
Now, we focus on the second moment. Again, as before, conditioning on D2, we have:
E (a(2)) 2 D2 = B(x2) TE T (a(A))((A)))) ((22).
From the definition of á(A1). we have:
B(A2) TE (((A1))(A1)) 3(2z)
= = (2) (8I+A11) Dianad Ei (2) + +111) -17 3(A2)
+ 3(22) T (8I+ $100 &ix, ($1 + +1111) -17 -1 3(22)
BB22)TTB E
(Si++111)
X,JEI I
T (E) + +211) -1 3(A2)
n
+ B(22) (54 -1 -I
ef (
X1 ( + +111 -1 - B(22)
n
The expectation of the second and the third term will be 0, e 11, Therefore,
3(22) T [) B(A2)
=E
(SLAE) (E) (Ei+Art) )100)
27
((2))T TE
(Zi+ X11)
X n12
(Ei (Ei+A11) -1 3(22)
E (3(20) (E (81+)11) -1
$1a0)
27
1 (A2)) TI (SI+ -1 ES ( + >11) - B(12)
Again, by the same calculation as in Claim 1 of Lemma E. we have:
3(22) T (SL A11) Σn Σ + X11) ((2z) = 110()))1
/
d d (x)
(x + A1)²
(D.3)
For the other term, we have, by the same calculation as of Equation F.. in the proof of Lemma E. (with un = an, wn, = B(22) and f(x) = x/
27
(((xeTT( (Ss + A11) Eιa0)
E
(D.4)
30

---

31))) + 2(a0 ((22))
/
(x + X1) d (2)
x²
(p + 2)
a0 B(A2))² |100 (111)(122))
27
p
p
E
din, (10)
((t)
1(0))
p-1
p+ 2
(o+ 2)
(x + X1)
1
p
(ag) [(22))² | 10o11/11 (211
E
x.2
dā (x)
p -
p 2
(x+2)
(x + x1
p
(1-2))
2
(an ((22)) + ||001213((22)11
E
(x (x dān,
x²
(p 1)(p + 2)
p
p²
(an) |100 121 (3))
27
E
din din)(1))
(D..)
1p-1)(p+ +2
(x + X1)
Combining Equation
D.3 and D.4), we obtain:
(a(A)) 3(22))
E
D22
(D.6)
P
(1-2
(ag T ((22)) + (1o1
[
x²
d (x)
w - 1)(x+ + 2)
p)
(ar + >n)2
p²
|00 (101)11((200113
27
+ (p 1)(p + +2)
(ag (X2)))
(x + (1) nā (x))
P
1 11(222))1
di da),
(x + \1
[V
(a) ((2))))
dff dffm.((1))
x +1 A1
+ (p- 1) +2) 2)
p
E Varxmtta
X
)
(oT (1-2) 2) ?) + +1001111/(221111 p (D.7)
x x+>1 +
1 1 113(22)112
TE
dān, (x))
(x + 21
It is immediate from Equation (.6) that, for the unconditional expectation, we need to compute the following expectations with respect to D2, Bl(ag (( and B[]l (12) The calculation is similar to before, with slight modifications. First, let compute B[(a, (A2))) From the definition of (A2), we have:
E (at ((x))) 21 = E
T ES X21)
E200 + and (Ieg + 221)
X__ 142
27
00
E2080 22 +E ao (Iz + +212)
X.J 1222 X2
E
an
T
SS + x21)
-
(Ig + +x22)
ad
n²
+ 2F
a
{S2 + +xo1
-1
[5e + +1211)
-1 Xx k2
n
E
Srz -221
-1
E22180
27
+
1
ao
{Sg + X221
-1
ES2
(Eg + d21)
n2
/
z²
||00121100 12
2a
30)
(a >2) d 2
(p + 2)
31

---

| 100111116 1211
27
P
p
(ag 1b)
IE
(x + )2) d(nna(a))
P 1
P + 2
(p 1-+22 2)
|100| 111.B11.1113
x2
p
(ag) .002
E
d d))2(0))
p
p 2
(p 1p+2) 2)
(x+ + xx))
+ n
11a01l2 EE
x
d 2
(x + 22)
p
(1--
(ag Bo)2 + |100 1111
/
(ar - x2) d 2()))(z))
x²
(p - 1) + 2)
p²
(an 30)2
| 1 1oll
E (/
I
d d(tn2(())
27
(p 1)(p + 22
(ot + X2)
1
//
+ π
| 11
da 2
(x + X2)
The following simplification of the above expression would be required later:
[
27
E
(00 (OTB8AA))
(ao 30)
dā 2
d2
X
(aT 80)2 (1-2) p
E
varx
+ -110o| 21.3011.8013
(p (p 2)
x A2
+ |1001 E
/
x
dit 2(z)
(I X2)2
[
A (ag T 30)
di dhizz(1))
+ RI.
(D.8)
x d2
For the expected squared norm of B(22):
||( + d21)
X 1b2 27
E
[18(22)11 [3]
E
E200 + (Eg + d2I)
12
-2
H2 X2
-2 X 12
B E
Ee
Ex AI)
Eg
3o E
Ei AI
n
X22 P2
+ 2,30 E
Li Sig +1
23
X
-2 X.
]
B E
03,20,. T
1,2
(2yi2 + 2)2
3o + E tr
Σg + +1)
A3.2
E
[3T 0322) 2
+ E tr
( + =11 -2
22)
(g,2 + X)2
11.6112
E
(x x2)² di (2:)
+ +c E
(x x22x ( dr )
x
(D.9)
Finally, we put all the pieces together to compute the variance. First, taking expectation with respect to D2 on the both sides of Equation (D6). we have:
(((A))3((22) 7
E (ag B(xz))? E (
27
x + A1 dRn,)(z))
32

---

p
X
)1
{
E [aj ((12))))) (1-?) 2)
|100 [11((22]1 }
+
E varx
(p (p + 2)
X + A1
p
E [18()b)11]
(x +112 (2)
[/]
)
27
[f
(a 3
d 2
x+ A1 dan.1())
+ RI
x + X1 dāt, dānn, (x))
x + 22
p
E
varxwata
X
)
E (a B(A2)))
(1-
?)
2
1 (11(222) 12
(p (p + 2)
X + X1
p
1 13 [11((N))13]
/
(x 21)2 dz)
(ag 3 [//
dim2(())) 27
[f + d1 dān, (2)
x A1 10)
R1,p
x A2
p
X+ }1{B
X
{E
B[ (aT B(A))))))
(1-?)
2
|100 [1((A22)1
E
varx
(p - 1) + 2)
p
E [10()))]1 E
21)2 dān, (2)
This, along with Equation
D.2 yields:
var (d(Ai)T((2a))
= (aa Vaar
I dā 1
/
dinn2()) + RI
(J x + A
dānn (x))
27
x
X2
+
p
E varx
X X+ X1 }
X
(E. ( ((2)))
(1
(1-2) 2
|1002 [113(22)1
(p () + 2)
p
+ E [113(1)]12]
/
diin (x)
(x + A1)
Ti + +
To compute li var ((aa)) we compute the limit of each T3 separately, From Theorem
1.1 of 28 we have,
lim x 7 = e² lim x var (
I
1
d
x
di dinn2(1)) 0.
1000
1 X1
x + A2
From the definition of Ri. in Equation
D.8), we have:
1 Rip varx=Burr
X
+ u² f
dFMp(x) /ddFar(())
lim n1.00
c
X
(x + X2)²
Therefore,
q² +
u20
X
x [FdMp(()) (x )2)²
(
ddFp An
lim n x T2 n1o0 Furthermore, from Equation D.9 we have:
var x X x Xx
+ +u²
+ (
x [SdFFre(e))
lim [11))()))1 o² oo
(x++>x))) 22)²
(x + 22)²
which implies.
lim n x T4 = (2/ 1oc
+ )
(2)))(x) ((=dFFre(())
) (
x dFM
(x + (2)²
(xt + X2)²
(x x1)²
33

---

Finally, for T3, first observe that,
lim E n700
=
x x2
This implies,
1
{ 0² )
X
(fa
( +
lim n x T3 ntoc
-varx-Fur
X X=11 Xn
x x
u² 0²
(
x² ddFarr (x)
x @ddFrr (2)
+
(x + 2²)²
+ (x+
So overall, the limit is:
lim n x var (G(A))T 3(12)) 2x.00
0 + 22
X
)
x (dFMr((x))
ddFrr((a + A1
+ +u²
varx~Far
X+
(x + )2)2
)
X
x. dF
X
x x X2
0² (J
x² dFFpp()) (
(fedFire(a) dFMp(z)
u²
(x 22)2
(x+ )2)²
x2 dFFap(z)
ddFap(z)
022
(I (x+ 22)²
te
(x (x+
(x+
Therefore, the limiting variance of @INT,ab is:
3ep
lim vår (ONNT n1000
② + +200
x [xdFu((x)
(
x ddF
= -Var(AY)++ + ( (1,, d2))2
+
X A22
(x x2)²
A1
X
x dFMP(E) (ddFFr(())
+ varx~Fer
+
X+
x A2
(J
x² dFMpp(I)
to (
(x x2) (
I (IdB((()))
2 v²
(x x2)
+ { 2
(f (F
+c (/
x ddMu(()) (x )2)²
dFMp(a))
(x (x+
(x+ >1)²
This completes the proof.
D.2 Limiting Variance for 2 dl
Recall that in the case of 02pp
@INT,ah
estimate or and 30 from Dj. The estimator takes the form:
(1
INT (Az, Az) 92.2ap
a(A1)TB(M))
Σ
A.Y.
9 2sp
INT
(x1, A2)
n
9122pp (21, A2)
INT
iEDo
Therefore,
INT (A1, 22)
-2
n x var (a(A1)
X var
(dbT,db) 02sp
92.26p
vAY(
gi,2pp (21,
(g1.25p (A1, A2e)
34

---

As consequence, like the three-split case, we need derive the limit of var (
but the only difference is that they are now estimated from the same subsample. We break the proof into two propositions.
Proposition D.I. The expected value of the conditional variance of à (A1)8(22)) given X2 satisfies:
+20+c1++ /²)) / E var (A1))((A2) x1)] =1
x² (x) (o + 21)3( + 22)²
lim tf00
Proof.
From the definition of the conditional variance, we have:
var (a (TT())B(Aa2) x1 =E ((TTA)))))))) |x| - ( ( xx1)²
Let us first start with conditional expectation. For notational simplicity, define R(A) = ($) + AT)-1 From the definition of â (A2))(A2), we have:
a (T(N1)((A2) Si00 X T
($1.00
XI
R(A))R(A))
X,
00 En R(A S Bo + T R()))
n
X 6 T
x 1
+ 30 Σ R R(AA)(()2) R(22)
R(A1)
n
n
n
T + T3 T3 TT
(D.10)
As we are interested in the conditional variance given X,, Ti, being a function of X,, will not contribute. Therefore,
var (aT(81)(A22) x1) var(72 + Ta + T4 X1)
E [T2 T3 + T X1]
(E T3 Ta) x,))²
As and are centered Normal random variable and independent of X, we have:
E B X,
E
X1 0.
Therefore,
[[[ T3 + T4)
X1] T X1]
E X
X X, Ptr (R(A))R(AA)E,
R(A))(2)))
n
(D.11)
Next, we consider [T2 + T3 + T4)²
X]. Expanding the square, we have:
(T2 Ta + T XJ E
X
E X + E X
+
X + T X + ETTTT, XX)
35

---

From the normality of (c, ()) and their independence with X, we have the following conditional expectation of the six terms above:
Σ ((R R((2 E (((A) R(AA))
E
X
80 En R(A)) R(AA) ((A)))(2)))))) R((1)
((A2)
E[Z3
X
X = (1 + x²) tr (SROA))RAA + (S
E
n² X d0 ΣS R(A))R((2)E R(A) ((2))))))
TT
n
E|Eu2]
ETT2
X]
00 T IX (AARA))X
n3
E|
TT7374
X
n
(D.12)
Combining Equation (D.1I) and D.12 conclude that:
var (aT())3(A2) X)
00 (i)((1 R((2) SS, R(A1) R(A))0 S (AAAAA)
+ a0 E EA
(1+ tr n
Ltr (SIRAT))RA))) ((AATRRR)))
R(R()) ES R(A))
AT
ag DE
p1+ tr (AR n
(D.13)
Therefore, we need to compute the expected value the four terms on the RHS the above equation. Towards that end, we first compute the expectation of general term of the form:
u DAAD
E
This would cover the first three terms of the RHS of Equation D.13). Suppose Σn 20=10,00. denote the eigendecomposition of Eg Then,
E ROA
83
E
E
(u (v)
(Ag + A1) + 2₂
(D.14)
From the rotational symmetry of the Normal distribution, we know that the eigenvectors and eigen- values E are independent. Moreoever each eigenvector 0y is uniformly distributed on S Using this fact, we obtain:
X3 (u (v T T0,) (4g 2)a
X
E
(u (v (vTT))
(Λ₃ + X1 ( + X2)2
v u
E
p
()g + A1 (), =22)
36

---

In the last equality, we have used the fact that for random variable 7, uniformly distributed on S
we have [[( () m)] = (u v)pp. Therefore, going back to Equation (D.44) we have:
u T HAAD
u
V
X3
E
E p
(22 + X1 (2 + x2)
x² dit (x)
(D.15)
n
(x + >1 + X²)
We now consider the expectation of the fourth term of the RHS of Equation (D.13) decomposition of Σ we have:
Using the eigen
(1 p²) tr (ER(A))RA2))) (1 p²)
23
n
n²
(2y + 21 x2)2
c(1 + p²) E
83
n
\1 (2 12)²
c(1 + p²)
x2 d#n (I)
n
(x
2 + x2)3
(D.16)
Finally, combining Equation (D.13 (D.15) and (6. we conclude that:
n E var (A1()))(Aa) }) (Ilao + Boll2 c(1 + +D)))
x²2 d (x)
(x A ) + )2)
(D.17)
Now, using DCT and the fact that ESD of ΣΙ follows Marchenko-Pastur law, we conclude the proof of the Proposition.
Proposition D.2. The expected value of the conditional variance of à (A1))(A22) given X satisfies:
lim n X var [E a (Az)
x)
[u20² + e²) varx (X+ + +xo))
X2
x100
Proof. is immediate from Equation (D.0)) that
E à T (A1))(222
x)) = var
R(A, (()) E[T x))
var
var (a) E1 R(A))( ((2))) Ltr (AA))))(()aE))
Next, we compute var [aT S. R(A)) {1,0) and then argue that the variance of the second term is negligible order. Towards that end, an application of Lemma E. yields:
lim var (aT (())))))) u² 22 o² of000
X2
varx~Fur
(x - A1) + +22)
c
Therefore, to conclude the proof of the Proposition, we need to show that
(Ltr (((A)) 0.
lim n x var
nooo
n
Towards that goal, we will again use Theorem 1.1 of (28
Ltr
((A))))()))
lim n X var cp / n1000
I d+n((x)
lim n x var
nocc
n
(x A1) + A2))
x dir (xr
c lim n var
(
nfo0
X1)x + 2z)
Here, the last inequality follows from Theorem 1.1 of 28
This concludes the proof of the Proposition.
Finally, combining the results of Proposition D.I and D.2 variance of 02pp an
we conclude the proof of the limiting
37

---

D.3 Limiting Variance of @DR,db 2sp
Recall that (DR,db has the following form:
H1(
@DR a(\1) ((22))2A1))
1 Σ (Y x. (A -(AA)) 8(A2)H2(()) (((A)B(A))))))))))
(1
71
iEDs
(0o a(A1)) (603)))0-0)>))))) + 1 Σ
= (A)
(x. (3o - 3(A2)))
IED2
Σ iED3
(X (ao - a(Ai))
Σ SDDs
p.c. a(\1) ((A2))(11)
next use the tower property conditional on Dj The conditional expectation is:
E
DR,db
D. =
(1)) (00 - a(A))) - B(22)) + 0. - (x)
2xp
Taking the variance with respect D1, we have:
E @DR,db
|D1))
var
(A)vvr ((an (olo))(A1))) (
(A)vvr (a(AI)T 3(A2)(1 - H2(A)) - a(AI)* 30 a0 3(22)
-A] [L (1- (L--HH))) ( + var((())) 30) + var(a 3(x2))
-2(1 H2(A)) (a(AL) ((A2)) a(A1)T TB) -2(1- - -Ho()))cov (a(A1)T B(22), og 3(222))
+2cov(á(AI)) Bo, an 3(A2))
We now derive the limit of each of the summands. For the first term, we have from Proposition D. and D.2
lim n x var(á(Ai) = (n² + + +0 +0²+20+c(1++1 ++1(++10 p?)) / ntec
x²
d.
(xc + 21)2(ar + 22)
+ - lu222
+ e²1 varx~Fxp
X2
(x+>i)(X Xxx++N +
For the second summand, we have:
T Bo) var ((RAA))SSOD+ +((A))
X T
R(A1)
Su
XTa
B R(11)
var (N. ((A:)88)))) + 1B N ((AA))
var (a. (1)1; + 11.00122 E
/
From Lemma E.I we have:
1 -0² + u2v2) var varx~Fae
X
lim n x var (a (aTi;A(A))8))) 1700
x+
38

---

Therefore,
lim var(á(Ai)) 36) ( 1²) varx~Far n1=00
X
+ /
(2)
c
x+ X1
(x \1)²
A similar calculation yields:
X X + x222
+u² f
x (dFMp(2) (x + 22)²
lim n x var ((22) = (e² + u2 var Varx~FMP #f00 Next, we compute the limits of covariance terms:
cov (á(A)) B(A2), a(A1) 80)
(a) (())) X X
30 R(Xi) X
of ((21))))))))) + cov
+ cov (8T X 3 ((A))
61 n
+ cov
Mi [1 X1
R(AI)
X
.g & ((())8)))
n
n
For the first covariance term, an application of Lemma E.3 yields:
cov (n E, ((a))) ag ((A))3))
X²
X
(120² + g²) covxxFin
(x+ +
X+
For the second term, have:
XT
ST R(12) XTG
o/ T ((A)))))
cov
( Xim
(30
30 R(A)) XT
E
0 STT R(AAEERRRARR
Go
BoE /
x2 1 (I) (x+>)) (c+tA1)'( 22)
ag
Hence,
X M1 0g R(A))RA))
X,E11
x² dFMp(()) (1+
lim n cov 2100
90 R(\1)
00o
n
n
For the third term,
XTE EI cov (30 R(A)))
TTE
X
X E1 n
30
T ((A)))(())))
n
n
n
E
3 R(2))
|
x² d (x)
E
(r) A2)
Therefore,
X 3T R()
X
222 /
x² d MP (x) (1 X2)
lim n X cov (())())) 190
n
For the fourth term:
11 X1
X TÉ, 1(2)))))))
((A))
og
m
E
p1 X
R(A2)
X E
(AT1)
7
n
39

---

pi 1
E
R(A))
E
1000
E (()11))))
n
1E tr (((AI)
E tr
(R(A))
00 En (((:))).
00 S. (()1))1)))) ).80
n
[//
ddnn (x)
cEE /
x d (2)
cE
ag ((,)))
[ E R(A1))
(1) + X2)
(x + + + A2)
(ag 3o) {
[
din (x:
(2) (x (x+
n. (x) (r+ A1) (x + A2)
x. din (2r) (a + X1)
E
IE'
(I+ A2)
0 0.
The last conclusion follows from Theorem 1.1 of (28].
Therefore, we conclude:
lim n x cov (a(\1) ((x2), a(A1) (8) n1000
(Ix+ (IXXI
X²
X
= (u + e²) covx~Far
X+11 x² dFMMP(I)
/
x² (F)
+ + /
+ +00
(x (1) + X2)
(x+ + x2)
Now, we analyze the second covariance term:
cov
(G(A))T ag ((22))
XT on R(A2) M1 12
= cov(a) (() 11 ag R(A2) + Cov (ATE))(A))R()))))
T
X 14 + cov [aJ R(x2)
P R(A)))) X[61 + Cov (4
XTE
R(A))
.00 R()2)
T
n
Same calculation as for the previous term yields:
lim n x cov (a(A1) 3(22), an B(2)) 100
X²
X
(u2 + (²) cOVX~PMP
(X+ X +
/
(x) + X2)2 + +u² /
x² (F)
x² d. dFMp((2
+ 00e
(1 + X2)
Finally, for the last covariance term:
cov(á(A1)] So. ag (A22))
cov (at R(A1) af R(A2) + cov ag R(22) X 11 B R(11) X
Same calculation as before yields:
lim X cov(á(>1)) Bo, T 3(22)) xx000
= L 222 + e²) covX~Fai x
X
X x_
x+xa + /
x dFMp((x)
(Ir - >1) x2)
Combining the limits, obtain:
(E aDDRdbb DI))
lim n x var
whoc
+2 c(1 c(l+ p²)) /
x²
- -H2()2)))2()) (u²
+ (u²+00²
(x (x + 22)2 dFMp(#)
+ - ju2 = + e²1 varx~Fur
x²
(X+ +x)
40

---

+ (2) 1 00
(= + + varx~Fir
X
+u²
@dFM((())
X + A1
(a + (x)2
+ (2) (= + uu warkwaan
X
u2
x (FM
X X2
(x + Xx)2
-211 - Hz())) [(2202 + e²) cOVX~FRi
X²
X
(X+AI)(X + +22)²
X+
+000 /
(1) + x22) +1² /
xa (T)
x² (x)
(I+ X2)
X²
X
-217(A)(1 - H2()))
(u + +22) COVX~Fup
(X+ XX)(I + +22)'
X X+
xa (F)
x² dFMp(I)
+00e
+²²
(1) x²)²
(x+ x1)(1 )²)2
X
x d (2)
+ +2H2
(u²u² + x²) covX~FMP
X+>> X +
x1) + X2)
(D.18)
Next, we derive the limit of the expectation of the conditional variance of 02 variance given Dj is:
The conditional
@DR,db Di) var
(2(() var (V x 3A - x xT((A)) (D))
({(A) var (n00 - G(AI)) XXT (30 - B(x2))) + var (xT(3--B(aa)) B(22))c)
+ var (X
(ao -a(A1)))) + var(mE)
+ 2cov (x - -a(A))) X X - 3(xa))e))
H((() (1100- - a(A1)112118 - 3(22)11 + ((a0 - a(x))) (Bo- B(A22)) + 1000-l00e - @(A2))1
+ |180 B(2x)112 + 2p(c0 a(A1)) ((0 - B(22)) +1+ +1)
(A)
(Ti + + T3 T4 + To 1b+++p²)
In the second equality, rest of the covariance terms will be 0 because of independence between X and (e, v) and the fact that [| = Elu = Now we take the expectation of T; for 1 S S For notational simplicity, define R(A)= = (En + AI)-1 We start with the expectation of Ti:
R(Ar + R(A1) X E1
|
2
X MI
E T
E
R(A2) + R(22)
n
2
X2I R(A1) X 61
X
E
2x100 TA (21)
n
1211 + [(22) x P1 x
X. 14
2A23 R²222)
n
x, 14
(( 11112)20011 ( )000)) R((2a)
n
X E1 R()1)
+ R(21) X 2
X m1
+ R))))((22)36113
R()2)
n
n
12
41

---

X ci p X1 R² (22)80
+ 411 T (A1)
RRest have mean 0]
n
n
X + 1 + 13 + Ti4 + 41122lis.
Now, we find the limit of each of the above summands:
Tu B [[ [121 R(A2)
-200 u200 /
d (F))r(1) / (x+ x2)² dF)p(z).
1
(x (1)
Ti2 = E (100)))
R((2)
XT 11
27
1 -
[10(A))
(No())) ]
2
n1
1 a
c
ag n²2 (A1)a0
(Ny + 12)²
p (My \1) (
1
p
=c -clloo12 E
リ= (Ay 12)2
cu² /
dFup(a) (
/
(2F
(x: + X2)
similar calculation yields:
Tia - co² /
(a 22)2 d(wp(z) /
(x+ dF
x
For Ti4, we have:
R((A) X 1
X1
Ti4 = E
R((2)
X2 (AI)XX
(x2)
jel )
E
cT
11
n²
n²
1 tr
R22 (R((AIE)))
trr (R((DO)E))
- _ ((DD))
2 /
(x+ 21)a dFup(11) /
x
(2+12) dFMp(z).
x
Finally for Tis:
[ uu X1 R² (x2)30
Tis = E
T 00 (21)
=6p
E
0
T
R²
(X1)
ag
30
2
= 00
E
h
p
() X2)²
+0
Combining the limits we conclude:
/
X dFFp(x))
X3 ddFMp() (x+
+ cu² /
X2 dFv((())
x dFvp(()
E [i
22
(x + x1)²
(1+
(x 222
+ 2 2 /
X3 dFMp(())
x dF)M((I)
2 dFFMP(I)
x dF)
(x+11)² +
(x + X2)²
(x)
42

---

Now we focus on T2:
((a0 - a(Ai)) (60 3(A2)))881
E [ =E
X A1d2an R(N)))RAA)))()) Aa ((2))
E
n
-22 R(A)) X E R()))(())) X p1
n
E
A T R(A)))(2)))))))
+ +X2 (T)
X p1
X
( R(A))((A)
X 14
+ 2 8 ((A))
E TXI R(A))
X
A2 ((A))
N
X k1
X
+2A1d2 2 (((()))((A)))
30 ((A))))(()2)
n A X2A2T22 + A2 + X2T23 + T24 + 2A1 + +26)
We start with Tei:
[T(221]
-El(ag R(A))
(0) Bu)
=E
E
+ >1 + x22)
[ (00012113011 1301111 + 2(aTTBB)) LE / P(x) (x)
(p
|0013118.123
(/ f(x) dit ditm
+ p
p
(a] (aTBB)) 30)
E
P p+2 2
(p + 2)
1
p
(ag T 00)3
|c01| |0.131130003
E I (2(x)) dit,,
(Equation E.2
p 1 p + 2
(p 2)
where f(x) - 1/(x (x + X1 (x 22) Therefore,
e² (/
dFM
lim E[T21] nfoc
(I >1) + +22)
For T22:
T[222] E ag) R(A)))((())) X
E T RAAN
x d (e+
= =c
_ cua /
x dFFMr(I)
(1 X2)
Similarly, for Tex:
[23] + o22 {
x dFvs((()
+ >1) x2)
For T24, we have:
TX,
XT 27
Tea E
R(x))
n
43

---

tr (R(AA))((A)S))))
+ (1+ 63)
03 (
ddft,1 (x) (x x1) A2)
(1 + 06)
[/
x² d (2)
+
B
n
(x 21) 22)2
(s
x dd (x (1) + A2)
20
Now, for T25:
Ximi ) 71
)
(STR(A)RRCN)) X
2 E
ag R(A))))
n
c X1 (AA)ROA)BBOTTRAAAAAARA XTm
E
E
tr (Boad (AANAAA))A
n
E
R(A R() R(A,) 1(22)8)
x d
(aT 30) E
(1 22)
dF
c
(x (1) 22)
Finally, for T26:
a0 a ((AR) XTMI ) (80 (3T X
E [26] = E
E X1
R(X))R(A2)3B0) R(A))R(A)) X
E
dFMp (x)
cce
(x (1) + 22)
Combining all the terms, we conclude:
lim [[TT] = e² (/
A dFMp(()
(cu2 + cv²>3) (z)
(I XI) ( + tx2)
(D.19)
nooc
( X2)²
x (x) (x+ X1) 22) )
(j
4114200 x (x) + (x x2)
+ =20
+
(D.20)
For T33 we have:
E[Ta] E ||000 &(Ar)1l) =E ||-MeR(Waao + R(AI)
X Ei
n
X X Tx R((A1)
Jai R22 (Ai)aol +E
71
n
d
Ai11ao112E
R²2 (2<1)) + tr
(x + )1 (x)
n
ddtn (x)
= Af11a0012B
+ ccE
(x 11
44

---

Xi dl(
to /
I rdF(p(x)
u²
(x
(o (x+ X1)2
A similar calculation yields:
x² /
12 dg
+ ʃ
x (ddFp(()) (x+ )2)²
E E[aal
(x 12)²
Finally, for T3. we have:
T[ = E (oo - ((11)) - B(A2))]
(
-1 R(()))o + R((A)) x1 T
(-A2R(A2)88 + R((2)) X MI
E
X R(A))R((2))
XIM
(ag A1
R(A))R((2)33)))) +
di d
-1120 BBB
+ (((2))(()2)))))
An) X2)
d (x)
+cE /
x d (1) A2)
A1
BBB
(x A1) A2)
(F)
x (F)
- A1>20
(xr + A1) + A2) +c
(x + (x + +22
Combining the limits of (Ti...., Ts) we conclude that:
lim n x E var (@DR.ab [Di)] wh00
u²??? /
Xp (RdFMr(()
/
X3 ddFr(())
+ou² /
XP dfdFur(())
/
x dFMr(() (x + x2)2
= -2((2)
(x (1)2 X3 dfmp(z) (xc 22)
(x (2)
(x+11)
x dfMp(I)
x /
/ x dFfu((()))
+ cv²
+
(x xx)2
(x+
(x xx)?
+ 8 (/
X233 (x) (P (at x1) + X2)
(cu + ov² (2²)2)) x dF)
(x+ X1 ) 1 x2)²
(f df ( +22
2
1)1d2ce d (x) ( 2z2
208
+ 2² /
X2 dFFM((z)
+ /
dFFM(())
X (z) v2 /
+ /
x dFM
(1)
(1)
(x
+ (2)
280d1d20 /
A1) 22) \o) +c / An)
dFMp(1)
dFvp(e)
+ 1 + 08
(D.21)
Adding the limits in Equation D.18) and D.21
conclude the proof.
D.4 Limiting Variance of @NRd
Let us start with the definition of @NR.db;
02pp
1
NR.db
(2)
Yi(A: -1A:-XTTa -()))
n
VED
where the definition of functions H-),), H2(-) is immediate from Equation (36)). The term (A)
will be squared. As for the doubly-robust estimator, we start with the tower property of variance. condition on Dj:
var ([NR.ab) var (E NR, Dil) + (var (jNR,
[..]]
45

---

For the conditional expectation, we have:
E 8 @2pp.ab
Di (1)( [c00- ((co)) 30 0o a(\i)) ((A)))a()
Hence,
@NRdbb (Dil) (ajvaa (á(A)) 8o + (( ((.))))))))))
var
E
(2) (ar(((())) T 86) + ( 3(22))+
2H (Hg(A)cov((AA)) Bo. a(A1) ((22))
In the proof of the limiting variance of 2zpp
db have already established that:
/
x²
lim n x var (a(Ai) (((2))) (u² + 0² + 20 c(1 (1++17) +1))
dF
noo
(x A1 2 X2)
X²
+ lu= + e²1 varx~Fxp
(D.22)
(X+ + X2)
From the proof of the limiting variance of dR we have:
lim n x var vár( = 1 (e2 + u² varx~FuP
X
x dFF(M(()
+1²
1000
X+
(1)
X2
lim x cov(á(Ar) cov(a(>))To,,d()))) ( afoo
(u²r² + e²) COVX~PuP
(X+)i)(X + X2) X +
+ +00 /
x² dFMp (z) (x A1) x2)
+0² ʃ
x² dFMp(x)
(x (1 22)
Therefore, we obtain:
lim var 4t.00
E
@N (D])
2sp
X
+u² /
x dF
= -{
(e² + u varx~FsPp
x+
(o + 21
/
x²
+ (2)
(122 + 1² + 22 c11 + +p2)
dFMr(r)
x2)
X²
+ uu + e²) varx~Fau (X+ + X2)
+
(1 + e²) COVX~FMI
X2
X
+2Hz(A)
xx) + )2)
X X+1 X1
-0se /
x² dFMp(() + + (x+ X2)
x² dFMr(z) A2)
(D.23)
Next, we consider the expectation of the conditional variance. The conditional variance of 0 02pp NR.db given D1 is as follows:
H2(A) var (Y - X xTa(i)))
(ONB.al) db
Di)
HP(A)
var ((XT Bo + +o
+)(( (ao- a(A1)) +e))
n
HF(A)
[var X (ao - a(>1))) + var (c (cXT (b) + var (μ (uxT(an -((1))) (xxTB)
7
+var(pe) + 2cov (e X Bo, μ X (ao a(\:)))
46

---

(1((2)
(I100 - -(Ai))(( (1ao-d(Ai)) -(1)))1111801 + ((00 a(\i)) 80)² + |100 - a(Ai))
n
+1150112 + 2p(cu - -((A))) ((AAi))BBB Bo + 1+ 1%)
We have already established:
lim Billao ((1))11)) u² / 1100
XP dFMP(x)
tc /
I xdFMp(() (x (x)
(a 21)2
- =((()=((\\)) [o]= = =-10 / [
lim n700
+ f
lim Bi((ao & [6) Xie² (/
dF (x (x+11) A1)
xt00
Combining these we have:
(oNp- [Di)] -P((1)) {1r (n² f
22 (dFMp(())
(x (x+ to ʃ
x dFMM(()) (x (x
lim n x E var wf00
+ X²0² (/ (fdEtr(a))
(x X1)
/
>3 dFMP(x) + /
x d2)
+0² + u?
(x x
(x+
+200A1e /
dFMp(x)
+ +1+ + 03
(D.24)
A1
Finally, adding the expressions in Equation D.23 and D.24 we conclude the proof.
D.5 Limiting Variance of @DR.dh
We now establish the limiting variance of @D Equation (3.8)
Recall the definition of @DR defined in
@DR.db Σ -Y- -7 G() 1eDs
where the definition of H, (A) is immediate from Equation (3.8). As before, we will use the tower property, but now we will condition on D1.2 (D1, D2). The conditional expectation is:
E
@DR,db 3ap
1D:22
(an - a(\1))) (6o - 3(22)) + 6. - a(ha) a (1
= ((AA))(((A)) - (A)) - a. 3(2)) a(A,) 3. + a 30 + 0n
Therefore, we have:
var (B
(DR.db (1222)) = (1 - va) (G(AI) 3sp
(Ba2)) + var(((A)))
2((1 H, (A))coovv()(() B(()))((())))()),).o)))))
B(A2), 3(x2))
2((1 - (A)-H(()) B((2), a(A1) To)
We first use the independence between a(A,) and B(A²) to simplify the covariance terms. For the first covariance term,
cov(á(Ai)) B(x)) ((22)) = E [AA
((())))) ((2)) 3(22)
E [B[ā(Ai)] ( 3(22)) - - B[3()))]
EEL HI 3(22)
47

---

E ag EJ (A1 (1((2)) Blan 3(A2)]
dān (xx)
var (ar ((2)) = pn,ivar var (a0 B(x2))
E
x + X1
Similarly,
((A)) ((x))(A))T((N)),0( =E [ d (f var (á(21) Bo) == Mn, (a(A1)TAA) (b)
cov(á(A1)) Plugging these in the expression of the conditional variance, we have:
+ X2
var E (03p-1h (D1.2] (1 (A)) (a(\i) ((A2))
+ (1 2(1 H1 (A)) var var 3()2))
+ (1 - 2(1 - H1 (1)))) var (á(\1) 3b)
It is immediate that:
ddFp((a) + A1 (F)
|n,1
at00
2n,
+ X2
We next find the limit of each summand individually, and then add them up. From the proof of the limiting variance of @INT,dh we have:
3ep
lim n X var (((2)) atoc
1
X
{e² + uuu +u²
(x+ xe)2 dBaa(to)
x
(
x
A1 d(Far(x)
varX~FuP
X A2
{
X
dFanp(e))
+ Faar
X+>1
1
I x A2
+
u²
x²
dFhae(o)) tc (
(x (E+ X2)2 dFuue(z)
x
}}
(I
+ 22)2
(f
x²
dFhne(c))++ ( ( (Fin)(z)))
(x + X2)2
+o
(x + 21)2 dF
From the proof of the limiting variance of @DR,ah we have:
1
X
x dFMr((x) (x ( \1)
lim var(ā(21)
Bu)
- (e² + +uu varx~Fur
t
atoo
c
X A1
1
X
+ /
dFFp (x) (a \2)²
lim ((A2))
- (o² + uu varxwBar
atoc
X+ No
Therefore, combining all the terms, have:
lim var #100
(E
@DRR. (Dia)]
03p
(1 (1- Hi()))
X (x 1222
10" +200
tu² ʃ
(j
x d(For(()) 2
(o+ d2y7 d(war(s))
1
(x+xx)
(0=0
x ( dFiar(s) +
u² 0z (
( A222 dFaar(z) to (
x²
xx (( }}
(x 22)a
48

---

3 (ʃ
(x + x2)a dFhap(x)) +c (f
x²
(x + d2)² dFua((2)
(x + x1)2 dFsp(t))
1 2(1 ()(()) /
x -
a dedFac (x)
ee u²²²)) varx~FMp
+
x A2 x (ddharle)
X X2
(x X2)²
2(1 - -H((A))
/
(e + =²) varx
X
+
1 (ddia((z))
x
A2
(x 22)² S (D.25)
Next, we derive the limit of the expectation of the conditional variance. Towards that end.
( (D1:2) = var ((Y x - (A Ssp
var
var (a0- X (8o - - var (xT (B0 B()2))6)
n
+ var (xT(ao - (()))) + var(we)
+ 2cov (x (ao a(\i))m x XT - B(X2))c))
(100 a(A)) 3 - ((ao - -a(A1)) (00 - ((22)))
+|100 2 + 1130- B(22)1 + 200(a0 ((A1)) (Bo 3(22)) + +++ 0g)
Now, we derive the limit of each summand separately. As already established in the proof of the limiting variance of ODR.ab; we have:
lim E [1l00 a(11)l = =² / 1/00
Xi ddFx(())
x dFFM((r)
+c (I + X1)a
(1)
E [1.00 - B(22)113) = 2² /
1/ dF +c
x x
lim Nto0
(x +x+ x2)²
(x+
As ag and 3o are estimated from different subsamples, we have:
lim E |1000 a(\i) B(A2)13) - -{(1²> u²
X dFMu(()
dFMp(z)
+c
2000
(x + x1)2
(x+
/
2A ddd(((()
x 02
+c
(a + X2)²
(I x2)²
ddFar(a))
dFuup(t)
lim E [a00 - a(Ai)) B A1
noso
x
2 x X2
Finally, for the square of the inner product, we have:
((as ((A))))T(( (Bo B(2=)))²]
XgE (ao a(A1)) R2(A2) (2()2)(an - a(Ar)))
do E (an a(\i)) (2 M1 X2 (2(d2)(an a(Ai))
X U2 BJ R2(de)(a0 a(Ai))
- (ao - a(Ai)) R2()2)
X.J H2 uh X2 R2 (2( -
+ E
(a0 -(AA)) R2()2)
X E ((ao - T(A1)) Rs
((AAI))RR(AAA))) (2)
1
(aD a(A1)) R(
n
49

---

Let us analyze the above two summands separately. For the second summand, observe that,
E (ao - (((A))))"(A2)E22aa)
- @(\1))
_ E [1lao a(Ai) E
//
2(x)
(ar x2))
Therefore, this summand is O(n-1) and hence asymptotically negligible. For the first summand: ((ao - &(A1)) R2()2)
1
//
[B|l (A) + 2E[((00 o(A1)) -(AA))
(x+ din((r)
(p + 2)
p
p
Bi((o0 - a(A))) (6) (o) B a())) 2(x
(0++2)) dif(())
27
+
E
p
p +
(p 2)
(x + x2)
1
B[((00 - a(Ai))) (6) 1BIla0 - a(Aa))
11111
(x + )2)² dii (2))
p+2
(p+ 2)
p²
El((a0 - a(Ai)) B [V
1
27 dān dān(()) (x))
+ 0(p
(p- 2)
(x + A2)
As consequence,
lim E[[((0 a((i)) 80091} ((0))) ( nooe
d( 2 X2)
lim E [100 a(A1)) " (2
no00
Now, to find the limit of the right hand side,
Bi((a0 - a(Ai))" 36) 3
27
(i + (AI) X
=
30
n
(->ra [->iaaTRAAA))e (A1) + B1(B)(1)
X
27
n
X
X [[a]] Rí (0)) +E
R1 Po BR
n
By a similar calculation as before, we have:
p2
(
27
E [(a B1 = (p - 1)(p + 2)
(a 30)
(a ( dān dān 22)
+ +0p--t)
Therefore,
(fdFur(())
lim wttc
[[ Ri (2)) = =
(o+
Also,
E
T X1 R1(A))200 Rn(r1)
X[6]]
'
(( 11.
n
n
Hence, we have:
(fdfmp(a)) 2
( (z +x)
(fdFurlo)) (x + )x))
lim E [(000 [((0- a(A1)) RR((2))))) = =>ie² n0000
This, in turn, implies:
X2230² (
dfur(()) 2 (x [dffaele))(
(J
lim E [(ao - a(\i)) (3o (
(x + x2)
Collecting all terms, we conclude:
50

---

lim 72 x IE var
(122)
-
{{(1²
u2
Xi dl (F)))())
to /
I dF (2)
Sep
(x + A1)²
(x+
22 /
X3 ddFMp(a)
tc /
I addFMp(2)
x
(2)
(x+
/
A dF
v. (F)p(())
+ 2
+e (xr + X1)
(x+
13 ddFMP (x)
I ddF
+ x²
+c
(xr + X2)
+ (x)
dFuu((t)
{
dFMp(z)
+ 200A1220
x+ x1
x 22
(ʃ (
2
(
[dFpp(z))
(x+
(ar + A2)
+ o
(D.26)
Combining Equation (D25 and D.26
conclude the proof.
D.6 Limiting Variance of @NR.ab 3sp
The calculation of limiting variance of 0gp. is similar to that of @DR.cb. We again the tower property, conditioning on D1:2 = (D1, D2). Recall the definition in Equation 335
i dIDdbb
3ap
YI(A,
X, xT6(21)) a(A1) ((
72
iEDs
where the definition of (A) is evident from the definition. The conditional expectation is:
E
D1.2
30 (as a(\1)) 0. a(\i) T (
Therefore, have:
E
i dN
D1:22 var (8fa(A)) + +(()) ((()))
var ((1))) + H((A))ar(((AI))
a(A)), a(A1) B(x2)) T(>x)TB(X2))
We have already established in the proof of the limiting variance 3ep
@DR.ab that:
lim n var (á(Az) B(12))
(((2))
atoo
X
{e² [0"+ +u²
/
x
dixp(st) (J
I *
Xi dF )
varx=Foup
X+ X22
X2)2
X
x (/= driuple) 2 +
+ -varx=Fuap
e²
X X+>1
u2
2
x²
dFhae(())) tc
x
X2)2 dFuur(t)) }
+ 22)2
x²
d dFaap(t) +c ( (x 22)3 dFFnn(t))
(
+
2
x
d dFFp(z)
(I + 22)²
(a 21
Furthermore, we have also established:
lim x var(ā(A)) (b) = - + u200) varx~Far
X
+ + (dFFx(())
2100
x
[x }1
and
lim n x cov(@(A1) 2100
a(22), _(0))
51

---

dFFar(x) x 22
(-10² + v² war
X
+0² - =d (o (++>1)²
(X+ A1
Combining the terms, we get:
lim var
[B
B Dua))
10700
varx-Fup
X
[0"+
X+12/
(ʃ
xi dFter(s)) 2
+u²²
dFhup(t)
(x+ 22)2
X
+ PMM
(+
x+ d2 dFrro(2)
x²
(f x
22)2 d(far(t)) }
u
v²
(I x2)2 dFhre(s))
to
(1
+c (ʃ (oottap dFunlca)) (
(a X1 dFhop()))) }
+
0²
[ 22)2 (Faro(z))
(x+x) + 0² I
X
a dF)
(e" + uu²²²) varx~FMr
(x A1)2
dFup(x)
( + uuu?
X x+
x dF)
+ 2H1(2)
varx~Fri
+ 0²
(D.27)
(x + x1)2
Next, we establish the limit of the expectation of the conditional variance:
var (oNB.
(D1:2) = - var (Y xTa(A)))
vaa ((ag - ((A))) XX 3o) + var (XT
+ var (an - a(\))) + var(uc) + 2cov (X (xT(a0- (a00-((M)))) -((1))), x XT Boc)
12 (Ilos - a(>i) 1/ + ((o. - a(1))) T 30)
+||00- a(A:)11 + || + 200 a(Ai)) 3o + ++0 0))
In the proof of the limiting variance of @DR.db, we have already established that:
lim E [[|ao - a(Ai)l = u² / Xi dF + =x=0
/
(dFMp(x)
( X1)²
(x+
lim [[[((a- - a( 80) = =X3e² (/
[x X1)
lim E((o a(Ai)) T [0] = Aie / dFFa((z)
x+>1
Hence, combining them, we conclude:
lim n E var afoo
(ONB (
=0² '2 ʃ
X7 ddMr(()
+ f
x (F) (x + 21)
(x + 21)2
+ x2e² (ʃ
dlMm(xx) (x)
(x+ X1)
X2 (dFFM(())
+c /
f dFMM((x)
x
+ 02 + 2²
(x (x+
(x+11)²
52

---

Then for any two + 20021e /
(x) +0.
(D.28)
x + X
Combining the limits in Equation (D27 and (D28) we conclude the proof.
E Some Auxiliary Lemmas
Lemma E.1. Suppose X E RExP is a standard Gaussian random matrix, ie. Xij Lid. NO,,1) 1)
and p/n Define the corresponding covariance matrix E = (X (XXX)/n Suppose E admits the following eigendecomposition:
-=VVÁV Σ G5
where V O(p) is the eigenmatrix of E. and {islisisp are columns of V. sequences of vectors {u, JneN and (WaJneNs with
lim u 1h.000
lim || of00
lim u, Wn 2100
we have for any bounded function f:
lim var 112 V f(A)Vw.)
lim var nt00
Σ 60) CoJunnl (OOTTw))()s))
(e² + +22v varx (f(X))
c
Proof. The proof hinges on the following three key facts:
1. From rotational invariance of a Gaussian random variable, we know that V is independent of Å. Furthermore, V follows Haar distribution 0(p), the set of all p p orthonormal matrices.
2. For any 1 < i p. 0; (the ith eigenvector), marginally uniformly distributed on the sphere. As consequence = 1//l9/1, where g (N
3. For any i j. j, the marginal distribution of (0;, 02) same for all + which follows from Haar measure.
These facts are elementary and well-known. Based on these facts, we first establish a few claims:
Claim 1: For any i we have:
E [(T] un (6 w..]
Wx. P
The proof of the claim is simple by using symmetry argument, thanks the second key fact. As 0; has same marginal distribution for all i, the value of IE [(t, (o) w.)] should also be the same for all Denote common value by C. Then,
E.
[( u, (6 wo)] -... [(0 un (1, wn)]
9
E
un (67 w..)
p
1 -u [V
u Wn
u E
Σ
Wn
w,
p
p
53

---

The last equality follows from the fact that VV = Ip as V O(p), which completes the proof of the claim. our following two claims, we compute the expectation, the second moment, and cross moments of the inner product:
Claim 2: For any 1 s 1 S we have:
E [0]] un (1, W
plp - 2) [1 + 2( wn)']
To prove this claim, we again use the second key fact. Suppose 9 NO Ip).
Then for any
1 kisp
( un (g(g wn.)
[00 un (0) w.)]]
119122
Now as g/ /1 IL Llg11. we have:
(g u, 0 w,)
E [gg ( w.n)
= E
Bllle111
1 1/12
(g un,) 3 wn)²
E (,, ( Tw,,)
un
E
|191
E [1gg)
Now as ||l1ß² X we have Eillg]la]
p(p + 2). On the other hand, as NOO, Ip), we know
g T un
~ .V (()
Wn [| 11
g gwnn
Wn
u Wn
Therefore,
E
[(g T 2 (TT))
|
w 11 + 2( w,)
Therefore, we can conclude:
(g un )²(²)) wn )2)
E (g
un (g( wn)
E (1g un (6, w..
=E
E
19
E E[[g
1
[I|u, w 112 2( (i/,,,,,,, w.
plp +
Claim 3: For any 1 isifis p. we have:
||u, 11 1/ w..
E [0, (0) (0, u, (0. ww.] p(p 1) 1p 2 (n,) ,,,,,,,"
p
w,,
2)
Again, we prove this claim with simple symmetry argument using the third key fact. The main observation is that for any i * j. the value of the expectation is the same as the marginal distribution of (6e, 03) is the same for any i + j. Let us denote the common value by D₁. Then we have:
P
Σ [(0. (0, (i] u. (6) wn)] = 1
E [(0 un ) w.))]]
[6]] (o) w.. (o) un (i) w.)]
(
E [0]]] u, (o, w.) + Cip(p 1)
1=1
w. 1 + 2u w.))] + Cip(p 1).
pp + 2)
Here, the last equality follows from our conclusion in Claim 2. Now, if we consider, the LHS of the above equation, we have:
[00 u, (oT u,) (n)(oJun)()] (...)... =
[o) u, ((]] (.)(oJu, (1, w.)
54

---

E
(n) Wn))
E
un
Σ 0500
Wn
Vu. wn, ) (u,, w,)
E
Therefore, we have:
1
(un 3,, (p 2)
[| 12 w., 2( w,,) Cip(p - 1)
- p(p-1) 1) (1 w,,,
1
1 I| 12 w,,
+ 2 w,,)
(p+2) 2)
This completes the proof. Now, we use the conclusions from these three claims to complete the proof of the Lemma. First of all, from the definition of variance, we have:
)
Σ G un [0] (ooTw.))((s))) 1)
Σ o0 (6 ww))((s)) E
Σ 00 un) (.TTT.))())))
E
p
Let us first consider the second term, expectation. From the independence of eigenvalues and eigenvectors of Σ, we have:
Σ 60) u. (.)((Tww)(C
E
[0] un (0, wn)] E [((,)]
j=1
[f ((x) dā (x)
E [() (u, E
Here 7n is the empirical spectral distribution the eigenvalues of E. As we know that गn converges to Marchenko-Pastie Pasteur (MP) law and is a bounded function, using dominated convergence theorem, we can conclude:
1)
2
Σ 0.) un [0 (..
=(u,,,, w.,)²
E
f(0) [(oo)dinll))
(E.1)
Now, coming back the second moment:
un (67 w.))((h)))
E
E
Σ
(o) un (0 ww) u (0) (TTT)))(
1/
E [00 un (0,
w.)] E [((W)((5))
P
Σ
E [0 un (0, w (5.) + + E [0, un (n) w.)( un) w..] E [(()((8)))
|||u, || 1111 w..l
+( w..) 2
Σ (())))
p(p + 2)
55

---

1
p
[u, w..)² ||un|211 Wn
Σ
[
+
p(p - 1)
p 2
(p + 2)
1
[1|un|6
Wn 112 2(u,, wn) 1 / ((x) din(())
(p + 2)
-
1
p u w,,)²
||un12/|/ml 2)
{ E
(E
- ==1
--E
p(p
1)
p
1=1
1
[1lum|3 Wn 11 2(u,, )3] E / (a)) doin(a))
(p+ (0+2) 2)
| | w.
(Sol) )
p
+
(n wn)
TE
p
p 2
(p+2) 2)
1
un w.)
Wn
2
E f (xx) dit d2)
(E.2)
p
p p+2 2
(p+2)
This, along with Equation
yields:
Σ un
(0) (n))((A)
var
1
Wn 12 + 2(u,, )² E / (xxx dancco)
[||n
(p + 2)
| | w.
+
p
p
u wx)
IE
p
[p + 2
(p+2) 2)
1
Wi
E / (x(x) dit (20)
[u,, wn)²
p 1
p 2
(p+2)
| f(x) ditn(())) 1)
-
(u )²
E
p-2
(11 w,,) )
+ (p+ + 2)(p - 1) |1u.x
p
W 1
E I ((t) din
(p+ 2)(p 1)
IE [[ f(x) (f
27 ((()))(
p-2
(1 w,) 2 + (p- (p-1)(p+)) +
p
w. [2
(p - 1)(p + 2)
+ (u, wn |var (ʃ ((1))) di)
[( drnl(o) - [[ f(x) drinle))
2-2 (p+ 2)(p 1)
(1) w2) + (p+ 2)(p - 1) un w.,
p
(u,, w,) Pvar [(a)
Recall that our goal is to compute:
lim n x var nio0
Σ
Oo] un (tJ w())))))))))))
As p/n - c, we have:
p
lim n X var 8700
Σ 1=1
(0) u (0 w.))((g))
= lim p x var 1
56

---

Consequently, it is immediate to compute the limit with respect to Towards that goal, first, observe that we have,
dā (x) ((())dh(())) 0,
lim var
pfoo
by Theorem 1.1 of (28), as there the authors have proved that
(f f(x) ddn(()) - / f(x) dFar((t) - N(0,09).
p
Therefore, var (f fx) dzn(()) Opp 2) and consequently p var (f f(x) dān(z)) Fur-
thermore, by DCT, we have:
E f ((x) di (0) P(x)
- ((Io)dtale)) - / P(x) (Fx())x)())) ( (x) dFsal())
varxwFac (f(x))
Therefore, it is immediate:
lim p var E (n) (..lo)))))))))) -0² + +u²u²) varx~Aur (f(X)) 1ro0
and consequently,
lim var
E0 C0JJun (0) w.lr(ss)
(o"+
u²u varx (f(X))
x/000
This completes the proof.
Lemma E.2. Under the same setup as Lemma E. we have:
lim (I)VV un) (w, VV w.)) u² [ dFim(()) / fa(l) (x) nfoc
Proof:
The proof is essentially similar to that of Claim 2 and Claim 3 of the of the proof of Lemma First observe that:
un Vi(AVV (w.. w:)]]
)
20 m m1)0))))))))
Σ (oTTw) (...))la))))) (1T
Co
E
p
E [10]] un [0] wn 12| E [((8))12(2)]] +
[6 (0] (1((A(A))))))
E. u.
1=1
343
(I|u,) 1 w., 2( )) E / (())))
p +
+
||u
x+2 2 (I w.ll [u,,
Σ E [(()))(())))
p(p 1) 373'
+ 2 w.,) )2 / [(())1(()) dit
(1|1g) 1| wn
+ 2 p
+
||,,1121
w..
p+2 (1 u 1...ll//.,l1 w 2(1, ww)²)
P
Σ ((sss)
1(--1) Σ En((S)E(
x
p - 1
57

---

(1/ux wn 1 2(,,, w.,))) / f(())f(()) (x)
p +
+
|| 1..l 11
un w 2(u wn)t))
p+2
P
(f fa(x)
11- / (())
1
E
p-1
Therefore, it is immediate that:
lim E VAAAV un) VV2AAVV w.)] u²r? / f((0) dFur(()) f Fa(()) dFur(1)) wh0o0
This concludes the proof.
Lemma E.3. Under the same semup as Lemma E.1 we have:
lim cov (IA)V(( u (2V who0
(u'r² + e²) coux (f1()) =
Proof. The expectation of the product is as follows:
un
1 un (0) w.)h(())))
un Un] w.))((())) (o
E
Σ
E [10)] un
(ū, wn 19 E [1()))(()]]
/=1
(6T w..) TE [[(S))s(ss)]]
E
[||,,1211 11, w,,l + 2( w.,) )2] Σ E [1(()))5a(A))
p(p + 2]
(1, ) (p+2) 2) [|| Un 1 w,, 112 + 2(1, w.,) )²]
Σ E [((()))(()))
+
p(p- 11 J455"
1/2 + 2 wn wn] ]} E / [(()))a(()))) d()))())
[]| |unl w//lw
p
+
(1 w,))?
1
un 12 w, n 112 + 2(11, w..)]
(0
2)
(Enc))) )
(Erach)]
-E Enchsssssn))
1
E
p(p 1)
) E / f(())ff(=)) (x)
|
//w 11.1 + 2(u
P
(u, )²
1
un w. 112 + 2(1 w..)
(p + 2)
1 E
/ fi(z)) ( f fa(x) dā d()
p ---18/ E / (i()))a(o)) dmnn()))
1
p+1)
As for the product of the expectation:
E
u Vf
u B Va(d)/a
= ) f fi(() d()) E [ fa(#)) fa(a) dim)l())
58

---

As consequence, subtracting the product of the expectation from the expectation of product, we obtain:
(u, (I. u, V..(A)))...)
cov
=(u, (m) 2I [cova... (fi())f2(z)) + EE [ f((z))f(z)) dima(a))
||lunll Wn 112 2(,,, w(u,,wn) wn]
1
(u (u]
1
||| w. 11 2(u w., =2u,]w,)) w.)l))
p-1 p + ( ) If fi(2) 1a) / f2(z) dimi(a))
p +
(1 + 2)
1
+
p
| 11 Wr 12 2(u wn 2 / fi(t) dāmm(a)) / f2(2) dan)(
(p + 1) 2)
From Theorem 1.1 of 28 we have:
lim 2000
E coov.... (f1(z)f((z)) 0.
Hence, we have:
lim n x cov V u, (AA)) n100
1
- (u + coVVX~a f2()),
This completes the proof.
F Prediction-Optimal Tuning Parameters
In this section, we derive the nuisance functions p(X) X ag and b(X) X Bu and the ridge-regression based estimators
optimal tuning parameters Å1, Az. Throughout, denote the
Px, (X) = x ((1)) and bxp (X) = X x ((22). Lemma F.1. Under the conditions of Theorrm nuisance functions p and b are given by
3.4 the mean squared errors of the estimators of the
x²
[[(,, (X) ((x)) 110012
1+
dF dF)p(x)) -2
I + dFMsp(z)
(x 1122
+c
(x+ dFMp(z) + o(1)
x
x2
E[(b,(() - b(x)) 11
1+
(x (x+ dFMr(x)- - x
dFuar(t)
+ +c
(x (x) dFMp(z) + o(1)
x
Lemma F.2. The limiting mean squared error of the estimator of the nuisance function p given in Lemma F.1 is minimized (with respect to X1) at \ = c/u² where = lim ||oo||2. Similarly, the limiting mean squared error of the estimator of is c/u? where v lim |1,oo|2-
Proof of Lemma F. It is straightforward to see that
B((x,, (X) p(X))² = B(llá((1)) - 0013)
Plugging in á(A1) = (Si, + I
x
we have that
T a
Blll aol2) =
{ + {1
00
n
+ >1I -I + (5 + >1 - e/n - ao
59

---

3 x111) - San]
2E
ag 3 + 1111) a0
2
Σ 1pI 'XT E
+ +110o12
21E | E
X(5 + - 2 c
E
n²
Observe that I [a] c] = 0 and E
T +
- = -. Recall from Section D.2 that
(1)
E
T E + A1Ip) an
X1Ip)) _
|100| (
x²
dF + o(1)
(x + 2)2
In the same manner,
f
IE
Og
T (
*00|
||001/
d (z)
x+
|
I
dF (x) o(1)
+
Finally, it follows from the same steps used in the analysis of T4 in Section D.2 that
1
E
x + 111
2
E
71
(23 +
A1
p
()3 + 21)2
cE
/
(x+ \1)2 d dttu(a)))
x
(x (x+ dFMp(z) dFxx(z))((11) +0(1),
I
which concludes the proof of the lemma.
Proof of Lemma F.2) We here prove the limiting prediction risk of px. (X) is minimized at A c/u2, where u. Tim ||00||2- The proof for the minimizer the limiting prediction risk of
bx, (X) is the same and hence skipped. First of all, observe that we can rewrite the prediction risk of
px, (X) follows:
lim E((,x, ((X))²] u 1+ f 20000
x² dFup(() -2 #ddFan(())
+o )
x
dFmp(()) (x)
(x (x+>1))
x+>
x dFvp(() dFur(o)) +c
2
+
(x (1)
X dFMp(z)
(FMP(x)
2
+ +0
(xr A1)²
(x+
X² + (c/u²)
2
dFiur(t)) u² (²)()>).
(x + A1)²
Our goal is to obtain Xj such that:
x = arg g(A1)
The derivative of g with respect to x is:
g'(A1) = -/ /
d
x2+ (c/w²) (x)
di
(x + 21
60

---

(x 21) (Af + (c/u²)x 21) dFMM((x)
(x x
(r 11)11 (Ap + (c/u²)z) dFMp(z)
=2
(x+
A1 (c/u²)xr
=
( dF)
= 2(A1 - (c/v²)) /
x (z)
(1)
Therefore, the derivative is when x1 c/u². Furthermore,
d'(A))= = -6 - -cc/u2)) /
\1) +2 x dF)p((x)
ddFM((x)
(ar + \1)3
which implies,
dFFmp(x) 0. (xr + (c/u22)33
g" (c///²)) =2
Hence g(A1) is minimized at Ai = c/u². This completes the proof.
G Parametric Bootstrap
We use parametric bootstrap approach estimate the large-sample variance of debiased estimators, For illustration purpose, we consider INb,INT which is defined as:
T(
=
IND
3ap
Σ n2
YA
INT 33sp (A1,
iED3
where INT (A1, A2) =ss (dF(sp())) with aspect ratio c = p/n.
(SddFar(())
(J
and FMP denotes the Marchenko-Pastten law a
dFaa((() x
xt>1
Bootstrap algorithm (3-split). Given á and the ridge estimators (á(Ar), define the following:
1. Define the constants (ct, d1, c2 d2, ca)
Cj = /
x²
Ag)2 dFMP(z),
jE (1,2)
dj (p/n) / + dBMp(x),
j (
C3 + dFMp(x) f x dF)(())
Then define two transformed regression coefficients and / as:
a(A))
|
[a(A1)lla
a(A))
(a(Ar) (:A2))
118(Aa)13 d2 (a(A,) (G(A,)T8(Ag)) ((Aa))
[a(A))l2
|ac 11l3
Aa 3
where z is some unit vector perpendicular to â(A1).
2. Generate ((() ......R x(6)
L,),
e 1)
3. Generate (e) mh (b)) )..., (cl)) nm
(b)
~
0
61

---

4. Set A,00 x!"' ā + o- and y, xx!0²² 3 + n for S n.
5. Estimate (bb), using {(X!') y,u, ! ) Jisisn
6. Repeat Step 2-5 multiple (say B3 times and return:
B (
2
1
B
8 3ap
db,
var
B
3ap
B
b=1
Intuition: Before delving into the technical details, begin by presenting some intuition behind the motivation for introducing the transformed vectors (ā,, B). To start, recall that the limiting variance of gdb,INT depends on seven variables: 3sp
(05p)..ITT) o² u, 3ep
lim n. x var n100
where a = pooe ||0:1 = to ||3 tomo T and c is the limiting value of (p/n).
On the other hand, if we increase the bootstrap sample size B, then vår converges to the conditional variance of 3ep
INb,INT given the original data, i.e.,
lim var var (00).bb.IT (D) B
Therefore, the bootstrap variance estimation will be consistent we can show that:
dbb
(D) as o (,
lim var no0
(G.1)
However there is a small issue here: we know that neither ||á((i)) nor ((2(2) gives us consistent estimate of u of v. In particular can show that:
lim | eru + di
lim
(13(22)112 cz + d2
afoc
lim a(\1) a(Ai)TB(2a)) a.s. c3p.
where (ci, d1, C2, d2, ca) is same defined in the Step of our bootstrap algorithm. As a consequence, can show that:
lim var (00)),b,INN D) ass. o² (p
cse, cru d2, C2U + d2, c, A2)
3ap
which does not satisfy the consistency, as required in Equation G. a transformation of (á(A1), (Dx))) that yields consistent estimate of (u, 2). To that end, consider the definition of (ā., () provided in Step of the bootstrap algorithm. We have:
Therefore, we need to construct
2
a(\1)
||x(Ar)113 d
lim ||āl nfo00
lim
nfoo
[
C1
2
||á((r) di
lim ntoc
C
Ci u + d1
d1
C1
lim [| #1.00
(G.2)
a(Aa)
(a(A:) ((A2))
|((())) de
(a(Ar) ((xa))
-1.00
[
| 113
[ B --44
(G.3)
62

---

(á(A1)' (G(A1)"B(Aa)) B(A2))
I((()a) 1 d2
(d((1)) B(2a))
lim n0
[oCA)11
03
10 41
11
(G.4)
[0(Aa))1 d2
(G.5)
og dg
dg
(G.6)
22
Last, but not least,
lim à T 1100
a(xI)
|á(Ai)l - dd
a(Ai)
(
lim ntoo
[|1(21]]12
C1
||a(A))lla
C3
Ia(A
(á(A1)) T ((22))
lim nto0
C3
Therefore, if we use these transformed vectors (). in our bootstrap algorithm, we expect the bootstrap variance to converge to the right limit.
For the other estimators in the shree-split regime, we apply the same three-split bootstrap procedure described above. The only difference lies in the specific form of the estimators themselves (i.e., the analytical expressions for @db,NR and Ddb,DR), while the underlying bootstrap mechanism and tatt-ggenn steps remain identical.
lim var (0s0) nTo0
(1U),bb.NT/NRRDD)) = = (,
Since all three estimators share the same asymptotic structure and depend the limiting quantities (u, v, g). Thus, the consistency argument for the bootstrap variance estimation holds in exactly the same way for NR and DR for the INT estimator.
G.1 Two-Split Parametric Bootstrap
With two splits, the norms obey the same limits, but the cross term changes. Let
a(>1)
91
INT (A1, @ 9 IVT (>1, A2) p.
where we have 91,2<p = f 2 ddxr(x)
(=+d)) 92 (A1, A2) =
co (x) (=+A1)(2+\a)
2
9e INT (NIT(A1
dF),
02
INT (A1, (N A2) =
dF (x).
(x A1) c X2)
(x Az) + A2)
Let a =
and z be any unit vector orthogonal to Define
(6(A1)
dj
à
(G.7)
C1
T(A.)TB(As)) B(Aa) 926
110(2)1113 de
a(A:) 926
8 = à
91
16021)13-4
18(21) (11)))3-44
61
We rewrite
91 = gi
INT (>1, 22)
== 92
INT
(21:(A1
|| - c d1,
113((22)11
25,
C2U + d2,
910 92P. 22 a(x))T((b))
63

---

a(\1)
| d1
lim
|all2
lim a1oc
no
[| 1
C1
||ár(A))
112
d1
2 + d1
d1
lim 7700
u.
C1
C1
lim 110112
(G.8
61.00
a(hi)
a(A:) B(A2) 9
11
d2
a(Ar)T -22
lim
+
-1.00
[[a(A;]ll2
91
[otA))13
02
10(x)) 113
(G.9)
lim sTo0
T(Ar))B((ra)
92
18022)11 1112 d2
((a1)T (B(aa) 92
91
C2
91
10CA1)13
51
91
(G.10)
[18(2))11 d2
C2v + d2
d2
lim 0100
(G.11)
Last, but not least,
a(\1)
|a(At) di
a(\1)
a(x1) a(xi)TB(A2)) 92
lim T n1000
-lim n000
[A1(21
C1
[
91
|
C1
a(A,)T 3(12) 92 910 92P 92P
lim n700
2:
91
91
Here, the bootstrap procedure proceeds analogously, but with one key distinction: the transformed coefficients now depend explicitly on the estimated correlation parameter P. As result, the algorithm requires incorporating an additional estimation step for ò before constructing the transformed vectors (ā, B). Nevertheless, as established in Theorem 3.1 I a consistent estimator of the true p. ensuring consistency of the bootstrap. In Section H.4 we present our simulation results, which confirm that the bootstrap variance approximates the Monte Carlo variance well.
H Additional Details and Results for the Simulation Study
Here, we provide additional details and results of the simulation study. These results serve the following purposes:
1. We illustrate the asymptotic bias of the integral-based estimator, Newey-Robing estimator, and doubly robust estimator, and we verify that the debiased versions these estimators are indeed asymptotically unbiased.
2. We illustrate the asymptotic variances of the debiased estimators across a range of values for the nuisance parameters when 0.5 and == 2. We then compare the optimal tuning parameters (that minimize the variance) to those that minimize the prediction error (i.c., the prediction-optimal tuning parameters), in the simulations presented in the main text. For c c= 2, where we do not trim the y-axis (unlike the main text).
3. We verify the derivations in Appendix F regarding the asymptotic mean squared error the nuisance functions and the prediction-optime tuning parameters.
4. We illustrate that the parametric bootstrap approach performs well for estimating the asymp- totic variance of the debiased estimators.
We considered four scenarios by varying whether two or three sample splits are used and whether 2. In each scenario, recall that we generated 10, 000 independent data sets. For
0.5
64

---

each data set, we generated N iid copies of (A, Y) as follows. We let X have dimension p and generated X by X Normal Ip). Then, we generated A and Y by
yr X Normal ((x X av (o
!))
where = 0.5. In each scenario, the true values of ao and 60 were set by sampling its entries independently from a Uniform(0, 1) distribution and then re-scaling them so that ||aol|2 = ||
1.
Throughout simulations, consider the case where the nuisance functions are tuned identically, i.e., A = A2 = A. We consider discrete grid of 100 values for (ranging from 0,05 to 10). For each of the debiased estimators, find the value of À resulting in the lowest asymptotic variance for each estimator and compare it to the prediction-optimal À (i.e., X = A = c).
The code will be released at the GitHub link: http:://tttbbbooom////owwwg////p
Function- Tuning
H.1 Bias and Variance in Settings with Two Splits
H.1.1 Setup
The total sample size of each data set was set to N = 1000 and we split the sample into two disjoint subsamples of size n 500, denoted by Dr and D22. We have p = 250 in the setting with 0.5 and p 1000 in the setting with c = 2. For all estimators, we estimated ag and So using D2 and estimated using D3. In this case, the non-debiased estimators are given by
INT 2ap
Σ IED2
A.Y, - a(2) ((A)
NR Peep
Σ A,(Y, XT
IED2
DR P2ep
Σ (Y; X
IDD2
and the debiased estimators are given by
INT (A, X) 92.2ap
T 3(A)
P2sp
INT.
91. 11.2op
INT
(2,
A.Y:
9122sp (2,
SED2
INT 92.24p
(A, 2)g3ep NR (A, A)
92.2sp (X, A)o A)
INT
NR,db
NR P2ep
aCA)T B(x)
P2sp
g1,2ap (A,, A)
9 11,2pp (h1, A2)
22p (A, IINT (A,
a(A) (()) (2,
DR.db P2ap
+ 92.2%p (A
PDR
ITT (2,
9 11,2ap
The values of the constants were set by performing Monte Carlo integration with 10,000 iterations.
H.1.2 Results
We first summarize the results for the setting with c = 0.5. The bias of the estimators is given in Figure 2 In line with Theorems 3.1 3.2 and 3.3 the estimators without bias corrections have considerable degree of bias and the biaseedd versions of the estimators have nearly zero bias. The variance of the debiased estimators are given in Figure We observe clear difference between the value of for minimizing the variance of the estimator and the prediction-optimal X.
The results for the setting with 2 are summarized in Figures [igures[and] and Broadly, similar conclusions held: the debiased versions of the estimators generally had close to zero bias and there were differences between the optimal versus prediction-optine À values. As noted in the main text, the variance of the integral-based estimator blows up at around À 1.48, which may explain why the empirical bias of the debiased integral- based estimator was not 0 at that value of X.
65

---

Interestingly, observe that the value of can plays role in whether the cnddiction-otinaal values are greater than or less than the optimal A. Taking the plug-in estimator an example, the prediction-optimal value for À is less than the optimal one when 0.5 but is greater than the optimal one when e 2.
Integral-basee Estimators (Two Splits)
Newey and Robins Estimators (Two Splits)
Doubly Robust Estimators (Twe Splits)
0.6
0.0
0.2
022
02
Bas
-02
-02
-06
-2.6
-0.
Figure 2: Bias of the non-debiased estimators (black dots) and debiased versions (blue dots) in the setting with two splits and 0.5. The red lines illustrate the derived asymptotic bias of the estimators, with the horizontal line at zero.
Integral-based (Two Splits)
Newey (Two Splits)
Doubly Estimator Splits)
3.0
50
68
038
08
0.6
Verseree
0.6
Varlacee
0.
0.4
04
02
02
3
60
000
80
Figure 3: Variance of the debiased estimators in the setting with splits and 0.5. The red line
indicates the value of resulting in the smallest asymptotic variance for estimating pt The blue line for the prediction ootimall X.
66

---

Integral-based Estimators (Two Splits)
Newey and Robins Estimators (Two Splits)
Doubly Robust
Robust Estimators (Two Splits)
Bis
-01 0.0
0.0
3
-05
10
Figure 4: Bias of the non-debiased estimators (black dots) and the debiased versions (blue dots) in the setting with two splits and 2. The red lines illustrate the derived asymptotic bias of the estimators, with the horizontal line at zero.
Integral-bared Estimator (Two Splits)
Newey and Robins Estimator ITwo Splits)
Doubly Robust Esimmator (Two Splits)
014
0000
0.31
Vartercw
Variance
4000
2000
0.06
0.2
Figure 5: Variance of the debiased estimators the setting with two splits and c 2. The red line indicates the value of À resulting in the smallest asymptotic variance for estimating p; The blue line for the prediction-optimal A.
H.2
Bias and Variance in Settings with Three Splits
H.2.1
Setup
The total sample size of each data set was set to N 1500 and we split the sample into three disjoint
subsamples of size 500, denoted by D,, D2 and D3. Once again, have 250 in the setting with 0.5 and 1000 in the setting with e = 2. For all estimators, we estimated og using D,, 3o using D2, and p using D3. In this case, recall that the non-debiased estimators are given by
INT P3sp
Σ iED:
A.Y, a(2) B(A)
NR Psp
A,Y,, B(A))
IED:
DR
Σ (Y, X, xxT((A)
P3p
IEDS
and the debiased estimators are given by
á(A)T (())
Pasp
.NT.abb
Σ
A.Y.
33p (X, A2)
INT
iEDs
67

---

33ep (A, X)
NR.db
.NR
â(A)
P3ep
PSsp
IN'T (X,
93sp
DR (A, X)
DR,d
.DR
((A) (AA)
P3ep
P3ep
IN'T (A,
93pp
The values of the constants were set by performing Monte Carlo integration with 10,000 iterations.
H.2.2 Results
For the setting with c = 0.5, we present the bias results in Figure 6 and variance results in Figure The analogous results for the setting with c 2 are given Figures [ and We had similar findings in these simulations compared to the ones with three splits. That is, the debiased estimators indeed have bias of approximately and there are clear differences between the optimal and the prediciionoppimna À values.
Integral-based Estimetor (Three Splits)
Newey and Robins Estimator (Three Splits)
Doubly Rebust Estimator (Three Splits
15
Bisc
8
3
-.0
4.0
Figure 6: Bias of the non-debiased estimators (black dots) and the debiased versions (blue dots) in the setting with three splits and e = 0.5. The red lines illustrate the derived asymptotic bias of the estimators, with the horizontal line at zero.
Intogral-bassd Estimator (Three Splits)
Newey (Three Splits)
Doubly Splits)
0013
0013 0.000 0021 0.022 0023
0431
Verience
Verieree
Varlance
0.34
0.224
0013
0000
000
Figure Variance of the debiased estimators the setting with three splits and 0.5. The red line indicates the value of À resulting in the smallest asymptotic variance for estimating pt The blue line for the iitiectionoopiimml X.
68

---

Integral-based Estimator (Three Splits)
Newey and Robins Estimator (Three Splits)
Doubly Robust Estimator (Three Spiits)
8
0
Biss
8
3-
10
Figure 8: Bias of the non-debiased estimators (black dots) and the debiased versions (blue dots) in the setting with three splits and 2. The red lines illustrate the derived asymptotic bias of the estimators, with the horizontal line at zero.
Integral-based Estimator (Three Splits)
Newey and Robins Estimator (Three Splits)
Doubly Rebust Estimator (Three Splits)
0.042
01320
0.025
0.088
Verkeree
01226
Vartana
0.020
0.03
0.03
0.022
Figure 9: Variance of the debiased estimators in the setting with three splits and The red line indicates the value of À resulting in the smallest asymptotic variance for estimating The blue line for the prediction-optimal A.
H.3 Verification of Prediction-Optimal Tuning Parameters
presents the mean squared error (MSE) profiles for estimating the nuisance functions p(X) X Op (left panel) and b(X) X Bu (right panel) across range of regularization parameters À in the setting with 0.5. In each panel, black points represent the empirical average
Figure 10
prediction error across simulations, while the red curve shows the derived value of the limiting MSE
(Lemma F.T). The vertical red dashed line indicates the value of À that minimizes the empirical MSE. As shown Lemma F2 the value of that minimizes the MSE is c/ ||ool = c/ e/11% 0.5 for each nuisance function. We present the results for in Figure II where similar conclusions held.
69

---

Estimator of
Estimator of b
0.8
0.7
0.7
MSE
MSE
0.6
0.5
0.5
0.4
0
4
6
8
10
0
2
6
8
10
À
Figure 10: MSE of the nuisance function estimators in the setting with 0.5 (Left: or Right: 9)
Estimator of
Estimator of b
13
1.2
11
MSE
MSE
1.0
1.0
0.9
0.9
0.8
0.8
0
4
6
8
10
0
2
6
8
10
X
Figure 11: MSE of the nuisance function estimators in the setting with 2 (Left: Right: ()
H.4 Parametric Bootstrap for Estimating the Asymptotic Variance
Our next set results evaluate the performance of the parametric bootstrap approach described in
Appendix G In particular, compared the estimated variance of the debiased ECC estimators (with 10, 000 bootstrap samples) to their true variances, obtained by Monte Carlo integration with 10, 000 samples.
H.4.1 Settings with Two Splits
Figure 12 compares the estimated variance of the ECC estimators with the true variance in the setting with two splits and e 0.5. Table | summarizes the results across values. We find that the estimated variance was very close to the true variance for each À value.
70

---

Table 1: Ratio of the true standard error to the bootstrap estimated standard error of the debiased ECC estimators in the setting with two splits and 0.5. The entries show summary statisties of the ratios calculated across the 100 different À values. Ratios greater than indicate that the true standard error is greater than the bootstrap estimated standard error.
Estimator
Min
1st Qu.
Median
Mean
3rd Qu.
Max
INT (integral- based)
0.995
0.996
0.999
1.001
1.004
1.019
NR (Newey-Robing,
0.992
0.992
0.993
0.996
0.996
1.035
IF (doubly robust)
0.983
0.989
0.992
0.991
0.993
0.996
Integral-Basee Splits
Newey (Two Splits)
Doubly Splits)
12
10
10
08
Vanaroe
0.
Varlaane
0.6
0.5
04
0.
8
0. 02
0.2
0.0
80
Figure 12:
Bootstrap estimated (black dots) versus true (red line) variance of the debiased ECC estimators in the setting with two splits and c = 0.5 across the values.
Analogous results in c = setting are given in Figures [3 [3 and Table 2 As noted the main text, the variance blows up when À R 1.5 due to the bias correction constant diverging for the the Newey and Robins estimator
Integral-based estimator. We also noticed similar phenomenon for À close O. We plan to explore these subtitles in detail in the future. Compared to the 0.5 setting, the differences between the estimated and true variances were larger, especially for the integral-based estimator.
Integral-Ban -asadd Estimator (Two Spiits)
Newey and Rabins Estimator (Two Splits)
Doubly Robust E stimator (Twe Splits)
Vertanne
Variance
10
10
0
10
Figure 13: Bootstrap estimated (black dots) versus true (red line) variance of the debiased ECC estimators in the setting with two splits and c across the À values.
71

---

Table 2: Ratio of the true standard error to the bootstrap estimated standard error of the debiased ECC estimators the setting with two splits and The entries show summary statistics the ratios calculated across the 100 different À values. Ratios greater than indicate that the true standard error greater than the bootstrap estimated standard error.
Estimator
Min
1st Qu.
Median
Mean
3rd Qu.
Max
INT (integral-based)
0.636
0.787
0.813
0.831
0.824
0.995
NR (Neeey-Robing
0.862
0.872
0,913
0.920
0.965
1.005
(doubly robust)
0.894
0.917
0.936
0.941
0.966
0.995
H.4.2 Settings with Three Splits
Figure 14 and Table summarize the simulation results in the setting with three splits and e = 0.5. Analogously, Figure 15 and Table 4 the results for the setting with c 2.
Similar the results in the settings with two splits, the estimated variance was close to the true variance.
Table 3: Ratio of the true standard error to the bootstrap estimated standard error of the debiased ECC estimators in the setting with three splits and e 0.5. The entries show summary statistics of the ratios calculated across the 100 different À values. Ratios greater than indicate that the true standard error greater than the bootstrap estimated standard error.
Estimator
Min
1st Qu
Median
Mean
3rd Qu.
Max
Integral- dategral-based estimator
1.026
1.027
1.031
1.031
1.034
1.042
Sowey-Robins estimator
1.014
1.016
1.024
1.023
1.029
1.031
Doubly robust estimator
0.990
1.015
1.016
1.016
1.018
1.019
Integral-Based Estimator (Three Splits)
Newey and Robins Estimator (Three Splits)
Doubly Robust Estimator (Three Splits)
0.000
0,000
0.020
Verarre
Varancc
0.000
0.00
0.000
0.000
0.000
0.00
10
Figure 14: Bootstrap estimated (black dots) versus true (red line) variance of the debiased ECC estimators in the setting with three splits and 0.5 across the À values.
Table 4: Ratio of the true standard error the bootstrap estimated standard error of the debiased ECC estimators in the setting with three splits and 2. The entries show summary statistics of the ratios calculated across the 100 different values. Ratios greater than indicate that the true standard error is greater than the bootstrap estimated standard error.
Estimator
Min
1st Qu
Median
Mean
3rd Qu.
Max
INT (integral- baasegal--base) based)
0.934
0.935
0.939
0.941
0.944
0.978
NR (Newey-Robing
0.935
0.935
0.940
0.942
0.946
0.977
IF (doubly robust)
0.935
0.936
0.943
0.946
0.951
0.994
72

---

Integral-Based Estimator (Three Splits)
Newey and Robins Estimator (Three Splits)
Doubly Robust Estimator (Three Spiits)
004
0.04
0.04
0.03
0103
003
Vasiarnce
Varlance
0.02
0.02
0.02
0.01
0.02
003
0.00
0.00
0.09
Figure 15: Bootstrap estimated (black dots) versus true (red line) variance of the debiased ECC
estimators in the setting with three splits and 2 across the À values.
73


Reviews:
[{"review_id": "Igaqpoexp8", "reviewer": "Reviewer_FNJX", "summary": "The authors study the expected conditional covariance (ECC) under proportional asymptotics. The regressions are assumed to be linear with independent Gaussian noise. For some results the covariates are assumed to be independent and sub-Gaussian. For the main results they are assume to be independent and Gaussian.\nThe first result proves that bias corrected ridge estimators are n^{1/2} consistent. There are six variations of the estimator.\nThe second results gives the asymptotic variances of the six estimators.", "questions": "What is the step by step method that a researcher should use to tune the nuisance functions in the inconsistency regime?\n\nHow does this step by step method perform in coverage simulations?", "limitations": "Yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Quality\n\nThe main weakness of the paper is that the results do not line up to the title and seem incomplete. First, the paper is about ECC not general functional estimation. Second, the assumptions in (2.1) are already very strong. Third, asymptotic normality is not proved, instead normality of all variables is assumed with independent covariates. Fourth, the way to operationalize (3.11) is not described. The variance seems to contain ECC=rho and also two-norm limits that are unknown, so it does not seem to given a feasible estimate of lamba1 and lambda2. Fifth, there is no analysis of a data-driven choice of lambda1 and lambda2. Sixth, the debiasing functions called g are not given in the main text so it is unclear what they depend on and how to implement them.\n\nIf I understand correctly, optimality is never established, even though it appears in the title. For a given ridge estimator on Gaussian data, the best ridge regularization is described as the minimizer of some objective. But this ridge estimator is not optimal, and so terminology  like “sharp asymptotic analyses” seems misleading. \n\nClarity\n\nThe main strength is that the paper summarizes the literature and ideas of the inconsistent regime well. The ideas are clearly presented.\n\nSignificance\n\nI am concerned about how significant the result is because it ultimately does not derive the asymptotic variance without assuming all the data are Gaussian. The result also does not give practical guidance on choosing lambda1 and lambda2, since they seem to depend on unknown limits and maybe a preliminary estimate of ECC, which is not discussed.\n\nOriginality\n\nThe originality depends on how much the derivations of the debiasing g functions departs from the literature on the inconsistent regime. I have not studied the appendix closely. If it is using new techniques, then there are originality points. If it is using the same techniques, the less so.", "was_revised": true, "final_justification": "In my assessment, the main results do not provide a complete enough answer to the main question posed in the paper. I will keep my score.", "rebuttal": "We greatly appreciate the reviewer’s feedback and suggestions, and would like to address them as follows: \n\n> 1. **ECC, not general functional:**  \n\nWe chose ECC as the main parameter, as in the literature it is often considered a starting point for analyzing more general doubly robust functionals (see refs. [7,10,14,15,19,25] of our paper). ECC is studied extensively in both fixed-dimensional nonparametric and ultra-high-dimensional setup under sparsity constraints (e.g., ref [19]). Moreover, as noted in those references, deriving necessary and sufficient ways of tuning nuisance functions for optimal doubly robust functional estimation has been carried out in the nonparametric fixed $p$ regime -- often using the ECC functional as a template. In contrast, we work in a more challenging proportional asymptotic regime ($p/n \\to c \\in (0, \\infty)$), where dimension can exceed sample size and no sparsity is assumed. Here, there are no results for the necessary and sufficient ways of tuning nuisance functions. We view ECC as a foundational step for extending this theory to more general functionals, as has been achieved in other asymptotic regimes. That said, we appreciate your feedback and will revise the paper title to avoid confusion.\n\n> 2. **Assumption (2.1) and asymptotic normality:**  \n\nWe would first like to point out, for the majority of our results (Theorem 3.1 - 3.3, which shows that the estimators are $\\sqrt{n}$-consistent), we do not need asymptotic normality of the covariates or errors; all we need is: (i) $X_{ij}$ are i.i.d. with mean $0$ and variance $1$ and finite $(4 + \\delta)$ moment, and (ii) $(\\epsilon, \\mu) \\overset{i.i.d.}{\\sim} F$ where $F$ with mean $0$, variance $1$, correlation $\\rho$, and finite $(2 + \\delta)$ moment for some $\\delta > 0$.\n\nWe believe the results can be extended to settings where $X_i$ is a subgaussian random variable in $\\mathbb{R}^p$ with bounded sub-gaussian norm using longer Martingale arguments, but we leave it as future work, as it doesn't change the main message of the paper and allows us to keep our proofs relatively compact. \n\nFor exact asymptotic variance, we use the normality of the covariates to shorten the length of the proof significantly. \nOne may once again use Martingale arguments along with deterministic equivalents for suitable functionals of reolvents of large covariance matrices, but it would yield a much longer proof without changing the message.  \n\nAs for the linearity assumption in regression, one could extend the analysis to feature space linear models or consider kernel ridge regression for nuisance functions, leveraging tools from [1]. While the overall framework would remain similar, the extension would require handling more involved deterministic equivalents of the associated Gram matrices, leading to significantly more complex computations. Given the current length and scope of our paper, we leave this promising direction for future work.\n\nLastly, while we do not provide asymptotic normality, we note that the main crux of the proof shares the same complexity as that of obtaining precise asymptotic variance as we describe above. \n\n> 3. **Practical implementation of (3.11) and a data-driven optimal estimator:** \n\nThe analytic expression of the variances indicates that given any $(\\lambda_1, \\lambda_2)$, the limiting variance is a deterministic function of three unknowns: $(||\\alpha_0||_2^2, ||\\beta_0||_2^2, \\alpha_0^\\top \\beta_0)$. The final goal is to choose $(\\lambda_1, \\lambda_2)$ which minimizes these variance expressions. This also echoes the main point of our paper: the prediction-optimal tuning parameters, i.e., the value of $\\lambda_1$ (resp. $\\lambda_2$) that yields the minimum prediction risk of $\\alpha_0$ (resp. $\\beta_0$), may not yield an estimator of ECC with minimum asymptotic variance. This contrasts with the conventional wisdom in double machine learning (DML), where nuisance parameters are typically estimated by minimizing prediction error, a point which is further highlighted in our simulation section. We next present two practical strategies for consistently estimating the limiting variance and selecting optimal tuning parameters:\n\n(i) **consistent estimation of unknown quantities**  One may consistently estimate $(||\\alpha_0||\\_2^2, ||\\beta_0||\\_2^2, \\alpha\\_0^\\top \\beta\\_0)$ and plug them into the variance expressions. To estimate $||\\alpha_0||\\_2^2$, one can first compute a simple plugged-in estimator $||\\hat \\alpha(\\lambda)||\\_2^2$, ($\\hat \\alpha(\\lambda)$ is a ridge estimator of $\\alpha\\_0$) with tuning parameter $\\lambda$, and then apply bias correction to obtain a consistent estimator. Similar procedures apply to $ ||\\beta_0||\\_2^2$ and $\\alpha\\_0^\\top \\beta\\_0$. \n\n(ii) **Bootstrapping** Alternatively, a parametric bootstrap approach can be employed: we first simulate noise terms $(\\varepsilon, \\mu)$ from a bivariate Gaussian distribution with mean $0$, variance $1$, and correlation $\\hat \\theta$ and add them to signals (with $\\hat \\alpha(\\lambda_1)$ and $\\hat \\beta(\\lambda_2)$) to generate synthetic responses, from which a bootstrap estimate $\\hat \\theta^{\\rm Boot}$ is computed. Repeating this procedure multiple times (say $B$ times) yields a collection of bootstrap estimates, say $\\hat \\theta\\_j^{\\rm Boot}$ for $1 \\le j \\le B$, whose empirical variance can then be used as an estimate of the variance of $\\hat \\theta$. Finally, to ensure consistency of this variance estimator, a bias-correction step is needed, for which we need to estimate $(||\\alpha_0||_2^2,  ||\\beta_0||_2^2, \\alpha_0^\\top \\beta_0)$ consistently as before. This method circumvents the need to evaluate the complex analytical form of the variance. We would be happy to provide further details in final version.\n\nFinally, we evaluate the estimated variance over a grid of $(\\lambda_1, \\lambda_2)$ values and select the pair minimizing estimated variance. This would give us an estimator of $\\hat \\theta$, which is $\\sqrt{n}$-consistent and has \\emph{minimum asymptotic variance} among all estimators in our proposed class. This aids in inference via giving a confidence interval with a small width.\n\n**Simulation** To validate, we run simulations for $\\hat{\\theta}^{{db, INT}}_{{3sp}}$ using bootstrap. In particular, we compute the ratio of the bootstrap estimator of the limiting standard deviation (s.d.) to that of the gold-standard oracle Monte-Carlo s.d. (obtained from repeated draws from the true data-generating process). We set $p = 250, n = 500, \\rho = 0.5, B = 10000$ and a grid of 100 $\\lambda$ values from $0.05$ to $10$. We have the summary of this ratio (mean = $1.024$):  \n\n|Min|$Q_1$|Median|$Q_3$|MAX|\n|--------|--------|---|---|--------|\n|1.014|1.016|1.023|1.029|1.051|\n\nThis shows that the bootstrap estimator closely matches the true asymptotic variance, and we have already shown (see Figure 2,4, and 6 in the appendix) that the bias is negligible. Thus, the data-driven procedure yields valid coverage. \n\n> 4. **Debiasing function g:** \n\nThe debiasing functions $g$ are defined using $\\triangleq$, e.g., in line 184 $$g^{\\rm INT}\\_{\\rm 3sp}(\\lambda_1, \\lambda_2)= \\left( \\int \\frac{x \\, dF_{\\rm MP}(x) }{x+\\lambda_1}\\right) \\left( \\int \\frac{x \\, dF_{\\rm MP}(x) }{x + \\lambda_2}\\right),$$ and in line 198 $$g_{1, \\rm 2sp}^{\\rm INT}(\\lambda_1, \\lambda_2) = \\int \\frac{x^2 \\ dF_{\\rm MP}(x)}{(x + \\lambda_1)(x + \\lambda_2)} \\,,$$ where $F_{\\rm MP}$ is Marchenko-Pasteur (MP)distribution, the limiting spectral distribution of eigenvalues of sample covariance matrices. In practice, we calculated by 10000 Monte Carlo samples of MP distribution.\n\n> 5. **Optimality of ridge estimator and sharp asymptotic analysis:** \n\nIn parameter estimation, optimality typically refers to either i) rate optimality in the minimax sense, and ii) semi-parametric efficiency. As our estimator is proven to be $\\sqrt{n}$-consistent, it achieves the former, as it attains the gold-standard parametric estimation rate. \n\nIn classical fixed-dimensional settings, semiparametric efficiency typically refers to the smallest achievable asymptotic variance among all regular estimators. In contrast, our analysis operates under the proportional asymptotic regime (e.g., see ref. [2] in our paper), where the covariate dimension $p$ grows proportionally larger than the sample size $n$, i.e., $p/n \\to c \\in (1, \\infty)$, without assuming sparsity in $E[Y|X]$ or $E[A|X]$.\n\nIn this high-dimensional context, foundational concepts like regularity of estimators and Hajek–LeCam local minimax theory are not yet well developed. Thus, rather than pursuing classical efficiency bounds, we focus on identifying the best estimator within a given estimation strategy. Specifically, we ask: given a debiasing approach with regularized nuisance estimation and a fixed sample-splitting scheme (e.g., 1/2 or 1/3 splits), which configuration yields the lowest asymptotic variance? \n\nCan that be achieved by merely a priori choosing the level of regularization from a prediction optimality perspective -- as is the case for DML estimators under fixed $p$ or ultra-high dimensional sparse regimes? Our notion of sharp optimality is grounded in this perspective. We argue that when ridge regularization is used to estimate the nuisance functions, the regularization parameter should be chosen by minimizing the asymptotic variance of $\\theta_0$ directly, rather than the prediction optimal one (as advocated by DML). That said, it remains possible that alternative regularization schemes, such as LASSO, may yield even lower variance. However, identifying the minimum possible achievable variance in this regime is highly nontrivial and is unknown even for simple linear regression. Therefore, this remains an important open problem for future investigation. We would be happy to include this discussion in the final version and are open to incorporating any other suggestions you may have."}, {"review_id": "6jCwM8PoNC", "reviewer": "Reviewer_Wn4b", "summary": "The paper studies the optimal tuning parameter choice in ridge regression to estimate the Expected Conditional Covariance in the setting of linear data-generating model and the proportional asymptotic regime (where $p>n$ is allowed). It studies three estimators and two data-splitting methods, and proves the $\\sqrt{n}-$consistency and derives the asymptotic variances of the estimators.", "questions": "The proofs rely heavily on results from Random Matrix Theory. To what extent can the techniques be extended beyond the restrictive assumptions of a linear model and the ridge estimator?\n\nThe paper provides analytic expressions for the asymptotic variances. Do these expressions offer any intuitive understanding of the influence of the parameters involved? Can they help explain the variance blowup observed in the experiment?", "limitations": "The paper discusses the limitation on Gaussian assumption. The limitations of the proving technique applied to linear model and ridge estimator merits consideration.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper clearly explained the novelties compared with existing work and proves novel results for proportional asymptotic regime. It reveals that prediction-optimal tuning parameters may not lead to the lowest asymptotic variance of the ECC estimator. \n\nThe proposed sample-splitting approach appears to sacrifice the sample efficiency of DML. In addition, there is a considerable body of work on high-dimensional linear regression, where RMT is widely applied. The paper can benefit from discussing connections to this literature.", "was_revised": true, "final_justification": "Thank you for addressing the concerns. I will raise the score.", "rebuttal": "We greatly appreciate the reviewer’s feedback and suggestions, and would like to address them as follows. \n\n> 1. **The sample-splitting approach appears to sacrifice the sample efficiency of DML:** \n\nThank you for raising this important point. You are absolutely right that sample splitting generally sacrifices statistical efficiency, and cross-fitting is a widely adopted strategy to address this issue.\nHowever, our analysis is conducted under the proportional asymptotic regime where $p/n \\to c \\in (1, \\infty)$. In this setting, a rigorous analysis of cross-fitting becomes substantially more challenging. To the best of our knowledge, recent work (ref. [23] of our paper) analyzed cross-fitting in the context of AIPW estimators, but only in the regime $p/n \\to c \\in (0, 1)$, where the dimensionality is smaller than the sample size. The main technical difficulty is that, in classical fixed-$p$ settings (or in high-dimensional settings with sparsity constraints), the estimators obtained from different sample splits are only weakly correlated and the correlation is asymptotically negligible. However, in the proportional regime, this correlation is often non-negligible, rendering the analysis of cross-fitting significantly more challenging. Moreover, whether cross-fitting effectively reduces variance, i.e., whether the non-negligible correlation between estimators from different splits is negative, can depend on several factors, including the sample splitting scheme, the type of estimator used, and the specific parameter settings. In fact, recent work (ref. [23] of our paper) has observed a striking phenomenon: the correlation between estimators from different splits can be positive and non-vanishing, leading to increased variance of the cross-fitted estimator. This highlights the nuanced and context-dependent behavior of cross-fitting in the proportional asymptotic regime. While we believe it may be possible to adapt the techniques from [23] and extend them carefully to this setting, doing so is non-trivial and would likely require substantial new insights. As such, we leave this as an important direction for future work. We expect that a complete theoretical treatment of cross-fitting under the regime $p/n \\to c \\in (1, \\infty)$ would constitute a contribution in its own right.\n\n\n> 2. **Extension beyond linear model and ridge:** \n\nThank you for raising this interesting point! While, as a first step, we focus on a linear data-generating model, we agree that extending the analysis to more expressive non-linear models—such as kernel ridge regression (KRR) or random feature (RF) models is an important direction for future work.\nRecent advances (e.g., [1]) have analyzed KRR and RF estimators under the proportional asymptotic regime, leveraging tools from random matrix theory. Therefore, we believe that it is possible to construct a consistent estimator of $\\theta_0$ when the conditional mean function is a polynomial. However, extending these results to enable $\\sqrt{n}$-consistent estimation and drawing a valid inference remains challenging. In particular, debiasing non-linear estimators in the proportional regime is a highly non-trivial task, and to the best of our knowledge, no prior work has addressed this issue, even in the context of simple regression. We believe this represents a promising but technically demanding research avenue for future work. \n\nIf we replace the squared error loss with another loss function (e.g., Huber loss), it is still possible to leverage similar tools from random matrix theory (RMT) to arrive at the desired conclusions. This is supported by existing work on general M-estimation problems in high dimensions using RMT techniques (e.g., [2]). However, if the penalty function is changed to the $\\ell_1$ norm or other convex penalties, the analysis becomes more involved. In such settings, the tools from Approximate Message Passing (AMP), Convex Gaussian minmax theorem (CGMT), or Stein's identity become useful to obtain a $\\sqrt{n}$-consistent and asymptotically normal (CAN) estimator of $\\theta_0$. Debiasing methods based on these techniques have been studied in the context of linear models (ref. [25] of our paper) and doubly robust estimators such as AIPW under the regime $p < n$ (ref. [23] of our paper). These developments suggest that similar techniques could be adapted to our setting. We leave this direction as a promising avenue for future investigation.\n\n\n> 3. **Asymptotic variance:** \n\nThe analytic expression of the variances indicates that given any $(\\lambda_1, \\lambda_2)$, the limiting variance is a deterministic function of three unknowns: $(||\\alpha_0||_2^2, ||\\beta_0||_2^2, \\alpha_0^\\top \\beta_0)$. The final goal is to choose $(\\lambda_1, \\lambda_2)$ which minimizes these variance expressions. This also echoes the main point of our paper: the prediction-optimal tuning parameters, i.e., the value of $\\lambda_1$ (resp. $\\lambda_2$) that yields the minimum prediction risk of $\\alpha_0$ (resp. $\\beta_0$), may not yield an estimator of ECC with minimum asymptotic variance. This contrasts with the conventional wisdom in double machine learning (DML), where nuisance parameters are typically estimated by minimizing prediction error, a point which is further highlighted in our simulation section. We next present two practical strategies for consistently estimating the limiting variance and selecting optimal tuning parameters:\n\n(i) **consistent estimation of unknown quantities**  One may consistently estimate $(||\\alpha_0||\\_2^2, ||\\beta_0||\\_2^2, \\alpha\\_0^\\top \\beta\\_0)$ and plug them into the variance expressions. To estimate $||\\alpha_0||\\_2^2$, one can first compute a simple plugged-in estimator $||\\hat \\alpha(\\lambda)||\\_2^2$, ($\\hat \\alpha(\\lambda)$ is a ridge estimator of $\\alpha\\_0$) with tuning parameter $\\lambda$, and then apply bias correction to obtain a consistent estimator. Similar procedures apply to $ ||\\beta_0||\\_2^2$ and $\\alpha\\_0^\\top \\beta\\_0$. \n\n(ii) **Bootstrapping** Alternatively, a parametric bootstrap approach can be employed: we first simulate noise terms $(\\varepsilon, \\mu)$ from a bivariate Gaussian distribution with mean $0$, variance $1$, and correlation $\\hat \\theta$ and add them to signals (with $\\hat \\alpha(\\lambda_1)$ and $\\hat \\beta(\\lambda_2)$) to generate synthetic responses, from which a bootstrap estimate $\\hat \\theta^{\\rm Boot}$ is computed. Repeating this procedure multiple times (say $B$ times) yields a collection of bootstrap estimates, say $\\hat \\theta\\_j^{\\rm Boot}$ for $1 \\le j \\le B$, whose empirical variance can then be used as an estimate of the variance of $\\hat \\theta$. Finally, to ensure consistency of this variance estimator, a bias-correction step is needed, for which we need to estimate $(||\\alpha_0||_2^2,  ||\\beta_0||_2^2, \\alpha_0^\\top \\beta_0)$ consistently as before. This method circumvents the need to evaluate the complex analytical form of the variance. We would be happy to provide further details in final version.\n\nFinally, we evaluate the estimated variance over a grid of $(\\lambda_1, \\lambda_2)$ values and select the pair minimizing estimated variance. This would give us an estimator of $\\hat \\theta$, which is $\\sqrt{n}$-consistent and has \\emph{minimum asymptotic variance} among all estimators in our proposed class. This aids in inference via giving a confidence interval with a small width.\n\nOur simulation suggests that the bootstrap estimator works well, but we cannot add it due to space constraints. Kindly see our response to Comment 3 of Reviewer FNJX and Comment 6 of AoFP for the details. \n\n> 4. **Variance blowup:** \n\nThe variance blow-up is primarily driven by the bias correction factor. For instance, consider the definition of $\\hat{\\theta}\\_{2sp}^{\\rm INT, db}$ in Equation (3.4). It is immediate from the definition that the variance of this estimator diverges when $$ \\left(1 - \\frac{g\\_{2, 2sp}^{\\rm INT}(\\lambda_1, \\lambda_2)}{g\\_{1, 2sp}^{\\rm INT}(\\lambda_1, \\lambda_2)}\\right) \\approx 0 \\,.$$\nThis scenario arises in our simulations when $\\lambda_1 = \\lambda_2 = \\lambda$ is around 1.5 (Top left picture of Figure 1), where $g\\_{1, 2sp}^{\\rm INT}(\\lambda, \\lambda) \\approx g\\_{2, 2sp}^{\\rm INT}(\\lambda, \\lambda)$, causing the variance to explode. However, this phenomenon does not affect our method. Since our goal is to select $(\\lambda_1, \\lambda_2)$ that minimizes the asymptotic variance, such values where the variance is unstable are naturally excluded from consideration.\n\n[1] *Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression*. Theodor Misiakiewicz.\n\n[2] *Asymptotics For High Dimensional Regression M-Estimates: Fixed Design Results*. Lihua Lei, Peter J. Bickel, Noureddine El Karoui."}, {"review_id": "stHG44ohpp", "reviewer": "Reviewer_AoFP", "summary": "- DML relies on estimating nuisance functions (eg. Propensity score, outcome regression) accurately. However, in high-dimensional settings, ridge, Lasso, and many other estimators become inconsistent. Besides, in the propotional regime, the invalidity of orthogonalization may be invalid, thus the product of two inconsistent nuisance estimators can introduce substantial bias. Furthermore, cross-fitting of DML becomes problematic: estimates from different folds can be correlated;\n\n- This paper studies how to optimally tune nuisance function estimators in high-dimensional settings of the ECC. They derive bias-corrected versions of three ECC estimators (plug-in, Newey-Robins, and doubly robust) that achieve $/sqrt{n}$-consistency, each applied under two distinct sample-splitting strategies. They also provide exact asymptotic variance formulas for these estimators, showing that the tuning parameters that are optimal for prediction do not necessarily minimize the variance of the final ECC estimator.\n\n- Simulations validate the theoretical findings and provide practical insights into selecting tuning parameters for reliable inference in high-dimensional settings.", "questions": "See my questions in the above “Weakness” part and below:\n\n* Section 3.2 and Section 4, no concrete method is provided for how practitioners can implement this in real-data setting when ground truth is unavailable. Can you add empirical study or a practical algorithm or heuristic for tuning under real constrains?\n* Can you add a simple simulation example under eg non-Gaussian error, nonlinearity in E[Y|X] and E[A|X]? This would strengthen the empirical case with robustness to non-Gaussian and misspecified settings.\n* In the two-split simulations, the authors note that the variance “blow up” around a certain $\\lambda$, but this is unexplained? Any insights?", "limitations": "See the limitations also in the above “Weakness” and “Questions” part.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The paper is technically strong. It addresses a challenging of inference for doubly robust functionals when nuisance functions cannot be consistently estimated due to high-dimensionality.\n- The authors provide rigorous bias correction techniques for three estimators (integral-based, Newey-Robins, and doubly robust), and prove $\\sqrt{n}$-consistency and asymptotic normality.\n- The asymptotic variance analysis is detailed, revealing a nuanced dependence of estimator performance on tuning parameters and sample splitting schemes.\n- The simulation studies illustrating theoretical findings.\n- The insights regarding inference-optimal and prediction-optimal can be useful for the causal and robust statistical inference applications. \n- The paper builds on concepts like sample splitting and debiasing from classical DML literature, its methodological contributions—especially the derivation of explicit variance formulas under ridge regularization—are novel.\n\n**Weakness:**\n\n- This paper only analyzes ridge regression for estimating nuisance functions $E[A|X]$, and $E[Y|X]$ under the linearity assumptions. While this helps derive the closed-form asymptotic variance but narrows the applicability. \n- The results do not generalize to model nonlinearity or non-Gaussian errors. It is unclear whether similar variance or bias correction methods extend to more realistic scenarios. \n- The asymptotic results depend on the assumption of Gaussian and sub-Gaussian. It would be great if the author could add a discussion of the robustness of the results to deviations from Gaussianity. Though it can be a future work for the theoretical study, adding empirical verification of performance under violations of this assumption might be helpful to explain the robustness of the results.\n- Any minimax lower bound for estimating ECC under proportional asymptotic? \n- How sensitive are confidence intervals to variance estimation? \n- In theory, it’s elegant to minimize asymptotic variance w.r.t. $\\lambda_1$, $\\lambda_2$, but these variance expressions are complicated, require knowledge of unknown population parameters (eg. Inner product between nuisance coefficients). However, there’s no practical method is proposed for running parameter sections, may not be practically estimable in finite samples.", "was_revised": true, "final_justification": "I would like to increase my score to 5.", "rebuttal": "We greatly appreciate the reviewer’s feedback and suggestions, and would like to address them as follows.\n\n> 1. **Linearity assumption and ridge regression**:\n\nWe agree that the linear model assumption may appear restrictive at first glance. However, we would like to emphasize that our analysis includes the challenging proportional asymptotic regime where the number of covariates $p$ is larger than the sample size $n$, and $p/n \\to c \\in (1, \\infty)$. This regime is fundamentally different from the standard high-dimensional setting, where valid inference typically relies on some form of sparsity assumption. Theoretical understanding of the proportional regime is still in its early stages. In fact, results on simple linear regression [1] and AIPW estimators under the linear outcome regression and propensity score model and $p < n$ (ref. [23] of our paper) have only recently been established. From this perspective, focusing on the linear model is not merely a simplifying assumption, but rather a necessary and meaningful first step toward developing a deeper understanding of more complex models in this regime. \n\nA natural next step is to extend the linear regression setting to regression in a general feature space, such as kernel ridge regression. Since our analysis relies on deterministic equivalents of functionals of Gram matrices, the techniques can be carried over, provided similar results hold for Gram matrices in the feature space setting.\n    \nRegarding the ridge penalty, we believe that similar analyses can be conducted in the presence of other convex penalties (e.g., $\\ell_1$ norm), borrowing tools from Approximate Message Passing (AMP) theory. Debiasing methods based on AMP have been studied in the context of linear models [1] and doubly robust estimators such as AIPW under the regime $p < n$ of ref. [23]. These developments suggest that similar techniques could be adapted to our setting. \n\n> 2. **Gaussian assumption:** \n\nThank you for raising this concern. We would first like to point out that for the majority of our results (Theorems 3.1 - 3.3, which show that the estimators are $\\sqrt{n}$-consistent), we do not need normality of the covariates or the errors. The following two conditions are sufficient for these three estimators: (i) $X_{ij}$ are i.i.d. with mean $0$ and variance $1$ and finite $(4 + \\delta)$ moment, and (ii) $(\\epsilon, \\mu) \\overset{i.i.d.}{\\sim} F$ where $F$ with mean $0$, variance $1$, correlation $\\rho$, and has finite $(2 + \\delta)$ moment for some $\\delta > 0$.\n\nRegarding allowing dependence between the covariates, we believe that our results can be extended to the settings where $X_i$ is a subgaussian random variable in $\\mathbb{R}^p$ with bounded sub-gaussian norm using more modern techniques. However, we leave this as future work, as we do not believe it will add anything substantial to the main message of the paper, and the current analysis is already quite complicated. \n\nTheorem 3.4 indeed relies on the assumption of Gaussian covariates and error distribution to derive the exact form of the limiting variance. That said, we believe these assumptions could potentially be relaxed by appealing to suitable universality results. Nonetheless, doing so would require substantial theoretical development and a careful analysis to rigorously establish the necessary universality results, which is a challenging task in random matrix theory. \n\nWe added additional simulations below to show robustness against the Gaussian assumption. See our response to ``Additional simulations\" below. \n\n> 3. **Minimaxity:** \n\nIn this paper, we establish that ECC can be estimated at the $\\sqrt{n}$ rate, which is known to be minimax optimal for estimating finite-dimensional parameters under standard assumptions. Consequently, our estimator is minimax rate optimal. Obtaining sharp efficiency bounds beyond rate minimaxity for functional estimation under proportional asymptotics remains an open direction. Therefore, we explore optimal estimation under a given strategy of estimation, i.e., ridge tuning, sample splitting, and given choice of estimators such as INT, NR, or DR. \n\n> 4. **Confidence interval coverage:** \n\nIn the analyses in the current paper, the bias of the debiased estimators is negligible compared to their standard errors. We also discuss standard error estimation in the following bullet point, which we found to be within 5\\% of the true standard error. Together, these suggest near-nominal confidence interval coverage. However, we would be happy to report how sensitive confidence intervals are to variance estimation in the revised paper. \n\n> 5. **Practical methods for selecting tuning parameters:** \n\nThank you for raising this concern. While the expressions for the limiting variances may appear complex, they are in fact smooth deterministic functions of three unknown quantities: $(||\\alpha_0||_2^2, ||\\beta_0||_2^2, \\alpha_0^\\top \\beta_0)$. This allows practitioners to adopt one of two practical strategies, as outlined below:\n\n(i) **consistent estimation of unknown quantities**  One may consistently estimate $(||\\alpha_0||\\_2^2, ||\\beta_0||\\_2^2, \\alpha\\_0^\\top \\beta\\_0)$ and plug them into the variance expressions. To estimate $||\\alpha_0||\\_2^2$, one can first compute a simple plugged-in estimator $||\\hat \\alpha(\\lambda)||\\_2^2$, ($\\hat \\alpha(\\lambda)$ is a ridge estimator of $\\alpha\\_0$) with tuning parameter $\\lambda$, and then apply bias correction to obtain a consistent estimator. Similar procedures apply to $ ||\\beta_0||\\_2^2$ and $\\alpha\\_0^\\top \\beta\\_0$. \n\n(ii) **Bootstrapping** Alternatively, a parametric bootstrap approach can be employed: we first simulate noise terms $(\\varepsilon, \\mu)$ from a bivariate Gaussian distribution with mean $0$, variance $1$, and correlation $\\hat \\theta$ and add them to signals (with $\\hat \\alpha(\\lambda_1)$ and $\\hat \\beta(\\lambda_2)$) to generate synthetic responses, from which a bootstrap estimate $\\hat \\theta^{\\rm Boot}$ is computed. Repeating this procedure multiple times (say $B$ times) yields a collection of bootstrap estimates, say $\\hat \\theta\\_j^{\\rm Boot}$ for $1 \\le j \\le B$, whose empirical variance can then be used as an estimate of the variance of $\\hat \\theta$. Finally, to ensure consistency of this variance estimator, a bias-correction step is needed, for which we need to estimate $(||\\alpha_0||_2^2,  ||\\beta_0||_2^2, \\alpha_0^\\top \\beta_0)$ consistently as before. This method circumvents the need to evaluate the complex analytical form of the variance. We would be happy to provide further details in the final version.\n\nFinally, we evaluate the estimated variance over a grid of $(\\lambda_1, \\lambda_2)$ values and select the pair minimizing estimated variance. This would give us an estimator of $\\hat \\theta$, which is $\\sqrt{n}$-consistent and has \\emph{minimum asymptotic variance} among all estimators in our proposed class. This aids in inference via giving a confidence interval with a small width. Due to space constraints, we omit the technical details here, but we would be happy to elaborate upon request and add a discussion in the final version of the paper. \n\n> 6. **Additional simulations**\n\n**Parametric Bootstrap:** We run simulations for $\\hat{\\theta}^{{\\rm INT, db}}_{{\\rm 3sp}}$ using bootstrap. In particular, we compute the ratio of the bootstrap estimator of the limiting standard deviation (s.d.) to that of the gold-standard oracle Monte-Carlo s.d. (obtained from repeated draws from the true data-generating process). We set $p = 250, n = 500, \\rho = 0.5, B = 10000$ and and a grid of 100 $\\lambda$ values from $0.05$ to $10$.  Below is the summary of this ratio (mean = $1.024$) over $\\lambda$:  \n\n|Min|$Q_1$|Median|$Q_3$|MAX|\n|--------|--------|---|---|--------|\n|1.014|1.016|1.023|1.029|1.051|\n\nThis shows that the bootstrap estimator closely matches the true asymptotic variance across different $\\lambda$, and we have already shown (see Figure 2,4, and 6 in the appendix) that bias is negligible. Thus, our data-driven procedure yields valid coverage. \n\n**Non-gaussian error:** We perform simulations with non-Gaussian errors for $\\hat{\\theta}^{{\\rm INT, db}}_{{\\rm 3sp}}$, where we now generate errors from a bivariate Laplace distribution with mean $0$, marginal variances $1$, and correlation $\\rho=0.5$, same as the Gaussian setting in the paper. The de-biased estimators had negligible bias in this case, and the following table provides a summary of the ratio (Laplace to Gaussian) of s.d. over 10000 Monte Carlo iterations: \n\n|Min|$Q_1$|Median|$Q_3$|MAX|\n|--------|--------|---|---|--------|\n|1.039|1.041|1.048|1.053|1.06|\n\nThis implies our results continue to hold beyond Gaussianity. We would be happy to elaborate upon request and add this to the final version of the paper.\n\n> 7. **Variance blowup:** \n\nThe variance blow-up is primarily driven by the bias correction factor. For instance, consider the definition of $\\hat{\\theta}\\_{2sp}^{\\rm INT, db}$ in Equation (3.4). It is immediate from the definition that the variance of this estimator diverges when $$ \\left(1 - \\frac{g\\_{2, 2sp}^{\\rm INT}(\\lambda_1, \\lambda_2)}{g\\_{1, 2sp}^{\\rm INT}(\\lambda_1, \\lambda_2)}\\right) \\approx 0 \\,.$$ This scenario arises in our simulations when $\\lambda_1 = \\lambda_2 = \\lambda$ is around 1.5 (Top left picture of Figure 1), where $g\\_{1, 2sp}^{\\rm INT}(\\lambda, \\lambda) \\approx g\\_{2, 2sp}^{\\rm INT}(\\lambda, \\lambda)$, causing the variance to explode. However, this phenomenon does not affect our method. Since our goal is to select $(\\lambda_1, \\lambda_2)$ that minimizes the asymptotic variance, such values where the variance is unstable are naturally excluded from consideration.\n\n[1] Hastie, T., Montanari, A., Rosset, S., & Tibshirani, R. J. (2022). Surprises in high-dimensional ridgeless least squares interpolation. Annals of statistics, 50(2), 949."}, {"review_id": "OfqfPV6G1U", "reviewer": "Reviewer_Fm5s", "summary": "The authors study parameter tuning for estimation of the ECC in linear models under the asymptotic regime $\\frac{p}{n} \\rightarrow c$ as $n \\rightarrow \\infty$ (including $p > n$). In this setting, the linear model weights are nuisance parameters for which naive plug-in, Newey--Robins or doubly robust estimators have worse-than-$\\frac{1}{\\sqrt{n}}$ asymptotic bias. The authors propose debiased versions of these estimators. The authors derive asymptotic variances of their estimators---including estimators utilizing with additional subsampling---as a function of the ridge-regression regularization parameters for the linear weights.", "questions": "Overall, I am very happy with this paper. No questions/suggestions significant enough to influence my score come to mind. I am keen to see other reviewers' comments and will follow up with any significant questions during the next stage.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "The paper was a pleasure to read, with contributions well motivated and technical results clearly spelled out. The authors compare a wide range of approaches and make clear which results are novel and/or surprising. Applications to structure discovery and/or covariate selection are clear.", "was_revised": true, "final_justification": "`Reviewer FNJX` raised concerns about the validity of the authors' analysis, which, to my reading, have been addressed by the authors. \n\nAll three reviewers (except myself) have downweighted their scores based on concerns about the scope of the authors' investigation due to linear Gaussian assumptions and a focus on estimators of the ECC. The authors rebut that their derivation strategy shows promise for future work in extending their results. However, given the simplicity of the authors' argument for why Gaussian assumptions should be unnecessary, a short empirical investigation with, e.g., sub-Gaussian noise would have strengthened their rebuttal. \n\nThe authors reiterate the potential use of their approach in structure discovery without a priori sparsity assumptions, which is an exciting direction with possible applications in assessing the sensitivity of structure discovery approaches to these assumptions.", "rebuttal": "Thank you very much for your kind words! If you have any other comments/concerns during the discussion period, kindly do let us know. We would be happy to answer them."}]

Author rebuttal:

Meta-review: **Summary**
Double Machine Learning (DML) relies on estimating nuisance functions accurately. However, in high-dimensional settings, estimators such as ridge, LASSO, and other become inconsistent. Besides, the proportional regime ($p/n = c \longrightarrow (0, \infty)$), the product of two inconsistent nuisance estimators can introduce substantial bias. Furthermore, cross-fitting of DML becomes problematic since estimates from different folds can correlated.

This paper studies how to optimally tune nuisance function estimators in high-dimensional settings of the ECC. They derive bias-corrected version of three ECC estimators under two different sample-splitting strategies: (plug-in, Newey-Robins, and doubly-robust) for the ECC estimators, and (2-splitting and 3-splitting) for sample-splitting. All bias-corrected ECC estimators achieve $\sqrt{n}$-consistency. Next, the authors derive exact asymptotic variance formulas for these estimators, showing that the tuning parameters that are optimal for prediction do not necessarily minimize the variance of the final ECC estimator.

Simulations validate the theoretical findings and provide practical insights into selecting tuning parameters for reliable inference in high-dimensional settings.

**Strength**

Except Rev.1 (FNjX), all reviewers agree on the following (best summarized by Rev. 3 AoFP) :

1. The paper is technically strong; it addresses a the challenge of inference for doubly-robust robust functionals when nuisance functions cannot be consistently estimated due to high-dimensionality.

2. The authors provide rigorous bias correction techniques for three ECC estimators, and prove $\sqrt{n}$-consistency and asymptotic normality.

3. The asymptotic variance analysis is detailed, revealing a nuanced dependence of estimator performance on tuning parameters and sample splitting schemes.

4. Simulation studies illustrate theoretical findings.

5. The insights regarding inference-optimal and prediction-optimal can be useful for the causal and robust statistical inference applications.

6. The paper builds on concepts like sample splitting and debiasing from classical DML literature, the methodological contributions, especially the derivation of explicit variance formulas under ridge regularization, are **novel**.

The concerns raised by Rev.1 (FNjX) are marginal and do not reflect a deep comprehension for the topic and contribution discussed in this submission. The authors responded with a thorough rebuttal to the reviewer, yet I find the reviewer's comment to the authors (after the rebuttal) to be disconnected from the author's rebuttal. I believe Rev.1 (FNjX) did not read, or did not carefully read the authors rebuttal, and/or even try to review the submission in order to have a better understanding. 

Accordingly, the AC recommends acceptance for this nice and strong submission.
Decision: Accept (spotlight)