Title: Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference
Venue: NeurIPS 2023
Accepted: yes
Keywords: Adaptive linear regression

bandit algorithms

high dimensional statistics

statistical inference
Abstract: Estimation and inference in statistics pose significant challenges when data are collected adaptively. Even in linear models, the Ordinary Least Squares (OLS) estimator may fail to exhibit asymptotic normality for single coordinate estimation and have inflated error. This issue is highlighted by a recent minimax lower bound, which shows that the error of estimating a  single coordinate can be enlarged by a multiple of $\sqrt{d}$ when data are allowed to be arbitrarily adaptive, compared with the case when they are i.i.d. Our work explores this striking difference in estimation performance between utilizing i.i.d. and adaptive data. We investigate how the degree of adaptivity in data collection impacts the performance of estimating a low-dimensional parameter component in high-dimensional linear models. We identify conditions on the data collection mechanism under which the estimation error for a low-dimensional parameter component matches its counterpart in the i.i.d. setting, up to a factor that depends on the degree of adaptivity. We show that OLS or OLS on centered data can achieve this matching error. In addition, we propose a novel estimator for single coordinate inference via solving a Two-stage Adaptive Linear Estimating equation (TALE). Under a weaker form of adaptivity in data collection, we establish an asymptotic normality property of the proposed estimator.

Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference

Abstract

Estimation and inference in statistics pose significant challenges when data are collected adaptively. Even in linear models, the Ordinary Least Squares (OLS) estimator may fail to exhibit asymptotic normality for single coordinate estimation and have inflated error. This issue is highlighted by a recent minimax lower bound, which shows that the error of estimating a single coordinate can be enlarged by a multiple of √ d when data are allowed to be arbitrarily adaptive, compared with the case when they are i.i.d. Our work explores this striking difference in estimation performance between utilizing i.i.d. and adaptive data. We investigate how the degree of adaptivity in data collection impacts the performance of estimating a low-dimensional parameter component in high-dimensional linear models. We identify conditions on the data collection mechanism under which the estimation error for a low-dimensional parameter component matches its counterpart in the i.i.d. setting, up to a factor that depends on the degree of adaptivity. We show that OLS or OLS on centered data can achieve this matching error. In addition, we propose a novel estimator for single coordinate inference via solving a Two-stage Adaptive Linear Estimating equation (TALE). Under a weaker form of adaptivity in data collection, we establish an asymptotic normality property of the proposed estimator.

1 Introduction

Estimating a low-dimensional parameter component in a high-dimensional model is a fundamental problem in statistics and machine learning that has been widely studied in e.g., semiparametric statistics [35, 8], causal inference [16, 15] and bandit algorithms [1, 27]. When data are independently and identically distributed (i.i.d.), it is often possible to derive estimators that are asymptotically normal with a rate of convergence of √ n , and that achieve the semi-parametric variance lower bound

that is independent of the dimension. There is now a rich body of literature that studies this problem under various scenarios [5, 6, 33, 36, 4, 8, 41].

In this work we are interested in the same estimation and inference problem but under the setting where the i.i.d. data assumption fails. Specifically, we consider an adaptive collection framework where the data collected at time i is allowed to be dependent on the historical data collected up to time i -1 . This adaptive framework incorporates datasets originated from applications in many fields, including sequential experimental design [15], bandit algorithm [27], time series modeling [7], adaptive stochastic approximation schemes [13, 24].

1.1 An interesting lower bound

To see the intrinsic difference between the i.i.d. and the adaptive data collection settings, we consider the canonical example of linear model y = x ⊤ θ ∗ + ε , where the parameter θ ∗ = ( θ ∗ 1 , θ ∗ 2 ) ∈ R 1 × R d -1 , ε iid ∼N (0 , 1) . Clearly, when the covariates { x i } i ≤ n are deterministic, a straightforward calculation yields

where θ ols is the OLS estimator and S n := ∑ n x i x ⊤ is the sample covariance matrix.

However, somewhat surprisingly, when the covariates { x i } i ≤ n are allowed to be collected in an arbitrary adaptive manner, in a recent work [20] the authors proved the following (informal) counterintuitive minimax lower bound on the scaled-MSE (defined in Definition 2.2)

where the extra d -factor enters the estimation of a single coordinate. This lower bound indicates that a dimension independent single coordinate estimation is infeasible when the data are collected arbitrarily adaptively. This is undesirable especially in the high dimensional scenario where d →∞ , since a √ n -consistent estimation is unattainable. Motivated by the contrast between i.i.d. and adaptive data collection, we pose the following question in this work:

Can we bridge the gap between iid and adaptive data collection, and obtain an estimator for a low-dimensional parameter component in linear models, such that its performance depends on the degree of adaptivity?

1.2 Contributions

In this work, we initiate the study of how the adaptivity of the collected data affects low-dimensional estimation in a high-dimensional linear model. We explore the previously posed question and provide an affirmative answer.

We begin by introducing a general data collection assumption, which we term ( k, d ) -adaptivity . Broadly speaking, ( k, d ) -adaptivity implies that the data pairs { ( x i , y i ) } n i =1 ∈ R d × R are collected in a way that the first k coordinates of x i (denoted by x ad i ) are chosen adaptively based on the historical data, while the remaining d -k coordinates of x i (denoted by x nad i ) are i.i.d. across time i ∈ [ n ] .

Assume the collected data are ( k, d ) -adaptive from a linear model y = x ⊤ θ ∗ + ε . We analyze the lower-dimensional estimation problem under the scenarios where the i.i.d. non-adaptive components x nad i are either zero-mean or nonzero-mean. In the zero mean case, we show that the ordinary least squares estimator (OLS) for the first k -coordinate yields a scaled mean squared error (scaledMSE) of k log( n ) (Theorem 3.1). For the nonzero-mean case, a similar result is achieved using the OLS estimator on centered data (Theorem 3.2). Consequently, we find that the degree of adaptivity significantly impacts the performance of single coordinate estimation, in the sense that the scaled-MSE is inflated by a factor of k , where k denotes the number of adaptive coordinates (see Corollary 3.3).

Although OLS for a single coordinate has a dimension independent scaled-MSE when the collected data are (1 , d ) -adaptive, it should be noted that OLS may exhibit non-normal asymptotic behavior [13, 20] when data are adaptively collected. Therefore, we propose a novel estimator by solving a Twostage Adaptive Linear Estimating Equation (TALE). When the collected data are (1 , d ) -adaptive and the non-adaptive component is zero mean, we show that our new estimator is asymptotically normal and has a comparable scaled-MSE as the naive OLS estimator (see Theorem 3.4).

2 Problem set up

Consider a linear model

where the parameter θ ∗ ∈ R d , and ε is a zero mean noise variable. Given access to a data set { ( x i , y i ) } i ≤ n from the model (3), we are interested in the estimation and inference problem of a low-dimensional parameter component θ ∗ ad ∈ R k , where θ ∗ = ( θ ∗⊤ ad , θ ∗⊤ nad ) ⊤ .

In this paper, we are interested in adaptive data collection regime. Concretely, we assume that the data are collected adaptively in the following way

Definition 2.1 ( ( k, d ) -adaptivity) The collected samples { ( x i , y i ) } i ≤ n forms a filtration {F} ∞ i =0 with F 0 = ∅ and F i = σ ( x 1 , y 1 , . . . , x i , y i ) . Let P be an unknown distribution on R d -k . We assume that at each stage, i ≥ 1

The adaptive component x ad i = x i, 1: k is collected from some unknown distribution that could depend on F i -1 .

The non-adaptive component x nad i = x i,k +1: d is a sample from P and independent of ( x ad i , F i -1 ) .

When k = 0 , Definition 2.1 reduces to an i.i.d. data collection strategy; when k = d , it corresponds to the case where the data are allowed to be collected arbitrarily adaptively. Consequently, ( k, d ) -adaptivity connects two extreme scenarios, and the degree of adaptivity increases as k increases.

Example 2.1 (Treatment assignment) As a concrete example, consider the problem of treatment assignment to patients. At round i , we observe the health profile of the patient i , which we denote by x i ∈ R d -1 . Our job to assign a treatment A i ∈ { 0 , 1 } based on the patient's health profile x i and also our prior knowledge of effectiveness of the treatments. It is natural to capture our prior knowledge using F i = σ ( A 1 , x 1 , y 1 , . . . , A i -1 , x i -1 , y i -1 ) -the sigma field generated by previous data-points. As already pointed out in (2) , in the adaptive regime the estimator error for treatment effect scales as √ d/n ; in words, we have to pay for a dimension factor √ d even if we are only interested in estimating a one-dimensional component. While for our treatment assignment example, the dimension d -1 of the covariate vector x i is large in practice, it is natural to assume that the treatment assignment is dependent on k -1 ≪ d -1 , a few (unknown) components. Under this assumption, it is easy to see that this treatment assignment problem is ( k, d ) -adaptive. We show that the treatment effect can be estimated at a rate √ k/n ≪ √ d/n .

2.1 Statistical limits

Before we discuss how to obtain estimators for a low-dimensional parameter component of θ ⋆ , we establish some baselines by recalling existing lower bounds. Throughout this section, we assume the noise ϵ i iid ∼N (0 , σ 2 ) . We start with defining the metric for comparison.

Definition 2.2 (scaled mean squared error (scaled-MSE)) Given a subset I ⊆ [ d ] . We define the scaled-MSE of an estimator ̂ θ I for θ ∗ I ∈ R |I| to be E [( ̂ θ I -̂ θ I ) ⊤ [( S -1 n ) II ] -1 ( ̂ θ I -̂ θ I )] , where S n = ∑ n i =1 x i x ⊤ i is the sample Gram matrix.

Roughly speaking, when the covariates x i are all fixed, the scaled-MSE compares the performance of ̂ θ I against the estimator with minimal variance (OLS). Moreover, we have the following result:

Proposition 2.2 (A simplified version of Theorem 2 in Khamaru et al. [20])

(a). Given a set I ⊆ [ d ] . Suppose the data { ( x i , y i ) } i =1 n are i.i.d. ( (0 , d ) -adaptive) from model (3) . Then the scaled-MSE satisfies

Furthermore, the equality holds when choosing ̂ θ I to be the OLS estimator for θ ∗ I .

(b). Suppose the data points { ( x i , y i ) } n i =1 are allowed to be arbitrarily adaptive ( ( d, d ) -adaptive). For any ( n, d ) with d ≥ 2 and n ≥ c · d 3 , and any non-empty set I ∈ [ d ] , there exists a data collection algorithm such that

where c, c ′ > 0 are some universal constants.

Proposition 2.2 exhibits the striking difference between two extreme data collection mechanisms. While the scaled-MSE scales as O ( |I| ) when data are i.i.d., the scaled-MSE for even a single coordinate (e.g., setting I = { 1 } ) can be of the order O ( d ) if the data are allowed to be collected arbitrarily adaptively.

Let I c = [ d ] \ I . By the matrix inverse formula, we have

where P X I c denotes the projection onto the column space of X I c ∈ R n ×|I| . In this work, we are often interested in the special cases where I = [ k ] (or { ℓ } for ℓ ∈ [ k ] ), which denote (a single coordinate of) the adaptive component.

2.2 Related work

Adaptive linear model In the early works by Lai et al. [24, 23], the authors studied regression models when the data are adaptively collected. They established the asymptotic normality of OLS under a stability assumption on the covariate matrix. However, the stability assumption might be violated under various setting, including data collected from online bandit algorithms such as UCB [27, 2, 31, 34, 42], forecasting and autoregressive models [14, 38, 24]. Recent works [13, 20] addressed this issue and proposed debiasing estimators with inferential guarantee in linear models with fixed dimension. While allowing for arbitrarily adaptively collected data, their results impose an additional √ d factor in the error bound for single coordinate estimation, limiting their applicability in linear models with increasing dimensions [19, 28].

Parameter estimation in bandit algorithms Though with the primary goal being achieving a low regret, the problem of parameter estimation under adaptive data are also studied when designing online bandit algorithms [1, 26, 29, 27]. Many online bandit algorithms are built based on the estimation or construction of adaptive confidence sets for the reward function [2, 9, 17], which can be viewed as finite sample estimation and inference of the unknown parameter of a model. Most related to our paper, in linear bandits, the works [1, 26] derived non-asymptotic upper bound on scaled-MSE of OLS for the whole parameter vector, as well as for a single coordinate. However, the upper bound on scaled-MSE for estimating a single coordinate is inflated by a factor of d compared with the i.i.d. case, as suggested by the lower bound in [20].

Inference using adaptively collected data The problem of estimation and inference using adaptively collected data has also been studied under other settings. The work by Hadad et al. [15] and Zhan et al. [40] proposed a weighted augmented inverse propensity weighted (AIPW, [32]) estimator for treatment effect estimation that is asymptotic normal. Zhang et al. [43] analyzed a weighted M -estimator for contextual bandit problems. Lin et al. [30] proposed a weighted Z -estimator for statistical inference in semi-parametric models. While investigating more intricate models, these works are built on the strong assumption that the adaptive data collection mechanism is known. In contrast, our ( k, d ) -adaptivity assumption allows the adaptive component to be collected in an arbitrary adaptive way.

√

Semi-parametric statistics A central problem in semi-parametric statistics is to derive n -consistent and asymptotic normal estimators of a low-dimensional parameter component in highdimensional or semi-parametric models [5, 6, 33, 36, 4, 8, 41]. Most works in this literature assume i.i.d. data collection and aim to obtain estimators that achieve the optimal asymptotic variance (i.e., semi-parametric efficient [35, 16]). On the other hand, our work focuses on a complementary perspective, with the goal of understanding how the data collection assumption affects the statistical limit of low-dimensional parameter estimation.

2.3 Notations

In the paper, we use the bold font to denote vectors and matrices (e.g., x , x , X , θ , ε ), and the regular font to denote scalars (e.g., x, θ, ε ). Given data { ( x i , y i ) } n i =1 from the linear model (3) that are ( k, d ) -adaptive, we use x ad i ∈ R k , x nad i ∈ R d -k to denote the adaptive and non-adaptive covariates. We also write θ ∗ = ( θ ∗⊤ ad , θ ∗⊤ nad ) ⊤ to denote the components that correspond to the adaptive and non-adaptive covariates. Let X = ( x ⊤ 1 , . . . , x ⊤ n ) ⊤ ∈ R n × d be the covariate matrix, with X ad (or X nad ) representing the submatrices consisting of the adaptive (or non-adaptive) columns. We use x j to denote the j -th column of the covariate matrix.

For a matrix M with n rows, let M -j be the matrix obtained by deleting the j -th column of M . We define the projection operator P M := M ( M ⊤ M ) -1 M ⊤ and the (columnwise) centered matrix ˜ M := ( I n -P 1 n ) M , where I ∈ R n × n is the identity matrix and 1 n ∈ R n is the all-one vector. For a symmetric M ⪰ 0 , we define ∥ x ∥ M := √ x ⊤ M x . Lastly, we use c, c ′ , c ′′ > 0 to denote universal constants and C, C ′ , C ′′ > 0 to denote constants that may depend on the problem specific parameters but not on k, d, n . We allow the values of the constants to vary from place to place.

3 Main results

This section is devoted to our main results on low-dimensional estimation and inference. In Section 3.1 and 3.2 we discuss the problem of estimating a low-dimensional component of θ ⋆ , and Section 3.3 is devoted to inference of low-dimensional components.

3.1 Low-dimensional estimation

Suppose the collected data { ( x i , y i ) } n i =1 are ( k, d ) -adaptive. In this section, we are interested in estimating the adaptive parameter component θ ∗ ad ∈ R k .

In addition to ( k, d ) -adaptivity, we introduce the following assumptions on the collected data { ( x i , y i ) } n i =1 .

Assumption A

(A1) There exists a constant U x > 0 such that

(A2) The non-adaptive components { x nad i } n i =1 are i.i.d. sub-Gaussian vectors with parameter ν > 0 , that is, for any unit direction u ∈ S d -1 ,

(A3) There exist some constants 0 ≤ σ min ≤ σ max such that the covariance matrix of the non-adaptive component Σ := Cov[ x nad i ] satisfies,

(A4) Conditioned on ( x i , F i -1 ) , the noise variable ε i in (3) is zero mean sub-Gaussian with parameter v > 0 , i.e.,

and has conditional variance σ 2 = E [ ε 2 | x i , F i -1 ] for all i ∈ [ n ] .

Let us clarify the meaning of the above assumptions. Assumption (A1) is the regularity assumption on the adaptive component. Roughly speaking, we allow the adaptive component to be arbitrarily adaptive as long as X ⊤ ad X ad is not close to be singular and x ad i has bounded ℓ 2 -norm. This is weaker than the assumptions made in [15, 43, 30], which assume that the conditional distribution of X ad is known. Assumption (A2), (A3) on the non-adaptive component, assume its distribution is non-singular and light-tailed. Assumption (A4) is a standard assumption that characterizes the tail behavior of the zero-mean noise variable. We remark that the equal conditional variance assumption in Assumption (A4) is mainly required in Theorem 3.4, while it is sufficient to assume σ 2 being a uniform upper bound of the conditional variance in Theorem 3.1 and 3.2.

3.1.1 Warm up: zero-mean non-adaptive component

We start with discussing a special case where the non-adaptive component x nad i is zero-mean. In this case, we prove that the Ordinary Least Squares (OLS) estimator on ( X , y ) for θ ∗ ad is near-optimal; see the discussion in Section 2.1. Denote the OLS estimator by ̂ θ = ( ̂ θ ⊤ ad , ̂ θ ⊤ nad ) ⊤ . Throughout, we assume that the sample size n and dimension d satisfies the relation

where C is an independent of ( n, d ) but may depend on other problem specific parameters. With this set up, our first theorem states

Theorem 3.1 Given data points { ( x i , y i ) } n i =1 from a ( k, d ) -adaptive model, and tolerance level δ ∈ (0 , 1 / 2) . Let, assumption (A1)-(A4) and the bound (6) in force, and the non-adaptive component x nad i is drawn from a zero-mean distribution. Then, we have

with probability at least 1 -δ .

See Appendix A.3 for a detailed proof. A few comments regarding Theorem 3.1 are in order. One might integrate both sides of the last bound to get a bound on the scaled-MSE. Comparing the bound (7b) with the lower bound from Proposition 2.2, we see this bound is tight in a minimax sense, up to some logarithmic factors.

It is now worthwhile to compare this bound with the existing best upper bounds in the literature. Invoking the concentration bounds from [26, Lemma 16] one have that

One might argue that the first inequality is loose as we only want to estimate a low-dimensional component θ ∗ ad ∈ R k . However, invoking the lower bound from Proposition 2.2, we see that the bound (8) is the best you can hope for if we do not utilize the ( k, d ) -adaptivity structure present in the data. See also the scaled-MSE bound for a single coordinate estimation in [26, Theorem 8] which also has a dimension dependence in the scaled-MSE bound.

3.1.2 Nonzero-mean non-adaptive component

In practice, the assumption that the non-adaptive covariates are drawn i.i.d. from a distribution P with zero mean is unsatisfactory. One would like to have a similar result where the distribution P has an unknown non-zero mean.

Algorithm 1 Centered OLS for k adaptive coordinates ( X , y )

2: ˜ X ad = X ad -1 n ˆ µ ⊤ ad , ˜ X nad = X nad -1 n ˆ µ ⊤ nad

3: Run OLS on centered response vector y -y · 1 n and centered covariate matrix ˜ X = ( ˜ X ad , ˜ X nad ) ∈ R n × d ; obtain the estimator ˜ θ = ( ˜ θ ⊤ ad , ˜ θ ⊤ nad ) ⊤ .

Before we state our estimator for the nonzero-mean case, it is helpful to understand the proof intuition of Theorem 3.1. A simple expansion yields

We show that the interaction term X ⊤ ad P X nad is small compared to the other terms under ( k, d ) -adaptivity and zero-mean property of X nad . In particular, under zero-mean property, each entry of

the matrix X ⊤ ad X nad is a martingale difference sequence, and can be controlled via concentration inequalities [1]. This martingale property is not true when the columns of X nad have a nonzero mean.

As a remedy, we consider the mean-centered linear model:

where ˜ X ad = X ad -1 n 1 n ⊤ n X ad , and ˜ X nad = X nad -1 n 1 n ⊤ n X nad are centered version of the matrices X ad and X nad , respectively. The centering in (9) ensures that X nad is approximately zero-mean, but unfortunately, it breaks the martingale structure present in the data. For instance, the elements of ˜ X ⊤ ad ˜ X nad are not a sum of martingale difference sequence because we have subtracted the column mean from each entry. Nonetheless, it turns out that subtracting the sample mean, while breaks the martingale difference structure, does not break it in an adversarial way, and we can still control the entries of ˜ X ⊤ ad ˜ X nad . See Lemma A.1 part (b) for one of the key ingredient in the proof. We point out that this finding is not new. Results of this form are well understood in various forms in sequential prediction literature, albeit in a different context. Such results can be found in earlier works of Lai, Wei and Robbins [21, 22, 24] and also in the later works by several authors [11, 12, 10, 1] and the references therein.

Our following Theorem 3.2 ensures that the intuition developed in this section so far is useful to characterize the performance of the solution obtained from the centered OLS.

Theorem 3.2 Given data points { ( x i , y i ) } n i =1 from a ( k, d ) -adaptive model, and tolerance level δ ∈ (0 , 1 / 2) . Let, assumption (A1)-(A4) and the bound (6) be in force. Then, ˜ θ ad obtained from Algorithm 1, satisfies

with probability at least 1 -δ .

See Appendix A.4 for a proof. Note that the variance of ˜ θ ad is given by ˜ X ⊤ ad ( I n -P ˜ X nad ) ˜ X ad . The covariance matrix is the same as X ⊤ ad ( I n -P X nad ) X ad when the all one vector 1 n belongs to the column space of ˜ X nad .

3.2 Single coordinate estimation: Application to treatment assignment

Let us now come back to Example 2.1 that we started. Let, at every round i , the treatment assignment A i depends on k -1 (unknown) coordinate of the covariates x i ∈ R d -1 . We assume that the covariates x ′ i s are drawn i.i.d. from some unknown distribution P . Assuming the response is related to the treatment and covariates via a linear model, it is not hard to see that this problem satisfies a ( k, d ) -adaptivity property. The following corollary provides a bound on the estimation error of estimating the (homogeneous) treatment effect.

75

50

25

0

Dimension of adaptive covariates

0

100

200

Dimension of adaptive covariates

0

25

50

75

Scaled MSE

COLS

0

100

200

OLS

Scaled MSE

Corollary 3.3 Suppose the assumptions from Theorem 3.2 are in force, and ℓ ∈ [ k ] be an index corresponding to one of the adaptive coordinates. Then, the ℓ th coordinate of the the centered OLS estimator from Algorithm 1 satisfies

The bounds above hold with probability at least 1 -δ , and ˜ X -ℓ denote the matrix obtained by removing the ℓ th column from ˜ X .

See Appendix A.5 for a proof of this Corollary. We verify the result of Corollary 3.3 via simulations as shown in Figure 1. From the figure we see that the scaled MSE increases linearly as the number of adaptive covariates increases, matching with our theoretical predictions. See Appendix B for more details about the simulation.

3.3 Inference with one adaptive arm

In this section, we provide a method for constructing valid confidence intervals for θ ∗ ad . To simplify the problem, we restrict our attention to the case of (1 , d ) -adaptivity and E [ x nad i ] = 0 .

A two-stage estimator

Our goal is to derive an asymptotically normal estimator for a target parameter in presence of a nuisance component. We call our estimator a 'Two-stage-adaptive-linear-estimating-equation' based estimator, or TALE-estimator for short. We start with a prior estimate ̂ θ Pr nad of θ ∗ nad , and define our estimate ̂ θ TALE for θ ∗ ad as a solution of this

̸

Recall that ̂ θ TALE is a scalar and the equation has a unique solution as long as ∑ 1 ≤ i ≤ n w ad i x i = 0 .

The weights { w i } i ≤ n in equation (11) are a set of predictable random scalars (i.e. w i ∈ σ ( x ad i , F i -1 ) ). Specifically, we start with s 0 > 0 and s 0 ∈ F 0 , and define

Let us first gain some intuitions on why TALE works. By rewriting equation (11), we have

Following the proof in [39], we have v n d -→ N (0 , σ 2 ) . Besides, one can show that with a proper choice of prior estimator ̂ θ Pr nad , the bias term b n converges to zero in probability as n goes to infinity. It is important to note that [39] considers the linear regression model where the number of covariates is fixed, and the sample size goes to infinity. In this work, however, we are interested in a setting where the number of covariates can grow with the number of samples. Therefore, our approach, TALE-estimator , has distinctions with the ALEE estimator proposed in [39]. The above intuition is formalized in the following theorem. Below, we use the shorthand ̂ θ OLS nad to denote the coordinates of the least squares estimate of ̂ θ OLS corresponding to the non-adaptive components.

Theorem 3.4 Suppose 1 /s 0 + s 0 /s n = o p (1) , n/ (log 2 ( n ) · d 2 ) →∞ , and assumptions (A1)-(A4) are in force. Then, the estimate ̂ θ TALE, obtained using weights from (12a) and ̂ θ Pr nad = ̂ θ OLS nad , satisfies

where ̂ σ is any consistent estimate of σ . Moreover, the asymptotic variance ̂ θ TALE is optimal up to logarithmic-factors.

See Appendix A.6 for a proof of this theorem. The assumption 1 /s 0 + s 0 /s n = o p (1) in the theorem essentially requires s 0 grows to infinity in a rate slower than s n . Therefore, in order to construct valid confidence intervals for ̂ θ TALE, one has to grasp some prior knowledge about the lower bound of s n . In our experiments in Section 4 we set s 0 = log log( n ) . Finally, it is also worth mentioning that one can apply martingale concentration inequalities (e.g. [1]) to control the terms b n and v n in equation (13), which in turn yields the finite sample bounds for ̂ θ TALE estimator. Finally, a consistent estimator of σ can be found using [24, Lemma 3].

4 Numerical experiments

In this section, we investigate the performance of TALE empirically, and compare it with the ordinary least squares (OLS) estimator, W-decorrelation proposed by Deshpande et al. [13], and the nonasymptotic confidence intervals derived from Theorem 8 in Lattimore et al. [26]. Our simulation set up entails the motivating Example 2.1 of treatment assignment. In our experiments, at stage i , the treatments A i ∈ { 0 , 1 } are assigned on the sign of ̂ θ ( i ) 1 , where ̂ θ ( i ) = ( ̂ θ ( i ) 1 , ̂ θ ( i ) 2 , . . . , ̂ θ ( i ) d ) is the least square estimate based on all data up to the time point i -1 ; here, the first coordinate of ̂ θ ( i ) 1 is associated with treatment assignment. The detailed data generation mechanism can be found in Appendix. From Figure 2 (top) we see that both TALE and W-decorrelation have valid empirical coverage (i.e., they are close to or above the baseline), while the nonasymptotic confidence intervals are overall conservative and the OLS is downwardly biased. In addition, TALE has confidence intervals that are shorter than those of W-decorrelation, which indicates a better estimation performance. Similar observations occur in the high-dimensional model in Figure 2 (bottom), where we find that both the OLS estimator and W-decorrelation are downwardly biased while TALE has valid coverage.

0.7

0.8

0.9

Target coverage probability (1

)

0.7

0.8

0.9

1.0

Empirical coverage probability

Upper tail coverage for

*

1

Baseline

TALE

OLS

W-decorr

Concen

0.7

0.8

0.9

Target coverage probability (1

)

0.7

0.8

0.9

1.0

Empirical coverage probability

Lower tail coverage for

*

1

Baseline

TALE

OLS

W-decorr

Concen

0.7

0.8

0.9

Target coverage probability (1

)

0.05

0.10

0.15

0.20

0.25

Width of two-sided CI for

*

1

TALE

OLS

W-decorr

Concen

0.7

0.8

0.9

Target coverage probability (1

)

0.6

0.7

0.8

0.9

1.0

Empirical coverage probability

Upper tail coverage for

*

1

Baseline

TALE

OLS

W-decorr

Concen

0.7

0.8

0.9

Target coverage probability (1

)

0.7

0.8

0.9

1.0

Empirical coverage probability

Lower tail coverage for

*

1

Baseline

TALE

OLS

W-decorr

Concen

0.7

0.8

0.9

Target coverage probability (1

)

0.050

0.075

0.100

0.125

0.150

Width of two-sided CI for

*

1

TALE

OLS

W-decorr

4

2

0

2

4

(

1

*

)/

Var(

1)

0.0

0.2

0.4

0.6

Density

Histogram of the scaled errors for

*

1

TALE

OLS

4

2

0

2

4

(

1

*

)/

Var(

1)

0.0

0.2

0.4

0.6

Density

Histogram of the scaled errors for

*

1

TALE

OLS

1

1

5 Discussion

In this paper, we investigate the statistical limits of estimating a low-dimensional component in a high dimensional adaptive linear model. We start by recalling a recent lower bound [20], which states that we need to pay for the underlying dimension d even if we want to estimate a low (one)-dimensional component. Our main result is to show that in order to estimate a low-dimensional component, we need to pay only for the degree of adaptivity k , which can potentially be much smaller than the underlying dimension d . Additionally, we propose a two-stage estimator for the one-dimensional target component, which is asymptotically normal. Finally, we demonstrate the effectiveness of this two-stage estimator via numerical simulations. For the future work, there are several avenues for further exploration that can contribute to a more comprehensive understanding of adaptive regression models. First of all, it would be interesting to generalize the ( k, d ) -adaptivity for the case when the number of adaptive components may vary between samples. It is also interesting to investigate if the current assumptions can be relaxed or not. For statistical inference part, it would be interesting to extend the TALE estimator to the case when the number of adaptive components is great than one.

Acknowledgments

This work was partially supported by the National Science Foundation Grants DMS-2311304, CCF1934924, DMS-2052949 and DMS-2210850.

References

Yasin Abbasi-yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems , volume 24. Curran Associates, Inc., 2011.

Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research , 3(Nov):397-422, 2002.

Zhidong Bai and Jack W Silverstein. Spectral analysis of large dimensional random matrices , volume 20. Springer, 2010.

Alexandre Belloni, Victor Chernozhukov, and Ying Wei. Post-selection inference for generalized linear models with many controls. Journal of Business & Economic Statistics , 34(4):606-619, 2016.

Peter J Bickel. On adaptive estimation. The Annals of Statistics , pages 647-671, 1982.

Peter J Bickel, Chris AJ Klaassen, Peter J Bickel, Ya'acov Ritov, J Klaassen, Jon A Wellner, and Ritov. Efficient and adaptive estimation for semiparametric models , volume 4. Springer, 1993.

George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control . John Wiley & Sons, 2015.

Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal , 21(1):C1-C68, 2018.

Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages 208-214. JMLR Workshop and Conference Proceedings, 2011.

Varsha Dani, Thomas P. Hayes, and Sham M. Kakade. Stochastic linear optimization under bandit feedback. In Annual Conference Computational Learning Theory , 2008.

Victor H. de la Peña, Michael J. Klass, and Tze Leung Lai. Self-normalized processes: exponential inequalities, moment bounds and iterated logarithm laws. The Annals of Probability , 32(3):1902 - 1933, 2004.

Victor H de la Pena, Michael J Klass, and Tze Leung Lai. Theory and applications of multivariate self-normalized processes. Stochastic Processes and their Applications , 119(12):4210-4227, 2009.

Yash Deshpande, Lester Mackey, Vasilis Syrgkanis, and Matt Taddy. Accurate inference for adaptive linear models. In International Conference on Machine Learning , pages 1194-1203. PMLR, 2018.

David A. Dickey and Wayne A. Fuller. Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association , 74(366):427-431, 1979.

Vitor Hadad, David A Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey. Confidence intervals for policy evaluation in adaptive experiments. Proceedings of the National Academy of Sciences , 118(15):e2014602118, 2021.

Jinyong Hahn. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica , 66(2):315-331, 1998.

Botao Hao, Tor Lattimore, and Mengdi Wang. High-dimensional sparse linear bandits. Advances in Neural Information Processing Systems , 33:10753-10763, 2020.

Roger A Horn and Charles R Johnson. Matrix analysis . Cambridge university press, 2012.

Adel Javanmard and Andrea Montanari. Confidence intervals and hypothesis testing for highdimensional regression. The Journal of Machine Learning Research , 15(1):2869-2909, 2014.

Koulik Khamaru, Yash Deshpande, Lester Mackey, and Martin J Wainwright. Near-optimal inference in adaptive linear regression. arXiv preprint arXiv:2107.02266 , 2021.

T_ L_ Lai and Herbert Robbins. Strong consistency of least-squares estimates in regression models. Proceedings of the National Academy of Sciences , 74(7):2667-2669, 1977.

T L Lai, Herbert Robbins, and C Zi Wei. Strong consistency of least squares estimates in multiple regression ii. Journal of multivariate analysis , 9(3):343-361, 1979.

Tze Leung Lai. Asymptotic properties of nonlinear least squares estimates in stochastic regression models. The Annals of Statistics , pages 1917-1930, 1994.

Tze Leung Lai and Ching Zong Wei. Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. The Annals of Statistics , 10(1):154-166, 1982.

Tor Lattimore. A lower bound for linear and kernel regression with adaptive covariates. In The Thirty Sixth Annual Conference on Learning Theory , pages 2095-2113. PMLR, 2023.

Tor Lattimore and Csaba Szepesvari. The end of optimism? an asymptotic analysis of finitearmed linear bandits. In Artificial Intelligence and Statistics , pages 728-737. PMLR, 2017.

Tor Lattimore and Csaba Szepesvári. Bandit algorithms . Cambridge University Press, 2020.

Lihua Lei, Peter J Bickel, and Noureddine El Karoui. Asymptotics for high dimensional regression m-estimates: fixed design results. Probability Theory and Related Fields , 172:9831079, 2018.

Lihong Li, Yu Lu, and Dengyong Zhou. Provably optimal algorithms for generalized linear contextual bandits. In International Conference on Machine Learning , pages 2071-2080. PMLR, 2017.

Licong Lin, Koulik Khamaru, and Martin J Wainwright. Semi-parametric inference based on adaptively collected data. arXiv preprint arXiv:2303.02534 , 2023.

Xinkun Nie, Xiaoying Tian, Jonathan Taylor, and James Zou. Why adaptively collected data have negative bias and how to correct for it. Advances in Neural Information Processing Systems , 84:1261-1269, 2018.

James M Robins. Correcting for non-compliance in randomized trials using structural nested mean models. Communications in Statistics-Theory and methods , 23(8):2379-2412, 1994.

Peter M Robinson. Root-n-consistent semiparametric regression. Econometrica: Journal of the Econometric Society , pages 931-954, 1988.

Jaehyeok Shin, Aaditya Ramdas, and Alessandro Rinaldo. On the bias, risk, and consistency of sample means in multi-armed bandits. SIAM Journal on Mathematics of Data Science , 3(4):1278-1300, 2021.

Anastasios A Tsiatis. Semiparametric theory and missing data. 2006.

Mark J Van der Laan, Sherri Rose, et al. Targeted learning: causal inference for observational and experimental data , volume 10. Springer, 2011.

Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge University Press, 2019.

John S. White. The limiting distribution of the serial correlation coefficient in the explosive case ii. The Annals of Mathematical Statistics , 30(3):831-834, 1959.

Mufang Ying, Koulik Khamaru, and Cun-Hui Zhang. Adaptive linear estimating equations. arXiv preprint arXiv:2307.07320 , 2023.

Ruohan Zhan, Vitor Hadad, David A Hirshberg, and Susan Athey. Off-policy evaluation via adaptive weighting with data from contextual bandits. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pages 2125-2135, 2021.

Cun-Hui Zhang and Stephanie S Zhang. Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology , 76(1):217-242, 2014.

Kelly Zhang, Lucas Janson, and Susan Murphy. Inference for batched bandits. Advances in Neural Information Processing Systems , 33:9818-9829, 2020.

Kelly Zhang, Lucas Janson, and Susan Murphy. Statistical inference with m-estimators on adaptively collected data. Advances in Neural Information Processing Systems , 34:7460-7471, 2021.

Appendix

Table of Contents

A Proofs

Throughout the proof, we use c, c ′ , c ′′ > 0 to denote constants universal constants. We use C, C ′ , C ′′ > 0 to denote constants that may depend on the problem specific parameters. Concretely, we use them to denote constants that only depends polynomially on (1 , σ max , 1 /σ min , v, ν, U x ) . We allows the values of the constants to vary from place to place.

A.1 Auxiliary lemmas

Before stating our main theorems, we list some useful lemmas, which can be of independent interest. All the proofs of lemmas can be found in Appendix A.

Lemma A.1 Given n ≥ d ≥ 1 . Let { ( a i , b i ) } n i =1 be a sequence of pairs such that a i ∈ R d are F i -1 -measurable and b i ∈ R are F i -measurable w.r.t. some filtration {F i } n i =1 . Assume in addition that b i are zero-mean sub-Gaussian random variables with parameter σ conditioned on F i -1 , i.e.,

Let A = [ a 1 , . . . , a n ] ⊤ and b = [ b 1 , . . . , b n ] ⊤ . Suppose that 1 ≤ σ min ( A ⊤ A ) ≤ σ max ( A ⊤ A ) ≤ nB for some constant B > 0 .

(a). (A simplified version of Theorem 1 in Abbasi et al. [1].) With probability over 1 -δ

for some universal constant c > 0 .

(b). Let ˜ A := A -P 1 n A be the centered matrix, then with probability over 1 -δ

for some universal constant c > 0 .

In the proofs we choose ( A , b ) = ( X nad , ε ) , ( X ad , ε ) , ( X ad , x j ) for j ∈ [ k + 1 , d ] . It is readily verified that conditions in Lemma A.1 are satisfied under assumptions in Theorem 3.1 and 3.2. See the proof of this lemma in Section A.7.

Lemma A.2 Suppose the data set { ( x i , y i ) } n i =1 is ( k, d ) -adaptive and satisfies Assumption (A1)(A4). Suppose the sample size condition (6) is in force. Adopt the notations in Section 3.1.2. Define X nad = X nad -E [ X nad ] and recall that ˜ X nad = ( I n -P 1 n ) X nad . For the matrices X nad , ˜ X nad , we have the following with probability over 1 -δ

for some parameter-dependent constants C > 0 . Moreover, equation (14a) , (14b) also hold when replacing X nad with the zero-mean matrix X nad defined in Section 3.1.1.

See the proof in Section A.8.

Lemma A.3 Under assumptions in Theorem 3.1, with probability over 1 -δ

for some parameter-dependent constant C > 0 .

See the proof in Section A.9.

Lemma A.4 Under assumptions in Theorem 3.2, with probability over 1 -δ

for some parameter-dependent constant C > 0 .

See the proof in Section A.10.

A.2 Proof of Proposition 2.2

Part (a). Proposition 2.2(a) is a standard result on the minimax optimality of the OLS estimator in linear models. A proof of this result using a Bayes argument can be found in the proof of Theorem 2(a) in Khamaru et al. [20].

By properties of the OLS estimator, we have ( ̂ θ ols -θ ∗ ) ∼ N (0 , σ 2 ( S n ) -1 ) conditioned on X . Therefore ( ̂ θ ols , I -θ ∗ I ) ∼ N (0 , σ 2 ( S -1 n ) II ) conditioned on X , where ( S -1 n ) II denotes the submatrix of S -1 n that consists of the coordinates in I . It follows immediately that

for any θ ∗ ∈ R d . As a result, the OLS estimator attains the minimax lower bound when the data are i.i.d.

Part (b). When |I| = 1 , part (b) follows immediately from Theorem 2(b) of Khamaru et al. [20] with v chosen to be the one-hot vector supported on I . When |I| > 1 , w.l.o.g. assume I consists of the first |I| coordinates of [ d ] . It suffices to show the scaled-MSE for I is always no less than the scaled-MSE for the first coordinate.

This follows from properties of Schur complement and the projection operator,

where P ⊥ M denote the projection onto the orthogonal space of the column space of M and

Using the properties of projection operator and linear space decomposition, it can be verified that

Therefore ∥ ∥ ∥ ̂ θ I -θ ∗ I ∥ ∥ ∥ 2 [( S -1 n ) II ] -1 ≥ ∥ ∥ ∥ ̂ θ 1 -θ ∗ 1 ∥ ∥ ∥ 2 [( S -1 n ) 11 ] -1 . This completes the proof.

A.3 Proof of Theorem 3.1

By the definition of the OLS estimator, we have

Applying the block matrix inverse formula, we obtain

To simplify notation, we define

and let

Therefore

We claim the following results which we prove later. With probability over 1 -δ

Taking these claims as given, we establish

where equation (16) uses claim (15a) and equation (17) uses claim (15b), (15c). Putting the last two displays together completes the proof.

Proof of claim (15a) The first and third inequality follows from the definition of R 1 and R 1 . The second inequality follows from Lemma A.3 and noting that C ( d -k ) k log( n/δ ) n < 1 / 2 under the sample size assumption (6) with C in (6) chosen sufficiently large.

Proof of claim (15b) Define ˜ P A := ( A ⊤ A ) -1 / 2 A ⊤ for any A ∈ R n × d . Then we have ∥ ˜ P A b ∥ 2 = ∥ P A b ∥ 2 for any b ∈ R n . Note that

Applying equation (14a) and Lemma A.1 in the last display, we continue

where the fourth line uses Lemma A.1 and equation (14a), the last line follows from the sample size assumption (6).

Proof of claim (15c) This is a direct consequence of Lemma A.1 since ε i are conditionally zeromean sub-Gaussian by Assumption (A4).

A.4 Proof of Theorem 3.2

Let µ ∗ = ( µ ∗⊤ ad , µ ∗⊤ nad ) ⊤ denote the mean vector of E [ x i ] and define

We write X nad = [ x k +1 . . . x d ] . The proof of this theorem follows the same basic steps as the proof of Theorem 3.1.

Recall that ̂ θ ad ∈ R k denotes the non-adaptive component of the centered OLS estimator. By definition and the matrix inverse formula, we have

To simplify notation, we introduce

and

Consequently,

Again, we claim the following results which we prove later. With probability over 1 -δ

With the claims at hand, we obtain

where equation (19) uses claim (18a) and equation (20) uses claim (18b), (18c). Combining the last two displays concludes the proof.

Proof of claim (18a) The first and third inequality follows from the definition of ˜ R 3 and R 3 . For the second inequality, we have

where the last line uses Lemma A.2, A.4 and noting that C ( d -k ) k log( n/δ ) n < 1 / 4 under our sample size assumption (6) with C in (6) chosen sufficiently large.

Proof of claim (18b) Recall that we define ˜ P A := ( A ⊤ A ) -1 / 2 A ⊤ for any A ∈ R n × d . Note that

where

We next bound W i ( i = 1 , 2 , 3 , 4) respectively.

For W 1 and W 2 , we have from Lemma A.1 and equation (14f) that

For W 3 , similar to the proof of claim (15b), we have

where third line follows from Lemma A.1 and the last line follows from the sample size assumption (6). Likewise,

where the third inequality follows from Lemma A.1 again. Putting pieces together yields the desired result.

Proof of claim (18c) This is a direct consequence of Lemma A.1.

A.5 Proof of Corollary 3.3

W.l.o.g. assume ℓ = 1 . By Schur decomposition, we have

Therefore by Lemma A.4, the sample size assumption (6), and Theorem 3.2, we establish

Corollary 3.3 follows immediately from the fact that I n -P ˜ X -1 ⪯ I n -P ˜ X ad , -1 since ˜ X ad , -1 consists of some columns of ˜ X -1 .

A.6 Proof of Theorem 3.4

We start with the following decomposition

To prove the theorem, it suffices to show

where d -→ stands for convergence in distribution and α is a constant that is specified in equation (21).

Proof of v n d -→ N (0 , α · σ 2 ) : The proof of this part directly follows from [39, Theorem 3.1]. For completeness, we provide a proof here. Note that function f is a positive decreasing function and satisfies properties

Furthermore, it can be shown that

Next we compute

In equation ( i ) , we consider the mean value theorem where ∫ s i /s 0 s i -1 /s 0 f 2 ( x ) dx = f 2 ( ξ i /s 0 )( x ad i ) 2 /s 0 . Consequently, we have

We conclude that

By noticing max 1 ≤ i ≤ n w 2 i = max 1 ≤ i ≤ n f 2 ( s i /s 0 )( x ad i ) 2 /s 2 0 = o p (1) , we conclude from martingale central limit theorem that

Moreover, applying Slutsky's theorem yields

Proof of b n p -→ 0 : To simplify notations, let w = ( w 1 , . . . , w n ) ⊤ . Without loss of generality, we consider the first column of the design matrix is collected adaptively. By the definition of b n , we observe that

Analysis of b n, 1 : By the construction of the weights { w i } 1 ≤ i ≤ n , we have

Applying Lemma A.1 with A = w and b = x nad i , we conclude that with probability at least 1 -δ ,

where c is a universal constant. Therefore, with probability at least 1 -δ ,

Analysis of b n, 2 : note ̂ θ Pr nad = ̂ θ ols nad is the OLS estimate. Therefore, we can use block-wise matrix inverse formula to get its expression. Precisely, we have

Therefore, we can upper bound b n, 2 by

To analyze terms b (1) n, 2 , b (2) n, 2 , and b (3) n, 2 , we make use of the results from Lemma A.1, Lemma A.2, and Lemma A.3. Specifically, we have with probability 1 -δ , the following statements hold

where c ′ , c ′′ and c ′′′ are universal constants that are independent of n and d . In inequatity ( i ) , we use Lemma A.3 to obtain a lower bound for ∥ x 1 -P X nad x 1 ∥ 2 2 and apply Lemma A.1 and Lemma A.2 to control the other three terms separately. Inequality ( ii ) makes use of the fact that x ij ε i is sub-exponential with parameter ( cνv, cνv ) conditioned on F i -1 for j = 2 , . . . , d . Therefore, by Azuma-Bernstein inequality and the sample size assumption, we obtain for 2 ≤ j ≤ d ,

with probability over 1 -δ/d for any 2 ≤ j ≤ d . Applying a union bound to j = 2 , . . . , d , we have

for some constant C ′ . Inequality ( iii ) makes use of the bound for b (2) n, 2 and Lemmas A.1 and A.2. Therefore, when d 2 log 2 ( n ) /n → 0 , we conclude

With b n p -→ 0 at hand, a direct application of Slutsky's theorem yields

Putting things together, we conclude that

A.7 Proof of Lemma A.1

Proof of Part (a) The proof follows immediately from choosing V = I d in Theorem 1 of Abbasi et al. [1].

Proof of Part (b) Define S 1 := ˜ A ⊤ ˜ A and A aug := [ A 1 n ] . Then

for all k +1 ≤ j ≤ d with probability over 1 -δ . Here the first equality comes from the definition of P ; the second equality uses the fact that the coefficients of A in the ordinary least squares (OLS) estimator for the linear model b ∼ A + 1 equals the OLS estimator for the centered linear model b ∼ ˜ A , i.e.,

the third line is due to the fact that [ S 1 0 k 0 ⊤ k 0 ] ⪯ A ⊤ aug A aug ; the fifth line follows from Lemma A.1 and the last line exploits the Schur complement of ˜ A ⊤ ˜ A .

A.8 Proof of Lemma A.2

Let µ ∗ = ( µ ∗⊤ ad , µ ∗⊤ nad ) ⊤ denote the mean vector of E [ x i ] and define

Proof of (14a) and (14b) . Applying the concentration result for a sample covariance matrix of sub-Gaussian ensemble (see e.g., Theorem 6.5 in Wainwright [37]), we obtain

with probability over 1 -δ . Using Weyl's theorem (see e.g., Theorem 4.3.1 in Horn et al. [18]), the sample size assumption and the last display, we find that

Similarly, we have | | | X nad | | | op = ( σ max ( X ⊤ nad X nad )) 1 / 2 ≤ √ n ( σ max + σ min / 2) ≤ √ 2 nσ max . This concludes (14a), (14b).

Proof of (14c) . Recall that we use ˆ µ = ( ˆ µ ∗⊤ ad , ˆ µ ∗⊤ nad ) ⊤ to denote the empirical average of { x i } n i =1 . From Assumption (A2) and properties of sub-Gaussian vectors, we have with probability over 1 -δ

It follows immediately that

The second inequality in (14c) follows from equation (14a) and the sample size assumption 6.

Proof of equation (14d) and (14e) . By Woodbury's matrix identity, we have

By equation (14a), (14b), (14c) and a standard triangular inequality, we find

where the second inequality follows from the sample size assumption (6). Therefore, we have

and hence

This gives equation (14d). Moreover, note that

It follows immediately from equation (14a), (14b), (14c), (14d) and (37) that with probability over 1 -δ

This yields equation (14e).

Proof of equation (14f) . Define ∆ := ˆ µ -µ ∗ . We have

Since | 1 ⊤ n ε | ≤ cv √ n log(1 /δ ) with probability over 1 -δ by Assumption (A4) and concentration of sub-Gaussian variables, and

by the triangular inequality, it follows immediately from combining equation (14a), (14b), (14c), (14d), (36), (37) that

∥ ( P X nad -P ˜ X nad ) ε ∥ 2 ≤ C [ √ ( d -k ) log( n/δ ) log(1 /δ ) n + √ ( d -k ) 2 log 2 ( n/δ ) n ] ≤ C with probability over 1 -δ .

A.9 Proof of Lemma A.3

Denote ( X ⊤ ad X ad ) -1 / 2 X ⊤ ad P X nad X ad ( X ⊤ ad X ad ) -1 / 2 by B . By our construction and noting that | | | B | | | op ≤ tr( B ) , it suffices to show

with probability over 1 -δ for some C > 0 . Plugging in the definition of P X nad , we obtain

with probability over 1 -δ , where the third line follows from von Neumann's trace inequality (see e.g., Theorem A.15 in Bai et al. [3]), and the last line uses equation (14a). Write X ad = [ x 1 , x 2 , . . . , x k ] ∈ R n × k and X nad = [ x k +1 , x k +2 , . . . , x d ] ∈ R n × ( d -k ) . Following the calculation, we further have

It follows from Lemma A.1 that

for all k +1 ≤ j ≤ d with probability over 1 -δ . The desired result follows immediately from combining equation (40) with (38) and (39).

A.10 Proof of Lemma A.4

Following the same arguments as in the proof of Lemma A.3 with X ad replaced by ˜ X ad := X ad -P 1 n X ad , it suffices to show

for all k +1 ≤ j ≤ d with probability over 1 -δ . This follows immediately from Lemma A.1 (b).

B Simulation set up

We provide implementation details of our simulations in this section. The code is available at https://github.com/licong-lin/low-dim-debias .

B.1 Single coordinate estimation

In this section, we detail the simulation set up that was used to generate the Figure 1. The goal of this simulation is to display the effect of degree of adaptivity k on the estimation of a single coordinate, and provide empirical validation to the theory developed in the paper (c.f. Corollary 3.3).

We want to design an adaptive data collection mechanism, which can capture the estimation lower bound. Therefore, we adopt a similar data collection procedure as provided in Khamaru et al. [20]. We also refer readers to Lattimore [25] for related information.

Simulation set-up:

Sample size n = 1000 , d = 300 .

The degree of adaptivity k varies from 2 to 200 with step size equal to 3 .

Replication number 20 for each level of adaptivity ( k, d ) .

θ ad 1 = 1 and other coefficients are generated independently from N (0 , 1) .

x nad i is generated independently from uniform distribution on the sphere S d -k -1 . If x nad i has mean not equal to zero, then we consider x nad i plus E x nad i , where E x nad i is generated from N ( 1 , I d -k ) .

Data collection method: Here we modified the data collection algorithm from Section 5 . 2 . 2 in Khamaru et al. [20]. The only difference between our data collection algorithm and the one in Khamaru et al. [20] is that we replace m u,v := ∑ v w =1 b w ( y u,w -a u,w ) by

Figure 1 shows that empirical relation between the MSE of the centered OLS estimate of the first coordinate and the degree of dependence k .

B.2 Single coordinate inference

In this section, we detail the simulation set up that is used to generate the Figure 2 and 3.

We generate a dataset { ( x i , y i ) } n i =1 that satisfies the assumptions in Theorem 3.4. On this simulated dataset, we compare our method with the ordinary least squares (OLS) estimator, W-decorrelation proposed by Deshpande [13], and the non-asymptotic confidence intervals derived from Theorem 8 in Lattimore et al. [26].

We begin by describing our data generating mechanism. We assume the data { ( x i , y i ) } n i =1 ∈ R d × R are generated from a linear model y i = x ⊤ i θ ∗ + ε i , where ε i iid ∼N (0 , σ 2 ) . We generate the covariates { x i } n i =1 in the following way

We assume the non-adaptive component x i, 2: d are i.i.d N (0 , I d -1 ) across i ∈ [ n ] .

For the adaptive coordinate, we choose x 1 , 1 = 1 and assume x i, 1 ∈ { 0 , 1 } for all i ∈ [ n ] .

At each stage i ≥ 2 , denote by ̂ θ ( i ) 1 the OLS estimator for the first coordinate θ ∗ 1 obtained using the first i -1 samples ( X 1: i -1 , y 1: i -1 ) . With probability p we choose x i, 1 = 1 if ̂ θ ( i ) 1 > 0 and x i, 1 = 0 if otherwise; with probability 1 -p we simply choose x i, 1 = 1 to encourage exploration.

Recalling Example 2.1 on treatment assignment, in the simulated data, we use the OLS estimator to obtain an prior estimate of the treatment effect θ ∗ 1 and assign the treatment to the i -th patient if the prior estimation suggests that the treatment has a positive effect ( i.e., ̂ θ ( i ) 1 > 0) . Moreover, to encourage exploration, we assign the treatment (i.e., x i, 1 = 1 ) with some small probability 1 -p , regardless of the prior estimation.

Throughout the simulation we choose θ ∗ 2: d = 1 d -1 / √ d -1 and θ ∗ 1 = 0 , which corresponds to the case where no treatment effect is presented. We choose the noise level σ = 0 . 3 and the probability p = 0 . 8 . In the simulations we assume the noise level is known for simplicity. We run our simulations on both a low-dimensional model ( n = 1000 , d = 10) and a high-dimensional model ( n = 500 , d = 50) .

Comparison with W-decorrelation by Deshpande et al. [13] In our implementation of Wdecorrelation, we follow Algorithm 1 in [13], with the parameter λ · log( n ) be the 1 /n -quantile of σ min ( X ⊤ X ) . To estimate the quantile, we use the sample estimate from 1000 i.i.d. data matrices X 's to estimate the quantile.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the statistical limits of some adaptive linear models.\n\nThe paper defines the notion of $(k,d)$-adaptivity (Definition 2.1), and then it proves, under some conditions and for a $(k,d)$-adaptive model and failure probability $\\delta$, that\n- (Theorem 3.1) the estimation error of the adaptive components is bounded by $k\\log(n/\\delta)$ if the non-adaptive component has zero mean,\n- (Theorem 3.2) and similar results hold if the mean of the non-adaptive component is not zero;\n- (Theorem 3.4) moreover, there exists an estimator (TALE) that enjoys asymptotic normality.\n\nThe experiments present an interesting phenomenon: The TALE estimator coincides well with the normal distribution.\n\nSTRENGTHS:\nThe paper is written clearly and the presentation is smooth and relatively easy to follow.\n\nWEAKNESSES:\nI have a major concern about whether the paper is technically solid and I wish to read the response from the authors:\n- Regarding Theorems 3.1 and 3.2, could the authors justify how the proof techniques differ from prior works? It appears to me that the two results are basic extensions of prior proofs (so, correct me if I am wrong).\n- Theorem 3.4 holds only for a single adaptive coordinate. Could the authors elaborate on the difficulty of extending the results for general $(k,d)$-adaptivity?\n\n\nMinor:\n- the same symbol $\\sigma$ is used to represent sigma field, variance of a random variable, and singular values. In my humble opinion, this might be confusing.\n\n\n**NOTE**: I am not an expert on statistics in general and on this particular line of research, so it is not for me to say whether the paper is significant or novel.\n\nQUESTIONS:\n- From the paper it is a bit hard for me to understand what \"adaptivity\" precisely means. The definition of $(k,d)$-adaptivity only specifies that the adaptive components $x_i^{ad}$ depend on the sigma field $F_{i-1}$. Would it be much simpler and clearer to say \"dependency\" instead of \"adaptivity\"?\n\nLIMITATIONS:\nThe authors discussed future works and limitations at the end of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn \"Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference\" the authors consider the problem of estimating a low-dimensional signal in a high-dimensional linear model where data collection is allowed to be adaptive. The notion of adaptivity employed in this paper restricts itself to k components, meaning that the k first (wlog) covariates are allowed to be adaptive while the remaining are assumed i.i.d. This is in contrast to prior work where all components are allowed to be adaptive which yields very detrimental minimax lower bounds even in the case where only one component is to be estimated. Based on this new notion of adaptivity, the authors propose a scheme to estimate the low-dimensional signal yielding beneficial scaling guarantees, both in mean square error and asymptotic normality.\n\nSTRENGTHS:\n1. Originality: the authors study a novel scenario and provide a novel scheme with reasonable guarantees. The work has sufficient novelty.\n2. Quality: the paper is technically sound and the results are appropriately stated.\n3. Clarity: The main results of the paper and some intuition on how they are established are clear. The example provided could be a bit clearer. I expand on this in the Questions section.\n4. Significance: This may be to my own ignorance but the significance of the considered model is not entirely clear to me. I again will expand on this in the Questions section.\n\nWEAKNESSES:\nI merge this with the Questions section.\n\nQUESTIONS:\n1. Regarding Example 2.1.: Based on how the filtration is defined and the dimensionality of the quantities one can infer that A_i is considered to be the first coordinate of the covariates in the linear model. I don't see the loss of generality in instead defining x_i to include the treatment assignment as x_1i. This makes the mapping to the linear model more obvious.\n2. Regarding significance of the model. The underlying assumption is that variable selection is not necessary in this case, i.e. the low-dimensional coordinates to be estimated are known prior, and they are the only ones that are affected by adaptive covariates while the remaining are i.i.d.. Can the authors expand on the treatment assignment example to provide with a situation in which this would be the case?\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the issue of adaptive data collection in a linear regression model. To summarize the main idea, let us focus on the leading example in the paper (that is, Example 2.1, treatment assignment). In this example, a patient is treated based on effectiveness of the previous treatments as well as a small number of covariates. When the treatment is assigned adaptively in an unknown way, the treatment effect can be estimated via OLS only at a rate $\\sqrt{d/n}$, where $d$ is the dimension of the entire covariate vector (not the dimension of the covariates used for treatment assignment) and $n$ is the sample size. This result is shown by previous work in Khamaru et al. [21] and a simplified version is given as Proposition 2.2 in the paper. One of the main results in the paper is that the centered OLS estimator of the treatment effect attains a better rate of convergence, that is, $\\sqrt{k/n}$, where $k-1$ is the dimension of the covariates used for treatment assignment. The inference problem is further studied for the case that $k=1$. That is, the treatment assignment mechanism does not depend on the covariates but on the effectiveness of the previous treatments. The paper proposes an adaptive estimator called Two-stage Adaptive Linear Estimating Equation (TALE) Estimator. The adaptive weights are constructed in a particular fashion to develop asymptotic normality (see Theorem 3.4). Numerical experiments show potential usefulness of the proposed TALE estimator.\n\nSTRENGTHS:\n- This paper considers a highly important problem in the literature: adaptive data collection (e.g., bandits) is increasingly important in a number of fields. \n- The paper clarifies the important open question in the literature, that is, \"Can we obtain a good estimator for a low-dimensional parameter component in linear models when the degree of adaptivity is given?\".\n- The proposed TALE estimator has desirable theoretical properties and shows promising numerical results.\n\nWEAKNESSES:\n- The non-adaptive component $x_i^{\\mathrm{nad}}$ is assumed to be independent of the adaptive component $x_i^{\\mathrm{ad}}$ (see lines 84-85). This seems quite strong in the sense that if this is the case, we could just drop the non-adaptive component $x_i^{\\mathrm{nad}}$ in the regression model and then we automatically obtain the $\\sqrt{k/n}$ rate, provided that the variance of the new regression error, which now includes the omitted part $\\theta^\\top x_i^{\\mathrm{nad}}$, is bounded by a constant that is independent of $d$. Using the scenario in Example 2.1 with $k=1$ (that is, the treatment assignment mechanism depends only on the effectiveness of the previous treatments), it might be preferable to consider the difference-in-means estimator (that is, to include only the intercept term and a treatment indicator) instead of estimating the treatment effect via regression adjustment. It would be useful to carefully discuss the issue of independence between the adaptive and nonadaptive components.\n\nQUESTIONS:\n- Line 191: it seems that the conditional variance $\\sigma^2$ is a constant, meaning that it does not depend on $(x_i, \\mathcal{F}_{i-1})$. This is a restrictive assumption and could be commented in line 198.\n- The centered OLS is a proposed solution in the paper. I am wondering whether this estimator is the same as one that includes the intercept term. In other words, since it is conventional to use the intercept term in regression models, I am curious whether the standard practice already solves the research question raised in the paper (without fully realizing the importance of including the constant term in the regression model). \n- The centered OLS algorithm on page 6 is not fully implementable in an online fashion. This is because computation of the sample means requires access to the full dataset. It might be useful to add some remarks regarding how to carry out online estimation for the centered OLS.\n- Lots of notations are used before section 2.3. It might be better to move the notations section to improve readability of the paper.\n- The TALE estimator is highly related to the concurrent work [2] entitled \"Adaptive Linear Estimating Equations\". It would be useful to clarify the differences between this work and the current paper.\n\nLIMITATIONS:\nThe numerical results are promising but there is no theoretical result in the paper that implies that the TALE estimator should perform strictly better than W-decorrelation. It might be helpful to fully discuss what numerical results are predicted by asymptotic theory and what are not.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper introduces a new data collection assumption that captures the partially adaptive data and then derives a bound for scaled MSE of order $k\log n$, where $k$ is the number of entries that are collected adaptively.  Finally they also introduce a novel estimator for single coordinate inference which has  an asymptotic normality property.

STRENGTHS:
The paper is clearly structured and well-written. The theorems seem solid, and the assumptions are general. The authors also provide a concrete example of $\textit{treatment assignment}$ to showcase the power of their theorems.

WEAKNESSES:
1. The numerical section is focused on the performance of TALE, missing out the numerical verifications for Theorem 3.1 and 3.2, which are the main theorems of the paper. 

2. There still seem to be some fundamental limitations for the definition of $(k,d)$-adaptivity. For example, $(k,d)$-adaptivity requires a fixed number of entries in covariates $x$ are adaptively collected, which ignore the important scenario when such the number and indices of such entries may vary. 

3. While the example of $\textit{treatment assignment}$  is very helpful, no comparison has been made against the state-of-the-art statistical tools and no numerical experiments are provided.

4. Typo in line 106.

QUESTIONS:
It is not very clear to me what is the technical difficulty to generalize the original bound involving $d$ (Lemma 16 of Lattimore and Szepesvari) to the bound of order $k$, and how the proof in this paper solves it.

LIMITATIONS:
See 2. in Weakness section.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

comments: SUMMARY:
As I have communicated with the area chair, I will not be reviewing due to a conflict of interest. Submitting default ratings intended to be ignored below.

STRENGTHS:
NA

WEAKNESSES:
NA

QUESTIONS:
NA

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates degree of adaptivity in data impacts the performance of estimating a low-dimensional parameter component in high-dimensional linear models.  The main result is giving an error bound of a low-dimensional component that does not have diension dependence. They propose an estimator TALE. For a special case when there is a single adaptive coordinate and non-adaptive components have zero mean, this estimator is asymptotically normal.\n\nThe manuscript is easy to read through and the technical content of the paper appears to be correct albeit some typos:\n\nLine 69: Adaptive\nLine 478 in the appendix: $X_{\\mathrm{ad}}$\n\nSTRENGTHS:\nTheoretical results:\n- The authors define (k,d)-adaptivity to quantify the level of adaptivity along with a concrete example. Using this idea, they can get the upper bound depending on $k$ in stead of full dimension $d$. These results could bridge the gap between iid and arbitrarily adaptive data collection.\n\nExperiments:\n- Compared to other methods like OLS, estimation errors for TALE are in good accordance with a norm distribution.\n\nWEAKNESSES:\nTheoretical results:\n- The main result in this paper demonstrates the advantages of utilizing the $(k,d)$-adaptivity structure, yielding a scaled-MSE bound in the order of $k\\log(n)$ instead of $d\\log(n). While this is a positive outcome, it's worth considering that the result is based on a different norm, which makes it less convincing.\n\nExperiments:\n- In the introduction,  the paper aims to obtain an estimator with performance dependent on the degree of adaptivity. Besides, the main results highlight that having $k$ in the upper bound. Therefore, I believe it would be better for the authors to experiment with different levels of adaptivity.\n\nQUESTIONS:\nNumerical experiments:\n- The authors highlight that TALE exhibits shorter confidence intervals (CIs) compared to W-decorrelation, suggesting better estimation performance. However, it's noteworthy that OLS achieves much shorter CIs than TALE, especially in the higher-dimensional scenario where $d=50$. CI of TALE is about 50% longer than that of OLS. It would be helpful if the authors address this observation to provide a comprehensive evaluation of the methods. Also, given that, how could the authors conclude that TALE outperforms OLS in terms of estimation performance? \n- From Figure 2, we can see that OLS is indeed downwardly biased. However, we can also observe that the magnitude of errors might be the same. The main results show a tighter bound (from $d=50$ to $k=1\"), but the practical benefit is not evident. Additional analysis or insights would help demonstrate its significance.\n\nLIMITATIONS:\nIn addition to the limitations mentioned earlier, the authors noted that they did not provide asymptotic normality guarantees when the adaptive component has more than one dimension.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-22"}