SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \left\| \widehat {\boldsymbol {\theta}} _ {\mathrm {a d}} - \boldsymbol {\theta} _ {\mathrm {a d}} ^ {*} \right\| _ {\mathbf {X} _ {\mathrm {a d}} ^ {\top} \left(\mathbf {I} _ {n} - \mathbf {P} _ {\mathbf {X} _ {\mathrm {n a d}}}\right) \mathbf {X} _ {\mathrm {a d}}} ^ {2} \leq C ^ {\prime} \log \left(n \det  \left(\mathbf {X} _ {\mathrm {a d}} ^ {\top} \mathbf {X} _ {\mathrm {a d}}\right) / \delta\right) (7a) \\ \leq C ^ {\prime \prime} k \log (n / \delta). (7b) \\ \end{array}
$$
See Appendix A.3 for a detailed proof. A few comments regarding Theorem 3.1 are in order. One might integrate both sides of the last bound to get a bound on the scaled-MSE. Comparing the bound (7b) with the lower bound from Proposition 2.2, we see this bound is tight in a minimax sense, up to some logarithmic factors.
It is now worthwhile to compare this bound with the existing best upper bounds in the literature. Invoking the concentration bounds from [26, Lemma 16] one have that
$$
\left\| \widehat {\boldsymbol {\theta}} _ {\mathrm {a d}} - \boldsymbol {\theta} _ {\mathrm {a d}} ^ {*} \right\| _ {\mathbf {X} _ {\mathrm {a d}} ^ {\top} \left(\mathbf {I} _ {n} - \mathbf {P} _ {\mathbf {X} _ {\mathrm {n a d}}}\right) \mathbf {X} _ {\mathrm {a d}}} ^ {2} \leq \left\| \widehat {\boldsymbol {\theta}} _ {\mathrm {a d}} - \boldsymbol {\theta} _ {\mathrm {a d}} ^ {*} \right\| _ {\mathbf {X} _ {\mathrm {a d}} ^ {\top} \mathbf {X} _ {\mathrm {a d}}} ^ {2} \leq c \cdot d \log (n / \delta) \tag {8}
$$
One might argue that the first inequality is loose as we only want to estimate a low-dimensional component  $\theta_{\mathrm{ad}}^{*} \in \mathbb{R}^{k}$ . However, invoking the lower bound from Proposition 2.2, we see that the bound (8) is the best you can hope for if we do not utilize the  $(k,d)$ -adaptivity structure present in the data. See also the scaled-MSE bound for a single coordinate estimation in [26, Theorem 8] which also has a dimension dependence in the scaled-MSE bound.
3.1.2 Nonzero-mean non-adaptive component
In practice, the assumption that the non-adaptive covariates are drawn i.i.d. from a distribution  $P$  with zero mean is unsatisfactory. One would like to have a similar result where the distribution  $P$  has an unknown non-zero mean.
Algorithm 1 Centered OLS for  $k$  adaptive coordinates  $(\mathbf{X},\mathbf{y})$
1:  $\hat{\pmb{\mu}}_{\mathrm{ad}}\gets \frac{\mathbf{X}_{\mathrm{ad}}^{\top}\mathbf{1}_{n}}{n},\hat{\pmb{\mu}}_{\mathrm{nad}}\gets \frac{\mathbf{X}_{\mathrm{nad}}^{\top}\mathbf{1}_{n}}{n}$
2:  $\tilde{\mathbf{X}}_{\mathrm{ad}} = \mathbf{X}_{\mathrm{ad}} - \mathbf{1}_n\hat{\boldsymbol{\mu}}_{\mathrm{ad}}^\top$ $\tilde{\mathbf{X}}_{\mathrm{nad}} = \mathbf{X}_{\mathrm{nad}} - \mathbf{1}_n\hat{\boldsymbol{\mu}}_{\mathrm{nad}}^\top$
3: Run OLS on centered response vector  $\mathbf{y} - \bar{\mathbf{y}}\cdot \mathbf{1}_n$  and centered covariate matrix  $\widetilde{\mathbf{X}} = (\widetilde{\mathbf{X}}_{\mathrm{ad}},\widetilde{\mathbf{X}}_{\mathrm{nad}})\in \mathbb{R}^{n\times d}$ ; obtain the estimator  $\widetilde{\pmb{\theta}} = (\widetilde{\pmb{\theta}}_{\mathrm{ad}}^{\top},\widetilde{\pmb{\theta}}_{\mathrm{nad}}^{\top})^{\top}$ .
Before we state our estimator for the nonzero-mean case, it is helpful to understand the proof intuition of Theorem 3.1. A simple expansion yields
$$
\begin{array}{l} \widehat {\boldsymbol {\theta}} _ {\mathrm {a d}} - \boldsymbol {\theta} _ {\mathrm {a d}} ^ {*} = \left(\mathbf {X} _ {\mathrm {a d}} ^ {\top} \mathbf {X} _ {\mathrm {a d}} - \mathbf {X} _ {\mathrm {a d}} ^ {\top} \mathbf {P} _ {\mathbf {X} _ {\mathrm {n a d}}} \mathbf {X} _ {\mathrm {a d}}\right) ^ {- 1} \left(\mathbf {X} _ {\mathrm {a d}} ^ {\top} \boldsymbol {\varepsilon} - \mathbf {X} _ {\mathrm {a d}} ^ {\top} \mathbf {P} _ {\mathbf {X} _ {\mathrm {n a d}}} \boldsymbol {\varepsilon}\right) \\ \approx \left(\mathbf {X} _ {\mathrm {a d}} ^ {\top} \mathbf {X} _ {\mathrm {a d}}\right) ^ {- 1} \mathbf {X} _ {\mathrm {a d}} ^ {\top} \boldsymbol {\varepsilon} + \text {s m a l l e r o r d e r t e r m s} \\ \end{array}
$$
We show that the interaction term  $\mathbf{X}_{\mathrm{ad}}^{\top}\mathbf{P}_{\mathbf{X}_{\mathrm{rad}}}$  is small compared to the other terms under  $(k,d)$ -adaptivity and zero-mean property of  $\mathbf{X}_{\mathrm{rad}}$ . In particular, under zero-mean property, each entry of
the matrix  $\mathbf{X}_{\mathrm{ad}}^{\top}\mathbf{X}_{\mathrm{nad}}$  is a martingale difference sequence, and can be controlled via concentration inequalities [1]. This martingale property is not true when the columns of  $\mathbf{X}_{\mathrm{nad}}$  have a nonzero mean.
As a remedy, we consider the mean-centered linear model:
$$
\mathbf {y} - \bar {y} \cdot \mathbf {1} _ {n} = \widetilde {\mathbf {X}} _ {\mathrm {a d}} ^ {\top} \boldsymbol {\theta} _ {\mathrm {a d}} ^ {*} + \widetilde {\mathbf {X}} _ {\mathrm {n a d}} ^ {\top} \boldsymbol {\theta} _ {\mathrm {n a d}} ^ {*} + (\epsilon - \bar {\epsilon} \cdot \mathbf {1} _ {n}) \tag {9}
$$
where  $\widetilde{\mathbf{X}}_{\mathrm{ad}} = \mathbf{X}_{\mathrm{ad}} - \frac{\mathbf{1}_n\mathbf{1}_n}{n}\mathbf{X}_{\mathrm{ad}}$ , and  $\widetilde{\mathbf{X}}_{\mathrm{nad}} = \mathbf{X}_{\mathrm{nad}} - \frac{\mathbf{1}_n\mathbf{1}_n}{n}\mathbf{X}_{\mathrm{nad}}$  are centered version of the matrices  $\mathbf{X}_{\mathrm{ad}}$  and  $\mathbf{X}_{\mathrm{nad}}$ , respectively. The centering in (9) ensures that  $\mathbf{X}_{\mathrm{nad}}$  is approximately zero-mean, but unfortunately, it breaks the martingale structure present in the data. For instance, the elements of  $\widetilde{\mathbf{X}}_{\mathrm{ad}}^{\top}\widetilde{\mathbf{X}}_{\mathrm{nad}}$  are not a sum of martingale difference sequence because we have subtracted the column mean from each entry. Nonetheless, it turns out that subtracting the sample mean, while breaks the martingale difference structure, does not break it in an adversarial way, and we can still control the entries of  $\widetilde{\mathbf{X}}_{\mathrm{ad}}^{\top}\widetilde{\mathbf{X}}_{\mathrm{nad}}$ . See Lemma A.1 part (b) for one of the key ingredient in the proof. We point out that this finding is not new. Results of this form are well understood in various forms in sequential prediction literature, albeit in a different context. Such results can be found in earlier works of Lai, Wei and Robbins [21, 22, 24] and also in the later works by several authors [11, 12, 10, 1] and the references therein.
Our following Theorem 3.2 ensures that the intuition developed in this section so far is useful to characterize the performance of the solution obtained from the centered OLS.
Theorem 3.2 Given data points  $\{(\pmb{x}_i, y_i)\}_{i=1}^n$  from a  $(k, d)$ -adaptive model, and tolerance level  $\delta \in (0, 1/2)$ . Let, assumption (A1)-(A4) and the bound (6) be in force. Then,  $\widetilde{\theta}_{\mathrm{ad}}$  obtained from Algorithm 1, satisfies
$$
\begin{array}{l} \left\| \widetilde {\boldsymbol {\theta}} _ {\mathrm {a d}} - \boldsymbol {\theta} _ {\mathrm {a d}} ^ {*} \right\| _ {\widetilde {\mathbf {X}} _ {\mathrm {a d}} ^ {\top} \left(\mathbf {I} _ {n} - \mathbf {P} _ {\widetilde {\mathbf {X}} _ {\mathrm {n a d}}}\right) \widetilde {\mathbf {X}} _ {\mathrm {a d}}} ^ {2} \leq C ^ {\prime} \log \left(n \det  \left(\widetilde {\mathbf {X}} _ {\mathrm {a d}} ^ {\top} \widetilde {\mathbf {X}} _ {\mathrm {a d}}\right) / \delta\right) \tag {10} \\ \leq C ^ {\prime \prime} k \log (n / \delta). \\ \end{array}
$$
See Appendix A.4 for a proof. Note that the variance of  $\widetilde{\pmb{\theta}}_{\mathrm{ad}}$  is given by  $\widetilde{\mathbf{X}}_{\mathrm{ad}}^{\top}(\mathbf{I}_n - \mathbf{P}_{\widetilde{\mathbf{X}}_{\mathrm{rad}}})\widetilde{\mathbf{X}}_{\mathrm{ad}}$ . The covariance matrix is the same as  $\mathbf{X}_{\mathrm{ad}}^{\top}(\mathbf{I}_n - \mathbf{P}_{\mathbf{X}_{\mathrm{rad}}})\mathbf{X}_{\mathrm{ad}}$  when the all one vector  $\mathbf{1}_n$  belongs to the column space of  $\widetilde{\mathbf{X}}_{\mathrm{rad}}$ .
3.2 Single coordinate estimation: Application to treatment assignment
Let us now come back to Example 2.1 that we started. Let, at every round  $i$ , the treatment assignment  $A_{i}$  depends on  $k - 1$  (unknown) coordinate of the covariates  $\pmb{x}_i \in \mathbb{R}^{d - 1}$ . We assume that the covariates  $\pmb{x}_i^{\prime}s$  are drawn i.i.d. from some unknown distribution  $\mathcal{P}$ . Assuming the response is related to the treatment and covariates via a linear model, it is not hard to see that this problem satisfies a  $(k,d)$ -adaptivity property. The following corollary provides a bound on the estimation error of estimating the (homogeneous) treatment effect.
Figure 1: The plot depicts the empirical relation between the scaled MSE of the OLS and centered OLS estimate from Algorithm 1 and the number of adaptive covariates  $(k)$  for a carefully constructed problem. See Section B.1 for simulation details.
Corollary 3.3 Suppose the assumptions from Theorem 3.2 are in force, and  $\ell \in [k]$  be an index corresponding to one of the adaptive coordinates. Then, the  $\ell^{th}$  coordinate of the centered OLS estimator from Algorithm 1 satisfies
$$
| \widetilde {\boldsymbol {\theta}} _ {\mathrm {a d}, \ell} - \boldsymbol {\theta} _ {\mathrm {a d}, \ell} ^ {*} | \leq \frac {\sqrt {C \log (n \det (\widetilde {\mathbf {X}} _ {\mathrm {a d}} ^ {\top} \widetilde {\mathbf {X}} _ {\mathrm {a d}}) / \delta)}}{\sqrt {\widetilde {\mathbf {x}} _ {\ell} ^ {\top} (\mathbf {I} _ {n} - \mathbf {P} _ {\widetilde {\mathbf {X}} _ {- \ell}}) \widetilde {\mathbf {x}} _ {\ell}}} \leq \frac {C \sqrt {k \log (n / \delta)}}{\sqrt {\widetilde {\mathbf {x}} _ {\ell} ^ {\top} (\mathbf {I} _ {n} - \mathbf {P} _ {\widetilde {\mathbf {X}} _ {- \ell}}) \widetilde {\mathbf {x}} _ {\ell}}}.
$$
The bounds above hold with probability at least  $1 - \delta$ , and  $\widetilde{\mathbf{X}}_{-\ell}$  denote the matrix obtained by removing the  $\ell^{th}$  column from  $\widetilde{\mathbf{X}}$ .
See Appendix A.5 for a proof of this Corollary. We verify the result of Corollary 3.3 via simulations as shown in Figure 1. From the figure we see that the scaled MSE increases linearly as the number of adaptive covariates increases, matching with our theoretical predictions. See Appendix B for more details about the simulation.
3.3 Inference with one adaptive arm
In this section, we provide a method for constructing valid confidence intervals for  $\theta_{\mathrm{ad}}^{*}$ . To simplify the problem, we restrict our attention to the case of  $(1,d)$ -adaptivity and  $\mathbb{E}[\pmb{x}_i^{\mathrm{rad}}] = \mathbf{0}$ .
A two-stage estimator
Our goal is to derive an asymptotically normal estimator for a target parameter in presence of a nuisance component. We call our estimator a "Two-stage-adaptive-linear-estimating-equation" based estimator, or TALE-estimator for short. We start with a prior estimate  $\widehat{\pmb{\theta}}_{\mathrm{nad}}^{\mathrm{Pr}}$  of  $\pmb{\theta}_{\mathrm{nad}}^{*}$ , and define our estimate  $\widehat{\theta}_{\mathrm{TALE}}$  for  $\pmb{\theta}_{\mathrm{ad}}^{*}$  as a solution of this
$$
\text {T A L E - e s t i m a t o r :} \quad \sum_ {i = 1} ^ {n} w _ {i} \left(y _ {i} - x _ {i} ^ {\mathrm {a d}} \cdot \widehat {\theta} _ {\mathrm {T A L E}} - x _ {i} ^ {\mathrm {n a d} \top} \widehat {\boldsymbol {\theta}} _ {\mathrm {n a d}} ^ {\mathrm {P r}}\right) = 0. \tag {11}
$$
Recall that  $\widehat{\theta}_{\mathrm{TALE}}$  is a scalar and the equation has a unique solution as long as  $\sum_{1\leq i\leq n}w_i^{\mathrm{ad}}x_i\neq 0$
The weights  $\{w_i\}_{i \leq n}$  in equation (11) are a set of predictable random scalars (i.e.  $w_i \in \sigma(\boldsymbol{x}_i^{\mathrm{ad}}, \mathcal{F}_{i-1})$ ). Specifically, we start with  $s_0 > 0$  and  $s_0 \in \mathcal{F}_0$ , and define
$$
w _ {i} = \frac {f \left(s _ {i} / s _ {0}\right) x _ {i} ^ {\mathrm {a d}}}{\sqrt {s _ {0}}} \quad \text {w h e r e} \quad s _ {i} = s _ {0} + \sum_ {t \leq i} \left(x _ {t} ^ {\mathrm {a d}}\right) ^ {2} \text {a n d} \tag {12a}
$$
$$
f (x) = \frac {1}{\sqrt {x (\log e ^ {2} x) (\log \log e ^ {2} x) ^ {2}}} \quad \text {f o r} x > 1. \tag {12b}
$$
Let us first gain some intuitions on why TALE works. By rewriting equation (11), we have
$$
\sum_ {i = 1} ^ {n} w _ {i} x _ {i} ^ {\mathrm {a d}} \left(\widehat {\theta} _ {\mathrm {T A L E}} - \theta_ {\mathrm {a d}} ^ {*}\right) = \underbrace {\sum_ {i = 1} ^ {n} w _ {i} \epsilon_ {i}} _ {v _ {n}} + \underbrace {\sum_ {i = 1} ^ {n} w _ {i} x _ {i} ^ {\mathrm {n a d} \top} \left(\boldsymbol {\theta} _ {\mathrm {n a d}} ^ {*} - \widehat {\boldsymbol {\theta}} _ {\mathrm {n a d}} ^ {\mathrm {P r}}\right)} _ {b _ {n}}. \tag {13}
$$
Following the proof in [39], we have  $v_{n} \xrightarrow{d} \mathcal{N}(0, \sigma^{2})$ . Besides, one can show that with a proper choice of prior estimator  $\widehat{\pmb{\theta}}_{\mathrm{nad}}^{\mathrm{Pr}}$ , the bias term  $b_{n}$  converges to zero in probability as  $n$  goes to infinity. It is important to note that [39] considers the linear regression model where the number of covariates is fixed, and the sample size goes to infinity. In this work, however, we are interested in a setting where the number of covariates can grow with the number of samples. Therefore, our approach, TALE-estimator, has distinctions with the ALEE estimator proposed in [39]. The above intuition is formalized in the following theorem. Below, we use the shorthand  $\widehat{\pmb{\theta}}_{\mathrm{nad}}^{\mathrm{OLS}}$  to denote the coordinates of the least squares estimate of  $\widehat{\pmb{\theta}}^{\mathrm{OLS}}$  corresponding to the non-adaptive components.
Theorem 3.4 Suppose  $1 / s_0 + s_0 / s_n = o_p(1)$ ,  $n / (\log^2 (n)\cdot d^2)\to \infty$ , and assumptions (A1)-(A4) are in force. Then, the estimate  $\widehat{\theta}_{TALE}$ , obtained using weights from (12a) and  $\widehat{\pmb{\theta}}_{\mathrm{nad}}^{Pr} = \widehat{\pmb{\theta}}_{\mathrm{nad}}^{OLS}$ , satisfies
$$
\frac {1}{\widehat {\sigma} \sqrt {\sum_ {1 \leq i \leq n} w _ {i} ^ {2}}} \left(\sum_ {1 \leq i \leq n} w _ {i} x _ {i} ^ {\mathrm {a d}}\right) \cdot \left(\widehat {\theta} _ {T A L E} - \theta_ {\mathrm {a d}} ^ {*}\right) \xrightarrow {d} \mathcal {N} (0, 1),
$$
where  $\widehat{\sigma}$  is any consistent estimate of  $\sigma$ . Moreover, the asymptotic variance  $\widehat{\theta}_{\text{TALE}}$  is optimal up to logarithmic-factors.
See Appendix A.6 for a proof of this theorem. The assumption  $1 / s_0 + s_0 / s_n = o_p(1)$  in the theorem essentially requires  $s_0$  grows to infinity in a rate slower than  $s_n$ . Therefore, in order to construct valid confidence intervals for  $\widehat{\theta}_{\mathrm{TALE}}$ , one has to grasp some prior knowledge about the lower bound of  $s_n$ . In our experiments in Section 4 we set  $s_0 = \log \log (n)$ . Finally, it is also worth mentioning that one can apply martingale concentration inequalities (e.g. [1]) to control the terms  $b_{n}$  and  $v_{n}$  in equation (13), which in turn yields the finite sample bounds for  $\widehat{\theta}_{\mathrm{TALE}}$  estimator. Finally, a consistent estimator of  $\sigma$  can be found using [24, Lemma 3].
In this section, we investigate the performance of TALE empirically, and compare it with the ordinary least squares (OLS) estimator, W-decorrelation proposed by Deshpande et al. [13], and the non-asymptotic confidence intervals derived from Theorem 8 in Lattimore et al. [26]. Our simulation set up entails the motivating Example 2.1 of treatment assignment. In our experiments, at stage  $i$ , the treatments  $A_{i} \in \{0,1\}$  are assigned on the sign of  $\widehat{\theta}_{1}^{(i)}$ , where  $\widehat{\pmb{\theta}}^{(i)} = (\widehat{\theta}_{1}^{(i)}, \widehat{\theta}_{2}^{(i)}, \dots, \widehat{\theta}_{d}^{(i)})$  is the least square estimate based on all data up to the time point  $i-1$ ; here, the first coordinate of  $\widehat{\theta}_{1}^{(i)}$  is associated with treatment assignment. The detailed data generation mechanism can be found in Appendix. From Figure 2 (top) we see that both TALE and W-decorrelation have valid empirical coverage (i.e., they are close to or above the baseline), while the nonasymptotic confidence intervals are overall conservative and the OLS is downwardly biased. In addition, TALE has confidence intervals that are shorter than those of W-decorrelation, which indicates a better estimation performance. Similar observations occur in the high-dimensional model in Figure 2 (bottom), where we find that both the OLS estimator and W-decorrelation are downwardly biased while TALE has valid coverage.
Figure 2: Empirical coverage probability and the width of confidence intervals versus target coverage probability  $1 - \alpha$  for TALE, the OLS estimator, non-asymptotic concentration inequalities, and W-decorrelation. We select the noise level  $\sigma = 0.3$ . Top:  $n = 1000$ ,  $d = 10$ . Bottom:  $n = 500$ ,  $d = 50$ . At bottom right we do not display the result for concentration since the CIs are too wide. We run the simulation 1000 times and display the  $\pm 1$  standard deviation.
Figure 3: Histograms of the scaled errors for TALE and OLS. Left:  $n = 1000$ ,  $d = 10$ . Right:  $n = 500$ ,  $d = 50$ . We choose the noise level  $\sigma = 0.3$  and repeat the simulation 1000 times. Observe that the distribution of the OLS estimator is much more different than standard normal, and it exhibits a downwards bias [31], while TALE is in good accordance with a standard normal distribution.
In this paper, we investigate the statistical limits of estimating a low-dimensional component in a high dimensional adaptive linear model. We start by recalling a recent lower bound [20], which states that we need to pay for the underlying dimension  $d$  even if we want to estimate a low (one)-dimensional component. Our main result is to show that in order to estimate a low-dimensional component, we need to pay only for the degree of adaptivity  $k$ , which can potentially be much smaller than the underlying dimension  $d$ . Additionally, we propose a two-stage estimator for the one-dimensional target component, which is asymptotically normal. Finally, we demonstrate the effectiveness of this two-stage estimator via numerical simulations. For the future work, there are several avenues for further exploration that can contribute to a more comprehensive understanding of adaptive regression models. First of all, it would be interesting to generalize the  $(k,d)$ -adaptivity for the case when the number of adaptive components may vary between samples. It is also interesting to investigate if the current assumptions can be relaxed or not. For statistical inference part, it would be interesting to extend the TALE estimator to the case when the number of adaptive components is great than one.
[1] Yasin Abbasi-yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
[2] Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3(Nov):397-422, 2002.
[3] Zhidong Bai and Jack W Silverstein. Spectral analysis of large dimensional random matrices, volume 20. Springer, 2010.
[4] Alexandre Belloni, Victor Chernozhukov, and Ying Wei. Post-selection inference for generalized linear models with many controls. Journal of Business & Economic Statistics, 34(4):606-619, 2016.
[5] Peter J Bickel. On adaptive estimation. The Annals of Statistics, pages 647-671, 1982.
[6] Peter J Bickel, Chris AJ Klaassen, Peter J Bickel, Ya'acov Ritov, J Klaassen, Jon A Wellner, and Ritov. Efficient and adaptive estimation for semiparametric models, volume 4. Springer, 1993.
[7] George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
[8] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1-C68, 2018.
[9] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 208-214. JMLR Workshop and Conference Proceedings, 2011.
[10] Varsha Dani, Thomas P. Hayes, and Sham M. Kakade. Stochastic linear optimization under bandit feedback. In Annual Conference Computational Learning Theory, 2008.
[11] Victor H. de la Peña, Michael J. Klass, and Tze Leung Lai. Self-normalized processes: exponential inequalities, moment bounds and iterated logarithm laws. The Annals of Probability, 32(3):1902 - 1933, 2004.
[12] Victor H de la Pena, Michael J Klass, and Tze Leung Lai. Theory and applications of multivariate self-normalized processes. Stochastic Processes and their Applications, 119(12):4210-4227, 2009.
[13] Yash Deshpande, Lester Mackey, Vasilis Syrgkanis, and Matt Taddy. Accurate inference for adaptive linear models. In International Conference on Machine Learning, pages 1194-1203. PMLR, 2018.
[14] David A. Dickey and Wayne A. Fuller. Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74(366):427-431, 1979.
[15] Vitor Hadad, David A Hirshberg, Ruohan Zhan, Stefan Wager, and Susan Athey. Confidence intervals for policy evaluation in adaptive experiments. Proceedings of the National Academy of Sciences, 118(15):e2014602118, 2021.
[16] Jinyong Hahn. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica, 66(2):315-331, 1998.
[38] John S. White. The limiting distribution of the serial correlation coefficient in the explosive case ii. The Annals of Mathematical Statistics, 30(3):831-834, 1959.
[39] Mufang Ying, Koulik Khamaru, and Cun-Hui Zhang. Adaptive linear estimating equations. arXiv preprint arXiv:2307.07320, 2023.
[40] Ruohan Zhan, Vitor Hadad, David A Hirshberg, and Susan Athey. Off-policy evaluation via adaptive weighting with data from contextual bandits. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 2125-2135, 2021.
[41] Cun-Hui Zhang and Stephanie S Zhang. Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):217-242, 2014.
[42] Kelly Zhang, Lucas Janson, and Susan Murphy. Inference for batched bandits. Advances in Neural Information Processing Systems, 33:9818-9829, 2020.
[43] Kelly Zhang, Lucas Janson, and Susan Murphy. Statistical inference with m-estimators on adaptively collected data. Advances in Neural Information Processing Systems, 34:7460-7471, 2021.
A.1 Auxiliary lemmas 14
A.2 Proof of Proposition 2.2 15
A.3 Proof of Theorem 3.1 16
A.4 Proof of Theorem 3.2 17
A.5 Proof of Corollary 3.3 19
A.6 Proof of Theorem 3.4 20
A.7 Proof of Lemma A.1 22
A.8 Proof of Lemma A.2 23
A.9 Proof of Lemma A.3 25
A.10 Proof of Lemma A.4 25
B Simulation set up 25
B.1 Single coordinate estimation 25
B.2 Single coordinate inference 26
Throughout the proof, we use  $c, c', c'' > 0$  to denote constants universal constants. We use  $C, C', C'' > 0$  to denote constants that may depend on the problem specific parameters. Concretely, we use them to denote constants that only depends polynomially on  $(1, \sigma_{\max}, 1 / \sigma_{\min}, v, \nu, \mathrm{U}_{\mathrm{x}})$ . We allow the values of the constants to vary from place to place.
A.1 Auxiliary lemmas
Before stating our main theorems, we list some useful lemmas, which can be of independent interest. All the proofs of lemmas can be found in Appendix A.