SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left| \widehat {\theta} _ {\mathrm {A L E E}, k} - \theta_ {k} ^ {*} \right| \leq \sigma_ {g} \sqrt {\log (2 / \delta^ {2})} \frac {\sqrt {2 + \log \left(s _ {n , k} / s _ {0}\right)} \log \left\{2 + \log \left(s _ {n , k} / s _ {0}\right) \right\}}{\sqrt {s _ {n , k}} - \sqrt {s _ {0}}} \tag {23}
$$
provided  $s_0 > 1$ . See Section A.2 of the Appendix for a proof of this argument. Recall that  $\sqrt{s_{n,k}} = (s_0 + \sum_{i\leq n}x_{i,k}^2)^{1 / 2}$ , the bound is in the same spirit as existing finite sample bounds for the OLS estimator for arm means [1, 21]. In simple terms, the ALEE estimator behaves similarly to the OLS estimator in a non-asymptotic setting while still maintaining asymptotic normality.
3.2 Autoregressive time series
Next, we focus on an autoregressive time series model
$$
y _ {t} = \theta^ {*} y _ {t - 1} + \epsilon_ {t} \quad \text {f o r} t \in [ n ], \tag {24}
$$
where  $y_0 = 0$ . Note that the above model is a special case of the adaptive linear model (2). It is well-known that when  $\theta^* \in (-1, 1)$ , the time series model (24) satisfies a stability assumption (1). Consequently, one might use the OLS estimate based confidence intervals [19] for  $\theta^*$ . However, when  $\theta^* = 1$  — also known as the unit root case — stability condition (1) does not hold, and the
least squares estimator is not asymptotically normal [19]. In other words, when  $\theta^{*} = 1$ , the least squares based intervals do not provide correct coverage.
In this section, we apply ALEE-based approach to construct confidence intervals that are valid for  $\theta^{*}\in [-1,1]$ . Similar to previous sections, let  $s_0\in \mathcal{F}_0$  and denote  $s_t = s_0 + \sum_{1\leq i\leq t}y_{i - 1}^2$ . Following a construction similar to the last section, we have the following corollary.
Corollary 3.5. Assume the noise variables  $\{\epsilon_t\}_t$  are i.i.d with mean zero, variance  $\sigma^2$  and sub-Gaussian parameter  $\sigma_g^2$ . Then, for any  $\theta^* \in [-1,1]$ , the ALEE estimator, obtained using  $w_t = f(s_t / s_0) y_{t-1} / \sqrt{s_0}$  with function  $f$  from Corollary 3.2 and  $s_0 = n / \log \log(n)$ , satisfies
$$
\sqrt {\frac {4 \beta (\log 2) ^ {\beta}}{\log \left(s _ {n} / s _ {0}\right) \left\{\log \log \left(s _ {n} / s _ {0}\right) \right\} ^ {1 + \beta}}} \cdot \frac {\sqrt {s _ {n}} \left(\widehat {\theta} _ {\mathrm {A L E E}} - \theta^ {*}\right)}{\widehat {\sigma}} \xrightarrow {d} \mathcal {N} (0, 1). \tag {25}
$$
The proof of Corollary 3.5 can be found in Section A.3 of the Appendix.
3.3 Contextual bandits
In contextual bandit problems, the task of defining adaptive weights that satisfy the stability condition (6) while maintaining a large affinity is challenging. Without loss of generality, we assume that  $\| \pmb{x}_t\| _2\leq 1$ . Following the discussion around (11) and using  $\mathbf{S}_t$  as an approximation of  $\mathbf{S}_n$ , we see that a good choice for the weight is  $\pmb{w}_{t}\approx \mathbf{S}_{t}^{-\frac{1}{2}}\pmb{x}_{t}$ . However, it is not all clear at the moment why the above choice produces  $d$ -dimensional weights  $\pmb{w}_{t}$  satisfying the stability condition (6). It turns out that the success of our construction is based on the variability of certain matrix  $\mathbf{V}_t$ . For a  $\mathcal{F}_0$ -measurable  $d\times d$  symmetric matrix  $\pmb{\Sigma}_0\succeq \mathbf{I}_d$  and  $t\in [n]$ , we define
$$
\boldsymbol {\Sigma} _ {t} = \boldsymbol {\Sigma} _ {0} + \sum_ {i = 1} ^ {t} \boldsymbol {x} _ {i} \boldsymbol {x} _ {i} ^ {\top} \quad \text {a n d} \quad \boldsymbol {z} _ {t} = \boldsymbol {\Sigma} _ {t - 1} ^ {- \frac {1}{2}} \boldsymbol {x} _ {t}. \tag {26}
$$
For  $t\in [n]$  , we define the variability matrix  $\mathbf{V}_t$  as
$$
\mathbf {V} _ {t} = \left(\mathbf {I} _ {d} + \sum_ {i = 1} ^ {t} z _ {i} z _ {i} ^ {\top}\right) ^ {- 1} \quad (\text {V a r i a b i l i t y}). \tag {27}
$$
The variability matrix  $\mathbf{V}_t$  comes up frequently in finite sample analysis of the least squares estimator [16, 19, 2], the generalized linear models with adaptive data [23], and in online optimization [6]; see comments after Theorem 3.6 for a more detailed discussion on the matrix  $\mathbf{V}_t$ . Now, we define weights  $\{\pmb{w}_t\}_{t\geq 1}$  as
$$
\boldsymbol {w} _ {t} = \sqrt {1 + \boldsymbol {z} _ {t} ^ {\top} \mathbf {V} _ {t - 1} \boldsymbol {z} _ {t}} \cdot \mathbf {V} _ {t} \boldsymbol {z} _ {t}. \tag {28}
$$
Theorem 3.6. Suppose condition (3) holds and  $\| \pmb{\Sigma}_0^{-1}\|_{\mathrm{op}} + \| \mathbf{V}_n\|_{\mathrm{op}} = o_p(1)$ . Then, the ALEE estimator  $\widehat{\theta}_{\mathrm{ALEE}}$ , obtained using the weights  $\{\pmb{w}_t\}_{1\leq t\leq n}$  from (28), satisfies
$$
\left(\sum_ {t = 1} ^ {n} \boldsymbol {w} _ {t} \boldsymbol {x} _ {t}\right) \cdot \left(\widehat {\boldsymbol {\theta}} _ {\mathrm {A L E E}} - \boldsymbol {\theta} ^ {*}\right) \xrightarrow {d} \mathcal {N} \left(\mathbf {0}, \sigma^ {2} \mathbf {I} _ {d}\right).
$$
The proof of Theorem 3.6 can be found in Section A.4 of the Appendix. In Theorem B.4 in the appendix, we establish the asymptotic normality of a modified version of the ALEE estimator, which has the same asymptotic variance as the one in Theorem 3.6 under the assumption  $\| \pmb{\Sigma}_0^{-1}\|_{\mathrm{op}} = o_p(1)$ . In other words, the modified theorem B.4 does not assume any condition on the  $\| \mathbf{V}_n\|_{\mathrm{op}}$ .
To better convey the idea of our construction, we provide a lemma that may be of independent interest. This lemma applies to weights  $\boldsymbol{w}_t$  generated by (27) and (28) with general  $\boldsymbol{z}_t$ .
Lemma 3.7. Let  $\mathbf{w}_t$  be as in (28) with the variability matrix  $\mathbf{V}_t$  in (27). Then,
$$
\sum_ {t = 1} ^ {n} \boldsymbol {w} _ {t} \boldsymbol {w} _ {t} ^ {\top} = \mathbf {I} _ {d} - \mathbf {V} _ {n}, \quad \max  _ {1 \leq t \leq n} \| \boldsymbol {w} _ {t} \| _ {2} = \max  _ {1 \leq t \leq n} \| \mathbf {V} _ {t - 1} \boldsymbol {z} _ {t} \| _ {2} / (1 + \boldsymbol {z} _ {t} ^ {\top} \mathbf {V} _ {t - 1} \boldsymbol {z} _ {t}) ^ {1 / 2}. \tag {29}
$$
For  $\mathbf{z}_t \in \mathcal{F}_{t-1}$ , the stability condition (6) holds when  $\max_{1 \leq t \leq n} \mathbf{z}_t^\top \mathbf{V}_t \mathbf{z}_t + \| \mathbf{V}_n \|_{\mathrm{op}} = o_p(1)$ .
Proof. For any  $t \geq 1$ ,  $\mathbf{V}_t = \mathbf{V}_{t-1} - \mathbf{V}_{t-1} z_t z_t^\top \mathbf{V}_{t-1} / (1 + z_t^\top \mathbf{V}_{t-1} z_t)$ . It follows that  $\mathbf{V}_t \mathbf{z}_t = \mathbf{V}_{t-1} z_t / (1 + z_t^\top \mathbf{V}_{t-1} z_t)$  and  $\sum_{t=1}^n \mathbf{w}_t \mathbf{w}_t^\top = \sum_{t=1}^n \mathbf{V}_{t-1} (\mathbf{V}_t^{-1} - \mathbf{V}_{t-1}^{-1}) \mathbf{V}_t = \mathbf{I}_d - \mathbf{V}_n$ .
Comments on Theorem 3.6 conditions: It is instructive to compare the conditions of Theorem 3.1 and Theorem 3.6. The condition  $\| \pmb{\Sigma}_0^{-1}\|_{\mathrm{op}} = o_p(1)$  is an analogue of the condition  $1 / s_0 = o_p(1)$ . The condition  $\| \mathbf{V}_n\|_{\mathrm{op}} = o_p(1)$  is a bit more subtle. This condition is an analogue of the condition  $s_0 / s_{n,k} = o_p(1)$ . Indeed, applying elliptical potential lemma [2, Lemma 4] yields
$$
\frac {\log \left(\det \left(\boldsymbol {\Sigma} _ {0} + \mathbf {S} _ {n}\right)\right)}{\log \left(\det \left(\boldsymbol {\Sigma} _ {0}\right)\right)} \leq \operatorname {t r a c e} \left(\mathbf {V} _ {n} ^ {- 1}\right) - d = \sum_ {t = 1} ^ {n} \boldsymbol {x} _ {t} ^ {\top} \boldsymbol {\Sigma} _ {t - 1} ^ {- 1} \boldsymbol {x} _ {t} \leq 2 \cdot \frac {\log \left(\det \left(\boldsymbol {\Sigma} _ {0} + \mathbf {S} _ {n}\right)\right)}{\log \left(\det \left(\boldsymbol {\Sigma} _ {0}\right)\right)} \tag {30}
$$
where  $\mathbf{S}_n = \sum_{i=1}^n \boldsymbol{x}_i \boldsymbol{x}_i^\top$  is the Gram matrix. We see that for  $\| \mathbf{V}_n \|_{\mathrm{op}} = o_p(1)$ , it is necessary that the eigenvalues of  $\mathbf{S}_n$  grow to infinity at a faster rate than the eigenvalues of  $\boldsymbol{\Sigma}_0$ . Moreover, in the case of dimension  $d = 1$ , the condition  $\| \mathbf{V}_n \|_{\mathrm{op}} = o_p(1)$  is equivalent to  $s_0 / s_{n,k} = o_p(1)$ .
4 Numerical experiments
In this section, we consider three settings: two-armed bandit setting, first order auto-regressive model setting and contextual bandit setting. In two-armed bandit setting, the rewards are generated with same arm mean  $(\theta_1^*,\theta_2^*) = (0.3,0.3)$ , and noise is generated from a normal distribution with mean 0 and variance 1. To collect two-armed bandit data, we use  $\epsilon$ -Greedy algorithm with decaying exploration rate  $\sqrt{\log(t) / t}$ . The rate is designed to make sure the number of times each armed is pulled has order greater than  $\log(n)$  up to time  $n$ . In the second setting, we consider the time series model,
$$
y _ {t} = \theta^ {*} y _ {t - 1} + \epsilon_ {t}, \tag {31}
$$
where  $\theta^{*} = 1$  and noise  $\epsilon_{t}$  is drawn from a normal distribution with mean 0 and variance 1. In the contextual bandit setting, we consider the true parameter  $\theta^{*}$  to be 0.3 times the all-one vector. In the initial iterations, a random context  $x_{t}$  is generated from a uniform distribution in  $S^{d - 1}$ . Then, we apply  $\epsilon$ -Greedy algorithm to these pre-selected contexts with decaying exploration rate  $\log^2 (t) / t$ . For all of the above three settings, we run 1000 independent replications.
Figure 2: Two-armed bandit problem with equal arm mean  $\theta_1^* = \theta_2^* = 0.3$ . Error bars plotted are  $\pm$  standard errors.
To analyze the data we collect for these settings, we apply ALEE approach with weights specified in Corollary 3.2, 3.5 and Theorem B.4, respectively. More specifically, in the first two settings, we consider  $\beta = 1$  in Corollary 3.2. For two-armed bandit example, we set  $s_0 = e^2\log (n)$ , which is known to be a lower bound for  $s_{n,1}$ . For AR(1) model, we consider  $s_0 = e^2 n / \log \log (n)$ . For the contextual bandit example, we consider  $\Sigma_0 = \log (n)\cdot \mathbf{I}_d$ . In the simulations, we also compare ALEE approach to the normality based confidence interval for OLS estimator [19] (which may be incorrect), the concentration bounds for the OLS estimator based on self-normalized martingale sequence [1], and W-decorrelation [8]. Detailed implementations about these methods can be found in Appendix C.1.
In Figure 2, we display results for two-armed bandit example, providing the empirical coverage plots for the first arm mean  $\theta_{1}^{*}$  as well as average width for two-sided CIs. We observe that CIs based on
Figure 3: AR(1) with model coefficient  $\theta^{*} = 1$  and  $s_0 = e^2 n / \log \log (n)$ . Error bars plotted are  $\pm$  standard errors.
OLS undercover  $\theta_1^*$  while other methods provide satisfactory coverage. Notably, from the average CI width plot, we can see that W-decorrelation and concentration methods have relatively large CI widths. On the contrary, ALEE-based CIs achieve target coverage while keeping the width of CIs relatively small.
For AR(1) model, we display the results in Figure 3. For the context bandit example, we consider  $d = 20$  and summarize the empirical coverage probability and the logarithm of the volume of the confidence regions in Table 1, along with corresponding standard deviations. See Appendix C.2 for experiments with dimension  $d = 10$  and  $d = 50$ .
Table 1: Contextual bandit:  $\mathrm{d} = {20}$
Method	Level of confidence
0.8	0.85	0.9
Avg. Coverage	Avg. log(Volume)	Avg. Coverage	Avg. log(Volume)	Avg. Coverage	Avg. log(Volume)
ALEE	0.805 (± 0.396)	6.541 (± 0.528)	0.861 (± 0.346)	7.108 (± 0.528)	0.910 (± 0.286)	7.806 (± 0.528)
OLS	0.776 (± 0.417)	-2.079 (± 0.525)	0.830 (± 0.376)	-1.513 (± 0.525)	0.881 (± 0.324)	-0.815 (± 0.525)
W-Decorrelation	0.777 (± 0.416)	25.727 (± 0.518)	0.829 (± 0.377)	26.294 (± 0.518)	0.870 (± 0.336)	26.992 (± 0.518)
Concentration	1.000 (± 0.000)	17.374 (± 0.506)	1.000 (± 0.000)	17.408(± 0.506)	1.000 (± 0.000)	17.455 (± 0.506)
In this paper, we study the parameter estimation problem in an adaptive linear model. We propose to use ALEE (adaptive linear estimation equations) to obtain point and interval estimates. Our main contribution is to propose an estimator which is asymptotically normal without requiring any stability condition on the sample covariance matrix. Unlike the concentration based confidence regions, our proposed confidence regions allow for heavy tailed noise variables. We demonstrate the utility of our method by comparing our method with existing methods.
Our work leaves several questions open for future research. For example, it would be interesting to characterize the variance of the ALEE estimator compared to the best possible variance[13, 20] for  $d > 1$ . It would also be interesting to know if such results can be extended to non-linear adaptive models, e.g., to an adaptive generalized linear model [23]. Furthermore, our paper assumes a fixed dimension  $d$  for the problem while letting  $n \to \infty$ . It would be interesting to explore whether we can allow the dimension to grow with the number of samples at a specific rate.
This work was partially supported by the National Science Foundation Grants DMS-2311304, CCF-1934924, DMS-2052949 and DMS-2210850.
A Proof 12
A.1 Proof of Theorem 3.1 12
A.2 Proof of Remark 3.4 13
A.3 Proof of Corollary 3.5 14
A.4 Proof of Theorem 3.6 17
B Generalized Theorem 3.6 18
C Simulation 21
C.1 Simulation details 21
C.2 Tables for contextual bandits 22
C.3 Asymptotic normality with centered Poisson noise variables 22
A Proof
In Theorem 3.1, Corollary 3.3 and Remark 3.4, we deal with an arm with index  $k \in [K]$ . To simplify notations, we drop the subscript  $k$  in  $s_{t,k}, w_{t,k}, x_{t,k}, \widehat{\theta}_{\mathrm{ALEE},k}$  and  $\theta_k^*$  throughout the proof, and use  $s_t, w_t, x_t, \widehat{\theta}_{\mathrm{ALEE}}$  and  $\theta^{*}$ , respectively.
A.1 Proof of Theorem 3.1
Condition (17) serves as an important role in proving (18). Therefore, we start our proof by verifying the condition (17). Since function  $f$  is a positive decreasing function, we first have
$$
\max  _ {1 \leq t \leq n} f ^ {2} \left(\frac {s _ {t}}{s _ {0}}\right) \frac {x _ {t} ^ {2}}{s _ {0}} \leq f ^ {2} (1) \frac {1}{s _ {0}}. \tag {32}
$$
Furthermore, since function  $f' / f$  is increasing, we have
$$
\max  _ {1 \leq t \leq n} \left(1 - \frac {f \left(s _ {t} / s _ {0}\right)}{f \left(s _ {t - 1} / s _ {0}\right)}\right) = \max  _ {1 \leq t \leq n} \frac {f \left(s _ {t - 1} / s _ {0}\right) - f \left(s _ {t} / s _ {0}\right)}{f \left(s _ {t - 1} / s _ {0}\right)} \tag {33}
$$
$$
\stackrel {(i)} {\leq} \frac {1}{s _ {0}} \max  _ {1 \leq t \leq n} \frac {- f ^ {\prime} (s _ {t - 1} / s _ {0})}{f (s _ {t - 1} / s _ {0})} = \frac {1}{s _ {0}} \frac {- f ^ {\prime} (1)}{f (1)},
$$
where inequality  $(i)$  follows from mean value theorem and the monotonicity of the function  $f'/f$ . Thus, by assuming  $1/s_0 = o_p(1)$  and  $s_0/s_n = o_p(1)$ , condition (17) follows directly from equation (32) and equation (33).
By the construction of ALEE estimator, we have
$$
\left\{\sum_ {t = 1} ^ {n} w _ {t} x _ {t} \right\} \cdot \left(\widehat {\theta} _ {\mathrm {A L E E}} - \theta^ {*}\right) = \sum_ {t = 1} ^ {n} w _ {t} \epsilon_ {t}. \tag {34}
$$
$$
\sum_ {t = 1} ^ {n} w _ {t} x _ {t} = \sqrt {s _ {0}} \sum_ {t = 1} ^ {n} f \left(s _ {t} / s _ {0}\right) \frac {x _ {t} ^ {2}}{s _ {0}} = \sqrt {s _ {0}} \int_ {1} ^ {s _ {n} / s _ {0}} f (x) d x \cdot \frac {\sum_ {t \leq n} f \left(s _ {t} / s _ {0}\right) x _ {t} ^ {2} / s _ {0}}{\int_ {1} ^ {s _ {n} / s _ {0}} f (x) d x}. \tag {35}
$$
By the mean value theorem, we have that for  $t \in [n], \xi_t \in [s_{t-1}, s_t]$
$$
\int_ {s _ {t - 1} / s _ {0}} ^ {s _ {t} / s _ {0}} f (x) d x = f (\xi_ {t} / s _ {0}) \frac {x _ {t} ^ {2}}{s _ {0}}.
$$
$$
\frac {\sum_ {t \leq n} f (s _ {t} / s _ {0}) x _ {t} ^ {2} / s _ {0}}{\int_ {1} ^ {s _ {n} / s _ {0}} f (x) d x} = 1 + \underbrace {\frac {\sum_ {t \leq n} (\frac {f (s _ {t} / s _ {0})}{f (\xi_ {t} / s _ {0})} - 1) f (\xi_ {t} / s _ {0}) x _ {t} ^ {2} / s _ {0}}{\sum_ {t \leq n} f (\xi_ {t} / s _ {0}) x _ {t} ^ {2} / s _ {0}}} _ {\triangleq_ {R}}.
$$
$$
\begin{array}{l} | R | \leq \frac {\sum_ {t \leq n} \left| \frac {f \left(s _ {t} / s _ {0}\right)}{f \left(\xi_ {t} / s _ {0}\right)} - 1 \right| f \left(\xi_ {t} / s _ {0}\right) x _ {t} ^ {2} / s _ {0}}{\sum_ {t \leq n} f \left(\xi_ {t} / s _ {0}\right) x _ {t} ^ {2} / s _ {0}} \\ \leq \frac {\sum_ {t \leq n} \left| \frac {f \left(s _ {t} / s _ {0}\right)}{f \left(s _ {t - 1} / s _ {0}\right)} - 1 \right| f \left(\xi_ {t} / s _ {0}\right) x _ {t} ^ {2} / s _ {0}}{\sum_ {t \leq n} f \left(\xi_ {t} / s _ {0}\right) x _ {t} ^ {2} / s _ {0}} \\ \leq \max  _ {1 \leq t \leq n} \left(1 - \frac {f \left(s _ {t} / s _ {0}\right)}{f \left(s _ {t - 1} / s _ {0}\right)}\right) \stackrel {(i i)} {=} o _ {p} (1). \\ \end{array}
$$
Equality (ii) follows from condition (17). Consequently, applying Slutsky's theorem yields
$$
\frac {\sum_ {i = 1} ^ {n} w _ {t} x _ {t}}{\sqrt {s _ {0}} \int_ {1} ^ {s _ {n} / s _ {0}} f (x) d x} \stackrel {{p}} {{\longrightarrow}} 1.
$$
Similarly, we can derive
$$
\sum_ {t = 1} ^ {n} w _ {t} ^ {2} = \sum_ {t = 1} ^ {n} f ^ {2} \left(s _ {t} / s _ {0}\right) \frac {x _ {t} ^ {2}}{s _ {0}} = (1 + o _ {p} (1)) \int_ {1} ^ {s _ {n} / s _ {0}} f ^ {2} (x) d x = 1 + o _ {p} (1). \tag {36}
$$
Knowing  $\max_{1\leq t\leq n}w_t^2 = \max_{1\leq t\leq n}f^2 (s_t / s_0)x_t^2 /s_0 = o_p(1)$ , which is a consequence of equation (17), martingale central limit theorem together with an application of Slutsky's theorem yields
$$
(\widehat {\theta} _ {\mathrm {A L E E}} - \theta^ {*}) \cdot \int_ {1} ^ {s _ {n} / s _ {0}} \frac {\sqrt {s _ {0}}}{\widehat {\sigma}} f (x) d x \xrightarrow {d} \mathcal {N} (0, 1).
$$
Lastly, we recall that
$$
\frac {\widehat {\theta} _ {\mathrm {A L E E}} - \theta^ {*}}{\widehat {\sigma} \sqrt {\sum_ {t \leq n} w _ {t} ^ {2}}} \cdot \left(\sum_ {t = 1} ^ {n} w _ {t} x _ {t}\right) = \frac {1}{\widehat {\sigma} \sqrt {\sum_ {t \leq n} w _ {t} ^ {2}}} \sum_ {t = 1} ^ {n} w _ {t} \epsilon_ {t}.
$$
Therefore, equation (19) follows from martingale central limit theorem and Slutsky's theorem.
Remark A.1. Equation (18) sheds light on the asymptotic variance of the ALEE estimator, thereby aiding in the selection of a suitable function  $f$  to improve the efficiency of ALEE estimator. On the other hand, equation (19) offers a practical approach to obtaining an asymptotically precise confidence interval.
Remark A.2. Condition (17) is a general requirement that governs equation (18), and is not specific to bandit problems. However, the difficulty in verifying (17) can vary depending on the problem at hand.
A.2 Proof of Remark 3.4
Corollary 3.3 follows directly from Theorem 1 in [1]. In this section, we provide a proof of Remark 3.4. By considering  $\lambda_0 = 1$  in Corollary 3.3, we have with probability at least  $1 - \delta$
$$
\left| \sum_ {t = 1} ^ {n} w _ {t} x _ {t} \right| \cdot \left| \widehat {\theta} _ {\mathrm {A L E E}} - \theta^ {*} \right| \leq \sigma_ {g} \sqrt {\left(1 + \sum_ {t = 1} ^ {n} w _ {t} ^ {2}\right) \cdot \log \left(\frac {1 + \sum_ {t = 1} ^ {n} w _ {t} ^ {2}}{\delta^ {2}}\right)}. \tag {37}
$$
By the construction of the weights in Corollary 3.2, we have
$$
\sum_ {t = 1} ^ {n} w _ {t} ^ {2} = \sum_ {t = 1} ^ {n} f ^ {2} \left(\frac {s _ {t}}{s _ {0}}\right) \frac {x _ {t} ^ {2}}{s _ {0}} \leq \int_ {1} ^ {\infty} f ^ {2} (x) d x = 1. \tag {38}
$$
Therefore, to complete the proof, it suffices to characterize a lower bound for  $\sum_{1\leq t\leq n}w_{t}x_{t}$ . By definition, we have
$$
\begin{array}{l} \sum_ {t = 1} ^ {n} w _ {t} x _ {t} = \sum_ {t = 1} ^ {n} f (s _ {t} / s _ {0}) \frac {x _ {t} ^ {2}}{\sqrt {s _ {0}}} \\ \stackrel {(i)} {=} \sum_ {t = 1} ^ {n} \frac {x _ {t} ^ {2}}{(s _ {t} \log (e ^ {2} s _ {t} / s _ {0})) ^ {1 / 2} \log \log (e ^ {2} s _ {t} / s _ {0})} \\ \geq \frac {1}{(2 + \log \left(s _ {n} / s _ {0}\right)) ^ {1 / 2} \log \left(2 + \log \left(s _ {n} / s _ {0}\right)\right)} \sum_ {t = 1} ^ {n} \frac {x _ {t} ^ {2}}{\sqrt {s _ {t}}} \tag {39} \\ \stackrel {(i i)} {\geq} \frac {1}{(2 + \log (s _ {n} / s _ {0})) ^ {1 / 2} \log (2 + \log (s _ {n} / s _ {0}))} \cdot 2 (\sqrt {s _ {n}} - \sqrt {s _ {0}}) \sqrt {\frac {s _ {0}}{1 + s _ {0}}} \\ \stackrel {(i i i)} {\geq} \frac {1}{(2 + \log (s _ {n} / s _ {0})) ^ {1 / 2} \log (2 + \log (s _ {n} / s _ {0}))} \cdot \sqrt {2} (\sqrt {s _ {n}} - \sqrt {s _ {0}}). \\ \end{array}
$$
In equation (i), we plug in the expression of function  $f$  and hence  $\sqrt{s_0}$  cancels out. Since  $x_{t}$  is either 0 or 1, inequality (ii) follows from the integration of the function  $h(x) = 1 / \sqrt{x}$ . Inequality (iii) follows from  $s_0 > 1$ . Putting things together, we have
$$
\begin{array}{l} \left| \widehat {\theta} _ {\text {A L E E}} - \theta^ {*} \right| \leq \sigma_ {g} \frac {\sqrt {2 \log (2 / \delta^ {2})}}{\sum_ {1 \leq t \leq n} w _ {t} x _ {t}} \tag {40} \\ \leq \sigma_ {g} \sqrt {\log (2 / \delta^ {2})} \frac {\sqrt {2 + \log (s _ {n} / s _ {0})} \log \{2 + \log (s _ {n} / s _ {0}) \}}{\sqrt {s _ {n}} - \sqrt {s _ {0}}}. \\ \end{array}
$$
This completes our proof of Remark 3.4.
A.3 Proof of Corollary 3.5
Note that it suffices to verify the following condition (41)
$$
\max  _ {1 \leq t \leq n} f ^ {2} \left(\frac {s _ {t}}{s _ {0}}\right) \frac {y _ {t - 1} ^ {2}}{s _ {0}} + \max  _ {1 \leq t \leq n} \left(1 - \frac {f \left(s _ {t} / s _ {0}\right)}{f \left(s _ {t - 1} / s _ {0}\right)}\right) + \int_ {s _ {n} / s _ {0}} ^ {\infty} f ^ {2} (x) d x = o _ {p} (1) \tag {41}
$$
for  $\theta^{*}\in [-1,1]$  in order to complete the proof of Corollary 3.5. The other part of the proof can be adapted from the proof of Theorem 3.1. To simplify notations, we let
$$
T _ {1} \triangleq \max _ {1 \leq t \leq n} f ^ {2} \Big (\frac {s _ {t}}{s _ {0}} \Big) \frac {y _ {t - 1} ^ {2}}{s _ {0}}, T _ {2} \triangleq \max _ {1 \leq t \leq n} \left(1 - \frac {f (s _ {t} / s _ {0})}{f (s _ {t - 1} / s _ {0})}\right), \mathrm {a n d} T _ {3} \triangleq \int_ {s _ {n} / s _ {0}} ^ {\infty} f ^ {2} (x) d x.
$$
Therefore, proving equation (41) is equivalent to showing that  $T_{1}, T_{2}$ , and  $T_{3}$  converge to zero in probability. We will now demonstrate the convergence of each of these three terms to zero in probability.
$T_{1}$  with  $\theta^{*} = 1$ : To prove  $T_{1} = o_{p}(1)$ , we make use of a result in [19, Equation 3.23], which is
$$
\mathbb {P} \left(\lim  _ {n \rightarrow \infty} n ^ {- 2} (\log \log n) \sum_ {t = 1} ^ {n} y _ {t - 1} ^ {2} = \sigma^ {2} / 4\right) = 1. \tag {42}
$$
$$
\begin{array}{l} T _ {1} = \max _ {1 \leq t \leq n} f ^ {2} (\frac {s _ {t}}{s _ {0}}) \frac {y _ {t - 1} ^ {2}}{s _ {0}} = \max _ {1 \leq t \leq n} \frac {y _ {t - 1} ^ {2}}{s _ {t} \log (e ^ {2} s _ {t} / s _ {0}) \{\log \log (e ^ {2} s _ {t} / s _ {0}) \} ^ {1 + \beta}} \\ \leq \max  _ {1 \leq t \leq n} \frac {y _ {t - 1} ^ {2}}{s _ {t} \log \left(e ^ {2}\right) \left\{\log \log \left(e ^ {2}\right) \right\} ^ {1 + \beta}} \tag {43} \\ = \frac {1}{2 (\log 2) ^ {1 + \beta}} \max  _ {1 \leq t \leq n} \frac {y _ {t - 1} ^ {2}}{s _ {t}} \\ \leq \frac {1}{2 (\log 2) ^ {1 + \beta}} \max \{\max _ {1 \leq t \leq \lfloor n ^ {2} / 3 \rfloor} \frac {y _ {t - 1} ^ {2}}{s _ {0}}, \max _ {\lfloor n ^ {2} / 3 \rfloor + 1 \leq t \leq n} \frac {y _ {t - 1} ^ {2}}{s _ {\lfloor n ^ {2} / 3 \rfloor} - s _ {0}} \} \\ \end{array}
$$
In equation (43), we split the sequence into two parts and set different lower bounds for  $s_t$ . The major benefit of this step is to help us derive a better choice of  $s_0$ . Now we bound  $\max_{1 \leq t \leq \lfloor n^{2/3} \rfloor} y_{t-1}^2$  and
$\max_{[n^{2 / 3}] + 1\leq t\leq n}y_{t - 1}^2$  . Note that