SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ingvar Ziemann
George J. Pappas
Stephen Tu
Nikolai Matni
We derive upper bounds for random design linear regression with dependent  $(\beta$ -mixing) data absent any realizability assumptions. In contrast to the strictly realizable martingale noise regime, no sharp instance-optimal non-asymptotics are available in the literature. Up to constant factors, our analysis correctly recovers the variance term predicted by the Central Limit Theorem—the noise level of the problem—and thus exhibits graceful degradation as we introduce misspecification. Past a burn-in, our result is sharp in the moderate deviations regime, and in particular does not inflate the leading order term by mixing time factors.
Ordinary least squares (OLS) regression from a finite sample is one of the most ubiquitous and widely used technique in machine learning. When faced with independent data, there are now sharp tools available to analyze its success optimally under relatively general assumptions. Indeed, a non-asymptotic theory matching the classical asymptotically optimal understanding from statistics [van der Vaart, 2000] has been developed over the last decade [Hsu et al., 2012, Oliveira, 2016, Mourtada, 2022]. However, once we relax the independence assumption and move toward data that exhibits correlations, the situation is much less well-understood—even for a problem as seemingly simple as linear regression. While sharp asymptotics are available through various limit theorems, there are no general results matching these in the finite sample regime.
In this paper, we study the instance-specific performance of ordinary least squares in a setting with dependent data—and in contrast to much contemporary work on the theme—without imposing realizability. If in addition to a realizability assumption the noise forms a martingale difference sequence, it is now well-known that martingale methods can be used to demonstrate that dependent linear regression is no harder than its independent counterpart [Simchowitz et al., 2018]. Furthermore, as long as one maintains such an assumption on the noise, a similar observation even holds true for generalized linear and bilinear models [Kowshik et al., 2021, Sattar et al., 2022], and regression with square loss more generally [Ziemann and Tu, 2022].
However, barring any such strong realizability assumption, martingale methods are no longer directly available, and neither are there any sharp non-asymptotics in the learning theory literature. Absent martingale techniques, a natural approach is to use the blocking technique [Bernstein, 1927, Yu, 1994] to port concentration inequalities valid for independent data to the dependent setting. However, since blocking effectively reduces the sample size by a factor of the degree of dependence of the data, a judicious application is necessary in order to recover the correct noise level of the problem—the level predicted by the Central Limit Theorem (CLT).
$^{1}$ A distribution  $\mathsf{P}_{X,Y}$  is (linearly) realizable if the regression function  $x \mapsto \mathbf{E}[Y \mid X = x]$  is linear.
This paper serves to explain how the combination of two simple yet powerful observations sidestep the aforementioned issues with blocking. To better appreciate these observations, we recall that the analysis of random design linear regression decomposes into: (1) controlling the lower tail of the empirical covariance matrix; and (2) controlling the interaction between the noise and the covariates.
First, as noted by Mendelson [2014], the dominant contribution to the error rate is due to the interaction of the noise with the covariates via the hypothesis class. In linear regression this interaction term takes the form of a random walk (see (2.6) and (2.8) below). While one must also analyze the lower tail of the empirical covariance matrix, its contribution to the final error tends to be lower order. This is exactly the point: the empirical covariance matrix tends to dominate its population counterpart under very mild assumptions [see e.g. Koltchinskii and Mendelson, 2015, Oliveira, 2016, Simchowitz et al., 2018]. Hence deflating the sample size for this purpose by using dependency is of relatively minor consequence and only amounts to an additional burn-in.
Second, turning to the random walk—the noise-class interaction term—the above issue with blocking can be remedied if one restricts its use to control only the largest scale of deviation. This observation can be traced to the moderate deviations literature, but does not seem to have made its way into the learning theory literature [see e.g. Merlevède et al., 2011]. To explain this idea, let us recall Bernstein's inequality: for  $b > 0$ ,  $\delta \in (0,1)$ , and a sequence of  $n \in \mathbb{N}$  iid mean zero  $b$ -bounded scalar random variables  $V_{1:n}$ ,
$$
\mathbf {P} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} V _ {i} \geq 2 \sqrt {\frac {\mathbf {E} V _ {1} ^ {2} \ln (1 / \delta)}{n}} + \frac {4 b \log (1 / \delta)}{3 n}\right) \leq \delta . \tag {1.1}
$$
In the moderate deviations bandwidth  $(\delta \gtrsim \exp (-n\mathbf{E}V_1^2 /b^2))$ , the leading term of (1.1) is exactly of the expected order, seen from a central limit heuristic:  $\sqrt{\frac{\mathbf{E}V_1^2\ln(1 / \delta)}{n}}$ . Assume now for sake of argument that  $k\in \mathbb{N}$  divides  $n$  and set  $m = n / k$ . Applying (1.1) instead to the  $bk$ -bounded variables  $\bar{V}_{i:m}$ ,  $\bar{V}_i\triangleq \sum_{j = ik - k - 1}^{ik}V_j$  we find instead:
$$
\mathbf {P} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} V _ {i} \geq 2 \sqrt {\frac {k ^ {- 1} \mathbf {E} (\bar {V} _ {1}) ^ {2} \ln (1 / \delta)}{n}} + \frac {4 b k \log (1 / \delta)}{3 n}\right) \leq \delta . \tag {1.2}
$$
The (normalized) variance of iid random variables tensorizes nicely  $(k^{-1}\mathbf{E}(\bar{V}_1)^2 = \mathbf{E}V_1^2)$ , and so the only difference between (1.1) and (1.2) is that the large deviations term has been inflated by a factor  $k$ . More generally, however, (1.2) remains valid as long as every  $k$  samples are blockwise independent. The leading term of (1.2) already captures the correct variance term in the blockwise independent, one-dimensional and bounded setting.
The above two paragraphs illustrate the core of our argument: by combining the above two observations we can entirely relegate any dependence on mixing to additive burn-in factors. In the sequel, we produce a more general version of this argument. To allow for arbitrary dimensions and handle unbounded processes, we first replace Bernstein's inequality with a corollary to Talagrand's inequality due to Einmahl and Li [2008]. To allow for  $\beta$ -mixing processes, we replace the blockwise independence assumption with the blocking strategy of Yu [1994]. By combining with control of the lower tail, which as noted above holds under mild assumptions, this leads to our main result Theorem 3.1, captured informally below.
Informal version of Theorem 3.1. Past a mild burn-in, polynomial in relevant problem quantities including the  $\beta$ -mixing coefficients of the data, and for a fixed failure probability  $\delta \in (0,1)$ , OLS with one-dimensional targets and  $d_{\mathsf{X}}$ -dimensional covariates enjoys the following excess risk guarantee:
$$
\text {E x c e s s} \quad \text {R i s k} \left(\mathrm {O L S}\right) \lesssim \frac {\sigma^ {2} \left(d _ {\mathrm {X}} + \log (1 / \delta)\right)}{n}. \tag {1.3}
$$
Moreover, the term  $\sigma^2$  in (1.3) accurately captures the noise level of the problem solely via the relevant second order statistics; it is not inflated by any mixing times.
The crux of this result is that past a burn-in, the OLS excess risk does not directly depend on mixing times, but only on the relevant second order statistics. Put differently, the effect of slow mixing has
been relegated to a small additive term with higher order dependence on  $1 / n$ . This stands in stark contrast to the usual invocation of the blocking technique where the effect of mixing typically enters multiplicatively, thereby degrading the rate of convergence uniformly across all sample-sizes and past any burn-in times [see e.g. Steinwart and Christmann, 2009, Kuznetsov and Mohri, 2017, Wong et al., 2020, Roy et al., 2021].
Applicability. Before we proceed with the main development, we remark that the class of  $\beta$ -mixing is quite broad; a few examples where Theorem 3.1 can be instantiated are as follows:
- all  $\phi$ -mixing processes are  $\beta$ -mixing [Doukhan, 2012],
stationary uniformly ergodic Markov chains are  $\beta$ -mixing,
stationary Gaussian vector autoregressive moving average (ARMA) processes are  $\beta$ -mixing [Mokkadem, 1988],
- many other sub-classes of GARCH models, often studied in the economics and finance literature, are  $\beta$ -mixing [Carrasco and Chen, 2002].
The list is far from exhaustive and further examples can for instance be found in Doukhan [2012]. The stationarity assumptions above can also typically be dropped. We also point out that it is precisely because we can handle misspecification that our result is of interest for many of these examples.
Outline. The rest of this article is structured as follows. Section 2 fixes our notation and yields a more formal problem formulation. We provide our main result, Theorem 3.1, in Section 3. After stating our main theorem, we highlight its features and then proceed to compare it to related work in Section 3.1. We outline the proof of Theorem 3.1 and provide supporting results in Section 4, including separate analyses of the noise-interaction and the lower tail of the empirical covariance matrix. Section 5 concludes and technical details are relegated to Appendix A.
Notation. Expectation (resp. probability) with respect to all the randomness of the underlying probability space is denoted by  $\mathbf{E}$  (resp.  $\mathbf{P}$ ). For two probability measures  $\mathsf{P}$  and  $\mathsf{Q}$  defined on the same probability space, their total variation is denoted  $\| \mathsf{P} - \mathsf{Q}\|_{\mathsf{TV}}$ . Maxima (resp. minima) of two numbers  $a, b \in \mathbb{R}$  are denoted by  $a \vee b = \max(a, b)$  (resp.  $a \wedge b = \min(a, b)$ ). For an integer  $n \in \mathbb{N}$ , we also define the shorthand  $[n] \triangleq \{1, \ldots, n\}$ .
The Euclidean norm on  $\mathbb{R}^d$  is denoted  $\| \cdot \| _2$ , and the unit sphere in  $\mathbb{R}^d$  is denoted  $\mathbb{S}^{d - 1}$ . The standard inner product on  $\mathbb{R}^d$  is denoted  $\langle \cdot ,\cdot \rangle$ . We embed matrices  $M\in \mathbb{R}^{d_1\times d_2}$  in Euclidean space by vectorization:  $\operatorname {vec}M\in \mathbb{R}^{d_1d_2}$ , where  $\operatorname{vec}$  is the operator that vertically stacks the columns of  $M$  (from left to right and from top to bottom). For a matrix  $M$ , the Euclidean norm is the Frobenius norm, i.e.,  $\| M\| _F\triangleq \| \operatorname {vec}M\| _2$ . We similarly define the inner product of two matrices  $M,N$  by  $\langle M,N\rangle \triangleq \langle \operatorname {vec}M,\operatorname {vec}N\rangle$ . The transpose of a matrix  $M$  is denoted by  $M^{\top}$  and if  $M$  is square—tr  $M$  denotes its trace. We also write  $\| M\|_{\mathrm{op}}$  for the induced  $(\mathbb{R}^d,\| \cdot \| _2)\to (\mathbb{R}^d,\| \cdot \| _2)$  norm. For two symmetric matrices  $M,N$ , we write  $M\succ N$  ( $M\succeq N$ ) if  $M - N$  is positive (semi-)definite. If  $M$  is positive semidefinite we write  $\partial M\triangleq \{x\in \mathbb{R}^d\mid x^\top Mx = 1\}$  for the boundary of the ellipsoid induced by  $M$  (note that  $\mathbb{S}^{d - 1} = \partial I_d$ ).
Problem Formulation. We are given  $n$  input-output tuples:  $X_{1:n} \sim \mathsf{P}_{1:n}^{X}$  (taking values in  $\mathbb{R}^{d_x}$ ) and  $Y_{1:n} \sim \mathsf{P}_{1:n}^{Y}$  (taking values in  $\mathbb{R}^{d_y}$ ). Using these samples, the goal of the learner is to estimate the best linear hypothesis:
$$
M _ {\star} \in \underset {M \in \mathbb {R} ^ {d _ {\mathrm {Y}} \times d _ {\mathrm {X}}}} {\operatorname {a r g m i n}} \left\{\mathbf {E} \| Y - M X \| _ {2} ^ {2} \right\} \tag {2.1}
$$
where the distributions of  $X$  and  $Y$  in (2.1) are specified via:
$$
X \sim \frac {1}{n} \sum_ {i = 1} ^ {n} \mathrm {P} _ {i} ^ {X} \quad \text {a n d} \quad Y \sim \frac {1}{n} \sum_ {i = 1} ^ {n} \mathrm {P} _ {i} ^ {Y}. \tag {2.2}
$$
Note that (2.2) is equivalent to sampling from the uniform mixture over  $(X_{1:n},Y_{1:n})$  with the index  $i\in [n]$  sampled uniformly. The operator  $\Sigma_X\triangleq \mathbf{E}[XX^{\mathsf{T}}]$  is the averaged covariance operator (with  $X$  as in (2.2)). The excess risk of a linear hypothesis  $M$  can then be written as:
$$
\operatorname {E x c e s s} (M) \triangleq \mathbf {E} \| Y - M X \| _ {2} ^ {2} - \mathbf {E} \| Y - M _ {\star} X \| _ {2} ^ {2} = \| \left(M - M _ {\star}\right) \sqrt {\Sigma_ {X}} \| _ {F} ^ {2}. \tag {2.3}
$$
We now define the noise variable  $W_{i} \triangleq Y_{i} - M_{\star}X_{i}$  but, as mentioned above, do not impose any (conditional) mean zero assumptions on the noise. To simplify the exposition, we will henceforth assume that  $\Sigma_X\succ 0$ , but our results easily extend to the case  $\Sigma_X\succeq 0$  by restricting attention to the span of  $\Sigma_{X}$ . With these preliminaries in place, on the event that the design is nondegenerate, the OLS and its error equation can be specified as follows:
$$
\widehat {M} \triangleq \left(\sum_ {i = 1} ^ {n} Y _ {i} X _ {i} ^ {\mathsf {T}}\right) \left(\sum_ {i = 1} ^ {n} X _ {i} X _ {i} ^ {\mathsf {T}}\right) ^ {- 1} \quad \Longrightarrow \quad \widehat {M} - M _ {\star} = \left(\sum_ {i = 1} ^ {n} W _ {i} X _ {i} ^ {\mathsf {T}}\right) \left(\sum_ {i = 1} ^ {n} X _ {i} X _ {i} ^ {\mathsf {T}}\right) ^ {- 1}. \tag {2.4}
$$
Our task in the sequel is to establish that the choice  $\widehat{M}$  renders the excess risk (2.3) small. We note in passing that  $\widehat{M}$  is an empirical risk minimizer:
$$
\widehat {M} \in \operatorname * {a r g m i n} _ {M \in \mathbb {R} ^ {d _ {\Upsilon} \times d _ {\Upsilon}}} \left\{\frac {1}{n} \sum_ {i = 1} ^ {n} \| Y _ {i} - M X _ {i} \| _ {2} ^ {2} \right\}.
$$
The Noise Term. Let us also define the following prefiltered noise-class interaction variables:
$$
V _ {i} \triangleq W _ {i} X _ {i} ^ {\mathsf {T}} \Sigma_ {X} ^ {- 1 / 2} \quad i \in [ n ]. \tag {2.5}
$$
The square of the following (weighted and possibly biased) random walk effectively characterizes the noise level in our problem:
$$
S _ {n} \triangleq \frac {1}{n} \sum_ {i = 1} ^ {n} V _ {i}. \tag {2.6}
$$
We remark that by construction  $\mathbf{E}S_{n} = 0$  using the optimality of  $M_{\star}$  in (2.1). To see this, simply invoke the optimality equation for  $M_{\star}$  and note that  $\mathbb{R}^{d_{\times}\times d_{\vee}}$  induces a convex class in the corresponding  $L^2$ -space over the mixtures (2.2). Note however that the increments of (2.6) are not necessarily mean zero unless  $X_{1:n}$  and  $Y_{1:n}$  are stationary. However, since  $\mathbf{E}S_{n} = 0$ , we also have with  $\bar{V}_i\triangleq V_i - \mathbf{E}V_i$ :
$$
S _ {n} = \frac {1}{n} \sum_ {i = 1} ^ {n} \bar {V} _ {i}. \tag {2.7}
$$
In light of (2.4) and (2.6), we have that the empirical excess risk depends on the norm of:
$$
\left(\widehat {M} - M _ {\star}\right) \sqrt {\Sigma_ {X}} = S _ {n} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \Sigma_ {X} ^ {- 1 / 2} X _ {i} X _ {i} ^ {\top} \Sigma_ {X} ^ {- 1 / 2}\right) ^ {- 1}. \tag {2.8}
$$
Hence, we need to control the random walk in (2.7) and the lower tail of
$$
\tilde {\Sigma} _ {n} \triangleq \frac {1}{n} \sum_ {i = 1} ^ {n} \Sigma_ {X} ^ {- 1 / 2} X _ {i} X _ {i} ^ {\top} \Sigma_ {X} ^ {- 1 / 2}, \tag {2.9}
$$
the prefiltered empirical covariance matrix. As mentioned previously, lower uniform laws for (2.9) are valid under mild assumptions [Koltchinskii and Mendelson, 2015, Oliveira, 2016], and blocking such results does not incur more than a worsening of the burn-in. Hence, the noise level of the problem is very much dictated by the random walk (2.6).
$\beta$ -mixing and the Blocking Technique. In the sequel we demonstrate that the standard blocking device combined with a (functional) version of Bernstein's inequality allows us to pass the distributional (or coarse) measure of dependency to a higher order additive term, yielding non-asymptotic rates consistent with the CLT as described in Section 1. We will also use blocking to derive our lower uniform law, controlling the lower tail of (2.9). To make these ideas rigorous we require the following standard measure of dependence (take  $Z_{1:n} = (X,Y)_{1:n}$  below).
Definition 2.1. Let  $Z_{1:n}$  be a stochastic process. The  $\beta$ -mixing coefficients of  $Z$ ,  $\beta_Z(i)$ , are:
$$
\beta_ {Z} (i) = \sup  _ {t \in [ n ]: t + i \leq n} \mathbf {E} \| \mathrm {P} _ {Z _ {i + t}} (\cdot | Z _ {1: t}) - \mathrm {P} _ {Z _ {i + t}} \| _ {\mathrm {T V}}, \quad i \in [ n ]. \tag {2.10}
$$
Intuitively, the coefficients  $\beta_{Z}(i)$  in (2.10) measure the dependence at range  $i$  of the process  $Z_{1:n}$ . This measurement is done in total variation distance by comparing the distribution of  $Z_{t + i}$  with the conditional distribution  $Z_{t + i} \mid Z_{1:t}$ . More concretely, the notion of  $\beta$ -mixing allows us to use the blocking technique Yu [1994]. This technique splits the process  $Z_{1:n}$  into blocks, such that every other block is approximately independent, leaving us with two separate processes, consisting of odd and even blocks, that are almost independent (see (4.1) below). One then proceeds to use  $\beta$ -mixing to construct a "parallel" probability space, approximating the original one, but in which the odd and even blocks are independent. The price we pay for this is measured in terms of the coefficients (2.10). The essence of this idea—to use that data points sufficiently separated in time are often roughly independent—can be traced back to Bernstein [1927].
To give our main result for general target dimension we require one last preliminary notion. Given a  $d$ -dimensional square, symmetric positive semidefinite matrix  $M \in \mathbb{R}^{d \times d}$ , we say that its effective dimension is  $\operatorname{edim}(M) \triangleq \operatorname{tr} M / \|M\|_{\mathrm{op}}$ . Our main result is the following theorem.
Theorem 3.1. Fix  $\delta \in (0,1)$  and  $n,m\in \mathbb{N}$  with  $2m\leq n$  . Let  $a_{1:2m}$  be a monotone partition of  $[n]$  such that if  $k\in a_i$ $l\in a_j$  , and  $i > j$  , then  $k > l$  holds. Set  $a_{\mathrm{max}}\in \operatorname{argmax}_{i\in [2m]}|a_i|$  . Fix also a  $\beta$  -mixing sequence  $(X,Y)_{1:n}$  of which each element admits at least  $s\in [4,\infty)$  moments. Assume that there exists a positive number  $\mathsf{h}\in \mathbb{R}$  such that for every  $v\in \partial \Sigma_X$  and  $i\in [n]$  we have that  $\mathbf{E}\langle v,X_i\rangle^4\leq \mathsf{h}^2\langle v,\mathbf{E}[X_iX_i^\top ]v\rangle$  . Define also the noise interaction terms:
$$
\Sigma \triangleq \frac {1}{n} \sum_ {i = 1} ^ {2 m} \Sigma_ {i}, \quad \Sigma_ {i} \triangleq \mathbf {E} \left[ \operatorname {v e c} \left(\sum_ {j \in a _ {i}} \bar {V} _ {j}\right) \operatorname {v e c} \left(\sum_ {j \in a _ {i}} \bar {V} _ {j}\right) ^ {\top} \right], \quad \sigma^ {2} \triangleq \| \Sigma \| _ {\mathrm {o p}}. \tag {3.1}
$$
There exist universal positive constants  $c_{1}, c_{2}, c_{3}, c_{4}, c_{5}, c_{6}$  such that with probability at least  $1 - \delta$ :
$$
\left\| \left(\widehat {M} - M _ {\star}\right) \sqrt {\Sigma_ {X}} \right\| _ {F} ^ {2} \leq \frac {c _ {1} \sigma^ {2} (\operatorname {e d i m} (\Sigma) + \log (1 / \delta))}{n} \tag {3.2}
$$
as long as the following burn-in conditions hold:
$$
\frac {n}{\left| a _ {\max } \right|} \geq c _ {2} \left(d _ {\mathrm {X}} + \mathrm {h} ^ {2} \log (1 / \delta)\right); \quad \left(\frac {n}{\left| a _ {\max } \right|}\right) ^ {1 - 2 / s} \geq c _ {3} s ^ {2} \frac {\left(\frac {1}{m} \sum_ {i = 1} ^ {2 m} \mathbf {E} \left\| \frac {1}{\sqrt {\left| a _ {\max } \right|}} \sum_ {j \in a _ {i}} \bar {V} _ {j} \right\| _ {F} ^ {s}\right) ^ {2 / s}}{\operatorname {e d i m} (\Sigma) \sigma^ {2} \delta^ {2 / s}}; \tag {3.3}
$$
$$
c _ {4} ^ {- 1} <   \frac {\sum_ {i \in [ 2 m ] \cap 2 \mathbb {N}} | a _ {i} |}{\sum_ {i \in [ 2 m ] \cap (2 \mathbb {N} - 1)} | a _ {i} |} <   c _ {4}; \quad \sum_ {i \in [ 2 m ] \cap (2 \mathbb {N} - 1)} c _ {5} ^ {- 1} \Sigma_ {i} \preceq \sum_ {i \in [ 2 m ] \cap 2 \mathbb {N}} \Sigma_ {i} \preceq \sum_ {i \in [ 2 m ] \cap (2 \mathbb {N} - 1)} c _ {5} \Sigma_ {i}; \tag {3.4}
$$
$$
\sum_ {i = 2} ^ {2 m - 1} \beta_ {X, Y} (| a _ {i} |) \leq c _ {6} \delta ; \tag {3.5}
$$
and where  $\beta_{X,Y}$  are the  $\beta$ -mixing coefficients of  $(X,Y)_{1:n}$ .
To interpret Theorem 3.1 we proceed with a sequence of remarks, discussing its features. These remarks also serve to parse the terminology above and lead us to a simplified statement for stationary data and 1-dimensional targets. We present this simplified version as Corollary 3.1 below.
- The dimensional scaling in (3.2) is captured by the effective dimension term  $\mathrm{edim}(\Sigma)$ , which always lies in the interval  $[1, d_{\mathsf{X}}d_{\mathsf{Y}}]$ . For one-dimensional targets and benign noise interaction, and since the  $X_{i}$  in the noise term  $V_{i}$  have been whitened (see (2.5)), we expect  $\Sigma$  to be roughly isotropic. Indeed, whenever  $d_{\mathsf{Y}} = 1$ , the trivial bound  $\mathrm{edim}(\Sigma) \leq d_{\mathsf{X}}$  produces the familiar behavior:  $\| (\widehat{M} - M_{\star}) \sqrt{\Sigma_X} \|_2^2 \lesssim \sigma^2 d_{\mathsf{X}} / n$  with high probability.
- The scaling with  $d_{\mathrm{X}}, d_{\mathrm{Y}}, \sigma^2$  and  $n$  thus scales as expected in the iid regime, but also degrades gracefully with dependence. We reiterate that (3.2) does not depend directly on mixing in the leading order term. For comparison, under suitable regularity conditions, the noise term predicted by the CLT is:
$$
\Sigma_ {\mathrm {C L T}} \triangleq \operatorname * {l i m s u p} _ {n \rightarrow \infty} n ^ {- 1} \mathbf {E} \left[ \operatorname {v e c} \left(\sum_ {j = 1} ^ {n} \bar {V} _ {j}\right) \operatorname {v e c} \left(\sum_ {j = 1} ^ {n} \bar {V} _ {j}\right) ^ {\top} \right], \tag {3.6}
$$
and one can achieve  $\Sigma = \Sigma_{\mathrm{CLT}} + o(1)$  in most situations of practical interest by tuning the block-length; therefore, our analysis of OLS essentially matches the optimal asymptotics.