SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Syamantak Kumar<sup>1</sup> Purnamrita Sarkar<sup>2</sup>
$^{1}$ Department of Computer Science, UT Austin
$^{2}$ Department of Statistics and Data Sciences, UT Austin
syamantak@utexas.edu, purna.sarkar@austin.utexas.edu
Since its inception in 1982, Oja's algorithm has become an established method for streaming principle component analysis (PCA). We study the problem of streaming PCA, where the data-points are sampled from an irreducible, aperiodic, and reversible Markov chain starting in stationarity. Our goal is to estimate the top eigenvector of the unknown covariance matrix of the stationary distribution. This setting has implications in scenarios where data can solely be sampled from a Markov Chain Monte Carlo (MCMC) type algorithm, and the objective is to perform inference on parameters of the stationary distribution. Most convergence guarantees for Oja's algorithm in the literature assume that the data-points are sampled IID. For data streams with Markovian dependence, one typically downsamples the data to get a "nearly" independent data stream. In this paper, we obtain the first near-optimal rate for Oja's algorithm on the entire data, where we remove the logarithmic dependence on the sample size,  $n$ , resulting from throwing data away in downsampling strategies.
Streaming Principal Component Analysis (PCA) is an important and well studied problem where the principal eigenvector of the sample covariance matrix of a dataset is computed one data-point at a time. One of the most popular algorithms for streaming PCA was introduced by Erkki Oja in 1982 [29, 30]. Most existing analyses of Oja's algorithm are done when the data is sampled IID.
However, in many practical applications, the data-points are dependent and are sampled from an MCMC process converging to a target stationary distribution. This naturally arises in the context of token algorithms for Federated PCA settings [10, 12, 13] with multiple machines communicating via a fixed and connected graph topology. Each machine contains an arbitrary fraction of data-points and the goal is to design a streaming algorithm that respects this topology and returns the principal component of the whole dataset. This is typically achieved using a Metropolis-Hastings scheme that uses local information to design the transition matrix of a Markov chain with any desired stationary distribution. The stationary distribution,  $\pi$ , of the random walk is chosen so that the distribution of the samples under  $\pi$  matches the uniform distribution over data-points. Governed by this Markov chain, a random walker then travels the network of machines and samples one data-point at a time from the current machine and computes the update. However, even under the stationary distribution, the data-points are dependent, which deviates from the IID setup. Our goal is to obtain a near-optimal analysis of the  $\sin^2$  error of the estimated vector with respect to the true top eigenvector of the unknown covariance matrix in the Markovian setting.
Estimating the first principal component with streaming PCA: Let  $X_{t}$  be a mean zero  $d$  dimensional vector with covariance matrix  $\Sigma$ , and let  $\eta_{t}$  be a decaying learning rate. The update rule of Oja's algorithm is given as -
$$
w _ {t} \leftarrow \left(I + \eta_ {t} X _ {t} X _ {t} ^ {T}\right) w _ {t - 1}, \quad w _ {t} \leftarrow \frac {w _ {t}}{\| w _ {t} \| _ {2}} \tag {1}
$$
where  $w_{t}$  is the estimate of  $v_{1}$ , the top eigenvector of  $\Sigma$ , and  $\eta_t$  is the step-size at timestep  $t$ . We aim to analyse the  $\sin^2$  error of Oja's iterate at timestep  $t$ , defined as  $1 - \langle w_t, v_1 \rangle^2$ .
Paper	Markovian?	Online?	log-free main-term	sin2 error rate	Sample complexity
Jain et al. [16]	N	Y	Y	O(∇/gap2 1/n)	O(∇/gap2 1/ε)
N	N	O(∇log(d)/gap2 1/n)	O(∇log(d)/gap2 1/ε)
Chen et al. [3]	Y	Y	N	-	O(G/gap2 1/ε log2 (G/gap2 1/ε))
Neeman et al. [28]	Y	N	N	O(∇log(d2-π/4)/(1-|λ2(P)|) gap2 1/n)	O(∇log(d2-π/4)/(1-|λ2(P)|) gap2 1/ε)
Theorem 1	Y	Y	Y	O(∇/(1-|λ2(P)|) gap2 1/n)	O(∇/(1-|λ2(P)|) gap2 1/ε)
Table 1: Comparison of  $\sin^2$  error rates and sample complexities. Here gap :=  $(\lambda_1 - \lambda_2)$ , where  $\lambda_1, \lambda_2$  are the top 2 eigenvalues of  $\Sigma$  and  $\mathcal{V}$  represents a suitably defined variance parameter (see assumption 2). The sample complexity represents the number of samples required to achieve  $\sin^2$  error at most  $\epsilon$ . We note that [1] and [15] also match the online sample complexity bound provided in [16]. Further, for the offline algorithm with IID data, [17] removes the log  $(d)$  factor in exchange for a constant probability of success for large enough  $n$ .
Streaming PCA in the IID setting: For an IID data stream with  $\mathbb{E}[X_i] = 0$  and  $\mathbb{E}[X_i X_i^T] = \Sigma$ , there has been a lot of work on determining the non-asymptotic convergence rates for Oja's algorithm and its various adaptations [16, 1, 3, 38, 14, 15, 26, 21, 25]. Amongst these, [16], [1] and [15] match the optimal offline sample complexity bound suggested by the independent and identically distributed (IID) version of Theorem 1 (See Theorem 1.1 in [16]).
We consider Oja's algorithm in the setting where the data is generated from a reversible, irreducible, and aperiodic Markov chain with stationary distribution  $\pi$ . We denote by  $\mathbb{E}_{\pi}[.]$  the expectation under the stationary distribution. In this setting our goal is to estimate the principal eigenvector of  $\mathbb{E}_{\pi}\left[X_iX_i^T\right]$ . As in the IID setting,  $\mathbb{E}_{\pi}[X_i] = 0$ . The challenge is that the data, even when it reaches stationarity, is dependent. Here the degree of dependence is captured by the second eigenvalue in the magnitude of the transition matrix  $P$  (denoted as  $|\lambda_2(P)|$ ) of the Markov chain. This is closely related to the mixing time of a Markov chain [19], denoted as  $\tau_{\mathrm{mix}}$ , which is the time after which the conditional distribution of a state is close in total variational distance to its stationary distribution,  $\pi$  (See Section 2.1).
Our contribution: Using a series of approximations, we obtain an optimal error rate for the  $\sin^2$  error, which is worse by a factor of  $1 / (1 - |\lambda_2(P)|)$  from the corresponding error rate of the IID case. Previous work [3] has established rates worse by a poly-logarithmic factor by using downsampling, i.e. applying the update on every  $k^{th}$  datapoint. In Figure 1, we compare Oja's algorithm with its downsampled and offline variants (see Section 6 for more details on setup). We see that Oja's algorithm performs significantly better than the downsampled variant, and similarly to the offline variant where for the  $i^{th}$  data point we compute the eigenvector of the sample covariance matrix of all data-points up-to  $i$ . Our work provides a concrete and novel result that explains these observations. In Table 1, we compare our bounds with related analyses of Oja's algorithm. The last row shows that we are the first to obtain an error whose main term is free of logarithmic dependence on  $n$  or  $d$  for streaming PCA in the Markovian case.
We break the logarithmic barrier in previous work by considering a series of approximations of finer granularity which uses reversibility of the Markov chain and standard mixing conditions of irreducible and aperiodic Markov chains. Our rates are comparable to the recent work of [28] (Proposition 1) that establishes an offline error analysis for estimating the principal component of the empirical covariance matrix of Markovian data by using a Matrix Bernstein inequality. Therefore, our results are nearly optimal in terms of the dependence on sample size,  $n$ , the dimension  $d$ , and the model parameters  $\mathcal{V}$ ,  $(\lambda_1 - \lambda_2)$  and  $1 - |\lambda_2(P)|$  (See Section 2 for definitions). Our results also imply a linearly convergent decentralized algorithm for streaming PCA in a distributed setting. As a simple byproduct of our theoretical result, we also obtain a rate for Oja's algorithm applied on downsampled data, which is worse by a factor of  $\log n$ , as shown in Figure 1. To our knowledge, this is the first work that analyzes the Markovian streaming PCA problem without any downsampling that matches the error of the offline algorithm.
The crux of our analysis uses the mixing properties of the Markov chain. Strong mixing intuitively says that the conditional distribution of a state  $s$  in timestep  $k$  given the starting state is exponentially close to the stationary distribution of  $s$ , the closeness being measured using the total variation distance. All previous work on Markovian data exploits this property by conditioning on states many time steps before. However, it is crucial to a) adaptively find how far to look back and b) bound the error of the sequence of matrices we ignore between the current state and the state we are conditioning on. Observe that these two components are related. Looking back too far makes the dependence very small but increases the error resulting from approximating a larger matrix product of intermediate matrices. We present a fine analysis that balances these two parts and then uses spectral theory to bound the second part within a factor of a variance parameter that characterizes the variability of the matrices and shows up in the analysis of [16, 28].
Related work on streaming PCA and online matrix decomposition on Markovian data: Amongst recent work, [3] is very relevant to our setting, since it analyzes Oja's algorithm with Markovian Data samples. Inspired by the ideas of [8], the authors propose a downsampled version of Oja's algorithm to reduce dependence amongst samples and provide a Stochastic Differential Equation (SDE) based analysis to achieve a sample complexity of  $O\left(\frac{G}{(\lambda_1 - \lambda_2)^2}\frac{1}{\epsilon}\log^2\left(\frac{G}{(\lambda_1 - \lambda_2)^2}\frac{1}{\epsilon}\right)\right)$  for  $\sin^2$  error smaller than  $\epsilon$ , where  $G$  is a variance parameter. We obtain a similar rate in Corollary 1 through our techniques. However, comparing with Theorem 1, we observe that downsampling leads to an extra  $O(\log (n))$  factor. It is important to point out that [3] provides an analysis for estimating top  $k$  principal components, whereas this paper focuses on obtaining a log-free error rate for the first principal component.
[22] consider the harder problem of online non-negative matrix factorization for Markovian data. Their analysis establishes asymptotic convergence of error, but does not provide a rate.
Stochastic Optimization with Markovian Data: Markovian models are often considered in Reinforcement Learning and Linear Dynamic Systems[2, 5, 9, 31, 4, 35, 18, 24]. There have been many notable nonasymptotic bounds for stochastic gradient descent (SGD) methods for general convex and nonconvex functions with Markovian data [8, 32, 6, 7, 10, 39, 34]. The convergence rates (sample complexities) obtained in these works apply to more general problems but do not exploit the matrix product structure inherent to Oja's algorithm. In this work, we develop novel techniques to show that a sharper analysis is possible for the PCA objective. The paper is organized as follows. Section 2 contains the problem setup and preliminaries about Markov Chains. Section 3 contains Theorem 1. We present a sketch of the
Figure 1: Comparison of Oja's algorithm with downsampled and offline variants. The X-axis represents the sample size and the Y axis represents the  $\sin^2$  error of each algorithm's estimate of the leading eigenvector. The experimental setup is available in Section 6.
main technical tools in Section 4, intermediate theorems needed for the main theorem in Section 5 and conclude with simulations in Section 6.
This section presents the problem setup and outlines important properties of the Markov chain that will be utilized subsequently. We assume that:
Assumption 1. The Markov chain is irreducible, aperiodic, reversible, and starts in stationarity, with state distribution  $\pi^1$ .
In practice, one can let the Markov chain mix until a burn-in period of  $O(\tau_{\mathrm{mix}})$  steps to achieve state distributions close to the stationary one. A simulation experiment with different burn-in periods can be found in the Supplement Section S.6.
Such a Markov chain can arise in various situations, for e.g., while performing random walks on expander graphs which are used extensively in fields such as computer networks, error-correcting codes, and pseudorandom generators. Each state  $s$  of the Markov chain is associated with a distribution  $D(s)$  over  $d$ -dimensional vectors with mean  $\mu_s \in \mathbb{R}^d$  and covariance matrix  $\Sigma_s \in \mathbb{R}^{d \times d}$ .
For a random walk  $s_1, s_2, \dots, s_t$  on the Markov chain, we define the sequence of random variables  $X_1, X_2, \dots, X_t$ , where conditioned on the state  $s_i$ ,  $X_i \sim D(s_i)$ . We represent the mean as  $\mu := \mathbb{E}_{s \sim \pi}[\mu_s] = \sum_s \pi_s \mu_s$  and the covariance matrix as  $\Sigma \in \mathbb{R}^{d \times d}$ , which, for  $i \in [n]$  can be expressed as:
$$
\Sigma := \mathbb {E} _ {s _ {i} \sim \pi} \mathbb {E} _ {D (s _ {i})} \left[ (X _ {i} - \mu) (X _ {i} - \mu) ^ {T} \right] = \mathbb {E} _ {s \sim \pi} [ \Sigma_ {s} ] + \mathbb {E} _ {s \sim \pi} [ \mu_ {s} \mu_ {s} ^ {T} ] - \mu \mu^ {T}
$$
In this work, we assume  $\mu = 0$ , which is a common assumption in the IID setting (see [16, 1]). While it may be possible to extend our analysis to the non-zero mean case, it is out of the scope of this paper. Therefore,  $\Sigma = \mathbb{E}_{s_i \sim \pi} \mathbb{E}_{D(s_i)}[X_i X_i^T]$  for  $i \in [n]$
Let the eigenvalues of  $\Sigma$  be denoted as  $\lambda_1 > \lambda_2 \geq \lambda_3 \cdots \lambda_d$ . Let  $v_1$  denote the leading eigenvector of  $\Sigma$  and  $V_\perp$  denote the  $\mathbb{R}^{d \times (d-1)}$  matrix with the remaining eigenvectors as columns. In this work,  $\| \cdot \|_2$  denotes the Euclidean  $L_2$  norm for vectors and the operator norm for matrices unless otherwise specified and  $I$  denotes the identity matrix of the appropriate dimensionality. We proceed under the following standard assumptions for  $i \in [n]$ , (see for eg. [15]).
Assumption 2.  $\| \mathbb{E}_{s_i\sim \pi}\mathbb{E}_{D(s_i)}[(X_iX_i^T -\Sigma)^2 ]\| _2\leq \mathcal{V}.$
Assumption 3.  $\| X_{i}X_{i}^{T} - \Sigma \|_{2}\leq \mathcal{M}$  with probability 1.
Assumption 3 also implies \(\| \Sigma_s + \mu_s\mu_s^T -\Sigma \| _2\leq \mathcal{M}\) with probability 1. WLOG, we assume \(\mathcal{M} + \lambda_1\geq 1\) . We use \(\mathbb{E}[.] := \mathbb{E}_{s\sim \pi}\mathbb{E}_{D(s)}[.\] to denote the expectation over state \(s\sim \pi\) and over the state-specific distributions \(D(.)\) , unless otherwise specified. We observe that Assumption 3 can be relaxed to accommodate sub-gaussian data distributions with appropriate variance decay (see Proposition 1 in [21]) using standard truncation-based arguments. We do not pursue this here. Define the matrix product
$$
B _ {t} := \left(I + \eta_ {t} X _ {t} X _ {t} ^ {T}\right) \left(I + \eta_ {t} X _ {t - 1} X _ {t - 1} ^ {T}\right) \dots \left(I + \eta_ {1} X _ {1} X _ {1} ^ {T}\right) \tag {2}
$$
Unrolling the recursion in Eq 1, the output of Oja's algorithm at timestep  $t$  is given as  $w_{t} = \frac{B_{t}w_{0}}{\|B_{t}w_{0}\|_{2}}$ .
2.1 Markov chain mixing times
Now we will discuss some well-known properties of an irreducible, aperiodic, and reversible Markov chain (see [19] for details and derivations). Let  $|\lambda_2(P)|$  denote the second largest absolute eigenvalue of the Markov chain; let the state-distribution of the Markov chain at timestep  $t$  with  $s_1 = x$  be  $P^t(x, \cdot)$ . For any two probability distributions  $\nu_1$  and  $\nu_2$ , recall that the total variational distance is  $TV(\nu_1, \nu_2) \coloneqq \| \nu_1 - \nu_2 \|_{TV} \coloneqq \frac{1}{2} \sum_{x \in \Omega} |\nu_1(x) - \nu_2(x)|$ . The distance from  $\pi$  at the  $t^{\text{th}}$  timestep is defined as  $d_{\text{mix}}(t) \coloneqq \sup_{x \in \Omega} TV(P^t(x, \cdot), \pi)$ . For irreducible and aperiodic Markov chains, by Theorem 4.9 in [19], we have  $d_{\text{mix}}(t) \leq C \exp(-ct)$  for some  $C, c > 0$ . The mixing time of the Markov chain is defined as:
$$
\tau_ {\text {m i x}} (\epsilon) := \inf  \{t: d _ {\text {m i x}} (t) \leq \epsilon \} \tag {3}
$$
As defined in Section 4.5 of [19], we will denote  $\tau_{\mathrm{mix}} := \tau_{\mathrm{mix}}\left(\frac{1}{4}\right)$ . Then, we have  $\tau_{\mathrm{mix}}(\epsilon) \leq \lceil \log_2(1 / \epsilon) \rceil \tau_{\mathrm{mix}}$ . It is worth mentioning the useful relationship between  $d_{\mathrm{mix}}(.)$  and  $\tau_{\mathrm{mix}}$ , given as  $d_{\mathrm{mix}}(l\tau_{\mathrm{mix}}) \leq 2^{-l} \quad \forall l \in \mathbb{N}_0$ . These results about mixing time are valid for general irreducible and aperiodic Markov chains. A reversible Markov chain satisfies  $\forall x, y \in \Omega$ ,  $\pi(x)P(x,y) = \pi(y)P(y,x)$ . For a reversible, irreducible, and aperiodic Markov chain, the gap  $1 - |\lambda_2(P)|$ , is inversely proportional to (Theorem 12.4,  $\tau_{\mathrm{mix}}$  [19]).
In this section, we present our main result, a near-optimal convergence rate for Oja's algorithm on Markovian data. As a corollary, we also establish a rate of convergence for Oja's algorithm applied
on downsampled data, where every  $k^{\mathrm{th}}$  data-point is considered. Supplement Section S.5 contains comprehensive proofs of Theorem 1 and Corollary 1 while the proof of Proposition 1 can be found in Supplement Section S.2.
Theorem 1. Fix a  $\delta \in (0,1)$  and let the step-sizes be  $\eta_{i} := \frac{\alpha}{(\lambda_{1} - \lambda_{2})(\beta + i)}$  with  $\eta_0 \leq \frac{1}{e}, \alpha > 2$ . Under assumptions 1, 2 and 3, for sufficiently large numbers  $n$  such that  $\frac{n}{\log\left(\frac{1}{\eta_n}\right)} > \frac{\beta}{\log\left(\frac{1}{\eta_0}\right)}$ ,
$$
\beta := \frac {1 0 0 0 \alpha^ {2} \max  \left\{\tau_ {m i x} \log \left(\frac {1}{\eta_ {0}}\right) (\mathcal {M} + \lambda_ {1}) ^ {2} , \frac {\left(\frac {\nu}{1 - | \lambda_ {2} (P) |} + \lambda_ {1} ^ {2}\right)}{1 0 0} \right\}}{(\lambda_ {1} - \lambda_ {2}) ^ {2} \log \left(1 + \frac {\delta}{2 0 0}\right)} \tag {4}
$$
the output  $w_{n}$  of Oja's algorithm (1) satisfies
$$
1 - \left(w _ {n} ^ {T} v _ {1}\right) ^ {2} \leq \frac {C \log \left(\frac {1}{\delta}\right)}{\delta^ {2}} \left[ d \left(\frac {2 \beta}{n}\right) ^ {2 \alpha} + \frac {C _ {1} \mathcal {V}}{\left(\lambda_ {1} - \lambda_ {2}\right) ^ {2} \left(1 - \left| \lambda_ {2} (P) \right|\right)} \frac {1}{n} + \frac {C _ {2} \mathcal {M} \left(\mathcal {M} + \lambda_ {1}\right) ^ {2}}{\left(\lambda_ {1} - \lambda_ {2}\right) ^ {3}} \frac {\tau_ {m i x} \left(\eta_ {n} ^ {2}\right) ^ {2}}{n ^ {2}} \right]
$$
with probability at least  $(1 - \delta)$ . Here  $C$  is an absolute constant and
$$
C _ {1} := \frac {\alpha^ {2} (3 + 7 | \lambda_ {2} (P) |)}{2 \alpha - 1}, C _ {2} := \frac {3 5 \alpha^ {3}}{\alpha - 1}
$$
Remark 1. (Interpreting the  $\sin^2$  error) Theorem 1 simply establishes that, under suitable choices of  $\alpha, \beta$ , as long as assumptions 1, 2 and 3 hold, for decaying step sizes  $\eta_i = \frac{\alpha}{(\lambda_1 - \lambda_2)(\beta + i)}$ , with probability at least  $\frac{3}{4}$ , we have,
$$
1 - \left(w _ {n} ^ {T} v _ {1}\right) ^ {2} = O \left(\frac {\mathcal {V}}{\left(\lambda_ {1} - \lambda_ {2}\right) ^ {2} \left(1 - \left| \lambda_ {2} (P) \right|\right)} \frac {1}{n}\right),
$$
for large enough  $n$ . This can be seen by using  $\alpha = 6$ , setting  $\beta$  as in Eq 4, and taking  $n = \tilde{\Omega} \left(d^{0.1}(\beta)^{1.2}\right)$ , where  $\tilde{\Omega}$  hides logarithmic factors in  $n$ . These choices make the first and third terms of the upper bound on the  $\sin^2$  error in Theorem 1 a smaller order than the second term, which is  $O\left(\frac{1}{n}\right)$ . For the first term, we have  $d\left(\frac{2\beta}{n}\right)^{2\alpha} = O\left(\frac{1}{n^2}\right)$ . The third term involves  $\frac{\tau_{mix}(\eta_n^2)}{n^2}$ , which is  $O\left(\frac{\log^2(n)}{n^2}\right)$ , using the definition of  $\tau_{mix}(\epsilon)$  in Eq 3.
Remark 2. (Knowledge of mixing times) We note that setting the step-sizes  $\eta_{i}$  in Theorem 1 requires the knowledge of the mixing time of the Markov chain,  $\tau_{mix}$ . An offline algorithm for estimating the mixing time is provided in [37]. They show that at least a constant  $\frac{\tau_{mix}|\Omega|}{\epsilon^2}$  samples are needed for estimating  $\tau_{mix}$  within an absolute error  $\epsilon$ . Therefore, it is possible to use the first  $O\left(\frac{1}{\epsilon^2}\right)$  samples to estimate the mixing time and then set the learning rate accordingly.
Remark 3. (Regarding non-reversible Markov chains) We assume the reversibility of Markov chains to simplify our analyses. Our techniques and results extend to the non-reversible case using reversibilization tools (see [20], [11]). Specifically, one can obtain bounds in terms of the second-largest eigenvalue, in magnitude, of the modified transition matrix  $P_{*}P$ , the multiplicative reversibilization of  $P$ . Here,  $P_{*}$  represents the probability transition matrix of the time-reversed chain, given as  $\forall x,y\in \Omega$ ,  $P_{*}(x,y) = \frac{\pi(y)}{\pi(x)} P(y,x)$ . We leave this extension for future work.
Next, we compare the rate of convergence proposed in Theorem 1 with the offline algorithm having access to the entire dataset  $\{X_{i}\}_{i = 1}^{n}$  using a recent result from [28]. Here, the authors extend the Matrix Bernstein inequality [36, 33], to Markovian random matrices. Their setup is much like ours except that the matrix at any state is fixed, i.e., there is no data distribution  $D(s)$  as in our setup. However, it is easy to extend their result to our setting by observing that conditioned on the state sequence, the matrices  $X_{i}X_{i}^{T}, i\in [n]$  are independent under our model, and we can push in the expectation over the state-specific distributions,  $D(s)$ , whenever required. Therefore, we have the following result -
Proposition 1 (Theorem 2.2 of [28] + Wedin's theorem). Fix  $\delta \in (0,1)$ . Consider an irreducible and aperiodic Markov chain. Under assumptions 2 and 3, with probability  $1 - \delta$ , the leading eigenvector  $\hat{v}$  of  $\sum_{i=1}^{n} X_i X_i^T / n$  satisfies,
$$
1 - \left(\hat {v} ^ {T} v _ {1}\right) ^ {2} \leq C _ {1} ^ {\prime} \frac {\mathcal {V} \log \left(\frac {d ^ {2 - \frac {\pi}{4}}}{\delta}\right)}{\left(\lambda_ {1} - \lambda_ {2}\right) ^ {2}} \left(\frac {1 + | \lambda_ {2} (P) |}{1 - | \lambda_ {2} (P) |}\right). \frac {1}{n} + C _ {2} ^ {\prime} \left(\frac {\mathcal {M} \log \left(\frac {d ^ {2 - \frac {\pi}{4}}}{\delta}\right)}{\left(\lambda_ {1} - \lambda_ {2}\right) \left(1 - | \lambda_ {2} (P) |\right)}\right) ^ {2}. \frac {1}{n ^ {2}}
$$
for absolute constants  $C_1^\prime$  and  $C_2^\prime$
Observe that Theorem 1 matches the leading term  $\frac{\nu}{(\lambda_1 - \lambda_2)^2(1 - |\lambda_2(P)|)}$  in Theorem 1 except the  $\log (d)$  term. We believe, much like the IID case (also see footnote 1 in [16]), this logarithmic term in [28]'s result is removable for large  $n$  and a constant probability of success.
Remark 4. (Comparison with IID algorithm) Fix a  $\delta \in (0,1)$ . If the data-points  $\{X_i\}_{i=1}^n$  are sampled IID from the stationary distribution  $\pi$ , then under assumptions 2 and 3, using Theorem 4.1 from [16], we have that the output  $w_n$  of Oja's algorithm 1 satisfies,
$$
1 - \left(w _ {n} ^ {T} v _ {1}\right) ^ {2} \leq \frac {C \log \left(\frac {1}{\delta}\right)}{\delta^ {2}} \left[ d \left(\frac {\beta^ {\prime}}{n}\right) ^ {2 \alpha} + \frac {\alpha^ {' 2} \mathcal {V}}{\left(2 \alpha^ {\prime} - 1\right) \left(\lambda_ {1} - \lambda_ {2}\right) ^ {2}} \frac {1}{n} \right] \tag {5}
$$