Original: Robust Streaming PCA
Revised: Robust Streaming PCA

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We consider streaming principal component analysis (PCA) when the stochasticdata-generating model is subject to perturbations.
Revised: We consider streaming principal component analysis (PCA) when the stochasticdata-generating model is subject to perturbations.

Original: While existing models assume afixed covariance, we adopt a robust perspective where the covariance matrix belongsto a temporal uncertainty set.
Revised: While existing models assume afixed covariance, we adopt a robust perspective where the covariance matrix belongsto a temporal uncertainty set.

Original: Under this setting, we provide fundamental limitson any algorithm recovering principal components.
Revised: Under this setting, we provide fundamental limitson any algorithm recovering principal components.

Original: We analyze the convergenceof the noisy power method and Oja’s algorithm, both analyzed for the stationarydata generating model, and argue that the noisy power method is rate-optimal inour setting.
Revised: We analyze the convergenceof the noisy power method and Oja’s algorithm, both analyzed for the stationarydata generating model, and argue that the noisy power method is rate-optimal inour setting.

Original: Finally, we demonstrate the validity of our analysis through numericalexperiments.
Revised: Finally, we demonstrate the validity of our analysis through numericalexperiments.

Original:  Introduction
Revised:  Introduction

Original: Principal component analysis (PCA) is one of the most extensively studied methods for obtaining thelow-dimensional representation of observed data [21].
Revised: Principal component analysis (PCA) is one of the most extensively studied methods for obtaining thelow-dimensional representation of observed data [21].

Original: However, classical algorithms for PCA storeall the observations and use cubic-time complexity, thereby imposing prohibitively large computationtime and space requirements.
Revised: However, classical algorithms for PCA storeall the observations and use cubic-time complexity, thereby imposing prohibitively large computationtime and space requirements.

Original: Recently, several works on PCA have focused on the design and analysis of single-pass (streaming)algorithms with near-optimal memory and storage complexity [40, 43, 24, 28].
Revised: Recently, several works on PCA have focused on the design and analysis of single-pass (streaming)algorithms with near-optimal memory and storage complexity [40, 43, 24, 28].

Original: These algorithmsassume that all the observations belong to the same low-dimensional space.
Revised: These algorithmsassume that all the observations belong to the same low-dimensional space.

Original: However, this situationis unlikely when the unknown/unexplored alterations corrupt a system’s observations.
Revised: However, this situationis unlikely when the unknown/unexplored alterations corrupt a system’s observations.

Original: For instance,it is well known that the typical data attacks on power grids can significantly change the estimatedcovariance matrix of the data observed from sensors [8, 22, 9].
Revised: For instance,it is well known that the typical data attacks on power grids can significantly change the estimatedcovariance matrix of the data observed from sensors [8, 22, 9].

Original: Similarly, PCA can be used to explainstock returns in terms of the macroeconomic factors [26], product pricing taking into account crossproduct elasticity and demands [45], etc.
Revised: Similarly, PCA can be used to explainstock returns in terms of the macroeconomic factors [26], product pricing taking into account crossproduct elasticity and demands [45], etc.

Original: In all these scenarios the underlying data generating modelchanges at every time instant and the decisions are based on identifying the changed model.
Revised: In all these scenarios the underlying data generating modelchanges at every time instant and the decisions are based on identifying the changed model.

Original: Recent work considers perturbations of the data lying in a fixed low-dimensional space [49].
Revised: Recent work considers perturbations of the data lying in a fixed low-dimensional space [49].

Original: Theydetermine the worst-case position of the adversarial data-point to incur the maximum error in thesubspace estimated through PCA and measure the distance between the two subspaces using thenotion of the principal angle between them.
Revised: Theydetermine the worst-case position of the adversarial data-point to incur the maximum error in thesubspace estimated through PCA and measure the distance between the two subspaces using thenotion of the principal angle between them.

Original: Another line of work considers PCA through the lens ofstochastic optimization [51, 52, 42, 4].
Revised: Another line of work considers PCA through the lens ofstochastic optimization [51, 52, 42, 4].

Original: Our work differs from these approaches since we assume thatthe data generating model changes at every time instant.
Revised: Our work differs from these approaches since we assume thatthe data generating model changes at every time instant.

Original: Further, we propose near-optimal algorithmsfor recovering principal components under this framework.
Revised: Further, we propose near-optimal algorithmsfor recovering principal components under this framework.

Original: We assume a system relies on the time-series of p -dimensional vectors sampled from the time-variantmodel.
Revised: We assume a system relies on the time-series of p -dimensional vectors sampled from the time-variantmodel.

Original: The available observations are the vectors { x t } Tt =1 .
Revised: The available observations are the vectors { x t } Tt =1 .

Original: We assume that T
Revised: We assume that T

Original: ≫ p ; justifying theuse of streaming (single-pass) algorithms.
Revised: ≫ p ; justifying theuse of streaming (single-pass) algorithms.

Original: The noisy observation x t ∈ R p is a vector lying in thecolumn space of an unobserved full-rank matrix A t ∈ R p × k .
Revised: The noisy observation x t ∈ R p is a vector lying in thecolumn space of an unobserved full-rank matrix A t ∈ R p × k .

Original: Precisely, from the standard spikedcovariance model [35], we consider the time-dependent environment:
Revised: Precisely, from the standard spikedcovariance model [35], we consider the time-dependent environment:

Original: [Equation]
Revised: [Equation]

Original: where A t can vary with time.
Revised: where A t can vary with time.

Original: The parameter k ≪ p is the desired number of principal components.
Revised: The parameter k ≪ p is the desired number of principal components.

Original: Assuming A t belongs to a temporal uncertainty set (defined in (2)), our goal is to recover the top- kprincipal components of the terminal subspace A T .
Revised: Assuming A t belongs to a temporal uncertainty set (defined in (2)), our goal is to recover the top- kprincipal components of the terminal subspace A T .

Original: In financial applications, our model capturesthe market evolution, in terms of the changing A T with the ultimate goal of explaining the marketconditions for an appropriately chosen T .
Revised: In financial applications, our model capturesthe market evolution, in terms of the changing A T with the ultimate goal of explaining the marketconditions for an appropriately chosen T .

Original: Previous works on the stationary environment assume A t =
Revised: Previous works on the stationary environment assume A t =

Original: A for all t and have focused oncomputing a basis for the column space of the matrix A using streaming algorithms [43, 28, 38,32, 20, 36].
Revised: A for all t and have focused oncomputing a basis for the column space of the matrix A using streaming algorithms [43, 28, 38,32, 20, 36].

Original: The accurate reconstruction of the principal components for the standard streaming
Revised: The accurate reconstruction of the principal components for the standard streaming

Original: PCA problem depends on the magnitude of observation noise σ , the dimension of observations p ,the number of principal components k of the matrix A , and the spectral gap δ between k th andk +1 th singular value of A .
Revised: PCA problem depends on the magnitude of observation noise σ , the dimension of observations p ,the number of principal components k of the matrix A , and the spectral gap δ between k th andk +1 th singular value of A .

Original: In a marked departure from previous work, we study the case when thecolumn space of A varies across time, possibly perturbed by Γ .
Revised: In a marked departure from previous work, we study the case when thecolumn space of A varies across time, possibly perturbed by Γ .

Original: This paper explores these avenuesand proposes a tractable analysis framework for the streaming PCA problem, robust to perturbationsin the stochastic data-generating model.
Revised: This paper explores these avenuesand proposes a tractable analysis framework for the streaming PCA problem, robust to perturbationsin the stochastic data-generating model.

Original: Our contributions can be summarized as:(a)
Revised: Our contributions can be summarized as:(a)

Original: (Lower Bound; Section 4)
Revised: (Lower Bound; Section 4)

Original: Our first contribution is establishing a fundamental lower boundfor estimating the principal components when the covariance matrix belongs to a temporaluncertainty set.
Revised: Our first contribution is establishing a fundamental lower boundfor estimating the principal components when the covariance matrix belongs to a temporaluncertainty set.

Original: In Theorem 1, we derive the minimax bounds of the expected error forrecovering the top singular vectors for any algorithm in the class Φ .
Revised: In Theorem 1, we derive the minimax bounds of the expected error forrecovering the top singular vectors for any algorithm in the class Φ .

Original: A key componentof this analysis involves carefully constructed instances to elicit the worst-case behaviorof any algorithm in Φ .
Revised: A key componentof this analysis involves carefully constructed instances to elicit the worst-case behaviorof any algorithm in Φ .

Original: We show that for T
Revised: We show that for T

Original: =
Revised: =

Original: O (Γ − 2 / 3 ) , the minimax estimation errordecreases as O ( p 1 / 2 /δ √ T ) .
Revised: O (Γ − 2 / 3 ) , the minimax estimation errordecreases as O ( p 1 / 2 /δ √ T ) .

Original: On the other hand, for T = Ω(Γ − 2 / 3 ) , the estimation errorstagnates to O ( p 1 / 3 Γ 1 / 3 /δ ) and does not decrease upon collecting more observations 1 .
Revised: On the other hand, for T = Ω(Γ − 2 / 3 ) , the estimation errorstagnates to O ( p 1 / 3 Γ 1 / 3 /δ ) and does not decrease upon collecting more observations 1 .

Original: Since observations can belong to potentially different spaces, the estimation of principalcomponents associated with A T should be determined by a subset of samples.
Revised: Since observations can belong to potentially different spaces, the estimation of principalcomponents associated with A T should be determined by a subset of samples.

Original: This is insharp contrast to the standard spiked covariance model (where data is generated from thefixed model), observations from the far past become less important when the covariancematrix is time-variant, and it becomes imperative to find the block size B that can be usedto recover the principal components.(b) (Algorithm Analysis; Section 5)
Revised: This is insharp contrast to the standard spiked covariance model (where data is generated from thefixed model), observations from the far past become less important when the covariancematrix is time-variant, and it becomes imperative to find the block size B that can be usedto recover the principal components.(b) (Algorithm Analysis; Section 5)

Original: We then analyze two algorithms to recover the principalcomponents extensively used in the standard streaming PCA setting; the noisy powermethod [28] and Oja’s algorithm [48].
Revised: We then analyze two algorithms to recover the principalcomponents extensively used in the standard streaming PCA setting; the noisy powermethod [28] and Oja’s algorithm [48].

Original: These algorithms represent two very different designprinciples for computing principal components from data; processing data in blocks visà-vis single observations (However, note that both algorithms perform computations in astreaming fashion, i.e., have O ( kp ) storage requirements).
Revised: These algorithms represent two very different designprinciples for computing principal components from data; processing data in blocks visà-vis single observations (However, note that both algorithms perform computations in astreaming fashion, i.e., have O ( kp ) storage requirements).

Original: We first determine the optimalchoice for critical parameters: the block size in the noisy power method on Lemma 1 andthe learning rate in Oja’s algorithm on Lemma 2.
Revised: We first determine the optimalchoice for critical parameters: the block size in the noisy power method on Lemma 1 andthe learning rate in Oja’s algorithm on Lemma 2.

Original: Next, we leverage these results in orderto obtain an upper bound on the convergence error for these algorithms in Theorem 2 and
Revised: Next, we leverage these results in orderto obtain an upper bound on the convergence error for these algorithms in Theorem 2 and

Original: Theorem 3respectively.
Revised: Theorem 3 respectively.

Original: From these results, we have that: (i) the block size B for the noisypower method and inverse of the learning rate, 1 /ζ for Oja’s algorithm play a similar roleeven in the non-stationary environment; (ii) In either case, the optimal parameters scale with∼
Revised: From these results, we have that: (i) the block size B for the noisypower method and inverse of the learning rate, 1 /ζ for Oja’s algorithm play a similar roleeven in the non-stationary environment; (ii) In either case, the optimal parameters scale with∼

Original: Γ − 2 / 3 , where Γ is perturbation budget which will be defined later, and (iii) the derivedupper bound on the estimation error for noisy power method matches the minimax error interms of p, δ, and Γ and becomes rate-optimal if it satisfies mild condition.
Revised: Γ − 2 / 3 , where Γ is perturbation budget which will be defined later, and (iii) the derivedupper bound on the estimation error for noisy power method matches the minimax error interms of p, δ, and Γ and becomes rate-optimal if it satisfies mild condition.

Original: Notation We fix notation throughout the main body of the paper.
Revised: Notation We fix notation throughout the main body of the paper.

Original: Matrices are denoted by bolduppercase letters ( M ) and vectors are denoted by bold lowercase letters ( x ).
Revised: Matrices are denoted by bolduppercase letters ( M ) and vectors are denoted by bold lowercase letters ( x ).

Original: Let M ∈ R p × k .
Revised: Let M ∈ R p × k .

Original: Then
Revised: Then

Original: M i denote the i th column of M , M i : j denote the submatrix consists with i th ∼ j th column of M and
Revised: M i denote the i th column of M , M i : j denote the submatrix consists with i th ∼ j th column of M and

Original: M i,j be the
Revised: M i,j be the

Original: ( i, j ) -element of M .
Revised: ( i, j ) -element of M .

Original: ∥·∥ denote matrix 2-norm or equivalently operator norm for matricesand standard 2-norm for vectors.
Revised: ∥·∥ denote matrix 2-norm or equivalently operator norm for matricesand standard 2-norm for vectors.

Original: We use St k ( R p ) for the class of orthogonal matrices in R p × k .b
Revised: We use St k ( R p ) for the class of orthogonal matrices in R p × k .b

Original: ( M ) is the orthogonal matrix where the columns form basis for ran( M ) .
Revised: ( M ) is the orthogonal matrix where the columns form basis for ran( M ) .

Original: λ i ( M ) represents the i thlargest eigenvalue of the matrix M and and s i ( M ) be the i th largest singular value.
Revised: λ i ( M ) represents the i thlargest eigenvalue of the matrix M and and s i ( M ) be the i th largest singular value.

Original: The singularvalue decomposition of M is defined as SVD( M ) =
Revised: The singularvalue decomposition of M is defined as SVD( M ) =

Original: UDV ⊤ , where U ∈ St p ( R p ) ,
Revised: UDV ⊤ , where U ∈ St p ( R p ) ,

Original: V ∈ St k
Revised: V ∈ St k

Original: ( R k ) , and
Revised: ( R k ) , and

Original: D ∈ R p × k is a diagonal matrix whose i th diagonal element equals s i ( M ) .
Revised: D ∈ R p × k is a diagonal matrix whose i th diagonal element equals s i ( M ) .

Original: Therefore, we assumewithout loss of generality that the singular values and respective singular vectors are ordered fromlargest to smallest.
Revised: Therefore, we assumewithout loss of generality that the singular values and respective singular vectors are ordered fromlargest to smallest.

Original: We denote by M ⊥ the orthogonal projection onto the orthogonal complement of the range of M .
Revised: We denote by M ⊥ the orthogonal projection onto the orthogonal complement of the range of M .

Original: Therefore, if rk( M ) =
Revised: Therefore, if rk( M ) =

Original: k and SVD( M )
Revised: k and SVD( M )

Original: =
Revised: =

Original: UDV ⊤ , M ⊥ given by M ⊥ =
Revised: UDV ⊤ , M ⊥ given by M ⊥ =

Original: I − U 1: k U ⊤ 1: k .
Revised: I − U 1: k U ⊤ 1: k .

Original: Moreover, when rk( M ) = rk( ˜ M ) and SVD( ˜ M ) = ˜
Revised: Moreover, when rk( M ) = rk( ˜ M ) and SVD( ˜ M ) = ˜

Original: U
Revised: U

Original: ˜ D ˜ V ⊤ , the distance between ran( M ) and ran( ˜ M )is defined by:
Revised: ˜ D ˜ V ⊤ , the distance between ran( M ) and ran( ˜ M )is defined by:

Original: [Equation]
Revised: [Equation]

Original: We denote d (ran( M ) , ran( ˜ M )) by d ( M , ˜ M ) whenever clear from the context.
Revised: We denote d (ran( M ) , ran( ˜ M )) by d ( M , ˜ M ) whenever clear from the context.

Original: For
Revised: For

Original: A =
Revised: A =

Original: ( A 1 , . . .
Revised: ( A 1 , . . .

Original: , A T ) , we denote the expectation of f ( x 1 , . . .
Revised: , A T ) , we denote the expectation of f ( x 1 , . . .

Original: , x T ) over x t ∼ N ( 0 , A t
Revised: , x T ) over x t ∼ N ( 0 , A t

Original: A ⊤ t + σ 2 I )as E X∼A [ f ] .
Revised: A ⊤ t + σ 2 I )as E X∼A [ f ] .

Original: We denote ˜ O
Revised: We denote ˜ O

Original: as the big-
Revised: as the big-

Original: O notation with ignore the multiplicative dependency oflog( p ) + 2 log( T ) and 1 + Θ(log( T/ ∆) /k ) .
Revised: O notation with ignore the multiplicative dependency oflog( p ) + 2 log( T ) and 1 + Θ(log( T/ ∆) /k ) .

Original:  Related Work
Revised:  Related Work

Original: Principal component analysis has been extensively studied across operations research, computerscience, and other communities.
Revised: Principal component analysis has been extensively studied across operations research, computerscience, and other communities.

Original: We highlight how our work differs from existing literature.
Revised: We highlight how our work differs from existing literature.

Original: Robust
Revised: Robust

Original: PCA Robust PCA deals with the problem of retrieving the principal components robust to thepresence of outliers in the data.
Revised: PCA Robust PCA deals with the problem of retrieving the principal components robust to thepresence of outliers in the data.

Original: The cornerstone work in this direction is the Principal Component
Revised: The cornerstone work in this direction is the Principal Component

Original: Pursuit framework wherein they assume that the matrix of observations M can be decomposed interms of a low-rank matrix L and a sparse matrix (with entries of arbitrarily large magnitude) S [14].
Revised: Pursuit framework wherein they assume that the matrix of observations M can be decomposed interms of a low-rank matrix L and a sparse matrix (with entries of arbitrarily large magnitude) S [14].

Original: Several works [23, 25, 16, 46] consider the robust PCA problem and propose algorithms in the offline,batch, and online setting.
Revised: Several works [23, 25, 16, 46] consider the robust PCA problem and propose algorithms in the offline,batch, and online setting.

Original: Our work differs from this line of literature in two aspects: assumptionsabout the data generating model and convex optimization techniques.
Revised: Our work differs from this line of literature in two aspects: assumptionsabout the data generating model and convex optimization techniques.

Original: First, our data generation modelis unrelated to those considered in the robust PCA literature.
Revised: First, our data generation modelis unrelated to those considered in the robust PCA literature.

Original: Further, rich theories from convexoptimization can be used in the PCA framework to design efficient algorithms, but our problem is notamenable to such techniques.
Revised: Further, rich theories from convexoptimization can be used in the PCA framework to design efficient algorithms, but our problem is notamenable to such techniques.

Original: Streaming PCA Streaming algorithms for PCA have been proposed among other works on PCA [43,34, 28, 59, 2].
Revised: Streaming PCA Streaming algorithms for PCA have been proposed among other works on PCA [43,34, 28, 59, 2].

Original: Algorithms analyzed in this work, such as the noisy power method [28] and Oja’salgorithm [48], are iterative methods for estimating the principal components.
Revised: Algorithms analyzed in this work, such as the noisy power method [28] and Oja’salgorithm [48], are iterative methods for estimating the principal components.

Original: These iterative schemesare instances of stochastic approximation-based solutions for the optimization version of the PCAproblem [4].
Revised: These iterative schemesare instances of stochastic approximation-based solutions for the optimization version of the PCAproblem [4].

Original: The stochastic approximation is a root-finding framework extensively used for stochasticoptimization [37, 10].
Revised: The stochastic approximation is a root-finding framework extensively used for stochasticoptimization [37, 10].

Original: Oja’s algorithm, originally proposed by [48], was the first such scheme.
Revised: Oja’s algorithm, originally proposed by [48], was the first such scheme.

Original: This framework is also used to analyze gradient-type and proximal-type incremental methods akin toalgorithms for convex optimization.
Revised: This framework is also used to analyze gradient-type and proximal-type incremental methods akin toalgorithms for convex optimization.

Original: Stochastic gradient descent-based algorithms for the streaming
Revised: Stochastic gradient descent-based algorithms for the streaming

Original: PCA problem, where a single observation is used at every point in time to update the principalcomponents’ estimate, have been extensively studied.
Revised: PCA problem, where a single observation is used at every point in time to update the principalcomponents’ estimate, have been extensively studied.

Original: Along this line, [29] propose GRASTA, anincremental online gradient method for learning over different subspaces.
Revised: Along this line, [29] propose GRASTA, anincremental online gradient method for learning over different subspaces.

Original: Similarly, [5] proposes
Revised: Similarly, [5] proposes

Original: GROUSE, based on the idea of gradient updates over the Grassmannian manifold.
Revised: GROUSE, based on the idea of gradient updates over the Grassmannian manifold.

Original: In these and otherrelated works [53, 15, 60, 30, 30, 1, 58] consider Oja’s algorithm for the standard streaming PCAproblem.
Revised: In these and otherrelated works [53, 15, 60, 30, 30, 1, 58] consider Oja’s algorithm for the standard streaming PCAproblem.

Original: However, all these works are based on a completely different modeling assumption thanthis paper and do not provide theoretical guarantees for our setting.
Revised: However, all these works are based on a completely different modeling assumption thanthis paper and do not provide theoretical guarantees for our setting.

Original: Streaming and robust PCA algorithms are used in presence of outliers or data with a lot of missingentries [17, 54].
Revised: Streaming and robust PCA algorithms are used in presence of outliers or data with a lot of missingentries [17, 54].

Original: From this literature, closest to our work is the work on robust subspace tracking [2,59, 29].
Revised: From this literature, closest to our work is the work on robust subspace tracking [2,59, 29].

Original: However, the robustness considered their is against erasures or sparse outliers.
Revised: However, the robustness considered their is against erasures or sparse outliers.

Original: While thesealgorithms provide theoretical guarantees in that setting, those guarantees cannot be extended to ourmodel.
Revised: While thesealgorithms provide theoretical guarantees in that setting, those guarantees cannot be extended to ourmodel.

Original: Considerations of erasures and outliers under the model proposed in this paper is beyond thescope of this work and remains an interesting future direction.
Revised: Considerations of erasures and outliers under the model proposed in this paper is beyond thescope of this work and remains an interesting future direction.

Original:  Mathematical Framework
Revised:  Mathematical Framework

Original: Observations from the standard spiked covariance model belong to a fixed k -dimensional columnspace A ∈ R p .
Revised: Observations from the standard spiked covariance model belong to a fixed k -dimensional columnspace A ∈ R p .

Original: Previous work has focused on reconstructing this space from the observed timeseries.
Revised: Previous work has focused on reconstructing this space from the observed timeseries.

Original: Under our framework, we assume that the sequence of observations ( x t )
Revised: Under our framework, we assume that the sequence of observations ( x t )

Original: Tt =1 follows the timedependent spiked covariance model (1).
Revised: Tt =1 follows the timedependent spiked covariance model (1).

Original: We consider the problem of computing top-
Revised: We consider the problem of computing top-

Original: k singular vectorsof A T .
Revised: k singular vectorsof A T .

Original: We formulate this model by addressing a minimax optimization as a robust optimizationproblem.
Revised: We formulate this model by addressing a minimax optimization as a robust optimizationproblem.

Original: When the adversary is allowed to select a completely arbitrary sequence of matrices A 1 , . .
Revised: When the adversary is allowed to select a completely arbitrary sequence of matrices A 1 , . .

Original: A T , itis impossible to accurately recover the column space of A T since only the single vector x T providesinformation about this space.
Revised: A T , itis impossible to accurately recover the column space of A T since only the single vector x T providesinformation about this space.

Original: Instead, we define temporal uncertainty sets to restrict the power of theadversary.
Revised: Instead, we define temporal uncertainty sets to restrict the power of theadversary.

Original: Definition 1.
Revised: Definition 1.

Original: Let Γ , δ ≥ 0 .
Revised: Let Γ , δ ≥ 0 .

Original: We only allow the sequence of matrices
Revised: We only allow the sequence of matrices

Original: A 1 , . . .
Revised: A 1 , . . .

Original: A T that lie in antemporal uncertainty set Tu ( δ, Γ) defined as:
Revised: A T that lie in antemporal uncertainty set Tu ( δ, Γ) defined as:

Original: [Equation]
Revised: [Equation]

Original: That is, we constrain the difference between any two consecutive covariance matrices of the underlying(noise-free) process.
Revised: That is, we constrain the difference between any two consecutive covariance matrices of the underlying(noise-free) process.

Original: The assumption s k
Revised: The assumption s k

Original: ( A t A ⊤ t ) ≥ δ is crucial for establishing bounds of estimationerror.
Revised: ( A t A ⊤ t ) ≥ δ is crucial for establishing bounds of estimationerror.

Original: It is also justified in any setting where the underlying phenomenology guarantees covariance ofrank k; the stated lower bound excludes pathological cases of near-rank k − 1 or smaller.
Revised: It is also justified in any setting where the underlying phenomenology guarantees covariance ofrank k; the stated lower bound excludes pathological cases of near-rank k − 1 or smaller.

Original: Having described the constraints on the perturbation power, we now present the family of algorithmsand the performance metric of interest.
Revised: Having described the constraints on the perturbation power, we now present the family of algorithmsand the performance metric of interest.

Original: In particular, we consider algorithms that are (a) streaming,i.e., make a single pass through the time series in chronological order; only the previous samples arestored at any point in the past.
Revised: In particular, we consider algorithms that are (a) streaming,i.e., make a single pass through the time series in chronological order; only the previous samples arestored at any point in the past.

Original: These characteristics model the behavior of an algorithm that receivesdata in real-time and only stores samples in limited memory.
Revised: These characteristics model the behavior of an algorithm that receivesdata in real-time and only stores samples in limited memory.

Original: Among those streaming algorithms, weconsider the algorithms whose (b) output is a set of k orthonormal vectors in R p .
Revised: Among those streaming algorithms, weconsider the algorithms whose (b) output is a set of k orthonormal vectors in R p .

Original: The first requirementensures that the storage and computational memory requirements are independent of the number ofsamples T , and hence, the assumption T ≫ p is not prohibitive.
Revised: The first requirementensures that the storage and computational memory requirements are independent of the number ofsamples T , and hence, the assumption T ≫ p is not prohibitive.

Original: Let us denote by Φ the family ofalgorithms just described.
Revised: Let us denote by Φ the family ofalgorithms just described.

Original: Let A
Revised: Let A

Original: =
Revised: =

Original: ( A 1 , . . .
Revised: ( A 1 , . . .

Original: , A T ) ∈
Revised: , A T ) ∈

Original: Tu ( δ, Γ) be given.
Revised: Tu ( δ, Γ) be given.

Original: We formalize this definitionas follows:
Revised: We formalize this definitionas follows:

Original: Definition 2.
Revised: Definition 2.

Original: Σ( A ) is the time-dependent spiked covariance model represented by the A =(
Revised: Σ( A ) is the time-dependent spiked covariance model represented by the A =(

Original: A 1 , . . .
Revised: A 1 , . . .

Original: , A T ) .
Revised: , A T ) .

Original: Furthermore, X :=
Revised: Furthermore, X :=

Original: ( x 1 , . .
Revised: ( x 1 , . .

Original: , x T ) ∼ Σ( A ) is a sequence of observation vectorsdrawn from Σ( A ) .
Revised: , x T ) ∼ Σ( A ) is a sequence of observation vectorsdrawn from Σ( A ) .

Original: The output of algorithm ϕ for a given choice of the observations x t sampled using model (2) is a setof k orthonormal vectors which we denote by ϕ X ; abusing notation we will also view ϕ X as a matrixin R p × k or the subspace generated by ϕ X .
Revised: The output of algorithm ϕ for a given choice of the observations x t sampled using model (2) is a setof k orthonormal vectors which we denote by ϕ X ; abusing notation we will also view ϕ X as a matrixin R p × k or the subspace generated by ϕ X .

Original: Definition 3.
Revised: Definition 3.

Original: Let ϕ ∈ Φ ,
Revised: Let ϕ ∈ Φ ,

Original: A ∈ Tu ( δ, Γ) and X ∼ Σ( A ) .
Revised: A ∈ Tu ( δ, Γ) and X ∼ Σ( A ) .

Original: (a) Given A and X , the estimation error of ϕ given X is the distance between the space spannedby ϕ X and the column space of A T (= d ( ran ( A T ) , ϕ X ) ).
Revised: (a) Given A and X , the estimation error of ϕ given X is the distance between the space spannedby ϕ X and the column space of A T (= d ( ran ( A T ) , ϕ X ) ).

Original: The metric can be easily extended
Revised: The metric can be easily extended

Original: to the cumulative error.
Revised: to the cumulative error.

Original: Please see the Appendix A for the discussion.
Revised: Please see the Appendix A for the discussion.

Original: (b) R ϕ is the maximum expected estimation error of ϕ under A over all A ∈ Tu ( δ, Γ) .
Revised: (b) R ϕ is the maximum expected estimation error of ϕ under A over all A ∈ Tu ( δ, Γ) .

Original: Precisely,
Revised: Precisely,

Original: [Equation]
Revised: [Equation]

Original: (c) R ∗ is the minimax estimation error defined as the minimum of the largest expected estima-
Revised: (c) R ∗ is the minimax estimation error defined as the minimum of the largest expected estima-

Original: tion error incurred by ϕ ∈ Φ :
Revised: tion error incurred by ϕ ∈ Φ :

Original: [Equation]
Revised: [Equation]

Original: We note that minimax estimation problems have been widely studied [56, 13, 57].
Revised: We note that minimax estimation problems have been widely studied [56, 13, 57].

Original: d
Revised: d

Original: An algorithm
Revised: An algorithm

Original: ϕ ∈ Φ
Revised: ϕ ∈ Φ

Original: is rate-optimal
Revised: is rate-optimal

Original: if
Revised: if

Original: R ϕ ≤ C · R ∗ , where constant C > 0 is independentof the problem parameters Γ , T , δ , p , k .
Revised: R ϕ ≤ C · R ∗ , where constant C > 0 is independentof the problem parameters Γ , T , δ , p , k .

Original: In this paper we establish the minimax estimation error R ∗ and propose rate-optimal sublinear-time, single-pass algorithms for robuststreaming PCA.
Revised: In this paper we establish the minimax estimation error R ∗ and propose rate-optimal sublinear-time, single-pass algorithms for robuststreaming PCA.

Original:  Minimax Lower Bound
Revised:  Minimax Lower Bound

Original: When A t belongs to the temporal uncertainty set Tu ( δ, Γ) (Definition 1) an algorithm designed torecover the principal components of A T from the observations cannot guarantee zero estimation error.
Revised: When A t belongs to the temporal uncertainty set Tu ( δ, Γ) (Definition 1) an algorithm designed torecover the principal components of A T from the observations cannot guarantee zero estimation error.

Original: Our first goal is to obtain a lower bound on the estimation error in terms of the problem parameters δ, Γ , p and the number of observations T .
Revised: Our first goal is to obtain a lower bound on the estimation error in terms of the problem parameters δ, Γ , p and the number of observations T .

Original: We now establish a lower bound on the expected error inrecovering the top singular vector of A T from observations X =
Revised: We now establish a lower bound on the expected error inrecovering the top singular vector of A T from observations X =

Original: ( x t )
Revised: ( x t )

Original: Tt =1 .
Revised: Tt =1 .

Original: In order to establish the lower bound, we leverage the fundamental limit of hypothesis tests [56].
Revised: In order to establish the lower bound, we leverage the fundamental limit of hypothesis tests [56].

Original: The crux of the proof lies in constructing the set of worst-case hypotheses and establishing a lowerbound on the probability of error in distinguishing between these hypotheses using observed data.
Revised: The crux of the proof lies in constructing the set of worst-case hypotheses and establishing a lowerbound on the probability of error in distinguishing between these hypotheses using observed data.

Original: The complete proof is provided in Appendix C.
Revised: The complete proof is provided in Appendix C.

Original: Theorem 1 (Lower Bound) .
Revised: Theorem 1 (Lower Bound) .

Original: Assume 0 ≤ Γ < δ and T ≥ 1 .
Revised: Assume 0 ≤ Γ < δ and T ≥ 1 .

Original: For any algorithm ϕ ∈ Φ , there exists
Revised: For any algorithm ϕ ∈ Φ , there exists

Original: asequence A ∈ Tu ( δ, Γ) and constant C > 0 which is independent with Γ , δ, T, p, k such that:
Revised: asequence A ∈ Tu ( δ, Γ) and constant C > 0 which is independent with Γ , δ, T, p, k such that:

Original: [Equation]
Revised: [Equation]

Original: For the standard streaming PCA problem (Theorem 1 with the case of Γ = 0 ), the fundamental limitis Θ(( σ/δ )( p ( σ 2 + δ ) /T )
Revised: For the standard streaming PCA problem (Theorem 1 with the case of Γ = 0 ), the fundamental limitis Θ(( σ/δ )( p ( σ 2 + δ ) /T )

Original: 1 / 2 ) which has the expected Θ(1 / √ T ) dependence [13, 57].
Revised: 1 / 2 ) which has the expected Θ(1 / √ T ) dependence [13, 57].

Original: On the otherhand, in presence of perturbations ( Γ > 0 ), the lower bound exhibits a phase transition phenomenon,with the first term representing the effect of model ambiguity.
Revised: On the otherhand, in presence of perturbations ( Γ > 0 ), the lower bound exhibits a phase transition phenomenon,with the first term representing the effect of model ambiguity.

Original: To this end, define the critical time T cas
Revised: To this end, define the critical time T cas

Original: [Equation]
Revised: [Equation]

Original: For T
Revised: For T

Original: =
Revised: =

Original: O ( T c ) , the lower bound decreases with the rate of 1 / √ T .
Revised: O ( T c ) , the lower bound decreases with the rate of 1 / √ T .

Original: However, when T = Ω( T c ) , thefirst term dominates the second term and R ∗ becomes independent to the number of observations T .
Revised: However, when T = Ω( T c ) , thefirst term dominates the second term and R ∗ becomes independent to the number of observations T .

Original: In this regime, the error stagnates to O ((Γ /δ ) 1 / 3 (cid:0) pσ 2 ( σ 2 + δ ) /δ
Revised: In this regime, the error stagnates to O ((Γ /δ ) 1 / 3 (cid:0) pσ 2 ( σ 2 + δ ) /δ

Original: 2 (cid:1) 1 / 3 ) .
Revised: 2 (cid:1) 1 / 3 ) .

Original: This validates our intuitionthat in a dynamic environment past information quickly becoming stale.
Revised: This validates our intuitionthat in a dynamic environment past information quickly becoming stale.

Original: In Theorem 2 and 3, we will prove that the noisy power method and Oja’s algorithm attain a nearoptimal bound on the convergence guarantee.
Revised: In Theorem 2 and 3, we will prove that the noisy power method and Oja’s algorithm attain a nearoptimal bound on the convergence guarantee.

Original: Theorem 2 guarantees that if s 1 ( A t A ⊤ t )
Revised: Theorem 2 guarantees that if s 1 ( A t A ⊤ t )

Original: = δ , theupper bound for estimation error on the noisy power method is of the following order:
Revised: = δ , theupper bound for estimation error on the noisy power method is of the following order:

Original: [Equation]
Revised: [Equation]

Original: That is, if pσ 2 dominates kδ , the noisy power method becomes rate-optimal under the controlleduncertainty with Tu ( δ, Γ) .
Revised: That is, if pσ 2 dominates kδ , the noisy power method becomes rate-optimal under the controlleduncertainty with Tu ( δ, Γ) .

Original: This regime is the case of practical noisy situations, with σ 2 ̸≪ δ .
Revised: This regime is the case of practical noisy situations, with σ 2 ̸≪ δ .

Original: Observealso that we have p > k from the setting.
Revised: Observealso that we have p > k from the setting.

Original:  Convergence Analysis
Revised:  Convergence Analysis

Original: In this section, we analyze two algorithms for the robust streaming PCA problem.
Revised: In this section, we analyze two algorithms for the robust streaming PCA problem.

Original: A generic templatefor algorithms ϕ ∈
Revised: A generic templatefor algorithms ϕ ∈

Original: Φ of interest to us is as follows: (i) ϕ is initialized with a random matrix withorthonormal columns ˆ U ∈ R p × k ; (ii) A running estimate of the principal components is maintainedas the columns of ˆ U ; (iii) Observations are projected onto the column space of ˆ U to update thisestimate.
Revised: Φ of interest to us is as follows: (i) ϕ is initialized with a random matrix withorthonormal columns ˆ U ∈ R p × k ; (ii) A running estimate of the principal components is maintainedas the columns of ˆ U ; (iii) Observations are projected onto the column space of ˆ U to update thisestimate.

Original: We consider two algorithms: robust version of the noisy power method (Algorithm 1) and Oja’salgorithm (Algorithm 2).
Revised: We consider two algorithms: robust version of the noisy power method (Algorithm 1) and Oja’salgorithm (Algorithm 2).

Original: The key difference between the noisy power method and Oja’s algorithmis in the data used to estimate the principal components.
Revised: The key difference between the noisy power method and Oja’s algorithmis in the data used to estimate the principal components.

Original: In the noisy power method, the estimatesare updated after a batch of observations, whereas in Oja’s algorithm, the estimates are updatedafter scaling every observation with the learning rate.
Revised: In the noisy power method, the estimatesare updated after a batch of observations, whereas in Oja’s algorithm, the estimates are updatedafter scaling every observation with the learning rate.

Original: Therefore, the parameters determining theperformance of these algorithms are the batch size B for the robust power method and the learningrate ζ for Oja’s algorithm.
Revised: Therefore, the parameters determining theperformance of these algorithms are the batch size B for the robust power method and the learningrate ζ for Oja’s algorithm.

Original: The analysis of these algorithms cannot be readily established with existing techniques when thecovariance matrix belongs to a temporal uncertainty set since they rely on showing that the estimatesimprove every iteration.
Revised: The analysis of these algorithms cannot be readily established with existing techniques when thecovariance matrix belongs to a temporal uncertainty set since they rely on showing that the estimatesimprove every iteration.

Original: Further, the application of many concentration results requires randommatrices to be bounded which is not the case when the observations are sampled from (1).
Revised: Further, the application of many concentration results requires randommatrices to be bounded which is not the case when the observations are sampled from (1).

Original: Therefore,in order to simplify the analysis on both algorithms, we introduce Assumption 1, adapted from [34].
Revised: Therefore,in order to simplify the analysis on both algorithms, we introduce Assumption 1, adapted from [34].

Original: Assumption 1.
Revised: Assumption 1.

Original: Let ( A t )
Revised: Let ( A t )

Original: Tt =
Revised: Tt =

Original: 1 ∈ Tu ( δ, Γ) .
Revised: 1 ∈ Tu ( δ, Γ) .

Original: For ˜ δ ≥ δ and M , V > 0 , we consider the observations x tfor t ∈
Revised: For ˜ δ ≥ δ and M , V > 0 , we consider the observations x tfor t ∈

Original: [ T ] satisfy the following:
Revised: [ T ] satisfy the following:

Original: (a) E [ x t x ⊤ t ] =
Revised: (a) E [ x t x ⊤ t ] =

Original: A t A ⊤ t + σ 

Revised: A t A ⊤ t + σ

Original: I p × p
Revised: I p × p

Original: while
Revised: while

Original: ∥
Revised: ∥

Original: A t A ⊤ t ∥ ≤ ˜ δ ,
Revised: A t A ⊤ t ∥ ≤ ˜ δ ,

Original: (b) ∥ x t x ⊤ t − ( A t A ⊤ t + σ 

Revised: (b) ∥ x t x ⊤ t − ( A t A ⊤ t + σ

Original: I p × p ) ∥
Revised: I p × p ) ∥

Original: ≤ M almost
Revised: ≤ M almost

Original: surely, and
Revised: surely, and

Original: (c) (cid:13)(cid:13) E (cid:2) (
Revised: (c) (cid:13)(cid:13) E (cid:2) (

Original: [Equation]
Revised: [Equation]

Original: (cid:3)(cid:13)(cid:13)
Revised: (cid:3)(cid:13)(cid:13)

Original: [Equation]
Revised: [Equation]

Original: V
Revised: V

Original: [Equation]
Revised: [Equation]

Original: When the observations follow equation (1), we condition our analysis on event E ∆ .
Revised: When the observations follow equation (1), we condition our analysis on event E ∆ .

Original: Definition 4.
Revised: Definition 4.

Original: For z t ∼ N ( 0 k × 1 , I k × k ) and w t ∈ N ( 0 p × 1 , I p × p ) , the event E ∆ is defined as:
Revised: For z t ∼ N ( 0 k × 1 , I k × k ) and w t ∈ N ( 0 p × 1 , I p × p ) , the event E ∆ is defined as:

Original: [Equation]
Revised: [Equation]

Original: The observations from model (1) satisfy Assumption 1 with M = 

Revised: The observations from model (1) satisfy Assumption 1 with M =

Original: ( k ˜ δ
Revised: ( k ˜ δ

Original: + pσ 2 )(1+Θ(log( T/ ∆) /k )) ,
Revised: + pσ 2 )(1+Θ(log( T/ ∆) /k )) ,

Original: V = 2 M
Revised: V = 2 M

Original: (˜ δ
Revised: (˜ δ

Original: + σ 2 ) with probability P [ E ∆ ] ≥ 1 − ∆ .
Revised: + σ 2 ) with probability P [ E ∆ ] ≥ 1 − ∆ .

Original: Although after conditioning E [ x t x ⊤ t | E ∆ ] ̸ =
Revised: Although after conditioning E [ x t x ⊤ t | E ∆ ] ̸ =

Original: A t A ⊤ t + σ 2 I p × p , we can use all the results in Section 5.1 and 5.2 with a sufficiently small ∆ .
Revised: A t A ⊤ t + σ 2 I p × p , we can use all the results in Section 5.1 and 5.2 with a sufficiently small ∆ .

Original: Pleaserefer to Appendix H for the details.
Revised: Pleaserefer to Appendix H for the details.

Original: 5.1 Noisy Power Method
Revised: 5.1 Noisy Power Method

Original: The noisy power method is an iterative algorithm for computing the top- k principal components ofgiven matrix.
Revised: The noisy power method is an iterative algorithm for computing the top- k principal components ofgiven matrix.

Original: Starting from the random matrix ˆ U (0) in R p × k , the algorithm runs for L iterations,each processing B samples.
Revised: Starting from the random matrix ˆ U (0) in R p × k , the algorithm runs for L iterations,each processing B samples.

Original: By repeating this procedure, we expect the algorithm to reconstruct thecovariance matrix if the observations are derived from the fixed distribution.
Revised: By repeating this procedure, we expect the algorithm to reconstruct thecovariance matrix if the observations are derived from the fixed distribution.

Original: When observations are drawn from the model 1 under Assumption 1, the observations belong todifferent underlying subspace.
Revised: When observations are drawn from the model 1 under Assumption 1, the observations belong todifferent underlying subspace.

Original: Unlike the standard case, for any algorithm ϕ , the presence ofperturbation prevents the convergence of the columns of ˆ U ( ℓ ) to the singular vectors of the finalcovariance matrix U ( ℓ ) .
Revised: Unlike the standard case, for any algorithm ϕ , the presence ofperturbation prevents the convergence of the columns of ˆ U ( ℓ ) to the singular vectors of the finalcovariance matrix U ( ℓ ) .

Original: The main difficulty here is that the columns of ˆ U ( ℓ ) do not convergetowards a fixed set of vectors but keep tracking the time-varying principal components.
Revised: The main difficulty here is that the columns of ˆ U ( ℓ ) do not convergetowards a fixed set of vectors but keep tracking the time-varying principal components.

Original: Recall thatour ultimate objective is to recover the principal components associated with the terminal observation.
Revised: Recall thatour ultimate objective is to recover the principal components associated with the terminal observation.

Original: Hence, we decompose the covariance matrix in terms of the last observation and the remainingsamples.
Revised: Hence, we decompose the covariance matrix in terms of the last observation and the remainingsamples.

Original: For ℓ -th block, we denote the covariance matrix for ℓB -th observation as M ( ℓ ) and have:
Revised: For ℓ -th block, we denote the covariance matrix for ℓB -th observation as M ( ℓ ) and have:

Original: [Equation]
Revised: [Equation]

Original: Due to presence of perturbations in the robust model, E ( ℓ ) is a non-zero mean random variable.
Revised: Due to presence of perturbations in the robust model, E ( ℓ ) is a non-zero mean random variable.

Original: Wefirst decompose E ( ℓ ) in terms of the contribution due to inherent noise and the perturbations allowedby the robust model.
Revised: Wefirst decompose E ( ℓ ) in terms of the contribution due to inherent noise and the perturbations allowedby the robust model.

Original: In Lemma 1, we decompose the error ∥E ( ℓ ) ∥ with respect to the block size Band the allowed perturbations in the robust model.
Revised: In Lemma 1, we decompose the error ∥E ( ℓ ) ∥ with respect to the block size Band the allowed perturbations in the robust model.

Original: The complete proof is provided in Appendix D.
Revised: The complete proof is provided in Appendix D.

Original: Lemma 1 (Spectral norm of noise) .
Revised: Lemma 1 (Spectral norm of noise) .

Original: Assume that the observations ( x t )
Revised: Assume that the observations ( x t )

Original: Tt =1 generated according tothe Assumption 1.
Revised: Tt =1 generated according tothe Assumption 1.

Original: For probability greater than 1 − 1 T , the matrices E ( ℓ ) in the equation (7) arebounded by:
Revised: For probability greater than 1 − 1 T , the matrices E ( ℓ ) in the equation (7) arebounded by:

Original: [Equation]
Revised: [Equation]

Original: when the block size B is larger than M 2 log(2 pT 2 ) / V .
Revised: when the block size B is larger than M 2 log(2 pT 2 ) / V .

Original: Lemma 1 highlights the effect of allowing perturbations in the data generation model.
Revised: Lemma 1 highlights the effect of allowing perturbations in the data generation model.

Original: From classicalresults in statistics, our intuition tells that the effect of noise washes out as the block size increases,i.e., the error decays with the ‘inverse square root’ of the block size.
Revised: From classicalresults in statistics, our intuition tells that the effect of noise washes out as the block size increases,i.e., the error decays with the ‘inverse square root’ of the block size.

Original: Hence, barring memory anddata issues, a larger block size is better when Γ = 0 .
Revised: Hence, barring memory anddata issues, a larger block size is better when Γ = 0 .

Original: In contrast, covariance perturbations add errorproportional to the block size.
Revised: In contrast, covariance perturbations add errorproportional to the block size.

Original: Consequently, we have a trade-off between two terms in this case, andthere exists an optimal block size depending on the Γ .
Revised: Consequently, we have a trade-off between two terms in this case, andthere exists an optimal block size depending on the Γ .

Original: We establish the convergence guarantee of the robust power method in Theorem 2.
Revised: We establish the convergence guarantee of the robust power method in Theorem 2.

Original: In the proof of
Revised: In the proof of

Original: Theorem 2 (Appendix E), we bound the distance, d (cid:0) U ( L ) , ˆ U ( L ) (cid:1) between the output of Algorithm 1,ˆ U 1: k ( L ) and k -orthonormal vectors spanning the column space of A T , U 1:
Revised: Theorem 2 (Appendix E), we bound the distance, d (cid:0) U ( L ) , ˆ U ( L ) (cid:1) between the output of Algorithm 1,ˆ U 1: k ( L ) and k -orthonormal vectors spanning the column space of A T , U 1:

Original: k
Revised: k

Original: ( L ) .
Revised: ( L ) .

Original: We identify theoptimal block size B , which is the unique parameter for noisy power method and establish an upperbound on the estimation error of the noisy power method.
Revised: We identify theoptimal block size B , which is the unique parameter for noisy power method and establish an upperbound on the estimation error of the noisy power method.

Original: Theorem 2 (Robust power method) .
Revised: Theorem 2 (Robust power method) .

Original: Assume that δ ≥ 0 .
Revised: Assume that δ ≥ 0 .

Original: 71 σ 2 and Γ =
Revised: 71 σ 2 and Γ =

Original: O
Revised: O

Original: ( δ 3 / ( V log(2 pT 2 ))) .
Revised: ( δ 3 / ( V log(2 pT 2 ))) .

Original: Whenthe observations { x t } Tt =1 satisfies the assumption 1, for B =
Revised: Whenthe observations { x t } Tt =1 satisfies the assumption 1, for B =

Original: Θ( V 1 / 3 log(2 pT 2 ) 1 / 3 / Γ 2 / 3 ) we have:
Revised: Θ( V 1 / 3 log(2 pT 2 ) 1 / 3 / Γ 2 / 3 ) we have:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: with probability 1 − 1 /T − c Ω( p − k +1)
Revised: with probability 1 − 1 /T − c Ω( p − k +1)

Original: − exp( − Ω( p )) .
Revised: − exp( − Ω( p )) .

Original: When Γ = Ω( δ 3 / ( V log(2 pT 2 )))
Revised: When Γ = Ω( δ 3 / ( V log(2 pT 2 )))

Original: we have that ( V Γlog(2 pT 2 )) 1 / 3 δ = Ω(1) .
Revised: we have that ( V Γlog(2 pT 2 )) 1 / 3 δ = Ω(1) .

Original: Therefore, the conditionon Γ in Theorem 2 is necessary to avoid a trivial upper bound ( ≤ 1 ) on R NPM .
Revised: Therefore, the conditionon Γ in Theorem 2 is necessary to avoid a trivial upper bound ( ≤ 1 ) on R NPM .

Original: This condition encompasses several applications of interest alluded to earlier.
Revised: This condition encompasses several applications of interest alluded to earlier.

Original: For example, in the financial applicationsalluded to earlier, individual market changes of interest happen on millisecond time-scale and it is ofsignificant interest to terminally detect cumulative market changes.
Revised: For example, in the financial applicationsalluded to earlier, individual market changes of interest happen on millisecond time-scale and it is ofsignificant interest to terminally detect cumulative market changes.

Original: Our results hold for sufficientlylarge value of cumulative changes (allowing for seconds time-scale) allowing us to study them.
Revised: Our results hold for sufficientlylarge value of cumulative changes (allowing for seconds time-scale) allowing us to study them.

Original: Theyfurther imply that the noisy power method is rate-optimal for non-trivial values of Γ .
Revised: Theyfurther imply that the noisy power method is rate-optimal for non-trivial values of Γ .

Original: When theobservations follow model (1), under the event E ∆ , Theorem 2 shows that the robust power methodcan achieve a estimation error of:
Revised: When theobservations follow model (1), under the event E ∆ , Theorem 2 shows that the robust power methodcan achieve a estimation error of:

Original: [Equation]
Revised: [Equation]

Original: with probability 1 − ∆ − 1 /T − c Ω( p − k +1)
Revised: with probability 1 − ∆ − 1 /T − c Ω( p − k +1)

Original: − exp( − Ω( p ))
Revised: − exp( − Ω( p ))

Original: if ˜ δ equals to δ .
Revised: if ˜ δ equals to δ .

Original: This is order-wise identicalto the fundamental limit established in Theorem 1 when pσ 2 dominates kδ .
Revised: This is order-wise identicalto the fundamental limit established in Theorem 1 when pσ 2 dominates kδ .

Original: Establishing bounds on the estimation error when the underlying singular vectors change is intricatesince the subspace to which consecutive observations belong are potentially different.
Revised: Establishing bounds on the estimation error when the underlying singular vectors change is intricatesince the subspace to which consecutive observations belong are potentially different.

Original: Conventionalproofs that analyze noisy power methods or Oja’s algorithm show that under a variety of assumptionsat every iteration ℓ , the distance between the estimated and true subspace, d ( U ( ℓ ) , ˆ U ( ℓ )) decreases.
Revised: Conventionalproofs that analyze noisy power methods or Oja’s algorithm show that under a variety of assumptionsat every iteration ℓ , the distance between the estimated and true subspace, d ( U ( ℓ ) , ˆ U ( ℓ )) decreases.

Original: For instance, the proof in [28] requires ∥E ( ℓ ) U ( ℓ ) ∥
Revised: For instance, the proof in [28] requires ∥E ( ℓ ) U ( ℓ ) ∥

Original: =
Revised: =

Original: O ( (cid:112) k/p ) which does not hold under ourmodel since ∥ M ( ℓ ) − M ( ℓ − 1) ∥ is in general greater than (cid:112) k/p .
Revised: O ( (cid:112) k/p ) which does not hold under ourmodel since ∥ M ( ℓ ) − M ( ℓ − 1) ∥ is in general greater than (cid:112) k/p .

Original: Similarly, the concentrationapproach in [32] can be used only when the covariance matrix is time-invariant.
Revised: Similarly, the concentrationapproach in [32] can be used only when the covariance matrix is time-invariant.

Original: We briefly describeour proof technique to establish Theorem 2, deferring details to Appendix E. Let
Revised: We briefly describeour proof technique to establish Theorem 2, deferring details to Appendix E. Let

Original: M ( L ) denote theproduct (cid:81) Lℓ =1 (cid:0) M ( ℓ ) + E ( ℓ ) (cid:1) .
Revised: M ( L ) denote theproduct (cid:81) Lℓ =1 (cid:0) M ( ℓ ) + E ( ℓ ) (cid:1) .

Original: Then, the output of the algorithm ˆ U 1: k
Revised: Then, the output of the algorithm ˆ U 1: k

Original: ( L ) is an orthonormal basisof M ( L ) ˆ U (0) , which estimates the first k principal components U 1: k of the M ( L ) .
Revised: ( L ) is an orthonormal basisof M ( L ) ˆ U (0) , which estimates the first k principal components U 1: k of the M ( L ) .

Original: To addressthis, we construct sequences of k and ( p − k ) -dimensional subspaces of R p from observations{ ( x t ) ℓBt =( ℓ − 1) B } Lℓ =1 , denoted by { N ( ℓ ) } Lℓ =1 ∈
Revised: To addressthis, we construct sequences of k and ( p − k ) -dimensional subspaces of R p from observations{ ( x t ) ℓBt =( ℓ − 1) B } Lℓ =1 , denoted by { N ( ℓ ) } Lℓ =1 ∈

Original: St p − k
Revised: St p − k

Original: ( R p ) and { W ( ℓ ) } Lℓ =1 ∈ St k
Revised: ( R p ) and { W ( ℓ ) } Lℓ =1 ∈ St k

Original: ( R p ) respectively,such that for all iterations ℓ :
Revised: ( R p ) respectively,such that for all iterations ℓ :

Original: (a) ran (cid:0) ( M ( ℓ )+ E ( ℓ )) N ( ℓ ) (cid:1) ⊆ ran (cid:0) N ( ℓ +1) (cid:1) and ran (cid:0) ( M ( ℓ )+ E ( ℓ )) W ( ℓ ) (cid:1) ⊆
Revised: (a) ran (cid:0) ( M ( ℓ )+ E ( ℓ )) N ( ℓ ) (cid:1) ⊆ ran (cid:0) N ( ℓ +1) (cid:1) and ran (cid:0) ( M ( ℓ )+ E ( ℓ )) W ( ℓ ) (cid:1) ⊆

Original: ran (cid:0) W ( ℓ +1) (cid:1) ,
Revised: ran (cid:0) W ( ℓ +1) (cid:1) ,

Original: (b) ∥ ( M ( ℓ ) + E ( ℓ )) N ( ℓ ) ∥∥ (( M ( ℓ ) + E ( ℓ )) W ( ℓ ) ) − 1 ∥ < 1 ,
Revised: (b) ∥ ( M ( ℓ ) + E ( ℓ )) N ( ℓ ) ∥∥ (( M ( ℓ ) + E ( ℓ )) W ( ℓ ) ) − 1 ∥ < 1 ,

Original: (c) d ( U 1: k ( ℓ ) , W ( ℓ +1) )
Revised: (c) d ( U 1: k ( ℓ ) , W ( ℓ +1) )

Original: =
Revised: =

Original: O ( ∥E ( ℓ ) ∥ ) and d ( U k
Revised: O ( ∥E ( ℓ ) ∥ ) and d ( U k

Original: +1: p ( ℓ ) , N ( ℓ +1) )
Revised: +1: p ( ℓ ) , N ( ℓ +1) )

Original: =
Revised: =

Original: O ( ∥E ( ℓ ) ∥ ) .
Revised: O ( ∥E ( ℓ ) ∥ ) .

Original: The initial random matrix ˆU 1: k (0) consists of both N (1) and W (1) with high probability.
Revised: The initial random matrix ˆU 1: k (0) consists of both N (1) and W (1) with high probability.

Original: From thefirst two properties, at every iteration ℓ , the projection of ˆU 1: k ( ℓ − 1) in W ( ℓ ) is amplified more thanthat on N ( ℓ ) and thus ˆU 1: k ( L ) becomes very close to W ( L ) after sufficiently large L .
Revised: From thefirst two properties, at every iteration ℓ , the projection of ˆU 1: k ( ℓ − 1) in W ( ℓ ) is amplified more thanthat on N ( ℓ ) and thus ˆU 1: k ( L ) becomes very close to W ( L ) after sufficiently large L .

Original: From the lastproperty, we can conclude that W ( L ) is close to U 1: k ( L ) , where the distance between W ( L ) and
Revised: From the lastproperty, we can conclude that W ( L ) is close to U 1: k ( L ) , where the distance between W ( L ) and

Original: U 1: k ( L ) is proportional to ∥E ( L ) ∥ .
Revised: U 1: k ( L ) is proportional to ∥E ( L ) ∥ .

Original: Combining these ideas with Lemma 1 establishes the results.
Revised: Combining these ideas with Lemma 1 establishes the results.

Original: 5.2 Oja’s Algorithm
Revised: 5.2 Oja’s Algorithm

Original: 1: Input: { x 1 } Tt =1 , Learning rate: ζ , dimensions: p, k & Output: ˆU ( T ) 2: ˆU (0) ← p × k random matrix 3: for t = 1 : T do 4: ˆU ( t ) ← Gram-Schmidt (( I + ζ x t x ⊤ t ) ˆU ( t − 1)) 5: end for
Revised: 1: Input: { x 1 } Tt =1 , Learning rate: ζ , dimensions: p, k & Output: ˆU ( T ) 2: ˆU (0) ← p × k random matrix 3: for t = 1 : T do 4: ˆU ( t ) ← Gram-Schmidt (( I + ζ x t x ⊤ t ) ˆU ( t − 1)) 5: end for

Original: We now establish the convergence guarantee for Oja’s Algorithm (Algorithm 2) when observationsfollow (1).
Revised: We now establish the convergence guarantee for Oja’s Algorithm (Algorithm 2) when observationsfollow (1).

Original: Unlike the noisy power method, Oja’s Algorithm is multiplicative in its construction of theestimated subspace.
Revised: Unlike the noisy power method, Oja’s Algorithm is multiplicative in its construction of theestimated subspace.

Original: We extend the existing analysis for Oja’s algorithm [34, 32, 3] by considering avirtual block with B = B ζ = ζ − 1 observations.
Revised: We extend the existing analysis for Oja’s algorithm [34, 32, 3] by considering avirtual block with B = B ζ = ζ − 1 observations.

Original: Building upon the analysis framework for the noisypower method and intuition from binomial approximation (1 + ζx ) 1 /ζ ≃ (1 + x ) we establish theconvergence guarantees for Oja’s algorithm.
Revised: Building upon the analysis framework for the noisypower method and intuition from binomial approximation (1 + ζx ) 1 /ζ ≃ (1 + x ) we establish theconvergence guarantees for Oja’s algorithm.

Original: Like the previous section, we decompose the block withtarget M ( ℓ ) and error matrix E ′ ( ℓ ) as:
Revised: Like the previous section, we decompose the block withtarget M ( ℓ ) and error matrix E ′ ( ℓ ) as:

Original: [Equation]
Revised: [Equation]

Original: where E ′ is scaled error matrix.
Revised: where E ′ is scaled error matrix.

Original: In Lemma 2, we provide bound for scaled error matrix with respect tothe learning parameter ζ , or the virtual block size B ζ = 1 /ζ similar to Lemma 1.
Revised: In Lemma 2, we provide bound for scaled error matrix with respect tothe learning parameter ζ , or the virtual block size B ζ = 1 /ζ similar to Lemma 1.

Original: The proof is providedin Appendix F.
Revised: The proof is providedin Appendix F.

Original: Lemma 2 (Spectral norm of noise, Oja’s algorithm case) .
Revised: Lemma 2 (Spectral norm of noise, Oja’s algorithm case) .

Original: Assume that the observations ( x t ) Tt =1generated according to the Assumption 1.
Revised: Assume that the observations ( x t ) Tt =1generated according to the Assumption 1.

Original: For probability greater than 1 − 1 T , the matrices E ′ ( ℓ ) onthe equation (11) are bounded by:
Revised: For probability greater than 1 − 1 T , the matrices E ′ ( ℓ ) onthe equation (11) are bounded by:

Original: [Equation]
Revised: [Equation]

Original: when the virtual block size B ζ = ζ − 1 is larger than 2 M 2 log( pT 2 ) .
Revised: when the virtual block size B ζ = ζ − 1 is larger than 2 M 2 log( pT 2 ) .

Original: The difference in parameter M , rather than V as in the case of Lemma 1 arises due to use of adifferent concentration inequality.
Revised: The difference in parameter M , rather than V as in the case of Lemma 1 arises due to use of adifferent concentration inequality.

Original: The estimator of the noisy power method averages the outerproduct of vectors (equation (11)) making it straightforward to use Bernstein’s inequality.
Revised: The estimator of the noisy power method averages the outerproduct of vectors (equation (11)) making it straightforward to use Bernstein’s inequality.

Original: On theother hand, Oja’s algorithm averages the product of random matrices rather than the sum of randommatrices.
Revised: On theother hand, Oja’s algorithm averages the product of random matrices rather than the sum of randommatrices.

Original: Therefore, we introduce the multiplicative concentration inequality [31], which requires the(probabilistic) norm bound for matrices.
Revised: Therefore, we introduce the multiplicative concentration inequality [31], which requires the(probabilistic) norm bound for matrices.

Original: Combining multiplicative concentration inequalities with
Revised: Combining multiplicative concentration inequalities with

Original: Lemma 2 and the techniques developed for noisy power method, we obtain a convergence guaranteesfor Oja’s algorithm in Theorem 3.
Revised: Lemma 2 and the techniques developed for noisy power method, we obtain a convergence guaranteesfor Oja’s algorithm in Theorem 3.

Original: The complete proof is provided in Appendix G.
Revised: The complete proof is provided in Appendix G.

Original: Theorem 3 (Oja’s algorithm) .
Revised: Theorem 3 (Oja’s algorithm) .

Original: Assume that δ ≥ 0 .
Revised: Assume that δ ≥ 0 .

Original: 71 and Γ = O
Revised: 71 and Γ = O

Original: ( δ 3 / ( e 3(˜ δ − δ )
Revised: ( δ 3 / ( e 3(˜ δ − δ )

Original: M 2 log( pT 2 ))) .
Revised: M 2 log( pT 2 ))) .

Original: Whenthe observations { x t } T t =1 satisfies the assumption 1, for 1 /ζ =
Revised: Whenthe observations { x t } T t =1 satisfies the assumption 1, for 1 /ζ =

Original: Θ( M 2 / 3 log( pT 2 ) 1 / 3 / Γ 2 / 3 ) wehave:
Revised: Θ( M 2 / 3 log( pT 2 ) 1 / 3 / Γ 2 / 3 ) wehave:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: −
Revised: −

Original: Unlike the noisy power method the upper bound in Theorem 3 is O ( p 2 / 3 ) rather than optimaldependence of O
Revised: Unlike the noisy power method the upper bound in Theorem 3 is O ( p 2 / 3 ) rather than optimaldependence of O

Original: ( p 1 / 3 ) from Theorem 1.
Revised: ( p 1 / 3 ) from Theorem 1.

Original: It is unclear whether the upper bound can be improved.
Revised: It is unclear whether the upper bound can be improved.

Original: Sharpening our analysis with a two-phase strategy [32, 39] (wherein the first phase identifies a goodinitial point and the second phase establishes convergence given an initial point) might be an excellentdirection for future investigation.
Revised: Sharpening our analysis with a two-phase strategy [32, 39] (wherein the first phase identifies a goodinitial point and the second phase establishes convergence given an initial point) might be an excellentdirection for future investigation.

Original:  Numerical Results
Revised:  Numerical Results

Original: [Figure]
Revised: [Figure]

Original: Lastly, in order to verify the established results for both algorithms, we examine the performance ofalgorithms for a variety of environments .
Revised: Lastly, in order to verify the established results for both algorithms, we examine the performance ofalgorithms for a variety of environments .

Original: Theorem 2 and 3 show the (a) existence of the optimal blocksize B and the learning rate ζ to obtain the minimum recovery error, and (b) Γ − 3 / 2 dependencies ofthose optimal B and 1 /ζ .
Revised: Theorem 2 and 3 show the (a) existence of the optimal blocksize B and the learning rate ζ to obtain the minimum recovery error, and (b) Γ − 3 / 2 dependencies ofthose optimal B and 1 /ζ .

Original: We demonstrate this phenomenon in Figure 1a and Figure 1b respectively.
Revised: We demonstrate this phenomenon in Figure 1a and Figure 1b respectively.

Original: Experimental details are described in Appendix I.
Revised: Experimental details are described in Appendix I.

Original: Our first observation on the Figure 1a is that
Revised: Our first observation on the Figure 1a is that

Original: [Figure]
Revised: [Figure]

Original: the block size increases without any covariance perturbation ( Γ = 0 ).
Revised: the block size increases without any covariance perturbation ( Γ = 0 ).

Original: This behavior is expected since in the absence of rotation of underlying space, increasing the numberof past information results in better accuracy guarantees for the recovered space.
Revised: This behavior is expected since in the absence of rotation of underlying space, increasing the numberof past information results in better accuracy guarantees for the recovered space.

Original: However, if thecovariance perturbation exists ( Γ > 0 ), there exists the optimal learning parameter, and we have thesmaller optimal block size as the adversary stronger.
Revised: However, if thecovariance perturbation exists ( Γ > 0 ), there exists the optimal learning parameter, and we have thesmaller optimal block size as the adversary stronger.

Original: This phenomenon is also the expected behaviorsince an increase in the adversarial budget implies that the past information becomes less relevant.
Revised: This phenomenon is also the expected behaviorsince an increase in the adversarial budget implies that the past information becomes less relevant.

Original: Our observations also corroborate our theoretical results in the Theorem 1.
Revised: Our observations also corroborate our theoretical results in the Theorem 1.

Original: In Figure 1b, we focuson the optimal value of the block size B and inverse of the learning rate 1 /ζ and its variation withthe perturbation budget Γ .
Revised: In Figure 1b, we focuson the optimal value of the block size B and inverse of the learning rate 1 /ζ and its variation withthe perturbation budget Γ .

Original: We plot the empirically optimal learning rate for the case of the noisypower method and Oja’s algorithm with Γ − 2 / 3 .
Revised: We plot the empirically optimal learning rate for the case of the noisypower method and Oja’s algorithm with Γ − 2 / 3 .

Original: We observe that the optimal block size and theinverse learning rate is proportional to Γ − 2 / 3 .
Revised: We observe that the optimal block size and theinverse learning rate is proportional to Γ − 2 / 3 .

Original: This experimental dependency of Γ − 2 / 3 verifies thetheoretically prescribed results in Theorem 2 and Theorem 3.
Revised: This experimental dependency of Γ − 2 / 3 verifies thetheoretically prescribed results in Theorem 2 and Theorem 3.

Original:  Conclusion
Revised:  Conclusion

Original: In this paper, we analyzed the fundamental limit of the minimax error and convergence of the noisypower method and Oja’s algorithm on the streaming PCA settings with time-varying covariance.
Revised: In this paper, we analyzed the fundamental limit of the minimax error and convergence of the noisypower method and Oja’s algorithm on the streaming PCA settings with time-varying covariance.

Original: Under this framework, when there exists no perturbation ( Γ = 0 ), our theoretical result on a lowerlimit coincides with the order of the traditional outcome.
Revised: Under this framework, when there exists no perturbation ( Γ = 0 ), our theoretical result on a lowerlimit coincides with the order of the traditional outcome.

Original: Furthermore, when perturbation exists( Γ > 0 ), we have a non-avoidable positive minimax recovery error, although the time horizonbecomes arbitrarily long.
Revised: Furthermore, when perturbation exists( Γ > 0 ), we have a non-avoidable positive minimax recovery error, although the time horizonbecomes arbitrarily long.

Original: Next, we found that the noisy power method order-wisely achieves thisrecovery error and becomes rate-optimal if the kδ ≪ pσ 2 .
Revised: Next, we found that the noisy power method order-wisely achieves thisrecovery error and becomes rate-optimal if the kδ ≪ pσ 2 .

Original: Finally, we showed that optimal block sizeand inverse learning rate minimizing recovery error were proportional to Γ − 3 / 2 both theoreticallyand experimentally.
Revised: Finally, we showed that optimal block sizeand inverse learning rate minimizing recovery error were proportional to Γ − 3 / 2 both theoreticallyand experimentally.

Original: All our results are based on the spectral gap assumption.
Revised: All our results are based on the spectral gap assumption.

Original: However, a recent lineof literature considers effective rank [11, 50] as a crucial metric.
Revised: However, a recent lineof literature considers effective rank [11, 50] as a crucial metric.

Original: Establishing guarantees for robuststreaming PCA in terms of effective rank is an interesting direction for future work.
Revised: Establishing guarantees for robuststreaming PCA in terms of effective rank is an interesting direction for future work.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] The datasets are public datasetsand the code is MIT license
Revised: [Yes] The datasets are public datasetsand the code is MIT license

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes] We explain our new framework and contributions in
Revised: [Yes] We explain our new framework and contributions in

Original: both abstract and introduction.
Revised: both abstract and introduction.

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] There is a result which is partially
Revised: [Yes] There is a result which is partially

Original: suboptimal compare to the edge works.
Revised: suboptimal compare to the edge works.

Original: We also describe and study the reason for that.
Revised: We also describe and study the reason for that.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: We provideour assumptions on the Section 3 and Assumption
Revised: We provideour assumptions on the Section 3 and Assumption

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] Detailed Proofs are inthe appendix.3.
Revised: [Yes] Detailed Proofs are inthe appendix.3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: Those areincluded in the supplemental material.
Revised: Those areincluded in the supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] Those are included in the Appendix.
Revised: [Yes] Those are included in the Appendix.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] Yes, we repeated every experiment 10 times and reportederror bars or uncertainty markers.
Revised: [Yes] Yes, we repeated every experiment 10 times and reportederror bars or uncertainty markers.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g.,
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g.,

Original: type of GPUs, internal cluster, or cloud provider)?
Revised: type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] Those are included in the
Revised: [Yes] Those are included in the

Original: Appendix.
Revised: Appendix.

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A] We only used synthesized random matrix data.
Revised: [N/A] We only used synthesized random matrix data.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]