Title: Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spectral norm

low-rank approximation

differentially private PCA

contour integration

matrix analysis
Abstract: A central challenge in machine learning is to understand how noise or measurement errors affect low-rank approximations, particularly in the spectral norm. This question is especially important in differentially private low-rank approximation, where one aims to preserve the top-$p$ structure of a data-derived matrix while ensuring privacy. Prior work often analyzes Frobenius norm error or changes in reconstruction quality, but these metrics can over- or under-estimate true subspace distortion. The spectral norm, by contrast, captures worst-case directional error and provides the strongest utility guarantees. We establish new high-probability spectral-norm perturbation bounds for symmetric matrices that refine the classical Eckart--Young--Mirsky theorem and explicitly capture interactions between a matrix $A \in \mathbb{R}^{n \times n}$ and an arbitrary symmetric perturbation $E$. Under mild eigengap and norm conditions, our bounds yield sharp estimates for $\| (A + E)_p - A_p \|$, where $A_p$ is the best rank-$p$ approximation of $A$, with improvements of up to a factor of $\sqrt{n}$. As an application, we derive improved utility guarantees for differentially private PCA, resolving an open problem in the literature. Our analysis relies on a novel contour bootstrapping method from complex analysis and extends it to a broad class of spectral functionals, including polynomials and matrix exponentials. Empirical results on real-world datasets confirm that our bounds closely track the actual spectral error under diverse perturbation regimes.

Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy

Abstract

A central challenge in machine learning is to understand how noise or measurement errors affect low-rank approximations-particularly in the spectral norm. This question is especially important in differentially private low-rank approximation, where one aims to preserve the topp structure of a data-derived matrix while ensuring privacy. Prior work often analyzes Frobenius norm error or changes in reconstruction quality, but these metrics can over- or under-estimate true subspace distortion. The spectral norm, by contrast, captures worst-case directional error and provides the strongest utility guarantees. We establish new high-probability spectral-norm perturbation bounds for symmetric matrices that refine the classical Eckart-Young-Mirsky theorem and explicitly capture interactions between a matrix A ∈ R n × n and an arbitrary symmetric perturbation E . Under mild eigengap and norm conditions, our bounds yield sharp estimates for ∥ ( A + E ) p -A p ∥ , where A p is the best rankp approximation of A , with improvements of up to a factor of √ n . As an application, we derive improved utility guarantees for differentially private PCA, resolving an open problem in the literature. Our analysis relies on a novel contour bootstrapping method from complex analysis and extends it to a broad class of spectral functionals, including polynomials and matrix exponentials. Empirical results on real-world datasets confirm that our bounds closely track the actual spectral error under diverse perturbation regimes.

1 Introduction

Low-rank approximation is a foundational technique in machine learning, data science, and numerical linear algebra, with applications ranging from dimensionality reduction and clustering to recommendation systems and privacy-preserving data analysis [1, 4, 5, 14, 21, 23, 24, 42, 45]. A common setting involves a real symmetric matrix A ∈ R n × n , such as a sample covariance matrix derived from high-dimensional data. Let λ 1 ≥ ··· ≥ λ n denote the eigenvalues of A , with corresponding orthonormal eigenvectors u 1 , . . . , u n . The best rankp approximation of A is denoted by A p := ∑ p i =1 λ i u i u ⊤ i . This approximation solves the optimization problem A p = arg min rank ( B ) ≤ p ∥ A -B ∥ , where the norm can be any unitarily invariant norm [7, 10]. In particular, A p minimizes both the spectral norm ∥·∥ , measuring worst-case error, and the Frobenius norm ∥ · ∥ F , measuring average deviation.

In many applications, the matrix A is not directly available-it may be corrupted by noise, compressed for efficiency, or randomized to preserve privacy. A standard model introduces a symmetric perturbation E , yielding the observed matrix ˜ A := A + E . The approximation ˜ A p , computed from ˜ A , is often used in downstream learning and inference. This leads to a central question: How does the perturbation E affect the topp approximation A p ? Understanding the deviation ∥ ˜ A p -A p ∥ is critical for ensuring the reliability and robustness of low-rank methods under noise.

∗

Motivating application: differential privacy. The stability under perturbations is especially important when the matrix A encodes sensitive information , such as user behavior or medical data. In such settings, even low-rank approximations of A can inadvertently leak private information [6]. To address this risk, differential privacy (DP) [14] has become the standard framework for designing privacy-preserving algorithms. Several mechanisms have been developed to release private low-rank approximations while satisfying DP guarantees [8, 9, 15, 25, 29, 31, 34, 39]. A canonical method, introduced in [15], adds a symmetric noise matrix E with i.i.d. Gaussian entries to the input matrix A , yielding the perturbed matrix ˜ A = A + E . The algorithm then releases ˜ A p as the privatized output. The utility of such mechanisms is typically assessed by comparing ˜ A p to the ideal (non-private) approximation A p . Two standard metrics are: (1) the Frobenius norm error ∥ ˜ A p -A p ∥ F , and (2) the change in reconstruction error |∥ A -A p ∥ ⋆ -∥ A -˜ A p ∥ ⋆ | , which measures how much the quality of low-rank approximation degrades due to noise, for a norm ∥ · ∥ ⋆ [3, 11, 15, 29]. These metrics offer insight into the effect of noise on overall variance or total reconstruction error. However, as we explain next, they may fail to capture worst-case directional misalignment , which is often critical for downstream tasks and algorithmic guarantees.

Limitations of existing utility metrics. The Frobenius norm error and reconstruction error may not be appropriate in applications that rely on the geometry of the topp eigenspace. In particular, the Frobenius norm may overestimate the impact of noise by up to a factor of √ p when the perturbation E lies largely in directions orthogonal to the topp subspace. The reconstruction error metric can underestimate subspace deviation-sometimes dramatically. In some cases, it remains small (or even zero) despite substantial rotation in the topp eigenspace. (See Sections B for concrete illustrations.) These limitations motivate the use of the spectral norm ∥ ˜ A p -A p ∥ , which captures the worst-case directional deviation between the two low-rank approximations. The spectral norm also governs algorithmic robustness in many downstream applications, such as PCA-based learning, private clustering, and subspace tracking.

A classical spectral norm bound, derived from the Eckart-Young-Mirsky theorem [7, 16], states that ∥ ˜ A p -A p ∥ ≤ 2( λ p +1 + ∥ E ∥ ) , which holds for arbitrary matrices and noise. However, such bounds are often pessimistic and fail to exploit the structure of A and E . More refined bounds exist in the Frobenius norm setting. For example, recent work [29, 30] shows that when A is positive semidefinite and has a nontrivial eigengap δ p := λ p -λ p +1 ≥ 4 ∥ E ∥ , and when E is drawn from a complex Gaussian ensemble, one obtains: E ∥ ˜ A p -A p ∥ F = ˜ O ( √ p · ∥ E ∥ · λ p δ p ) , which improves on the earlier reconstruction-error-based bounds of [15] by a factor of √ p . However, these bounds have important limitations: They hold only in expectation and do not yield high-probability guarantees; They often assume Gaussian noise distributions; They are not spectral norm bounds and therefore do not directly quantify the worst-case impact on the eigenspace. These limitations prompt the following open question, raised in [29, Remark 5.3]: Can one obtain high-probability spectral norm bounds for ∥ ˜ A p -A p ∥ under natural structural assumptions on A and realistic noise models?

Our contributions. We resolve the open question posed in [29, Remark 5.3], proving new highprobability spectral norm bounds for low-rank approximation under symmetric perturbations. Our results rely on natural structural assumptions on A and E and yield the first such guarantees for differentially private PCA (DP-PCA).

Two high-probability spectral norm bounds. Under the same eigengap condition as [29], δ p := λ p -λ p +1 ≥ 4 ∥ E ∥ , we prove ∥ ˜ A p -A p ∥ = O ( ∥ E ∥ · λ p δ p ) and ∥ ˜ A p -A p ∥ = ˜ O ( ∥ E ∥ + r 2 x · λ p δ p ) , where r is the halving distance (a measure of spectral decay) and x := max i,j ≤ r | u ⊤ i Eu j | quantifies noise-eigenspace alignment (Theorems 2.1-2.2). In addition, our contour-based framework extends to a broader class of spectral functionals f ( A ) (beyond f ( A ) = A ), encompassing matrix powers, exponentials, and trigonometric transforms; see Theorem 2.3.

Spectral utility bounds for DP-PCA. Our first bound yields a high-probability spectral norm utility guarantee for differentially private PCA under sub-Gaussian noise, improving existing Frobenius-norm bounds by up to a factor of √ p (Corollary 2.4). While prior work has achieved spectral norm guarantees in iterative or multi-pass settings [17, 18], our contribution concerns the direct noise-addition model, where this appears to be the first such result. For matrices with low stable rank and weak eigenspace-noise interaction, our second bound further improves by up to √ n .

Novel analytical technique: contour bootstrapping. Our proof relies on a contour bootstrapping argument (Lemma 3.1), which provides a new way to analyze the contour representation of perturbations [19, 26, 35], enabling analysis of a broader class of spectral functionals (Theorem 2.3). The bootstrapping argument here is a generalization of the argument used to handle eigenspaces perturbation introduced in [37].

Empirical validation. We benchmark our bounds on real covariance matrices under both Gaussian and Rademacher noise. Across datasets and noise regimes, the predicted error closely matches empirical behavior and consistently surpasses classical baselines, confirming the sharpness and robustness of our theoretical results (Section 4).

2 Main results

Main spectral norm bound. For clarity, we state our main bounds assuming A ∈ R n × n is positive semi-definite (PSD); extensions to symmetric matrices appear in Section D. Let λ 1 ≥ · · · ≥ λ n ≥ 0 be the eigenvalues of A , with corresponding orthonormal eigenvectors u 1 , . . . , u n , and define the eigengap δ k := λ k -λ k +1 . Given a real symmetric perturbation matrix E , we let ˜ A := A + E , and define A p and ˜ A p as the best rankp approximations of A and ˜ A , respectively. Our goal is to bound the spectral error ∥ ˜ A p -A p ∥ .

The O ( · ) notation here hides a small universal constant (less than 7), which we have not optimized; see Section D.1 for the proof of the generalization to the symmetric setting, of which this theorem is a special case. For Wigner noise-i.e., a symmetric matrix E with i.i.d. sub-Gaussian entries of mean 0 and variance 1-we have ∥ E ∥ = (2 + o (1)) √ n with high probability [41, 43], so Theorem 2.1 reduces to ∥ ˜ A p -A p ∥ = O ( √ n λ p δ p ) . The right-hand side is explicitly noisedependent, addressing a key limitation of the classical Eckart-Young-Mirsky bound. Moreover, in many widely studied structured models (e.g., spiked covariance, stochastic block, and graph Laplacian models), one typically has λ p = O ( δ p ) , yielding the clean bound O ( ∥ E ∥ ) . This rate is theoretically tight: for instance, when A is a PSD diagonal matrix and E = µI n for some µ > 0 , we have ∥ ˜ A p -A p ∥ = µ = ∥ E ∥ .

Gap condition. Our assumption 4 ∥ E ∥ < δ p aligns with standard conditions in prior work, including [29, 30], and is satisfied in many well-studied matrix models-such as spiked covariance (Wishart) models, deformed Wigner ensembles, stochastic block models, and kernel matrices for clustering. It also arises naturally in classical perturbation theory [12, 26, 28]. Empirical analyses [29, Section B] further show that this condition holds for real-world datasets commonly used in private matrix approximation (e.g., the 1990 U.S. Census and the UCI Adult dataset [3, 11]). Hence, Theorem 2.1 operates under a mild and broadly applicable assumption, satisfied across both theoretical models and practical benchmarks.

Comparison to the Eckart-Young-Mirsky bound. Using λ p = δ p + λ p +1 , Theorem 2.1 rewrites as ∥ ˜ A p -A p ∥ = O ( ∥ E ∥ + λ p +1 · ∥ E ∥ δ p ) . This improves on the E-Y-M bound O ( ∥ E ∥ + λ p +1 ) when λ p +1 ≫∥ E ∥ , by a factor of min { λ p +1 ∥ E ∥ , δ p ∥ E ∥ } . For example, consider a matrix with spectrum { 10 n, 9 n, . . . , n, n/ 2 , 1 , . . . , 1 } and p = 10 . For Gaussian noise with ∥ E ∥ = O ( √ n ) , E-Y-M yields O ( n ) error, while our bound gives O ( √ n ) , a √ n -factor gain.

Comparison to Mangoubi-Vishnoi bounds [29, 30]. Our bound also improves upon the Frobenius norm bounds of [29, 30], which under the same gap assumption yield: E ∥ ˜ A p -A p ∥ F = ˜ O ( √ p ∥ E ∥ · λ p δ p ) . We eliminate the √ p factor, upgrade from expectation to high probability, and support real-valued, non-Gaussian noise models. A more detailed comparison appears later in this section (Corollary 2.4), where we analyze implications for differentially private PCA.

Proof technique: contour bootstrapping. Unlike prior analyses [29, 30], which rely on Dyson Brownian motion and tools from random matrix theory (see Section A, our proof of Theorem 2.1 uses a contour-integral representation of the rankp projector. This approach, which we call contour bootstrapping , isolates the topp eigenspace via complex-analytic techniques and avoids powerseries or Davis-Kahan-type expansions. It enables tighter, structure-aware spectral bounds and ex-

tends naturally to refined perturbation results (Theorem 2.2) and general spectral functionals (Theorem 2.3). Full details appear in Section 3.

Refined bound via eigenspace interaction. To sharpen our analysis, we incorporate fine-grained structure of the eigenspace and its interaction with the noise. Inspired by the recent works [33, 38], we start with the observation that the rankp perturbation is primarily influenced by the cluster of eigenvalues near λ p , and the interaction between E and the corresponding eigenvectors. To control these factors, we define the halving distance r (w.r.t the index p ) as the smallest integer such that λ r +1 ≤ λ p / 2 , and interaction term x := max 1 ≤ i,j ≤ r | u ⊤ i Eu j | , measuring the alignment between the noise E and the topr eigenvectors of A . This yields a refined spectral norm bound:

See Section D.2 for the proof and its generalization to the symmetric setting. This bound improves upon the basic eigengap bound O ( ∥ E ∥ · λ p δ p ) when the interaction term r 2 x is small. This occurs, for instance, when (i) A has low stable rank or clustered eigenvalues (e.g., spiked models, multi-cluster Laplacians), (ii) the noise E is random and approximately orthogonal to the leading eigenspace, or (iii) λ p /δ p is large but x = ˜ O (1) and r = ˜ O (1) . In such regimes, the bound simplifies to ˜ O ( ∥ E ∥ + λ p δ p ) , yielding up to a √ n -factor improvement over Theorem 2.1. This highlights the benefit of explicitly incorporating spectral decay and noise-eigenspace alignment when analyzing noise-robust low-rank approximations.

In practice, many public DP datasets (e.g., Census, Adult, KDD) have small dimensions and modest eigenspace decay, the simple bound is more effective. However, the refined bound becomes especially informative in large-scale or synthetically structured settings. Thus, the two bounds are best viewed as complementary : the first is robust and broadly applicable, while the second highlights structural regimes where stronger stability is provable.

Extension to spectral functionals. Beyond approximating A itself, many applications involve lowrank approximations of spectral functions f ( A ) , such as A k , exp( A ) , or cos( A ) ; see [7, 44]. Our contour-based analysis extends naturally to this broader setting. Let f p ( A ) := ∑ p i =1 f ( λ i ) u i u ⊤ i denote the best rankp approximation of f ( A ) . We obtain the following general perturbation bound.

Theorem 2.3 ( Perturbation bounds for general functions). If 4 ∥ E ∥ ≤ δ p , then

where Γ 1 is the rectangle with vertices ( x 0 , T ) , ( x 1 , T ) , ( x 1 , -T ) , ( x 0 , -T ) with

The O ( · ) notation hides a small universal constant (less than 4), which we have not attempted to optimize; see Section F for details. For example, let f ( z ) = z 3 , so that f p ( ˜ A ) and f p ( A ) correspond to the best rankp approximations of ˜ A 3 and A 3 , respectively. Since max z ∈ Γ 1 ∥ f ( z ) ∥ ≤ 64 ∥ A ∥ 3 , Theorem 2.3 yields ∥ ˜ A 3 p -A 3 p ∥ = O ( ∥ A ∥ 3 · ∥ E ∥ /δ p ) . This result applies to many important classes of functions-e.g., polynomials, exponentials, and trigonometric functions-and hence we expect it to be broadly useful. However, Theorem 2.3 does not apply to non-entire functions such as f ( z ) = z c for non-integer c , where singularities obstruct the contour representation (1). In particular, when c < 0 , the expression f p ( A ) is no longer the best rankp approximation to f ( A ) , so the conclusion of Theorem 2.3 is not meaningful in that setting. We note that in a related work [36], the first two authors present an extension of the setting f ( z ) = z -1 .

Application: differentially private low-rank approximation. We now apply our spectral norm bound to analyze a standard differentially private (DP) mechanism for releasing a low-rank approximation of a sensitive matrix A , commonly assumed to be a sample covariance matrix and hence PSD. Under ( ε, δ ) -DP [14], the Gaussian mechanism releases ˜ A := A + E , where E is a symmetric matrix with i.i.d. Gaussian entries scaled to sensitivity ∆ = O ( √ log(1 /δ ) /ε ) . A common postprocessing step is to compute ˜ A p , the best rankp approximation of ˜ A . Prior analyses [3, 15, 30] focused primarily on Frobenius norm or reconstruction error. For instance, [30] showed that under complex Wigner noise and a moderate eigengap, E ∥ ˜ A p -A p ∥ F ≤ √ pnλ p δ p up to lower-order terms.

Since ∥ ˜ A p -A p ∥ ≤ ∥ ˜ A p -A p ∥ F , the above inequality implies an expected spectral norm error of ˜ O ( √ pn λ p δ p ) . In contrast, our bound yields the following high-probability spectral norm guarantee:

Corollary 2.4 ( Application to differential privacy). Let A be PSD and E be a real or complex Wigner matrix. If δ p ≥ 8 . 01 √ n , then with probability 1 -o (1) , ∥ ˜ A p -A p ∥ ≤ O ( √ n · λ p δ p ) .

This follows directly from Theorem 2.1, using the fact that ∥ E ∥ = O ( √ n ) with high probability for Wigner matrices [40, 43]. Compared to [30], this result provides a spectral norm (rather than Frobenius) guarantee, holds with high probability instead of in expectation, applies to both real and complex Wigner noise, removes the log log log n n factor, and eliminates restrictive assumptions such as λ 1 ≤ n 50 . It also improves the dependence on p by a factor of √ p , thereby resolving the open question posed in [30, Remark 5.3].

The spectral norm better captures subspace distortion, which is critical in applications like private PCA. Unlike Frobenius or reconstruction error-both of which may remain small even when ˜ A p deviates significantly from the true topp eigenspace-the spectral norm reflects worst-case directional error and is thus a more reliable utility metric. This distinction is empirically validated in Figure 3. Moreover, Corollary 2.4 further yields high-probability Frobenius norm and reconstruction error bounds on the perturbation of low-rank approximations:

Finally, while Corollary 2.4 is stated for sub-Gaussian noise, Theorem 2.1 extends to any symmetric perturbation satisfying the norm and gap conditions, including subsampled or quantized Gaussians and Laplace noise. We leave the detailed analysis of these settings to future work.

Alternative methods for approximating A p . Hardt and Price [17, 18] proposed a random iterative method which, under the condition δ p ≫ √ n log n , produces a rankk approximation A ′ of A p with k = p + O (1) , satisfying the trade-off bound ∥ A ′ -A p ∥ = ˜ O ( √ n λ 1 δ p max 1 ≤ i ≤ n ∥ u i ∥ ∞ ) , where u i denotes the eigenvectors of A .

If at least one eigenvector u i is localized (i.e., max 1 ≤ i ≤ n ∥ u i ∥ ∞ = 1 / ˜ O (1) ), this simplifies to ˜ O ( √ n λ 1 δ p ) . In this regime, Theorem 2.1 achieves a smaller bound by a factor of ˜ O ( λ 1 /λ p ) -up to √ n when λ 1 = Θ( n ) and λ p = Θ( √ n ) . Furthermore, Theorem 2.2 provides an additional improvement by a factor of O ( min { √ n r 2 , λ 1 δ p }) , which can reach √ n when r = ˜ O (1) and δ p = Θ( √ n ) -a common regime in high-dimensional data.

If all eigenvectors u i are delocalized (i.e., max 1 ≤ i ≤ n ∥ u i ∥ ∞ = ˜ O (1) / √ n ), the Hardt-Price bound reduces to ˜ O ( λ 1 /δ p ) . Theorem 2.1 achieves a comparable rate when σ 1 = Θ( n ) and λ p = c δ p = Θ( √ n ) , while Theorem 2.2 yields an improvement by a factor of λ 1 /λ p whenever r = ˜ O (1) , i.e., when A is approximately low-rank.

3 Proof outline

In the preceding section, we stated our main results-Theorems 2.1, 2.2, and 2.3. Here, we first sketch the key ideas behind the proof of Theorem 2.1, then adapt the same framework, with minor refinements, to derive Theorems 2.2 and 2.3.

The proof of Theorem 2.1 proceeds in three main steps. First, using the contour method, we obtain the contour-based bound of our perturbation ∥ ˜ A p -A p ∥ ≤ F ( z ) := 1 2 π i ∥ ∫ Γ z [( zI -˜ A ) -1 -( zI -A ) -1 ] ∥ dz. Here Γ is a contour on the complex plane, isolating the p -leading eigenvalues of A and ˜ A . This contour step captures the A -E interaction that the Eckart-Young-Mirsky bound omits (see Appendix A). Secondly, we develop the contour bootstrapping technique (Lemma 3.1), which under the gap assumption 4 ∥ E ∥ ≤ δ p , yields F ( z ) ≤ 2 F 1 ( z ) with F 1 ( z ) := ∫ Γ ∥ z ( zI -A ) -1 E ( zI -A ) -1 ∥| dz | . This technique (valid for any entire function f ) replaces the traditional series expansions and the heavy analysis of the matrix-derivative operator (the limitation of the Mangoubi-Vishnoi approach [29, 30], Appendix A) with a computable quantity. Third, we construct a bespoke contour Γ -onespecifically tailored so that the topp eigenvalues of A and ˜ A lie at prescribed distances from its sides. This precise alignment makes the integral defining F 1 ( z ) both tractable and essentially optimal, yielding a tight perturbation bound.

Step 1: Representing ∥ f p ( ˜ A ) -f p ( A ) ∥ via the classical contour method. Let λ 1 ≥ · · · ≥ λ n be the eigenvalues of A with the corresponding eigenvectors { u i } n i =1 . We now present the contour method to bound matrix perturbations in the spectral norm. Let Γ be a contour in C that encloses λ 1 , λ 2 , . . . , λ p and excludes λ p +1 , λ p +2 , . . . , λ n . Let f be any entire function and recall f p ( A ) = ∑ p i =1 f ( λ p ) u i u ⊤ i . Since f is analytic on the whole plane C , the well-known contour integral representation [19, 26, 35] gives us:

Let ˜ λ 1 ≥ · · · ≥ ˜ λ n denote the eigenvalue of ˜ A with the corresponding eigenvectors ˜ u 1 , ˜ u 2 , . . . , ˜ u n . The construction of Γ (presented later) and the gap assumption 4 ∥ E ∥ < δ p ensure that the eigenvalues ˜ λ i for 1 ≤ i ≤ p lie inside Γ , while all ˜ λ j for j > p remain outside. Then, similarly, we have 1 2 π i ∫ Γ f ( z )( zI -˜ A ) -1 dz = ∑ p i =1 f ( ˜ λ i )˜ u i ˜ u ⊤ i := f p ( ˜ A ) . Thus, we obtain the following contour identity for the perturbation:

Now we bound the perturbation by the corresponding integral

This inequality makes the interaction of A and E explicit and is widely used in functional perturbation analysis, e.g., [19, 26, 28, 32, 33, 37]. However, obtaining a sharp bound on its right-hand side remains a formidable analytical challenge.

Step 2: Bounding F ≤ 2 F 1 via the contour bootstrapping method. Attempts to control F ( f ) , the right-hand side of (2), often use series expansion and analytical tools. By repeatedly applying the resolvent formula, one can expand f ( z )[( zI -˜ A ) -1 -( zI -A ) -1 ] into ∑ ∞ s =1 f ( z )( zI -A ) -1 [ E ( zI -A ) -1 ] s . This yields the bound:

One needs to estimate F s ( f ) for each s . For example, when f ( z ) = 1 , [26, Part 2] bounds F s (1) by O ( ∥ E ∥ s ∫ Γ | dz | min i ∈ [ n ] | z -λ i | s +1 ) = O [( || E || /δ p ) s ] , where Γ is a union of vertical lines isolating { λ i , i ∈ p } , yielding the Davis-Kahan bound O ( ∥ E ∥ /δ p ) . However, for f ( z ) = z (relevant for low-rank perturbations), this approach fails as | z | → ∞ . These estimates are highly nontrivial and rely on deep analytical techniques, making generalization to arbitrary f challenging.

Moreover, for f ( z ) = 1 , under certain conditions, the dominant term is F 1 ( f ) , i.e., F ( f ) = O ( F 1 ( f )) ; see, e.g., [22, 27, 32, 33, 37]. In particular, using contour-bootstrapping technique, the authors in [37] proved F ( f ( z ) = 1) ≤ 2 F 1 ( f ( z ) = 1) . Inspired by this technique, we prove that F ( f ) ≤ 2 F 1 ( f ) for any entire function f .

Lemma 3.1 ( Contour bootstrapping for entire function f ). If δ p ≥ 4 ∥ E ∥ , then

Our contour bootstrapping argument is designed to prove Lemma 3.1. Our argument is concise and novel, avoiding the need for series expansion and convergence analysis. In the context of standard low-rank approximations, where f ( z ) ≡ z and f p ( A ) = A p , we write F ( z ) and F 1 ( z ) instead of F ( f ) and F 1 ( f ) respectively.

Step 3: Construction of Γ , F 1 ( z ) -estimation, and proof completion of Thm. 2.1. Given Lemma 3.1, we now need to carefully choose the contour Γ and estimate F 1 ( f ) . Constructing Γ (so that the perturbation analysis via contour integration provides a sharp bound) is delicate; for example, the classical pick of two vertically parallel lines and any Γ placed too near any λ i can blow up F 1 ( z ) to infinity. Indeed, we tailor Γ w.r.t F 1 ( z ) as follows. First, we choose Γ to be rectangular as this simplifies integration. To control the factor ( zI -A ) -1 in the expression of F 1 ( f ) , we need to ensure that the distance | z -λ i | for any z ∈ Γ and i ∈ [ n ] are relatively large. Since Γ separates λ p and λ p +1 , this minimal distance min z ∈ Γ ,i ∈ [ n ] | z -λ i | cannot exceed Θ( δ p ) . Thus, we simply construct Γ through the midpoint x 0 = λ p + λ p +1 2 . Finally, by setting the contour sufficiently high in the complex plane (while avoiding excessive height to prevent | f ( z ) | from diverging), we ensure that the primary contribution to the integral is from the vertical segments of Γ . This is because the distance | z -λ i | is minimized on these segments. Note that, under the assumption 4 ∥ E ∥ < δ p , this construction ensures that the p -leading eigenvalues of A and ˜ A are well aligned inside the contour.

Now, in particular, to prove Theorem 2.1, we will estimate

in which the contour Γ is set to be a rectangle with vertices ( x 0 , T ) , ( x 1 , T ) , ( x 1 , -T ) , ( x 0 , -T ) , where x 0 := λ p -δ p / 2 , x 1 := 2 λ 1 , T := 2 λ 1 . Then, we split Γ into four segments: Γ 1 := { ( x 0 , t ) | -T ≤ t ≤ T } ; Γ 2 := { ( x, T ) | x 0 ≤ x ≤ x 1 } ; Γ 3 := { ( x 1 , t ) | T ≥ t ≥ -T } ; Γ 4 := { ( x, -T ) | x 1 ≥ x ≥ x 0 } .

λ

p

+1

λ

p

λ

1

Γ

3

Γ

1

Γ

2

Γ

4

Given the construction of Γ , we have 2 πF 1 = ∑ 4 k =1 M k , where

Intuitively, we set T, x 1 large ( = 2 ∥ A ∥ ) so that the main term is the integral along Γ 1 , i.e., M 1 . Indeed, factoring our E and using the fact that | z -λ i | ≥ | z -λ p | = √ δ 2 p + t 2 for all 1 ≤ i ≤ n and z ∈ Γ 1 := { ( x 0 , t ) | -T ≤ t ≤ T } , we have M 1 ≤ ∫ Γ 1 ∥ E ∥ · | z | min i ∈ [ n ] | z -λ i | 2 | dz | ≤ ∥ E ∥ · ∫ T -T √ x 2 0 + t 2 ( δ p / 2) 2 + t 2 dt. Directly compute the integral ∫ T -T √ x 2 0 + t 2 ( δ p / 2) 2 + t 2 dt (see Section E.3), we obtain:

By a similar manner, replace Γ 1 by Γ 3 := { ( x 1 , t ) | -T ≤ t ≤ T } , we have

where the last inequality follows the fact that min i ∈ [ n ] | z -λ i | = | z -λ 1 | = √ ( x 1 -λ 1 ) 2 + t 2 = √ λ 2 1 + t 2 . Directly compute the integral ∫ T -T √ x 2 1 + t 2 λ 2 1 + t 2 dt (see Section E.3), we obtain:

Similarly, M 2 , M 4 = O ( ∥ E ∥ ) ( Section E.2). These estimates on M 1 , M 2 , M 3 , M 4 imply F 1 ( z ) = O ( ∥ E ∥ · λ p δ p ) , which together with Lemma 3.1 proves Theorem 2.1.

Proving the contour bootstrapping lemma (Lemma 3.1). The first observation is that using the Sherman-Morrison-Woodbury formula M -1 -( M + N ) -1 = ( M + N ) -1 NM -1 [20] and the fact that ˜ A = A + E , we obtain

Using this, we can rewrite

Using triangle inequality, we first see that F ( f ) is at most

Next is the key observation that the second term in the equation above can be rearranged and upperbounded as follows so that the original perturbation appears again:

Thus, we have

Now we need our gap assumption that 4 ∥ E ∥ < δ p and the construction of Γ , which imply min z ∈ Γ ,i ∈ [ n ] | z -λ i | ≥ δ p / 2 ≥ 2 ∥ E ∥ . Therefore, we have

Together with (3), it follows that F ( f ) ≤ F 1 ( f ) + 1 2 F ( f ) . Therefore, 1 2 F ( f, S ) ≤ F 1 ( f, S ) . This proves Lemma 3.1. 2

Remark 3.2. Using a similar strategy, one can prove that

see Appendix F. Together, this estimate and Lemma 3.1 prove Theorem 2.3.

Second upper bound of M 1 and proof of Theorem 2.2. The key idea of the second bound is to replace ( zI -A ) -1 by its spectral expansion ∑ n i =1 u i u ⊤ i z -λ i . Hence, M 1 is rewritten as ∫ Γ 1 ∥ ∑ 1 ≤ i,j ≤ n z ( z -λ i )( z -λ j ) u i u ⊤ i Eu j u ⊤ j ∥ dz.

There are n 2 terms in the expression, and the direct use of the triangle inequality cannot provide a good estimate. The next key trick is grouping up the r -top eigenvectors { u i } r i =1 . Formally, M 1 is at most

To estimate the first term, we apply the triangle inequality. For each term, we factor out components independent of z and carefully evaluate the integral. Specifically, by the triangle inequality, the first term is at most

Since max 1 ≤ i,j ≤ r | u ⊤ i Eu j | ≤ x, ∥ u i u ⊤ j ∥ = 1 , and Γ 1 := { z | z = x 0 + i t, -T ≤ t ≤ T } , the r.h.s. is at most

By the construction of Γ 1 , we have | x 0 -λ i | ≥ δ p 2 for all i ∈ [ n ] . Thus, the r.h.s. is bounded by r 2 x ∫ T -T | x 0 | + | t | t 2 +( δ p / 2) 2 dt, which by direct computation (see Appendix E.1 for full details) is less than or equals

To estimate the second term, we apply matrix-norm inequalities to factor out E from the integral: ∫ Γ 1 ∥ ∑ n i,j = r z ( z -λ i )( z -λ j ) u i u ⊤ i Eu j u ⊤ j ∥| dz | ≤ ∫ Γ 1 | z | · ∥ ∑ n ≥ i>r u i u ⊤ i z -λ i ∥ · ∥ E ∥ · ∥ ∑ n ≥ i>r u i u ⊤ i z -λ i ∥| dz | , which is at most ∥ E ∥ ∫ Γ 1 | z | min n ≥ i>r | z -λ i | 2 | dz | = ∥ E ∥ ∫ T -T √ x 2 0 + t 2 min n ≥ i>r [( x 0 -λ i ) 2 + t 2 ] dt. Moreover, by the construction of Γ 1 and the definition of r , | x 0 -λ i | = | ( λ p + λ p +1 ) / 2 -λ i | ≥ | ( λ p + λ p +1 ) / 2 -λ r +1 | ≥ λ p -λ r +1 2 ≥ λ p 4 , where the first inequality follows the fact i > r . Thus, the second term is at most

see Section E.1 for the detailed estimation.

Similar to estimating the second term, the last term is also ˜ O ( ∥ E ∥ ) . Combining the estimates on three parts of M 1 , we obtain M 1 ≤ ˜ O ( r 2 x λ p δ p + ∥ E ∥ ) . Consequently, by Lemma 3.1, we finally have

4 Empirical results

In this section, we empirically evaluate the sharpness of our spectral-gap bound (Theorem 2.1) in real-world settings central to privacy-preserving low-rank approximation. We compare: (1) the actual spectral error ∥ ˜ A p -A p ∥ , (2) our theoretical bound 3 7 ∥ E ∥ · λ p δ p , (3) and the classical Eckart-Young-Mirsky (EYM) bound 2( ∥ E ∥ + λ p +1 ) . Each quantity is computed over 100 trials and 20 noise levels. Because prior bounds [15, 29, 30] apply only to Gaussian noise and involve unspecified constants, we exclude them from this evaluation.

Setting. We study three covariance matrices A from the UCI Machine Learning Repository [13]: the 1990 US Census ( n = 69 ), the 1998 KDD-Cup network-intrusion data ( n = 416 ), and the Adult dataset ( n = 6 ). These matrices-henceforth Census , KDD , and Adult -are standard benchmarks in DP PCA [3, 11, 29]. The low-rank parameter p is chosen so that the Frobenius norm of A p contains > 99% of the Frobenius norm of A , giving p = 10 for A = Census , p = 2 for A = KDD , and p = 4 for A = Adult [29, Section B].

Each matrix is perturbed with either GOE noise E 1 or Rademacher noise E 2 , scaled by twenty evenly spaced factors ranging from 0 to 1 . Note that with high probability [41, 43], ∥ E 1 ∥ = ∥ E 2 ∥ = (2 + o (1)) √ n , so the gap condition 4 ∥ E k ∥ < δ p simplifies to 8 √ n < δ p . For Census ( n = 69 , p = 10 ), we have δ p ≈ 1433 . 99 > 8 √ 69 ≈ 66 . 45 . For KDD ( n = 416 , p = 2 ), we get δ p ≈ 351 . 3 > 8 √ 416 ≈ 163 . 2 . For Adult ( n = 6 , p = 4 ), we find δ p ≈ 37 . 02 > 8 √ 6 ≈ 19 . 6 . Hence 4 ∥ E k ∥ < δ p holds in all tested configurations.

Evaluation. Each data matrix is preprocessed as follows: non-numeric entries are replaced with 0 ; rows shorter than the maximum length are padded with zeros; each row is scaled to unit Euclidean norm; and each column is centered to have zero mean. We compute the covariance matrix A := M ⊤ M , where M is the processed data matrix. For each configuration ( A,E k , p ) , we run 100 independent trials. In each trial, we perturb A with noise E k ∈ { E 1 , E 2 } to form ˜ A = A + E k , compute its best rankp approximation ˜ A p , and measure the spectral error ∥ ˜ A p -A p ∥ . We compare this with our bound 7 ∥ E k ∥ · λ p /δ p and the classical EYM bound 2( ∥ E k ∥ + λ p +1 ) . Following standard practice, all reported values are averaged over 100 trials, with error bars shown for Actual Error and Our Bound (cap width = 3pt).

Result and conclusion. Across all experiments-the 69 × 69 US Census, the 416 × 416 KDDCup, and the 6 × 6 Adult matrix-our bound closely matches the empirical error for both Gaussian and Rademacher noise (Figs. 1-2), consistently outperforming the classical EYM estimate. (Note: the error bars for Census and KDD are too small to see.) Over all three benchmark datasets, two distinct noise models, and twenty escalation levels per model, our spectral-gap estimate never deviates from the observed error by more than a single order of magnitude. This uniform tightness, achieved without any dataset-specific tuning, demonstrates that the bound of Theorem 2.1 is not merely sufficient but practically sharp across matrix sizes spanning two orders of magnitude and privacy-motivated perturbations spanning the entire operational range. Consequently, the bound can serve as a reliable, application-agnostic error certificate for low-rank covariance approximation in both differential-privacy pipelines and more general noisy-matrix workflows.

104

103

102

103

Error (log scale)

(aje3s 6ol) jog

102

(a(ess 6ol) o.13

101

102

101

101

100

Actual Error

Actual error

Actual error

100

EYM bound$

Our bounds

100

EYM bound$

Our bound

Our bound

EYM bound

0.2

0.4

0.6

0.8

1.0

0.2

0.4

0.6

0.8

1.0

Z'0

0.4

0.6

0.8

1.0

Noise level α

Noise level α

Noise level o

Census-Rademacher Noise

104

103

102

103

Error (log scale)

Error (log scale)

102

(a(es bot) jo3

101

102

101

101

100

Actual Error

Actual error

Actual error

100

Our bounds

EYM bound$

100

Our bound

EYM bound$

Our bound

EYM bound

0.2

0.4

0.6

0.8

1.0

0.2

0.4

0.6

0.8

1.0

Z'0

0.4

0.6

0.8

1.0

Noise level α

Noise level α

Noise level o

5 Conclusion and future work

We established new spectral norm perturbation bounds for low-rank approximations that explicitly account for the interaction between a matrix A and its perturbation E . Our results extend the Eckart-Young-Mirsky theorem, improving upon prior Frobenius-norm-based analyses. A key contribution is a novel application of the contour bootstrapping technique, which simplifies spectral perturbation arguments and enables refined estimates. Our bounds provide sharper guarantees for differentially private low-rank approximations with high probability spectral norm bounds that improve upon prior results. We also extended our approach to general spectral functionals, broadening its applicability.

Several limitations and open questions remain. While spectral norm error bounds are standard and widely used in both theoretical and applied settings, can we extend our analysis to other structured metrics such as Schattenp norm, the Ky Fan norm, or subspace affinity norm? Can our bounds be further refined for matrices with specific spectral structures, such as polynomial or exponential decay? What can be the threshold for the gap assumption so that one still obtains a meaningful bound beyond the Eckart-Young-Mirsky theorem? 4 Additionally, real-world noise often exhibits structured dependencies-can our techniques be adapted to handle sparse or correlated perturbations?

Acknowledgments

This work was funded in part by NSF Award CCF-2112665, Simons Foundation Award SFI-MPSSFM-00006506, and NSF Grant AWD 0010308.

References

Dimitris Achlioptas and Frank McSherry. Fast computation of low-rank matrix approximations. Journal of the ACM (JACM) , 54(2):9-es, 2007.

U. Alon, N. Barkai, D. A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America , 96(12):6745-6750, 1999.

Kareem Amin, Travis Dick, Alex Kulesza, Andres Munoz, and Sergei Vassilvitskii. Differentially private covariance estimation. Advances in Neural Information Processing Systems , 32, 2019.

Y. Azar, A. Flat, A. Karlin, F. McSherry, and J. Saia. Spectral analysis of data. In Proceedings of the thirty-third annual ACM symposium on Theory of computing , pages 619-626, 2001.

Zhidong Bai and Jack William Silverstein. Spectral analysis of large dimensional random matrices . Springer, 2009.

James Bennett and Stan Lanning. The Netflix Prize. In Proceedings of KDD cup and workshop , volume 2007, page 35. New York, NY, USA., 2007.

Rajendra Bhatia. Matrix analysis , volume 169. Springer Science & Business Media, 2013.

Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. The Johnson-Lindenstrauss transform itself preserves differential privacy. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science , pages 410-419. IEEE, 2012.

Avrim Blum, Cynthia Dwork, Frank McSherry, and Kobbi Nissim. Practical privacy: the sulq framework. In Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems , pages 128-138, 2005.

Avrim Blum, John Hopcroft, and Ravindran Kannan. Foundations of data science . Cambridge University Press, 2020.

Kamalika Chaudhuri, Anand Sarwate, and Kaushik Sinha. Near-optimal differentially private principal components. Advances in neural information processing systems , 25:989-997, 2012.

C. Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. SIAM Journal on Numerical Analysis , 7:1-46, 1970.

Dheeru Dua and Casey Graff. UCI machine learning repository. https://archive.ics. uci.edu/ml , 2017.

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference , pages 265-284. Springer, 2006.

Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang. Analyze Gauss: Optimal bounds for privacy-preserving principal component analysis. In Proceedings of the forty-sixth annual ACM symposium on Theory of computing , pages 11-20, 2014.

G. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika 1 , pages 211-218, 1936.

Moritz Hardt. Robust subspace iteration and privacy-preserving spectral analysis. In 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton) , pages 1624-1626. IEEE, 2013.

Moritz Hardt and Eric Price. The noisy power method: A meta algorithm with applications. Advances in neural information processing systems , 27, 2014.

Nicholas J. Higham. Functions of Matrices: Theory and Computation . SIAM, 2008. See §1.3 and §7.6 for the Cauchy-Dunford integral.

R. A. Horn and C. R. Johnson. Matrix Analysis . Cambridge University Press, 2012.

M. Ivanovs, R. Kadikis, and K. Ozols. Perturbation-based methods for explaining deep neural networks: A survey. Pattern Recognition Letters , 150:228-234, 2021.

M. Jirak and M. Wahl. Perturbation bounds for eigenspaces under a relative gap condition. Proceedings of the American Mathematical Society , 148(2):479-494, 2020.

R. Kannan, H. Salmasian, and S. Vempala. The spectral method for general mixture models. SIAM Journal on Computing , 38(3):1141-1156, 2008.

R. Kannan and S. Vempala. Spectral algorithms. Foundations and Trends in Theoretical Computer Science , 4(3-4):157-288, 2009.

Michael Kapralov and Kunal Talwar. On differentially private low rank approximation. In Proceedings of the twenty-fourth annual ACM-SIAM symposium on Discrete algorithms , pages 1395-1414. SIAM, 2013.

Tosio Kato. Perturbation Theory for Linear Operators . Classics in Mathematics. Springer, New York, NY, 1980.

V. Koltchinskii and K. Lounici. Concentration inequalities and moment bounds for sample covariance operators. Bernoulli , 23:110-133, 2017.

Vladimir Koltchinskii and Dong Xia. Perturbation of linear forms of singular vectors under Gaussian noise. In High Dimensional Probability VII: The Carg` ese Volume , pages 397-423. Springer, 2016.

Oren Mangoubi and Nisheeth Vishnoi. Re-analyze Gauss: Bounds for private matrix approximation via Dyson Brownian motion. In Advances in Neural Information Processing Systems , volume 35, pages 38585-38599, 2022.

Oren Mangoubi and Nisheeth K. Vishnoi. Private low-rank approximation for covariance matrices, Dyson Brownian Motion, and eigenvalue-gap bounds for Gaussian perturbations. J. ACM , 72(2), March 2025.

Oren Mangoubi, Yikai Wu, Satyen Kale, Abhradeep Thakurta, and Nisheeth K Vishnoi. Private matrix approximation and geometry of unitary orbits. In Conference on Learning Theory , pages 3547-3588. PMLR, 2022.

Sean O'Rourke, Van Vu, and Ke Wang. Random perturbation of low rank matrices: Improving classical bounds. Linear Algebra and its Applications , 540:26-59, 2018.

Sean O'Rourke, Van Vu, and Ke Wang. Matrices with Gaussian noise: Optimal estimates for singular subspace perturbation. IEEE Transactions on Information Theory , 2023.

Or Sheffet. Old techniques in differentially private linear regression. In Algorithmic Learning Theory , pages 789-827. PMLR, 2019.

G. W. Stewart and Ji Guang Sun. Matrix Perturbation Theory . Academic Press, 1990. See Chap. III, §3.

Phuc Tran and Nisheeth K. Vishnoi. Perturbation bounds for low-rank inverse approximations under noise. In Proceedings of the 39th Conference on Neural Information Processing Systems (NeurIPS 2025) , 2025.

Phuc Tran and Van Vu. Davis-Kahan theorem under a moderate gap condition. Communications in Contemporary Mathematics , 2025. World Scientific, doi: 10.1142/S021919972550035X.

Phuc Tran and Van Vu. New matrix perturbation bounds with relative norm: Perturbation of eigenspaces. ArXiv preprint: 2409.20207 , 2026.

Jalaj Upadhyay. The price of privacy for low-rank factorization. Advances in Neural Information Processing Systems , 31, 2018.

Ramon Van Handel. On the spectral norm of Gaussian random matrices. Transactions of the American Mathematical Society , 369(11):8161-8178, 2017.

Sabine Van Huffel and Joos Vandewalle. On the accuracy of total least squares and least squares techniques in the presence of errors on all data. Automatica , 25(5):765-769, 1989.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018.

Van Vu. Spectral norm of random matrices. Combinatorica , 27(6):721-736, 2007.

Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge university press, 2019.

M.J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic view point . Cambridge Series in Statistical and Probabilistic Mathematics, 2019.

Hermann Weyl. Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen. Mathematische Annalen , 71(4):441-479, 1912.

Contents

A Limitations of prior approaches

This section explains why existing perturbation methods fail to yield spectral norm bounds of the form ∥ ˜ A p -A p ∥ that incorporate interaction between A and the perturbation E .

Eckart-Young-Mirsky: lack of interaction sensitivity. Let σ 1 ≥ σ 2 ≥ · · · ≥ σ n ≥ 0 denote the singular values of A . The Eckart-Young-Mirsky theorem gives ∥ A -A p ∥ = σ p +1 , and by the triangle inequality:

where the final step uses Weyl's inequality [46]. While this bound is assumption-free, it is uninformative in regimes where σ p +1 ≫∥ E ∥ , which are common in practice. The key limitation is that the triangle inequality treats A and E independently, failing to capture how structure or spectral gaps in A might mitigate the effect of E .

Mangoubi-Vishnoi: Frobenius only, spectral norm intractable. The strategy of [29, 30] models noise as a continuous-time matrix-valued Brownian motion:

where U ( t ) = [ u i ( t )] and λ 1 ( t ) ≥ · · · ≥ λ n ( t ) . The rankp approximation at time t is

The total perturbation is then expressed as an integral:

Using properties of Dyson Brownian motion and Itˆ o calculus, they derive a Frobenius-norm identity:

̸

̸

Bounding these expressions depends on repulsion properties of the eigenvalues; for GOE matrices, Weyl's inequality suffices, while for GUE matrices, stronger gap estimates are used.

Although this method captures the spectral structure of A and interaction with E , it only yields Frobenius-norm bounds. Extending it to the spectral norm would require controlling

which entails bounding the operator norm of the full stochastic process. This requires detailed control over the dynamics of U ( t ) and λ ( t ) , including their correlations-none of which are tractable with current techniques.

Moreover, for generalized functionals such as ∥ f p ( ˜ A ) -f p ( A ) ∥ , the problem becomes even harder: one must analyze ∫ 1 0 df p ( A ( t )) , which involves matrix-valued analytic functions under random perturbation, a setting far beyond existing random matrix tools.

In contrast, our approach bypasses these limitations by using a complex-analytic representation of spectral projectors that directly captures interaction between A and E , yielding sharp spectral norm bounds under broad assumptions.

with eigen-decomposition

40

100

40

35

80

30

30

25

60

20

20

15

40

10

10

20

5

000

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

Noise level α

Noise level o

Noise level α

B Comparison of error metrics

This section studies three common metrics for low-rank approximation under perturbation-namely: - the spectral-norm error ∥ ˜ A p -A p ∥ , - the Frobenius-norm error ∥ ˜ A p -A p ∥ F , and - the 'changein-error' ∣ ∣ ∥ A -A p ∥ - ∥ A -˜ A p ∥ ∣ ∣ .

We compare these metrics both empirically (through Monte Carlo simulations) and theoretically. Empirically, we examine how the metrics behave under Gaussian noise applied to both synthetic and real-world matrices (Figure 3). Theoretically, we analyze their interpretability and limitations, highlighting that while Frobenius norms capture aggregate error and change-in-error quantifies residual shifts, only the spectral norm controls worst-case subspace distortion.

A simple 2 × 2 example (Example B.1) further illustrates how residual-based measures can completely mask subspace drift, underscoring the robustness and interpretability of the spectral norm for tasks such as private low-rank approximation.

Empirical comparison of utility metrics. We perform three Monte Carlo experiments under additive Gaussian perturbations. The first uses a synthetic PSD matrix A ∈ R 50 × 50 with exponentially decaying eigenvalues λ i = 0 . 8 i , and sets p = 5 . The second and third use real-world covariance matrices derived from: - the 1990 US Census dataset ( n = 69 ), - the 1998 KDD-Cup dataset ( n = 416 ).

All datasets are drawn from the UCI Machine Learning Repository [13] and have been widely used in private matrix approximation and PCA [30, 29, 11].

In each setting, we compute the best rankp approximation A p , perturb A with symmetric Gaussian noise of varying standard deviation σ , and measure:

Spectral norm deviation: ∥ ˜ A p - A p ∥ ,

Frobenius norm deviation: ∥ ˜ A p - A p ∥ F ,

Change-in-error: ∣ ∣ ∣ ∥ A - A p ∥ - ∥ A - ˜ A p ∥ ∣ ∣ ∣ .

As shown in Figure 3, the Frobenius norm error grows fastest, reflecting total energy deviation. The change-in-error metric remains much smaller and, in the real-world cases, nearly flat, suggesting it may fail to capture meaningful distortion. Notably, in the synthetic case (left), the spectral norm error closely tracks the change-in-error-despite their differing intent-which may result from nearalignment of the top subspaces. However, such behavior is not guaranteed in general.

Theoretical distinction between utility metrics. Frobenius norm bounds of the form ∥ ˜ A p -A p ∥ F ≤ ε F aggregate squared deviations across all directions, but may hide large errors in in-

dividual components. Spectral norm bounds ∥ ˜ A p -A p ∥ ≤ ε directly constrain the worst-case deviation and are thus more reliable in sensitive applications such as differentially private PCA.

In contrast, residual-error metrics such as ∥ A -A p ∥ - ∥ A -˜ A p ∥ are commonly used for their analytical convenience. However, they reflect only changes in residual energy and are insensitive to subspace movement. In particular, this metric can be nearly zero even when the topp eigenspaces have shifted significantly.

Given the spectral decompositions

the change-in-error vanishes whenever U p U ⊤ p ≈ ˜ U p ˜ U ⊤ p and λ p +1 is large. Such conditions are typical when noise E is small and p ≤ sr( A ) := ∑ n i =1 λ i /λ 1 . Moreover, standard perturbation results imply

Example B.1 ( Rank-1 rotation in R 2 ) . Let

so that A p = A . Define the rotated matrix

Then ˜ A p = ˜ A , and although the top eigenspace has rotated by θ , the change-in-error is zero:

Yet the true subspace drift is visible in:

This example highlights the limitations of residual-based utility metrics and illustrates why spectral norm deviation provides a more reliable and interpretable signal of approximation quality under perturbation.

In summary, both our analysis and experiments support the use of the spectral norm as the most informative and robust error metric for evaluating private low-rank approximations. Unlike Frobenius and residual metrics, it captures the worst-case directional distortion and provides a tighter connection to subspace stability.

C Empirical evaluation beyond gap assumption

In this section, we empirically compare (1) the actual spectral error ∥ ˜ A p -A p ∥ , (2) our theoretical bound 7 ∥ E ∥ · λ p δ p , (3) and the classical Eckart-Young-Mirsky (EYM) bound 2( ∥ E ∥ + λ p +1 ) in the setting beyond the gap assumption that 4 ∥ E ∥ < δ p .

Setup. We conducted a simulation on a covariance matrix A with n = 2000 , derived from the Alon colon-cancer microarray dataset [2]. The low-rank parameter p is chosen so that the Frobenius norm of A p contains > 95% of the Frobenius norm of A , giving p = 9 with λ p ≈ 46 . 29 . We first computed δ p . Gaussian noise was then added in the form E = α · N (0 , I n ) , with α chosen over 11 evenly spaced values such that

For each α , we computed the following quantities:

the true error: ∥ ˜ A p -A p ∥ ,

the classical EYM bound: 2( ∥ E ∥ + σ p +1 ) ,

our bound: 7 ∥ E ∥ · λ p δ p ,

the ratios our bound true error and our bound classical bound .

Results. Table 2 summarizes the results. The ratio our bound true error remains remarkably stable even beyond the regime 4 ∥ E ∥ < δ p (i.e., ∥ E ∥ δ p < . 25 ), and our bound outperforms the classical bound precisely when 4 ∥ E ∥ < δ p (i.e., ∥ E ∥ δ p < . 25 ).

D Extensions of Theorem 2.1 and Theorem 2.2 to the symmetric matrices

In this section, we extend Theorem 2.1 and Theorem 2.2 to the setting where A is a symmetric matrix. These extensions are naturally important since the data in real-world applications is often arbitrary, making it natural for the eigenvalues of A to span both signs. While singular value decomposition (SVD) could be used to apply Theorem 2.1 or Theorem 2.2, singular value gaps are typically small. By working directly with eigenvalues, we exploit the fact that the eigenvalue gap δ k = λ k -λ k +1 is significantly large when λ k · λ k +1 < 0 .

D.1 Extension of Theorem 2.1 to the symmetric matrices

To simplify the presentation, we assume that the eigenvalues (singular values) are different, so the eigenvectors (singular vectors) are well-defined (up to signs). However, our results hold for matrices with multiple eigenvalues. Let A,E be n × n real symmetric matrices, and let 1 ≤ p ≤ n denote the rank of approximation. Let λ k be the k th largest eigenvalue of A and u k be the corresponding orthonormal eigenvector. Let ˜ A := A + E . Let A p , ˜ A p denote the best rankp approximations of A and ˜ A respectively. Define 1 ≤ k ≤ p such that the set of the top p singular values corresponds to { λ π (1) , . . . , λ π ( p ) } = { λ 1 , . . . , λ k > 0 ≥ λ n -( p -k )+1 , . . . , λ n } . In other words, the p th singular value of A is either λ k or | λ n -( p -k )+1 | . Let δ i := λ i -λ i +1 , for i ∈ [ n -1] . Theorem 2.1 is extended to the following result.

Theorem D.1 ( Extension of Theorem 2.1 to the symmetric matrices). If 4 ∥ E ∥ ≤ min { δ k , δ n -( p -k ) } , and 2 ∥ E ∥ < σ p -σ p +1 , then

Note that when A is not PSD, {| ˜ λ 1 | , . . . , | ˜ λ k | , | ˜ λ n -( p -k )+1 | , . . . , | ˜ λ n |} may not correspond to the p leading singular values of ˜ A . This issue is resolved by enforcing the singular-value gap condition σ p -σ p +1 > 2 ∥ E ∥ . Indeed, by Weyl's inequality, given σ p -σ p +1 > 2 ∥ E ∥ , we have

here δ = σ p -σ p +1 . By a similar argument, we also have | ˜ λ n -( p -k )+1 | > ˜ λ k +1 . Therefore,

as we want. Note that the gap condition of eigenvalues cannot guarantee this fact. For example, consider the following matrices

Here, clearly, S = { 1 } , ˜ S = { 1 } and | λ 1 | is the largest singular value of A , but | ˜ λ 1 | is not the largest singular value of ˜ A ( ˜ λ 1 is still the largest eigenvalue).

Proof of Theorem D.1 Let 1 ≤ k ≤ p be a natural number such that

Thus, we can split A p as A k + B p -k , in which

Similarly, ˜ A p = ˜ A k + ˜ B p -k . Therefore,

Applying the contour bootstrapping argument on ∥ ∥ ∥ ˜ A k -A k ∥ ∥ ∥ with contour Γ [1] and on ∥ ∥ ∥ ˜ B p -k -B p -k ∥ ∥ ∥ with another contour Γ [2] (we define these contours later), we obtain

We set Γ [1] and Γ [2] to be rectangles, whose vertices are

and

Now, we are going to bound F [1] 1 . First, we split Γ [1] into four segments:

λ

k

+1

λ

k

λ

1

Γ

3

Γ

1

Γ

2

Γ

4

Re

(

z

) = 0

Therefore,

Notice that we further obtain

in which

We use the following lemmas, whose proofs are delayed to the next section.

Lemma D.2. Under the assumption of Theorem D.1,

Lemma D.3. Under the assumption of Theorem D.1,

Lemma D.4. Under the assumption of Theorem D.1,

Since p < n , then k + 1 > n -( p -k ) + 1 and hence k + 1 / ∈ { π (1) , . . . , π ( p ) } . It means | λ k +1 | ≤ λ k . Thus 0 ≤ a 0 ≤ λ k , and hence

By the setting that a 1 = T = 2 σ 1 ,

Thus, using above estimates, we obtain

Applying a similar argument on contour Γ [2] , we obtain

Combining (4), (5) and (6), we complete our proof.

D.2 Extension of Theorem 2.2 to the symmetric matrices

Let A be a symmetric matrix with eigenvalues λ 1 ≥ λ 2 ≥ ·· · ≥ λ n , in which λ n is not necessarily positive. Recall the setting from the previous section that 1 ≤ k ≤ p is the positive integer such that the set of the top p singular values is { λ π (1) , . . . , λ π ( p ) } = { λ 1 , . . . , λ k > 0 ≥ λ n -( p -k )+1 , . . . , λ n } . To extend Theorem 2.2, we first generalize the definition of the halving distance r and interaction term x as follows. Let r 1 , r 2 respectively be the smallest positive integer satisfying λ k 2 ≤ λ k -λ r 1 +1 , and | λ n -( p -k )+1 | 2 ≤ λ n -r 2 +1 -λ n -( p -k )+1 . Define the 'halving distance' r := max { r 1 , r 2 } . Next, let x 1 := max 1 ≤ i,j ≤ r 1 | u ⊤ i Eu j | and x 2 := max 1 ≤ i,j ≤ r 2 | u ⊤ n -i +1 Eu n -j +1 | . Define the interaction parameter ¯ x := max { x 1 , x 2 } .

Theorem D.5 ( Extension of Theorem 2.2 to the symmetric matrices). Assume that 4 ∥ E ∥ ≤ min { δ k , δ n -( p -k ) } and 2 ∥ E ∥ < σ p -σ p +1 , then

Proof of Theorem D.5 First, we still split ( ˜ A p , A p ) into ( A k , B p -k , ˜ A k , ˜ B p -k ) and apply the contour bootstrapping argument on ∥ ∥ ∥ ˜ A k -A k ∥ ∥ ∥ , ∥ ∥ ∥ ˜ B p -k -B p -k ∥ ∥ ∥ . We also obtain

However, we will treat F [1] 1 , F [2] 1 a bit differently. Indeed,

in which

and

We additionally use the following lemma (its proof will be delayed in the next section).

Lemma D.6. Under the assumption of Theorem D.5,

Together with the estimates for N 2 , N 3 , N 4 from the previous section, we obtain

Thus,

Similarly,

Therefore, combining (4), (7), and (8), we finally obtain

E Estimating integrals over segments

In this section, we present in detail the integral estimations mentioned in the previous section: Lemma D.2, Lemma D.3, Lemma D.6 (integration over vertical segments); and Lemma D.4 (integration over horizontal segments) . We first present a technical lemma, which is used several times in the upcoming sections.

Lemma E.1. Let a, T be positive numbers such that a ≤ T . Then,

E.1 Estimating integrals over vertical segments for interaction-dependent bound

In this Section, we now estimate M 1 - integral over the left vertical segment (prove Lemma D.6) and estimate N 3 - the integral over the right vertical segment (prove Lemma D.3). First, we estimate M 1 as follows.

Using the spectral decomposition ( zI -A ) -1 = ∑ n i =1 u i u ⊤ i ( z -λ i ) , we can rewrite M 1 as

Define x 1 := max 1 ≤ i,j ≤ r 1 ∣ ∣ u ⊤ i Eu j ∣ ∣ . By the triangle inequality, M 1 is at most

Consider the first term, by the triangle inequality, we have

By the construction of Γ 1 , we have

Thus, the r.h.s. is at most

The second integral is estimated by what follows.

Therefore,

Next, we bound the second term as follows

Moreover, by the construction of Γ 1 and the definition of r 1 ,

where the second inequality follows the fact i > r 1 . Thus, the r.h.s. is at most

Similar to (10) and (11), we also have

It follows that

Now we consider the last term:

By (13) and (9), the r.h.s. is at most

Note that a 0 -λ r +1 ≥ δ k / 2 and a 0 -λ r +1 + δ k / 2 = λ k -λ r +1 ≥ λ k 2 . Therefore, a 0 -λ r +1 = max { δ k / 2 , a 0 -λ r +1 } ≥ λ k 4 . We further obtain that ∫ T 0 a 0 + t √ ( t 2 +( δ k / 2) 2 )( t 2 +( a 0 -λ r +1 ) 2 ) dt is at most

The estimates (16) and (17) together imply that the last term is at most

Combining (12), (15) and (18), we finally obtain that M 1 is at most

This proves Lemma D.6.

Next, we estimate N 3 . Notice that

E.2 Estimating integrals over horizontal segments

We are going to bound N 2 , N 4 - integral over top horizontal segment (prove Lemma D.4). We have

By similar arguments, we also obtain

These estimates on N 2 , N 4 prove Lemma D.4.

E.3 Estimating integrals over vertical segments for non-interaction bound

In this Section, we estimate N 1 , proving Lemma D.2. The estimation of N 3 follows the case of the interaction-dependent bound at the end of Section E.1.

F Perturbation of matrix functionals - Theorem 2.3

In this section, we complete the delayed proof of Theorem 2.3. By Remark 3.2, to prove Theorem 2.3, we need to show that

in which the contour Γ is set to be a rectangle with vertices

We split Γ into four segments:

Therefore, in which

By a similar strategy from previous section, we bound M 1 as follows. Notice that

Therefore, M 1 is at most

Next, we bound M 3 as what follows.

Next, we estimate M 2 as

Moreover, since Γ 2 := { z | z = x + i T, x 0 ≤ x ≤ x 1 } ,

Therefore, M 2 ≤ ∥ E ∥·| x 1 -x 0 | T 2 ≤ ∥ E ∥ λ 1 4 λ 2 1 = ∥ E ∥ 4 λ 1 . Similarly, we also obtain that M 4 = ∥ E ∥ 4 λ 1 . These estimates on M 1 , M 2 , M 3 , M 4 and Equation 20 imply

The last inequality follows the trivial fact that λ 1 > δ p for any PSD matrix A . We complete the proof.

G Some classical perturbation bounds

This section recalls standard results referenced in Section 2, Section 3, and Section A.

Theorem G.1 ( Eckart-Young-Mirsky bound [16]) . Let A, ˜ A ∈ R n × n , and let A p , ˜ A p denote their respective best rankp approximations. Set E := ˜ A -A . Then,

where σ p +1 is the ( p +1) st singular value of A .

Theorem G.2 ( Weyl's inequality [46]) . Let A,E ∈ R n × n be symmetric, and define ˜ A := A + E . Then, for any 1 ≤ i ≤ n ,

where λ i , ˜ λ i are the i th eigenvalues of A and ˜ A , and σ i , ˜ σ i are the corresponding singular values.

H Notation

This section summarizes the key notations used throughout the paper. Let A,E be symmetric n × n matrices, and define the perturbed matrix ˜ A := A + E . Let f be an entire function, and let s ∈ N .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction state that the paper provides new highprobability spectral norm bounds for low-rank approximation under symmetric perturbations, resolving key limitations of classical worst-case bounds and prior DP utility analyses, and the body of the paper rigorously proves and empirically validates this claim (Sections 2-4).

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Section 5 (Conclusion, Limitations, and Future Work) discusses the reliance on spectral quantities, the limitations of our results beyond the gap threshold, and the open questions of extending the framework to structured perturbations, including data matrices with specific spectral patterns or correlated noise.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Justification: All main theorems (e.g., Theorem 2.1) include clear assumptions, and full proofs are provided in Sections 3, Appendix D, Appendix E, and Appendix F.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Section 4 details the matrices used, noise models, parameter settings, evaluation metrics, and empirical setup to enable reproducibility.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The data (e.g., Census and 1998 KDD-Cup ) are publicly available and cited appropriately. Code and instructions are provided in the supplemental material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Section 4 and Section B describe matrix dimensions, truncation ranks, noise scales, trial counts, and the methods used to compute bounds.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Section 4 and Section B report error bars across 100 trials as mean ± standard deviation, with clear plots and captions.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The experiments are lightweight and run on standard CPU machines; resource requirements are described in the supplemental material.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research is theoretical and empirical, uses only publicly available datasets, and conforms to ethical standards.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a theoretical paper on spectral norm perturbation bounds with no direct societal or ethical impact pathways.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper does not release models or datasets with any risk of misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All datasets (e.g., Census, 1998 KDD-Cup, Adult) are properly cited (e.g., [29], [11], [3]) and are in the public domain or released under open academic licenses.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce new datasets, models, or other assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve any human subjects or crowdsourcing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The research does not involve human subjects and thus does not require IRB approval.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The research does not use LLMs for any component of the core methodology.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies how an additive noise matrix affects low-rank approximation. In particular, given a symmetric $n$-by-$n$ matrix $A$ and a symmetric noise matrix $E$, the goal is to bound $\\|(A+E)_p - A_p\\|$ for some norm $\\|\\cdot\\|$. (Given a symmetric matrix $M$, let $M_p$ denote its best rank-$p$ approximation.)\nPrevious works have studied error metrics such as the Frobenius norm $\\|(A+E)_p - A_p\\|_F$ and the change in reconstruction error $| \\|(A+E)_p - A\\| - \\|A_p - A\\| |$.\nThis paper studies the spectral norm error $\\|(A+E)_p - A_p\\|_2$, which is more suitable in certain cases. This is validated both theoretically and in experiments in the paper.\nThe main result of this paper is, assuming $A$ is PSD, if $\\|E\\| \\le (\\lambda_p - \\lambda_{p+1}) / 4$, then $\\| (A+E)_p - A_p \\| \\lesssim \\| E \\| \\cdot \\frac{\\lambda_p}{\\lambda_p - \\lambda_{p+1}}$, where $\\lambda_1 \\ge \\cdots \\ge \\lambda_n \\ge 0$ are the eigenvalues of $A$. (They also extend the result to general symmetric matrices. The bound is less clean than the PSD case.)\nTheir result is better than the Eckart–Young–Mirsky bound than a factor of $\\min\\{\\frac{\\lambda_{p+1}}{\\|E\\|}, \\frac{\\lambda_{p} - \\lambda_{p+1}}{\\|E\\|}\\}$, which is a meaningful improvement when $\\lambda_{p+1} \\gg \\|E\\|$ and $\\lambda_{p} - \\lambda_{p+1} \\gg \\|E\\|$.\nWhen translating the previous Frobenius norm bound by Mangoubi and Vishnoi to the spectral norm bound, it will be worse than the error bound in this paper by a factor of $\\sqrt{p}$. Moreover, the previous Frobenius norm bound by Mangoubi and Vishnoi only holds in expectation, while this paper obtains high probability bounds.\nOne important application of their main result is differentially private PCA. \nThis paper also gets two more results: a more fine-grained bound that takes into account a measure of spectral decay of $A$ and the alignment between the noise $E$ and the top eigenvectors of $A$; a perturbation bound for general matrix functions.\nThe authors prove these results by using a novel contour bootstrapping method from complex analysis and extends it to a broad class of spectral functionals, including polynomials and matrix exponentials.\n\nSTRENGTHS AND WEAKNESSES:\nI think the nice results obtained in this paper would be interesting at least for the filed of differential privacy and matrix perturbation theory. The spectral norm error bound is not only of interest itself, but also is more preferable in some applications. This paper is also very well-written. It motivates the spectral norm error bound well, presents a clear proof outline, and gives a good explanation of why previous techniques fail.\n\nQUESTIONS:\nWhy consider $\\|(A+E)_p - A_p\\|$ instead of $\\|(A+E)_p - A\\|$? It seems to me a more natural goal is to approximate $A$ instead of $A_p$. (Of course, $\\|(A+E)_p - A\\| \\le \\|(A+E)_p - A_p\\| + \\|A_p - A\\|$.)\n\nLIMITATIONS:\nno", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents the error bound (in terms of the spectral norm) for the rank-$p$ approximation of a symmetric matrix perturbed with a symmetric noise. The paper is written nicely with a clear motivation and description. The mathematical tools used to solve this hard problem namely contour bootstrapping are interesting in their own right and the authors seem to have found a novel application for this tool in this work. The simulation results also seem to highlight the utility of the derived bound when compared to classical baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The derived bound is tighter than the EYM bound and leverages the symmetry of data and noise matrices. Also, the presented bounds are high probability asymptotic bounds, which are preferable over the bounds on expectation. \n2. The paper adopts a new analytical approach, contour bootstrapping. \n\nWeaknesses:\n1. The paper highlights the importance of the spectral norm error bounds over the Frobenius norm error or reconstruction error. However, the spectral norm still does not present a complete picture. In my opinion, the more appropriate metric for the low-rank approximation problem would be the affinity between true and estimated subspaces (as defined in https://arxiv.org/abs/1301.2603) or, say, the Ky Fan norm.\n2. The technical assumption in line 66 for the bound to hold is too restrictive in DP since DP mechanisms usually add excessive noise that also scales with the dimension\n\nQUESTIONS:\nMajor comments:\n1. It is important to clarify the following: How different is the derived bound from that obtained by applying norm equivalence to the Frobenius error bound in https://arxiv.org/abs/2211.06418? Particularly, the $\\sqrt{p}$ improvement, as outlined in lines 119-123,  strongly suggests this.\n2. The results in Figures 1-2 should also consider the aforementioned bound from norm equivalence for comparison.\n3. Given that the result is presented with DP as the primary application, in light of the second weakness stated in the strength and weakness section, it is desirable to have an illustrative example showing where the condition $\\Vert E \\Vert \\leq \\delta_p/4$ would fail and where it will hold (especially with the increasing dimension $n$ for different values of DP parameters, $(\\epsilon,\\delta)$) along with an illustration on how $\\lambda_p$ and $\\lambda_{p+1}$ behave in such conditions.\n\nMinor comments:\n1. It has to be clearly indicated when the asymptotic bounds are probabilistic (e.g.,  $O_{\\texttt{p}}$ and $O_{\\texttt{p}}$) - please see https://arxiv.org/abs/2103.08721 and https://arxiv.org/abs/1108.3924.\n2. Provide a reference (e.g., \"High-Dimensional Statistics: A Non-Asymptotic View-point\" by M. J. Wainwright) for lines 151-154.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-28"}

comments: SUMMARY:
The paper studies rank-$k$ approximation under spectral norm. They give really great results and something that was definitely lacking in previous works.

STRENGTHS AND WEAKNESSES:
Beautiful paper and amazing result! 

I do not see any weakness in the paper. The writing is good. I am ready to champion the paper and do not have much to say. I would have accepted the paper just with one set of results (the bound on spectral norm error). I have thought about this problem for over three years, so I understand the limitations of previous approaches used when it comes to bound the spectral norm metric and get the high probability bound (instead of expected bound in the complex Dyson Brownian based analysis).

The weakness, if I can say, is that I was not able to read the proof, but I am looking forward to completely understanding the paper and engaging with the authors during the rebuttal phase with any questions I will have.

QUESTIONS:
I have one quick question: can we use the techniques in this paper to also give high probability bound in COLT 2023 paper where the authors study the Frobenius norm metric? In my opinion, that was the weak point of the paper. I do not see that using Dyson Brownian motion based method can be used to get a high probability bound (having extensively thought on this problem since the NeurIPS 2022 paper came out).

LIMITATIONS:
None.
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work provides new perturbation bounds for low rank approximations in the spectral norm, improving over classical results that follow from the Eckart-Young-Mirsky theorem. \n\nThe main feature of the bounds is that they are sensitive to the misalignment between the perturbation and the top singular vectors. This is important for applications in differential privacy, where the perturbation is typically Gaussian noise, and hence has small projection in the direction of any fixed eigenvector. The authors discuss the applications of their work to differentially private PCA.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is well written and provides helpful discussion throughout. The main results are elegant and natural, providing interesting extensions to classical results from numerical analysis and matrix theory. The proof technique sounds intriguing. Apparently, the authors have a new technique that avoids some of the classical tools like Davis-Kahan expansions. This sounds great, although I did not have time to dig into the details.\n\nUsing the new perturbation bounds, the authors give some improvements for one particular way of achieving differentially private PCA: Add noise to the original matrix and compute a low rank approximation on top of the noisy matrix. Whereas most previous results applied only to Frobenius norm, the new results apply to spectral norm. I agree with the authors that this is the much more meaningful guarantee in realistic settings.\n\nNote, though, that this is just one way of achieving differentially private PCA. It's a well studied problem and there are numerous ways of doing it. \n\nThe authors claim that they give the first high probability bounds for differentially private PCA in the spectral norm. This is, however, not true. High probability bounds in the spectral norm already follow from the work by Hardt and Price (NeurIPS 2015) and Hardt and Roth (STOC 2013). Thos papers analyze noisy subspace iteration. The bounds are sensitive to the misalignment between the noise (gaussian) and the spectrum of the matrix. In addition, they give results for overparameterization: approximating rank p by rank k > p matrices. It's worth discussing these classical results. In which ways do the new tools give improvements over these results?\n\nIn particular, in the empirical results it would be good to compare to these methods.\n\nQUESTIONS:\n1. How do your results for DP-PCA compare to those you can achieve with the power method or other iterative methods?\n2. Do your proof techniques give results in the overparameterized regime, where we try to capture the top p eigenvectors with k > p dimensions?\n3. It would be interesting to see additional experiments on a larger datasets and in cases where the assumptions may not hold exactly.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops general perturbation bounds for spectral functions of a matrix. Specifically, for a matrix $A$, with eigenvalues and eigenvectors $\\{(\\lambda_i, u_i): i = 1,2, \\dots N\\}$, let $f_p(A)$ denote the matrix $\\sum_{i=1}^p f(\\lambda_i) u_i u_i^\\top$. The main result of the paper provides a general purpose bound on $|| f(A+E) - f(A)||$ in terms $||E||$, where $E$ denotes a perturbation in the matrix. \n\nAs a primary application of their result, the authors analyze the utility (as measured by the operator norm error between the privatized best rank-$p$ approximation and the actual best rank-$p$ approximation)  for the Gaussian mechanism in differentially private PCA, improving existing results.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n— The paper is well-written and the authors have made a good effort to present their proofs in an accessible and systematic fashion.\n— The bound appears to be quite general, and could have many potential applications.  \n—The authors use a contour integral representation of spectral functions in terms of the resolvent to obtain their perturbation bound. This technique is well-known and classical in matrix perturbation theory. The primary contribution of this paper is to exploit this technique to develop a general-purpose bound which can be readily applied without a case-by-case analysis of the contour integral representation. To the best of my knowledge, I am not aware of a similar general purpose bound, so this contribution appears novel, although the contour integral technique is quite standard.\n\nWeaknesses:\n— The authors consider a single application of their bound, and moreover, the improvement in the application is not very significant (an operator norm bound rather than the Frobenius norm bound). Some additional applications could have improved the paper.\n\nQUESTIONS:\nQuestions and Comments:\n— I think the authors should discuss prior work on deriving perturbation bounds using the contour integral representation more prominently. Currently, those citations are easy to miss (in step 1 of the proof outline) and this sends the misleading impression that this technique is developed in this paper. It would be ideal to discuss these prior works as early as possible and in some detail.\n\n— Is it possible to state the bound in Theorem 2.3 with explicit constants? This would make the result user-friendly. Currently, it might not be very obvious to a reader what the constants hidden in the big-O notation depend on (without carefully reading the proof).\n\n— In the “Empirical Results” section the authors claim the bound $7 ||E|| \\cdot \\tfrac{\\lambda_p}{\\delta_p}$.  Where does the constant $7$ come from? (The results in the main paper are stated in big-O notation). \n\n— Does the bound apply to any submultiplicative norm, or is there something special about the operator norm? If the bound is general, it would be good to state it generally.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}