Original: When Does Differentially Private Learning Not Suffer in High Dimensions?
Revised: When Does Differentially Private Learning Not Suffer in High Dimensions?

Original: Abstract
Revised: Abstract

Original: Large pretrained models can be fine-tuned with differential privacy to achieve performance approaching that of non-private models.
Revised: Large pretrained models can be fine-tuned with differential privacy to achieve performance approaching that of non-private models.

Original: A common theme in these results is the surprising observation that high-dimensional models can achieve favorable privacy-utility trade-offs.
Revised: A common theme in these results is the surprising observation that high-dimensional models can achieve favorable privacy-utility trade-offs.

Original: This seemingly contradicts known results on the model-size dependence of differentially private convex learning and raises the following research question: When does the performance of differentially private learning not degrade with increasing model size?
Revised: This seemingly contradicts known results on the model-size dependence of differentially private convex learning and raises the following research question: When does the performance of differentially private learning not degrade with increasing model size?

Original: We identify that the magnitudes of gradients projected onto subspaces is a key factor that determines performance.
Revised: We identify that the magnitudes of gradients projected onto subspaces is a key factor that determines performance.

Original: To precisely characterize this for private convex learning, we introduce a condition on the objective that we term restricted Lipschitz continuity and derive improved bounds for the excess empirical and population risks that are dimensionindependent under additional conditions.
Revised: To precisely characterize this for private convex learning, we introduce a condition on the objective that we term restricted Lipschitz continuity and derive improved bounds for the excess empirical and population risks that are dimensionindependent under additional conditions.

Original: We empirically show that in private fine-tuning of large language models, gradients obtained during fine-tuning are mostly controlled by a few principal components.
Revised: We empirically show that in private fine-tuning of large language models, gradients obtained during fine-tuning are mostly controlled by a few principal components.

Original: This behavior is similar to conditions under which we obtain dimension-independent bounds in convex settings.
Revised: This behavior is similar to conditions under which we obtain dimension-independent bounds in convex settings.

Original: Our theoretical and empirical results together provide a possible explanation for the recent success of large-scale private fine-tuning.
Revised: Our theoretical and empirical results together provide a possible explanation for the recent success of large-scale private fine-tuning.

Original: Code to reproduce our results can be found at https://github.com/lxuechen/private-transformers/ tree/main/examples/classification/spectral_analysis .
Revised: Code to reproduce our results can be found at https://github.com/lxuechen/private-transformers/ tree/main/examples/classification/spectral_analysis .

Original:  Introduction
Revised:  Introduction

Original: Recent works have shown that large publicly pretrained models can be differentially privately finetuned on small downstream datasets with performance approaching those attained by non-private models.
Revised: Recent works have shown that large publicly pretrained models can be differentially privately finetuned on small downstream datasets with performance approaching those attained by non-private models.

Original: In particular, past works showed that pretrained BERT [DCLT18] and GPT-2 [RNSS18,
Revised: In particular, past works showed that pretrained BERT [DCLT18] and GPT-2 [RNSS18,

Original: RWC + 19] models can be fine-tuned to perform well for text classification and generation under a privacy budget of ε ∈ [2 , 6] [LTLH21, YNB + 21].
Revised: RWC + 19] models can be fine-tuned to perform well for text classification and generation under a privacy budget of ε ∈ [2 , 6] [LTLH21, YNB + 21].

Original: More recently, it was shown that pretrained
Revised: More recently, it was shown that pretrained

Original: ResNets [HZRS16] and vision-Transformers [DBK + 20] can be fine-tuned to perform well for
Revised: ResNets [HZRS16] and vision-Transformers [DBK + 20] can be fine-tuned to perform well for

Original: ImageNet classification under single digit privacy budgets [DBH + 22, MTKC22].
Revised: ImageNet classification under single digit privacy budgets [DBH + 22, MTKC22].

Original: One key ingredient in these successes has been the use of large pretrained models with millions to billions of parameters.
Revised: One key ingredient in these successes has been the use of large pretrained models with millions to billions of parameters.

Original: These works generally highlighted the importance of two phenomena: (i) large pretrained models tend to experience good privacy-utility trade-offs when fine-tuned, and (ii) the trade-off improves with the improvement of the quality of the pretrained model (correlated with increase in size).
Revised: These works generally highlighted the importance of two phenomena: (i) large pretrained models tend to experience good privacy-utility trade-offs when fine-tuned, and (ii) the trade-off improves with the improvement of the quality of the pretrained model (correlated with increase in size).

Original: While the power of scale and pretraining have been demonstrated numerous times in non-private deep learning [KMH + 20], one common wisdom in private learning had been that large models tend to perform worse.
Revised: While the power of scale and pretraining have been demonstrated numerous times in non-private deep learning [KMH + 20], one common wisdom in private learning had been that large models tend to perform worse.

Original: This intuition was based on (a) results in differentially private convex optimization, most of which predicted that errors would scale proportionally with the dimension of the learning problem in the worst case, and (b) empirical observations that the noise injected to ensure privacy tends to greatly exceed the gradient in magnitude for large models [YZCL21a, Kam20].
Revised: This intuition was based on (a) results in differentially private convex optimization, most of which predicted that errors would scale proportionally with the dimension of the learning problem in the worst case, and (b) empirical observations that the noise injected to ensure privacy tends to greatly exceed the gradient in magnitude for large models [YZCL21a, Kam20].

Original: For instance, consider the problem of differentially private convex empirical risk minimization (ERM).
Revised: For instance, consider the problem of differentially private convex empirical risk minimization (ERM).

Original: Here, we are given a dataset of n examples D = { s j } nj =1 ∈ S n , a convex set K ⊆ R d (not necessarily bounded), and the goal is to perform the optimization
Revised: Here, we are given a dataset of n examples D = { s j } nj =1 ∈ S n , a convex set K ⊆ R d (not necessarily bounded), and the goal is to perform the optimization

Original: [Equation]
Revised: [Equation]

Original: subject to differential privacy, where f ( · ; s ) is convex over K for all s ∈ S .
Revised: subject to differential privacy, where f ( · ; s ) is convex over K for all s ∈ S .

Original: For bounded K , past works presented matching upper and lower bounds that are dimension-dependent under the usual Lipschitz assumption on the objective [BST14, CMS11].
Revised: For bounded K , past works presented matching upper and lower bounds that are dimension-dependent under the usual Lipschitz assumption on the objective [BST14, CMS11].

Original: These results seem to suggest that the performance of differentially private ERM algorithms inevitably degrades with increasing problem size in the worst case, and present a seeming discrepancy between recent empirical results on large-scale fine-tuning.
Revised: These results seem to suggest that the performance of differentially private ERM algorithms inevitably degrades with increasing problem size in the worst case, and present a seeming discrepancy between recent empirical results on large-scale fine-tuning.

Original: To better understand the relation between problem size and the performance of differentially private learning, we study the following question both theoretically and empirically:
Revised: To better understand the relation between problem size and the performance of differentially private learning, we study the following question both theoretically and empirically:

Original: When does the performance of differentially private stochastic gradient descent (DP-SGD) not degrade with increasing problem dimension?
Revised: When does the performance of differentially private stochastic gradient descent (DP-SGD) not degrade with increasing problem dimension?

Original: On the theoretical front, we show that DP-SGD can result in dimension-independent error bounds even when gradients span the entire ambient space for unconstrained optimization problems.
Revised: On the theoretical front, we show that DP-SGD can result in dimension-independent error bounds even when gradients span the entire ambient space for unconstrained optimization problems.

Original: We identify that the standard dependence on the dimension of the ambient space can be replaced by the magnitudes of gradients projected onto subspaces of varying dimensions.
Revised: We identify that the standard dependence on the dimension of the ambient space can be replaced by the magnitudes of gradients projected onto subspaces of varying dimensions.

Original: We formalize this in a condition that we call restricted Lipschitz continuity and derive refined bounds for the excess empirical and population risks for DP-SGD when loss functions obey this condition.
Revised: We formalize this in a condition that we call restricted Lipschitz continuity and derive refined bounds for the excess empirical and population risks for DP-SGD when loss functions obey this condition.

Original: We show that when the restricted Lipschitz coefficients decay rapidly, both the excess empirical and population risks become dimension-independent.
Revised: We show that when the restricted Lipschitz coefficients decay rapidly, both the excess empirical and population risks become dimension-independent.

Original: This extends a previous work which derived rank-dependent bounds for learning generalized linear models in an unconstrained space [SSTT21].
Revised: This extends a previous work which derived rank-dependent bounds for learning generalized linear models in an unconstrained space [SSTT21].

Original: Our theoretical results shed light on the recent success of large-scale differentially private fine-tuning.
Revised: Our theoretical results shed light on the recent success of large-scale differentially private fine-tuning.

Original: We empirically show that gradients of language models during fine-tuning are mostly controlled by a few principal components — a behavior that is similar to conditions under which we obtain dimension-independent bounds for private convex ERM.
Revised: We empirically show that gradients of language models during fine-tuning are mostly controlled by a few principal components — a behavior that is similar to conditions under which we obtain dimension-independent bounds for private convex ERM.

Original: This provides a possible explanation for the observation that densely fine-tuning with DP-SGD need not necessarily experience much worse performance than sparsely fine-tuning [LTLH21].
Revised: This provides a possible explanation for the observation that densely fine-tuning with DP-SGD need not necessarily experience much worse performance than sparsely fine-tuning [LTLH21].

Original: Moreover, it suggests that DP-SGD can be adaptive to problems that are effectively low-dimensional (as characterized by restricted Lipschitz continuity) without further algorithmic intervention.
Revised: Moreover, it suggests that DP-SGD can be adaptive to problems that are effectively low-dimensional (as characterized by restricted Lipschitz continuity) without further algorithmic intervention.

Original: We summarize our contributions below.
Revised: We summarize our contributions below.

Original: (1) We introduce a condition on the objective function that we term restricted Lipschitz continuity.
Revised: (1) We introduce a condition on the objective function that we term restricted Lipschitz continuity.

Original: This condition generalizes the usual Lipschitz continuity notion and gives rise to refined analyses when magnitudes of gradients projected onto diminishing subspaces decay rapidly.
Revised: This condition generalizes the usual Lipschitz continuity notion and gives rise to refined analyses when magnitudes of gradients projected onto diminishing subspaces decay rapidly.

Original: (2) Under restricted Lipschitz continuity, we present refined bounds on the excess empirical and population risks for DP-SGD when optimizing convex objectives.
Revised: (2) Under restricted Lipschitz continuity, we present refined bounds on the excess empirical and population risks for DP-SGD when optimizing convex objectives.

Original: These bounds generalize previous dimension-independent results [SSTT21] and are broadly applicable to cases where gradients are full rank but most coordinates only marginally influence the objective.
Revised: These bounds generalize previous dimension-independent results [SSTT21] and are broadly applicable to cases where gradients are full rank but most coordinates only marginally influence the objective.

Original: (3) Our theory sheds light on recent successes of large-scale differentially private fine-tuning of language models.
Revised: Our theory sheds light on recent successes of large-scale differentially private fine-tuning of language models.

Original: We show that gradients obtained through fine-tuning mostly lie in a subspace spanned by a few principal components — a behavior similar to when optimizing a restricted Lipschitz continuous loss with decaying coefficients.
Revised: We show that gradients obtained through fine-tuning mostly lie in a subspace spanned by a few principal components — a behavior similar to when optimizing a restricted Lipschitz continuous loss with decaying coefficients.

Original: These empirical results provide a possible explanation for the recent success of large-scale private fine-tuning.
Revised: These empirical results provide a possible explanation for the recent success of large-scale private fine-tuning.

Original:  Preliminaries
Revised:  Preliminaries

Original: We define the notation used throughout this work and state the problems of differentially private empirical risk minimization and differentially private stochastic convex optimization.
Revised: We define the notation used throughout this work and state the problems of differentially private empirical risk minimization and differentially private stochastic convex optimization.

Original: Finally, we give a brief recap of differentially private stochastic gradient descent, and existing dimension-dependent and dimension-independent results in the literature.
Revised: Finally, we give a brief recap of differentially private stochastic gradient descent, and existing dimension-dependent and dimension-independent results in the literature.

Original: Notation & Terminology.
Revised: Notation & Terminology.

Original: For a positive integer n ∈ N + , define the shorthand [ n ] = { 1 , . . . , n } .
Revised: For a positive integer n ∈ N + , define the shorthand [ n ] = { 1 , . . . , n } .

Original: For a vector x ∈ R d , denote its ℓ 2 -norm by ∥ x ∥ 2 .
Revised: For a vector x ∈ R d , denote its ℓ 2 -norm by ∥ x ∥ 2 .

Original: Given a symmetric M ∈ R d × d , let λ 1 ( M ) ≥ λ 2 ( M ) ≥ · · · ≥ λ d ( M ) denote its eigenvalues.
Revised: Given a symmetric M ∈ R d × d , let λ 1 ( M ) ≥ λ 2 ( M ) ≥ · · · ≥ λ d ( M ) denote its eigenvalues.

Original: Given a positive semidefinite matrix A , let ∥ x
Revised: Given a positive semidefinite matrix A , let ∥ x

Original: ∥ A = ( x ⊤ Ax ) 1 / 2 denote the induced Mahalanobis norm.
Revised: ∥ A = ( x ⊤ Ax ) 1 / 2 denote the induced Mahalanobis norm.

Original: For scalar functions f
Revised: For scalar functions f

Original: and g , we write f ≲ g if there exists a positive constant C such that f ( x ) ≤ Cg ( x ) for all input x in the domain.
Revised: and g , we write f ≲ g if there exists a positive constant C such that f ( x ) ≤ Cg ( x ) for all input x in the domain.

Original: 2.1 Differentially Private Empirical Risk Minimization and Stochastic Convex Optimization
Revised: 2.1 Differentially Private Empirical Risk Minimization and Stochastic Convex Optimization

Original: Before stating the theoretical problem of interest, we recall the basic concepts of Lipschitz continuity, convexity, and approximate differential privacy.
Revised: Before stating the theoretical problem of interest, we recall the basic concepts of Lipschitz continuity, convexity, and approximate differential privacy.

Original: Definition 2.1 (Lipschitz Continuity) .
Revised: Definition 2.1 (Lipschitz Continuity) .

Original: The loss function h : K → R is G -Lipschitz with respect to the ℓ 2 norm if for all x, x ′ ∈ K , | f ( x ) − f ( x ′ )
Revised: The loss function h : K → R is G -Lipschitz with respect to the ℓ 2 norm if for all x, x ′ ∈ K , | f ( x ) − f ( x ′ )

Original: | ≤ G ∥ x − x ′ ∥ 2 .
Revised: | ≤ G ∥ x − x ′ ∥ 2 .

Original: Definition 2.2 (Convexity) .
Revised: Definition 2.2 (Convexity) .

Original: The loss function h : K → R is convex if h ( αx + (1 − α ) y )
Revised: The loss function h : K → R is convex if h ( αx + (1 − α ) y )

Original: ≤ αh ( x )
Revised: ≤ αh ( x )

Original: + (1 − α ) h
Revised: + (1 − α ) h

Original: ( y ) , for all α ∈ [0 , 1] , and x, y in a convex domain K .
Revised: ( y ) , for all α ∈ [0 , 1] , and x, y in a convex domain K .

Original: Definition 2.3 (Approximate Differential Privacy [DR + 14]) .
Revised: Definition 2.3 (Approximate Differential Privacy [DR + 14]) .

Original: A randomized algorithm M is ( ε, δ ) - differentially private if for all neighboring datasets D and D ′ that differ by a single record and all sets
Revised: A randomized algorithm M is ( ε, δ ) - differentially private if for all neighboring datasets D and D ′ that differ by a single record and all sets

Original: O ⊂ range( M ) , the following expression holds
Revised: O ⊂ range( M ) , the following expression holds

Original: [Equation]
Revised: [Equation]

Original: In this work, we study both differentially private empirical risk minimization (DP-ERM) for convex objectives and differentially private stochastic convex optimization (DP-SCO).
Revised: In this work, we study both differentially private empirical risk minimization (DP-ERM) for convex objectives and differentially private stochastic convex optimization (DP-SCO).

Original: In DP-ERM for convex objectives, we are given a dataset D = { s j } j ∈ [ n ] ∈ S n of n examples.
Revised: In DP-ERM for convex objectives, we are given a dataset D = { s j } j ∈ [ n ] ∈ S n of n examples.

Original: Each per-example loss f ( · ; s j ) is convex over the convex body K ⊆ R d and G -Lipschitz.
Revised: Each per-example loss f ( · ; s j ) is convex over the convex body K ⊆ R d and G -Lipschitz.

Original: We aim to design an ( ε, δ ) -DP algorithm that returns a solution x priv which approximately minimizes the empirical risk F ( x ; D ) :
Revised: We aim to design an ( ε, δ ) -DP algorithm that returns a solution x priv which approximately minimizes the empirical risk F ( x ; D ) :

Original: = 1 n (cid:80) s j ∈D f
Revised: = 1 n (cid:80) s j ∈D f

Original: ( x ; s j ) .
Revised: ( x ; s j ) .

Original: The term E x priv (cid:2) F ( x priv ; D ) − min x ∈K F ( x ; D ) (cid:3) is referred to as the excess empirical risk .
Revised: The term E x priv (cid:2) F ( x priv ; D ) − min x ∈K F ( x ; D ) (cid:3) is referred to as the excess empirical risk .

Original: In DP-SCO, we assume the per-example loss f ( · ; s ) is convex and G -Lipschitz for all s ∈ S , and we are given n examples drawn i.i.d.
Revised: In DP-SCO, we assume the per-example loss f ( · ; s ) is convex and G -Lipschitz for all s ∈ S , and we are given n examples drawn i.i.d.

Original: from some (unknown) distribution P .
Revised: from some (unknown) distribution P .

Original: The goal is to design an ( ε, δ ) -DP algorithm which approximately minimizes the population risk F ( x ; P ) :
Revised: The goal is to design an ( ε, δ ) -DP algorithm which approximately minimizes the population risk F ( x ; P ) :

Original: =
Revised: =

Original: E s ∼P
Revised: E s ∼P

Original: [ f ( x ; s )] .
Revised: [ f ( x ; s )] .

Original: The term E x priv (cid:2) F ( x priv ; P ) − min x ∈K F ( x ; P ) (cid:3) is referred to as the excess population risk .
Revised: The term E x priv (cid:2) F ( x priv ; P ) − min x ∈K F ( x ; P ) (cid:3) is referred to as the excess population risk .

Original: 2.2 Differentially Private Stochastic Gradient Descent
Revised: 2.2 Differentially Private Stochastic Gradient Descent

Original: Differentially Private Stochastic Gradient Descent (DP-SGD) [ACG + 16, SCS13] is a popular algorithm for DP convex optimization.
Revised: Differentially Private Stochastic Gradient Descent (DP-SGD) [ACG + 16, SCS13] is a popular algorithm for DP convex optimization.

Original: For the theoretical analysis, we study DP-SGD for unconstrained optimization.
Revised: For the theoretical analysis, we study DP-SGD for unconstrained optimization.

Original: To facilitate analysis, we work with the ℓ 2 regularized objective expressed as
Revised: To facilitate analysis, we work with the ℓ 2 regularized objective expressed as

Original: [Equation]
Revised: [Equation]

Original: To optimize this objective, DP-SGD independently samples an example in each iteration and updates the solution by combining the gradient with an isotropic Gaussian whose scale is proportional to G , the Lipschitz constant of f .
Revised: To optimize this objective, DP-SGD independently samples an example in each iteration and updates the solution by combining the gradient with an isotropic Gaussian whose scale is proportional to G , the Lipschitz constant of f .

Original: Algorithm 1 presents the pseudocode.
Revised: Algorithm 1 presents the pseudocode.

Original: ζ
Revised: t x

Revised: ) =

Revised: x ( t − 1) − η (cid:16) ∇ f ( x ( t − 1) ; s j t ) + α ( x ( t − 1) − x (0) )

Revised: + G · ζ t (cid:17) , ζ t ∼ N (0 , σ 2 I d ) 5 end 6 Return

Original: It is straightforward to show that Algorithm 1 satisfies differential privacy.
Revised: It is straightforward to show that Algorithm 1 satisfies differential privacy.

Original: The following lemma quantifies the overall privacy spending and builds on a long line of work on accounting the privacy loss of compositions [ACG + 16, BBG18].
Revised: The following lemma quantifies the overall privacy spending and builds on a long line of work on accounting the privacy loss of compositions [ACG + 16, BBG18].

Original: Lemma 2.4 ([KLL21]) .
Revised: Lemma 2.4 ([KLL21]) .

Original: There exists constants c 1 and c 2 such that for n ≥ 10 , ε < c 1 T/n 2 and δ ∈ (0 , 12 ] , DP-SGD (Algorithm 1) is ( ε, δ ) -DP whenever σ ≥ c√ [Equation]
Revised: There exists constants c 1 and c 2 such that for n ≥ 10 , ε < c 1 T/n 2 and δ ∈ (0 , 12 ] , DP-SGD (Algorithm 1) is ( ε, δ ) -DP whenever σ ≥ c√ T log(1 /δ ) εn .

Original: 2.3 On the Dimension Dependence of Private Learning
Revised: 2.3 On the Dimension Dependence of Private Learning

Original: Early works on bounding the excess empirical and population risks for privately optimizing convex objectives focused on a constrained optimization setup where algorithms typically project iterates back onto a fixed bounded domain after each noisy gradient update.
Revised: Early works on bounding the excess empirical and population risks for privately optimizing convex objectives focused on a constrained optimization setup where algorithms typically project iterates back onto a fixed bounded domain after each noisy gradient update.

Original: Results in this setting suggested that risks are inevitably dimension-dependent in the worst case.
Revised: Results in this setting suggested that risks are inevitably dimension-dependent in the worst case.

Original: For instance, it was shown that the excess empirical risk bound Θ( G ∥K∥ 2 (cid:112)
Revised: For instance, it was shown that the excess empirical risk bound Θ( G ∥K∥ 2 (cid:112)

Original: d log(1 /δ )
Revised: d log(1 /δ )

Original: n − 1 ε − 1 ) and excess population risk bound Θ( G ∥K∥ 2 ( n − 1 / 2 + (cid:

Revised: n − 1 ε − 1 ) and excess population risk bound Θ( G ∥K∥ 2 ( n − 1 / 2 + (cid:

Original: ) d log(1 /δ )
Revised: ) d log(1 /δ )

Original: n − 1 ε − 1 )) are tight when privately optimizing convex G -Lipschitz objectives in a convex domain of diameter ∥K∥ 

Revised: n − 1 ε − 1 )) are tight when privately optimizing convex G -Lipschitz objectives in a convex domain of diameter ∥K∥

Original: [BST14].
Revised: [BST14].

Original: Moreover, the lower bound instances in these works imply that such dimension-dependent lower bounds also apply when one considers the class of loss functions whose gradients are low-rank.
Revised: Moreover, the lower bound instances in these works imply that such dimension-dependent lower bounds also apply when one considers the class of loss functions whose gradients are low-rank.

Original: The body of work on unconstrained convex optimization is arguably less abundant, with the notable result that differentially private gradient descent need not suffer from a dimension-dependent penalty when learning generalized linear models with low-rank data (equivalently stated, when gradients are low-rank)
Revised: The body of work on unconstrained convex optimization is arguably less abundant, with the notable result that differentially private gradient descent need not suffer from a dimension-dependent penalty when learning generalized linear models with low-rank data (equivalently stated, when gradients are low-rank)

Original: [SSTT21].
Revised: [SSTT21].

Original: Our main theoretical results (Theorems 3.3 and 3.5) extend this line of work and show that dimension-independence is achievable under weaker conditions.
Revised: Our main theoretical results (Theorems 3.3 and 3.5) extend this line of work and show that dimension-independence is achievable under weaker conditions.

Original:  Refined Dimension-Dependence via Restricted Lipschitz Continuity
Revised:  Refined Dimension-Dependence via Restricted Lipschitz Continuity

Original: In this section, we formally introduce the restricted Lipschitz continuity condition and derive improved bounds for the excess empirical and population risks for DP-SGD when optimizing convex objectives.
Revised: In this section, we formally introduce the restricted Lipschitz continuity condition and derive improved bounds for the excess empirical and population risks for DP-SGD when optimizing convex objectives.

Original: Definition 3.1 (Restricted Lipschitz Continuity) .
Revised: Definition 3.1 (Restricted Lipschitz Continuity) .

Original: We say that the loss function h : K
Revised: We say that the loss function h : K

Original: → R is restricted Lipschitz continuous with coefficients G 0 , . .
Revised: → R is restricted Lipschitz continuous with coefficients G 0 , . .

Original: . , G d , if for all k ∈ [ d ] , there exists an orthogonal projection matrix P k with rank k such that
Revised: . , G d , if for all k ∈ [ d ] , there exists an orthogonal projection matrix P k with rank k such that

Original: [Equation]
Revised: [Equation]

Original: for all x ∈ K and all subgradients ∇ h ( x ) ∈ ∂h ( x ) .
Revised: for all x ∈ K and all subgradients ∇ h ( x ) ∈ ∂h ( x ) .

Original: Note that any G -Lipschitz function is also trivially restricted Lipschitz continuous with coefficients
Revised: Note that any G -Lipschitz function is also trivially restricted Lipschitz continuous with coefficients

Original: G = G 0 =
Revised: G = G 0 =

Original: G 1 = · · · = G d , since orthogonal projections never increase the ℓ 2 -norm of a vector (generally, it is easy to see that G = G 0 ≥ G 1 ≥ G 2 ≥ · · · ≥ G d = 0 ).
Revised: G 1 = · · · = G d , since orthogonal projections never increase the ℓ 2 -norm of a vector (generally, it is easy to see that G = G 0 ≥ G 1 ≥ G 2 ≥ · · · ≥ G d = 0 ).

Original: On the other hand, we expect that a function which exhibits little growth in subspaces of dimension k to have a restricted
Revised: On the other hand, we expect that a function which exhibits little growth in subspaces of dimension k to have a restricted

Original: Lipschitz coefficient G d − k of almost 0.
Revised: Lipschitz coefficient G d − k of almost 0.

Original: Our bounds on DP convex optimization will characterize errors through the use of restricted Lipschitz coefficients.
Revised: Our bounds on DP convex optimization will characterize errors through the use of restricted Lipschitz coefficients.

Original: We now summarize the main assumptions before presenting the core theoretical results.
Revised: We now summarize the main assumptions before presenting the core theoretical results.

Original: Assumption 3.2.
Revised: Assumption 3.2.

Original: The per-example loss function f ( · ; s ) is convex and G -Lipschitz continuous for all s ∈ S .
Revised: The per-example loss function f ( · ; s ) is convex and G -Lipschitz continuous for all s ∈ S .

Original: The average loss F ( · ; D ) is restricted Lipschitz 
Revised: The average loss F ( · ; D ) is restricted Lipschitz continuous with coefficients { G k } dk =1 .

Original: 3.1 Bounds for Excess Empirical Loss
Revised: 3.1 Bounds for Excess Empirical Loss

Original: We present the main theoretical result on DP-ERM for convex objectives.
Revised: We present the main theoretical result on DP-ERM for convex objectives.

Original: The result consists of two pieces: Equation (1) is a general bound that is applicable to any sequence of restricted Lipschitz coefficients; Equation (2) specializes the previous bound when the sequence of coefficients decays rapidly and demonstrates dimension-independent error scaling.
Revised: The result consists of two pieces: Equation (1) is a general bound that is applicable to any sequence of restricted Lipschitz coefficients; Equation (2) specializes the previous bound when the sequence of coefficients decays rapidly and demonstrates dimension-independent error scaling.

Original: Theorem 3.3 (Excess Empirical Loss) .
Revised: Theorem 3.3 (Excess Empirical Loss) .

Original: Let δ ∈ (0 , 12 ] , and ε ∈ (0 , 10] .
Revised: Let δ ∈ (0 , 12 ] , and ε ∈ (0 , 10] .

Original: Under Assumption 3.2, for any positive integer k ∈ [ d ] , setting T = Θ( n 2 + d [Equation] d ) , σ = Θ (cid:18)
Revised: Under Assumption 3.2, for any positive integer k ∈ [ d ] , setting T = Θ( n 2 + d log 2 d ) , σ = Θ (cid:18)

Original: [Equation]
Revised: [Equation]

Original: and α = 1 D (cid:113)(cid:80) Ss =1 s 2 2 s G 22 s − 1 k , where S =
Revised: and α = 1 D (cid:113)(cid:80) Ss =1 s 2 2 s G 22 s − 1 k , where S =

Original: ⌊ log( d/k ) ⌋ + 1 , DP-SGD (Algorithm 1) is ( ε, δ ) -DP, and
Revised: ⌊ log( d/k ) ⌋ + 1 , DP-SGD (Algorithm 1) is ( ε, δ ) -DP, and

Original: [Equation]
Revised: [Equation]

Original: where ∥ x (0) − argmin x F ( x ; D ) ∥ 2 ≤ D , x is the (random) output of DP-SGD (Algorithm 1), and the expectation is over the randomness of x .
Revised: where ∥ x (0) − argmin x F ( x ; D ) ∥ 2 ≤ D , x is the (random) output of DP-SGD (Algorithm 1), and the expectation is over the randomness of x .

Original: Moreover, if for some c > 1 / 2 , we have G k ≤ G 0 k − c for all k ∈ [ d ] , and in addition n ≥ ε − 1 (cid:112) log(1 /δ ) , minimizing the right hand side of (1) with respect to k yields
Revised: Moreover, if for some c > 1 / 2 , we have G k ≤ G 0 k − c for all k ∈ [ d ] , and in addition n ≥ ε − 1 (cid:112) log(1 /δ ) , minimizing the right hand side of (1) with respect to k yields

Original: [Equation]
Revised: [Equation]

Original: We include a sketch of the proof techniques in Section 3.3 and defer the full proof to Appendix A.
Revised: We include a sketch of the proof techniques in Section 3.3 and defer the full proof to Appendix A.

Original: Remark 3.4 .
Revised: Remark 3.4 .

Original: Consider DP-ERM for learning generalized linear models with convex and Lipschitz losses.
Revised: Consider DP-ERM for learning generalized linear models with convex and Lipschitz losses.

Original: When the (empirical) data covariance is of rank r < d , the span of gradients span ( {∇ x F ( x ) } ) is also of rank r .
Revised: When the (empirical) data covariance is of rank r < d , the span of gradients span ( {∇ x F ( x ) } ) is also of rank r .

Original: Thus, the average loss is restricted Lipschitz continuous with coefficients where G r ′ = 

Revised: Thus, the average loss is restricted Lipschitz continuous with coefficients where G r ′ =

Original: for all r ′ > r .
Revised: for all r ′ > r .

Original: Setting k = r in (1) yields the excess empirical risk bound of order O (cid:16) G 0 D (cid:112) r · log(1 /δ )
Revised: Setting k = r in (1) yields the excess empirical risk bound of order O (cid:16) G 0 D (cid:112) r · log(1 /δ )

Original: ε − 1 n − 1 (cid:17) .
Revised: ε − 1 n − 1 (cid:17) .

Original: This recovers the previous dimension-independent result [SSTT21].
Revised: This recovers the previous dimension-independent result [SSTT21].

Original: The restricted Lipschitz continuity condition can be viewed as a generalized notion of rank.
Revised: The restricted Lipschitz continuity condition can be viewed as a generalized notion of rank.

Original: The result captured in (2) suggests that the empirical loss achieved by DP-SGD does not depend on the problem dimension if the sequence of restricted Lipschitz coefficients decays rapidly.
Revised: The result captured in (2) suggests that the empirical loss achieved by DP-SGD does not depend on the problem dimension if the sequence of restricted Lipschitz coefficients decays rapidly.

Original: We leverage these insights to build intuition for understanding private fine-tuning of language model in Section 4.
Revised: We leverage these insights to build intuition for understanding private fine-tuning of language model in Section 4.

Original: 3.2 Bounds for Excess Population Loss
Revised: 3.2 Bounds for Excess Population Loss

Original: For DP-SCO, we make use of the stability of DP-SGD to bound its generalization error [BE02], following previous works [BFTT19, BFGT20, SSTT21].
Revised: For DP-SCO, we make use of the stability of DP-SGD to bound its generalization error [BE02], following previous works [BFTT19, BFGT20, SSTT21].

Original: The bound on the excess population loss follows from combining the bounds on the excess empirical risk and the generalization error.
Revised: The bound on the excess population loss follows from combining the bounds on the excess empirical risk and the generalization error.

Original: Theorem 3.5 (Excess Population Loss) .
Revised: Theorem 3.5 (Excess Population Loss) .

Original: Let δ ∈ (0 , 12 ] , and ε ∈ (0 , 10] .
Revised: Let δ ∈ (0 , 12 ] , and ε ∈ (0 , 10] .

Original: Under Assumption 3.2, for any positive integer k ∈ [ d ] , by setting T =
Revised: Under Assumption 3.2, for any positive integer k ∈ [ d ] , by setting T =

Original: Θ( n 2 + d log 2 d ) , σ = Θ (cid:18) √ T log(1 /δ )
Revised: Θ( n 2 + d log 2 d ) , σ = Θ (cid:18) √ T log(1 /δ )

Original: nε (cid:19) , η = (cid:113)
Revised: nε (cid:19) , η = (cid:113)

Original: D 2 T · G 20 ( T/n + kσ 2 ) and α = 1 D (cid:113)(cid:80)
Revised: D 2 T · G 20 ( T/n + kσ 2 ) and α = 1 D (cid:113)(cid:80)

Original: Ss =1 s 2 

Revised: Ss =1 s 2

Original: s G 22 s − 1 k , where S =
Revised: s G 22 s − 1 k , where S =

Original: ⌊ log( d/k ) ⌋ + 1 , DP-SGD (Algorithm 1) is ( ε, δ ) -DP, and
Revised: ⌊ log( d/k ) ⌋ + 1 , DP-SGD (Algorithm 1) is ( ε, δ ) -DP, and

Original: [Equation]
Revised: [Equation]

Original: where ∥ x (0) − argmin x F ( x ; P ) ∥ 2 ≤ D , x is the (random) output of DP-SGD (Algorithm 1), and the expectation is over the randomness of x .
Revised: where ∥ x (0) − argmin x F ( x ; P ) ∥ 2 ≤ D , x is the (random) output of DP-SGD (Algorithm 1), and the expectation is over the randomness of x .

Original: Moreover, if for some c > 1 / 2 , we have G k ≤ G 0 k − c for all k ∈ [ d ] , and in addition n > ε − 1 (cid:112) log(1 /δ ) , minimizing the above bound with respect to k yields
Revised: Moreover, if for some c > 1 / 2 , we have G k ≤ G 0 k − c for all k ∈ [ d ] , and in addition n > ε − 1 (cid:112) log(1 /δ ) , minimizing the above bound with respect to k yields

Original: [Equation]
Revised: [Equation]

Original: Remark 3.6 .
Revised: Remark 3.6 .

Original: Our result on DP-SCO also recovers the DP-SCO rank-dependent result for learning generalized linear models with convex and Lipschitz losses [SSTT21].
Revised: Our result on DP-SCO also recovers the DP-SCO rank-dependent result for learning generalized linear models with convex and Lipschitz losses [SSTT21].

Original: Remark 3.7 .
Revised: Remark 3.7 .

Original: When c > 1 / 2 , ε = Θ(1) and δ = 1 / poly( n ) , the population loss matches the (non-private) informational-theoretical lower bound Ω( G 0 D/ √ n ) [AWBR09].
Revised: When c > 1 / 2 , ε = Θ(1) and δ = 1 / poly( n ) , the population loss matches the (non-private) informational-theoretical lower bound Ω( G 0 D/ √ n ) [AWBR09].

Original: Remark 3.8 .
Revised: Remark 3.8 .

Original: Our results on DP-ERM and DP-SCO naturally generalize to (full-batch) DP-GD.
Revised: Our results on DP-ERM and DP-SCO naturally generalize to (full-batch) DP-GD.

Original: 3.3 Overview of Proof Techniques
Revised: 3.3 Overview of Proof Techniques

Original: The privacy guarantees in Theorems 3.3 and 3.5 follow from Lemma 2.4.
Revised: The privacy guarantees in Theorems 3.3 and 3.5 follow from Lemma 2.4.

Original: It suffices to prove the utility guarantees.
Revised: It suffices to prove the utility guarantees.

Original: We give an outline of the main proof technique and defer full proofs to the supplementary.
Revised: We give an outline of the main proof technique and defer full proofs to the supplementary.

Original: The following is a sketch of the core technique for deriving (2) in Theorem 3.3.
Revised: The following is a sketch of the core technique for deriving (2) in Theorem 3.3.

Original: By convexity, the error term of SGD is upper bounded as follows
Revised: By convexity, the error term of SGD is upper bounded as follows

Original: [Equation]
Revised: [Equation]

Original: where j ∈ [ n ] is the random index sampled at iteration t .
Revised: where j ∈ [ n ] is the random index sampled at iteration t .

Original: By definition of G k , we know that there is a k dimensional subspace U such that the gradient component orthogonal to U is small when G k is small.
Revised: By definition of G k , we know that there is a k dimensional subspace U such that the gradient component orthogonal to U is small when G k is small.

Original: Naïvely, one decomposes the gradient ∇ f j ( x ( t ) )
Revised: Naïvely, one decomposes the gradient ∇ f j ( x ( t ) )

Original: =
Revised: =

Original: g 1 + g 2 , where g 1 ∈ U and g 2 ∈
Revised: g 1 + g 2 , where g 1 ∈ U and g 2 ∈

Original: U ⊥ , and separately bounds the two terms g ⊤ 1 ( x ( t ) − x ∗ ) and g ⊤ 2 ( x ( t ) − x ∗ ) .
Revised: U ⊥ , and separately bounds the two terms g ⊤ 1 ( x ( t ) − x ∗ ) and g ⊤ 2 ( x ( t ) − x ∗ ) .

Original: Since g 1 lies in a k dimensional subspace, one can follow existing arguments on DP-SGD to bound g ⊤ 1 ( x ( t ) − x ∗ ) .
Revised: Since g 1 lies in a k dimensional subspace, one can follow existing arguments on DP-SGD to bound g ⊤ 1 ( x ( t ) − x ∗ ) .

Original: Unfortunately, this argument does not lead to dimension-independence.
Revised: Unfortunately, this argument does not lead to dimension-independence.

Original: Although ∥ g 2 ∥ 2 ≤ G k (which can be small for large k ), the term ∥ x ( t ) − x ∗ ∥ 2 can be as large as Ω( √ d ) with high probability due to the isotropic
Revised: Although ∥ g 2 ∥ 2 ≤ G k (which can be small for large k ), the term ∥ x ( t ) − x ∗ ∥ 2 can be as large as Ω( √ d ) with high probability due to the isotropic

Original: Gaussian noise injected in DP-SGD.
Revised: Gaussian noise injected in DP-SGD.

Original: Therefore, the naïve upper bound on | g ⊤ 2 ( x ( t ) − x ) | can be as large O ( √ dG k ) .
Revised: Therefore, the naïve upper bound on | g ⊤ 2 ( x ( t ) − x ) | can be as large O ( √ dG k ) .

Original: Our key idea is to partition the whole space R d into ⌊ log( d/k ) ⌋ + 2 orthogonal subspaces, expressing the error term ∇ f j ( x ( t ) )
Revised: Our key idea is to partition the whole space R d into ⌊ log( d/k ) ⌋ + 2 orthogonal subspaces, expressing the error term ∇ f j ( x ( t ) )

Original: ⊤ ( x ( t ) − x ∗ ) as the sum of individual terms each of which corresponds to a projection to a particular subspace.
Revised: ⊤ ( x ( t ) − x ∗ ) as the sum of individual terms each of which corresponds to a projection to a particular subspace.

Original: Fix the positive integer k ≤ d , and consider the following subspaces: Let U 

Revised: Fix the positive integer k ≤ d , and consider the following subspaces: Let U

Original: =
Revised: =

Original: range ( P k ) , U s be the subspace orthogonal to all previous subspaces such that (cid:76) si =

Revised: range ( P k ) , U s be the subspace orthogonal to all previous subspaces such that (cid:76) si =

Original: U i ⊇ range ( P 2 s k ) for s = 1 , 2 , · · · , ⌊ log( d/k ) ⌋ , and U S be the subspace such that the orthogonal direct sum of all subspaces { U i } Si =0 is R d where S =
Revised: U i ⊇ range ( P 2 s k ) for s = 1 , 2 , · · · , ⌊ log( d/k ) ⌋ , and U S be the subspace such that the orthogonal direct sum of all subspaces { U i } Si =0 is R d where S =

Original: ⌊ log( d/k ) ⌋ + 1 .
Revised: ⌊ log( d/k ) ⌋ + 1 .

Original: Here, P i is the orthogonal projection matrix with rank i promised by G i in Assumption 3.2.
Revised: Here, P i is the orthogonal projection matrix with rank i promised by G i in Assumption 3.2.

Original: Let Q s be the orthogonal projection to the subspace U s .
Revised: Let Q s be the orthogonal projection to the subspace U s .

Original: Observe that rank( Q s ) ≤ 2 s k and ∥ Q s ∇ F ( x ) ∥ 2 ≤
Revised: Observe that rank( Q s ) ≤ 2 s k and ∥ Q s ∇ F ( x ) ∥ 2 ≤

Original: G 2 s − 1 k for all x and all s ≥ 1 .
Revised: G 2 s − 1 k for all x and all s ≥ 1 .

Original: Rewriting the right hand side of (3) with this decomposition yields
Revised: Rewriting the right hand side of (3) with this decomposition yields

Original: [Equation]
Revised: [Equation]

Original: ∇ f j ( x ( t ) ) (cid:33) ⊤ ( x ( t ) − x ∗ ) .

Original: On the one hand, if G k decays rapidly, ∥ E j [ Q s ∇ f j ] ∥ 2 will be small for large s .
Revised: On the one hand, if G k decays rapidly, ∥ E j [ Q s ∇ f j ] ∥ 2 will be small for large s .

Original: On the other hand, we expect ∥ Q s ( x ( t ) − x ∗ ) ∥ 2 to be small for small s where Q s is an orthogonal projection onto a small subspace.
Revised: On the other hand, we expect ∥ Q s ( x ( t ) − x ∗ ) ∥ 2 to be small for small s where Q s is an orthogonal projection onto a small subspace.

Original: Thus, for each s , ∇ f j ( x ( t ) )
Revised: Thus, for each s , ∇ f j ( x ( t ) )

Original: ⊤ Q s ( x ( t ) − x ∗ ) is small either due to a small gradient (small Q s ∇ f j in expectation over the random index) or small noise (small Q s ( x ( t ) − x ∗ ) ), since noise injected in DP-SGD is isotropic.
Revised: ⊤ Q s ( x ( t ) − x ∗ ) is small either due to a small gradient (small Q s ∇ f j in expectation over the random index) or small noise (small Q s ( x ( t ) − x ∗ ) ), since noise injected in DP-SGD is isotropic.

Original: More formally, in Lemma A.1, we show that for any projection matrix Q with rank r , ∥ Q ( x ( t ) − x (0) )
Revised: More formally, in Lemma A.1, we show that for any projection matrix Q with rank r , ∥ Q ( x ( t ) − x (0) )

Original: ∥ 2 can be upper bounded by a term that depends only on r (rather than d ).
Revised: ∥ 2 can be upper bounded by a term that depends only on r (rather than d ).

Original:  Numerical Experiments
Revised:  Numerical Experiments

Original: The aim of this section is twofold.
Revised: The aim of this section is twofold.

Original: In Section 4.1, we study a synthetic example that matches our theoretical assumptions and show that DP-SGD attains dimension-independent empirical and population loss when the sequence of restricted Lipschitz coefficients decays rapidly—even when gradients span the entire ambient space.
Revised: In Section 4.1, we study a synthetic example that matches our theoretical assumptions and show that DP-SGD attains dimension-independent empirical and population loss when the sequence of restricted Lipschitz coefficients decays rapidly—even when gradients span the entire ambient space.

Original: In Section 4.2, we study a stylized example of privately fine-tuning large language models.
Revised: In Section 4.2, we study a stylized example of privately fine-tuning large language models.

Original: Building on the previous theory, we provide insights as to why dense fine-tuning can yield good performance.
Revised: Building on the previous theory, we provide insights as to why dense fine-tuning can yield good performance.

Original: 4.1 Synthetic Example: Estimating the Generalized Geometric Median
Revised: 4.1 Synthetic Example: Estimating the Generalized Geometric Median

Original: We privately estimate the geometric median which minimizes the average Mahalanobis distance.
Revised: We privately estimate the geometric median which minimizes the average Mahalanobis distance.

Original: Specifically, let x i ∈ R d for i ∈ [ n ] be feature vectors drawn i.i.d.
Revised: Specifically, let x i ∈ R d for i ∈ [ n ] be feature vectors drawn i.i.d.

Original: from some distribution P x , each of which is treated as an individual record.
Revised: from some distribution P x , each of which is treated as an individual record.

Original: Denote the entire dataset as D = { x i } ni =1 .
Revised: Denote the entire dataset as D = { x i } ni =1 .

Original: Subject to differential privacy, we perform the following optimization
Revised: Subject to differential privacy, we perform the following optimization

Original: [Equation]
Revised: [Equation]

Original: where we adopt the shorthand
Revised: where we adopt the shorthand

Original: f i ( x ) =
Revised: f i ( x ) =

Original: f
Revised: f

Original: ( x ; x i ) =
Revised: ( x ; x i ) =

Original: ∥ x − x i ∥ A .
Revised: ∥ x − x i ∥ A .

Original: When A = I d and α = 0 (without the regularization term), the problem reduces to estimating the usual geometric median (commonly known as center of mass).
Revised: When A = I d and α = 0 (without the regularization term), the problem reduces to estimating the usual geometric median (commonly known as center of mass).

Original: For this example, individual gradients are bounded since ∥∇ f i ( x ) ∥ 2 = ∥
Revised: For this example, individual gradients are bounded since ∥∇ f i ( x ) ∥ 2 = ∥

Original: A ( x − x i ) /
Revised: A ( x − x i ) /

Original: ∥ x − x i ∥
Revised: ∥ x − x i ∥

Original: A ∥ 2 ≤ λ 1 ( A 1 / 2 ) =
Revised: A ∥ 2 ≤ λ 1 ( A 1 / 2 ) =

Original: G 0 .
Revised: G 0 .

Original: More generally, the restricted Lipschitz coefficients of F ( x ) are the eigenvalues of
Revised: More generally, the restricted Lipschitz coefficients of F ( x ) are the eigenvalues of

Original: A 1 / 2 , since
Revised: A 1 / 2 , since

Original: [Equation]
Revised: [Equation]

Original: where Q k =
Revised: where Q k =

Original: I − P k is chosen to be the rank ( d − k ) orthogonal projection matrix that projects onto the subspace spanned by the bottom ( d − k ) eigenvectors of A 1 / 2 .
Revised: I − P k is chosen to be the rank ( d − k ) orthogonal projection matrix that projects onto the subspace spanned by the bottom ( d − k ) eigenvectors of A 1 / 2 .

Original: To verify our theory, we study the optimization and generalization performance of DP-SGD for minimizing (4) under Mahalanobis distances induced by different A as the problem dimension grows.
Revised: To verify our theory, we study the optimization and generalization performance of DP-SGD for minimizing (4) under Mahalanobis distances induced by different A as the problem dimension grows.

Original: The optimization performance is measured by the final training error, and the generalization performance is measured by the population quantity E x ∼ P x ,x [ ∥ x − x ∥ A ] , where x denotes the random output of DP-SGD.
Revised: The optimization performance is measured by the final training error, and the generalization performance is measured by the population quantity E x ∼ P x ,x [ ∥ x − x ∥ A ] , where x denotes the random output of DP-SGD.

Original: We study the dimension scaling behavior for A being one of
Revised: We study the dimension scaling behavior for A being one of

Original: [Equation]
Revised: [Equation]

Original: where diag : R d → R d × d maps vectors onto square matrices with inputs on the diagonal.
Revised: where diag : R d → R d × d maps vectors onto square matrices with inputs on the diagonal.

Original: In all cases, the span of gradients span ( {∇ F ( x ) } ) is the ambient space R d , since A is of full rank.
Revised: In all cases, the span of gradients span ( {∇ F ( x ) } ) is the ambient space R d , since A is of full rank.

Original: To ensure the distance from the initial iterate β (0) = 0 to the optimum is the same for problem instances of different dimensions, we let feature vectors { x i } ni =1 take zero values in any dimension k > d min , where d min is the dimension of the smallest problem in our experiments.
Revised: To ensure the distance from the initial iterate β (0) = 0 to the optimum is the same for problem instances of different dimensions, we let feature vectors { x i } ni =1 take zero values in any dimension k > d min , where d min is the dimension of the smallest problem in our experiments.

Original: Our theoretical bounds suggest that when the sequence of restricted Lipschitz coefficients is constant (when A = A const ), the excess empirical loss grows with the problem dimension, whereas when the sequence of k th-Lipschitz constants rapidly decays with k (when A =
Revised: Our theoretical bounds suggest that when the sequence of restricted Lipschitz coefficients is constant (when A = A const ), the excess empirical loss grows with the problem dimension, whereas when the sequence of k th-Lipschitz constants rapidly decays with k (when A =

Original: A sqrt or A =
Revised: A sqrt or A =

Original: A linear )
Revised: A linear )

Original: , the excess empirical loss does not grow beyond a certain problem dimension.
Revised: , the excess empirical loss does not grow beyond a certain problem dimension.

Original: Figure 1 empirically captures this phenomenon.
Revised: Figure 1 empirically captures this phenomenon.

Original: We include additional experimental setup details in Appendix C.
Revised: We include additional experimental setup details in Appendix C.

Original: 4.2 Why Does Dense Fine-Tuning Work Well for Pretrained Language Models?
Revised: 4.2 Why Does Dense Fine-Tuning Work Well for Pretrained Language Models?

Original: Stated informally, our bounds in Theorem 3.5 imply that DP-SGD obtains dimension-independent errors if gradients approximately reside in a subspace much smaller than the ambient space.
Revised: Stated informally, our bounds in Theorem 3.5 imply that DP-SGD obtains dimension-independent errors if gradients approximately reside in a subspace much smaller than the ambient space.

Original: Inspired by these results for the convex case, we now turn to study dense language model fine-tuning [LTLH21] and provide a possible explanation for their recent intriguing success — fine-tuning gigantic parameter vectors frequently results in moderate performance drops compared to non-private learning.
Revised: Inspired by these results for the convex case, we now turn to study dense language model fine-tuning [LTLH21] and provide a possible explanation for their recent intriguing success — fine-tuning gigantic parameter vectors frequently results in moderate performance drops compared to non-private learning.

Original: In the following, we present evidence that gradients obtained through fine-tuning mostly lie in a small subspace.
Revised: In the following, we present evidence that gradients obtained through fine-tuning mostly lie in a small subspace.

Original: We design subsequent experiments to work under a simplified setup.
Revised: We design subsequent experiments to work under a simplified setup.

Original: Specifically, we finetune DistilRoBERTa
Revised: Specifically, we finetune DistilRoBERTa

Original: [SDCW19, LOG + 19] under ε = 8 and δ = 1 / n 1 . 1 for sentiment classification on the SST-2 dataset [SPW + 13].
Revised: [SDCW19, LOG + 19] under ε = 8 and δ = 1 / n 1 . 1 for sentiment classification on the SST-2 dataset [SPW + 13].

Original: We reformulate the label prediction problem as templated text prediction [LTLH21], and fine-tune only the query and value matrices in attention layers.
Revised: We reformulate the label prediction problem as templated text prediction [LTLH21], and fine-tune only the query and value matrices in attention layers.

Original: [Figure]
Revised: [Figure]

Original: We focus on fine-tuning these specific parameter matrices due to the success of LoRA for non-private learning [HSW + 21] which focuses on adapting the attention layers.
Revised: We focus on fine-tuning these specific parameter matrices due to the success of LoRA for non-private learning [HSW + 21] which focuses on adapting the attention layers.

Original: Unlike LoRA, we fine-tune all parameters in these matrices rather than focusing on low-rank updates.
Revised: Unlike LoRA, we fine-tune all parameters in these matrices rather than focusing on low-rank updates.

Original: This gives a setup that is lightweight enough to run spectral analyses computationally tractably but retains enough parameters ( ≈ 7 million) such that a problem of similar scale outside of fine-tuning results in substantial losses in utility.
Revised: This gives a setup that is lightweight enough to run spectral analyses computationally tractably but retains enough parameters ( ≈ 7 million) such that a problem of similar scale outside of fine-tuning results in substantial losses in utility.

Original: For our setup, DP-SGD obtains a dev set accuracy approximately of 90% and 92% , privately and non-privately, respectively.
Revised: For our setup, DP-SGD obtains a dev set accuracy approximately of 90% and 92% , privately and non-privately, respectively.

Original: These numbers are similar to previous results obtained with the same pretrained model [YNB + 21, LTLH21].
Revised: These numbers are similar to previous results obtained with the same pretrained model [YNB + 21, LTLH21].

Original: We include the full experimental protocol and additional results in Appendix D.
Revised: We include the full experimental protocol and additional results in Appendix D.

Original: To provide evidence for the small subspace hypothesis, we sample gradients during fine-tuning and study their principal components.
Revised: To provide evidence for the small subspace hypothesis, we sample gradients during fine-tuning and study their principal components.

Original: Specifically, we “over-train” by privately fine-tuning for r = 2 × 10 3 updates and collect all the non-privatized average clipped gradients along the optimization trajectory.
Revised: Specifically, we “over-train” by privately fine-tuning for r = 2 × 10 3 updates and collect all the non-privatized average clipped gradients along the optimization trajectory.

Original: While fine-tuning for 200 and 2k updates have similar final dev set performance under our hyperparameters, the increased number of steps allows us to collect more gradients around the converged solution.
Revised: While fine-tuning for 200 and 2k updates have similar final dev set performance under our hyperparameters, the increased number of steps allows us to collect more gradients around the converged solution.

Original: This yields a gradient matrix H ∈ R r × p , where p ≈ 7 × 10 6 is the size of the parameter vector.
Revised: This yields a gradient matrix H ∈ R r × p , where p ≈ 7 × 10 6 is the size of the parameter vector.

Original: We perform PCA for H with the orthogonal iteration algorithm [Dem97] and visualize the set of estimated singular values σ i ( H )
Revised: We perform PCA for H with the orthogonal iteration algorithm [Dem97] and visualize the set of estimated singular values σ i ( H )

Original: =
Revised: =

Original: λ
Revised: λ

Original: i ( H ⊤ H ) 1 / 2 in terms of both (i) the density estimate, and (ii) their relation with the rank.
Revised: i ( H ⊤ H ) 1 / 2 in terms of both (i) the density estimate, and (ii) their relation with the rank.

Original: Figure 2 (a) shows the top 1000 singular values sorted and plotted against their rank k and the least squares fit on log-transformed inputs and outputs.
Revised: Figure 2 (a) shows the top 1000 singular values sorted and plotted against their rank k and the least squares fit on log-transformed inputs and outputs.

Original: The plot displays few large singular values which suggests that gradients are controlled through only a few principal directions.
Revised: The plot displays few large singular values which suggests that gradients are controlled through only a few principal directions.

Original: The linear fit suggests that singular values decay rapidly (at a rate of approximately k − 0 . 6 ).
Revised: The linear fit suggests that singular values decay rapidly (at a rate of approximately k − 0 . 6 ).

Original: To study the effects that different principal components have on fine-tuning performance, we further perform the following re-training experiment.
Revised: To study the effects that different principal components have on fine-tuning performance, we further perform the following re-training experiment.

Original: Given the principal components, we privately re-finetune with gradients projected onto the top k ∈ { 10 , 20 , 100 } components.
Revised: Given the principal components, we privately re-finetune with gradients projected onto the top k ∈ { 10 , 20 , 100 } components.

Original: Note that this projection applies only to the (non-privatized) average clipped gradients and the isotropic DP noise is still applied to all dimensions.
Revised: Note that this projection applies only to the (non-privatized) average clipped gradients and the isotropic DP noise is still applied to all dimensions.

Original: Figure 2 (b) shows that the original performance can be attained by optimizing within a subspace of only dimension k = 100 , suggesting that most of the dimensions of the 7 million parameter vector encode a limited learning signal.
Revised: Figure 2 (b) shows that the original performance can be attained by optimizing within a subspace of only dimension k = 100 , suggesting that most of the dimensions of the 7 million parameter vector encode a limited learning signal.

Original: While these empirical results present encouraging insights for the dimension-independent performance of fine-tuning, we acknowledge that this is not a complete validation of the restricted Lipschitz continuity condition and fast decay of coefficients (even locally near the optimum).
Revised: While these empirical results present encouraging insights for the dimension-independent performance of fine-tuning, we acknowledge that this is not a complete validation of the restricted Lipschitz continuity condition and fast decay of coefficients (even locally near the optimum).

Original: We leave a more thorough analysis with additional model classes and fine-tuning tasks to future work.
Revised: We leave a more thorough analysis with additional model classes and fine-tuning tasks to future work.

Original: [Figure]
Revised: [Figure]

Original:  Related Work
Revised:  Related Work

Original: DP-ERM and DP-SCO are arguably the most well-studied areas of differential privacy [CMS11,
Revised: DP-ERM and DP-SCO are arguably the most well-studied areas of differential privacy [CMS11,

Original: KST12, BST14, SCS13, WYX17, FTS17, BFTT19, MRTZ17, ZZMW17, WLK + 17, FKT20,
Revised: KST12, BST14, SCS13, WYX17, FTS17, BFTT19, MRTZ17, ZZMW17, WLK + 17, FKT20,

Original: INS + 19, BFGT20, STT20, LL21, AFKT21, BGN21, GTU22, GLL22].
Revised: INS + 19, BFGT20, STT20, LL21, AFKT21, BGN21, GTU22, GLL22].

Original: Tight dependence on the number of model parameters and the number of samples is known for both DP-ERM
Revised: Tight dependence on the number of model parameters and the number of samples is known for both DP-ERM

Original: [BST14] and
Revised: [BST14] and

Original: DP-SCO [BFTT19].
Revised: DP-SCO [BFTT19].

Original: In particular, for the error on general convex losses, an explicit polynomial dependence on the number of optimization variables is necessary.
Revised: In particular, for the error on general convex losses, an explicit polynomial dependence on the number of optimization variables is necessary.

Original: However, it is shown that if gradients lie in a fixed low-rank subspace M , the dependence on dimension d can be replaced by rank ( M ) which can be significantly smaller [JT14, STT20].
Revised: However, it is shown that if gradients lie in a fixed low-rank subspace M , the dependence on dimension d can be replaced by rank ( M ) which can be significantly smaller [JT14, STT20].

Original: We extend this line of work to show that under a weaker assumption (restricted Lipschitz continuity with decaying coefficients) one can obtain analogous error guarantees that are independent of d , but do not require the gradients of the loss to strictly lie in any fixed low-rank subspace M .
Revised: We extend this line of work to show that under a weaker assumption (restricted Lipschitz continuity with decaying coefficients) one can obtain analogous error guarantees that are independent of d , but do not require the gradients of the loss to strictly lie in any fixed low-rank subspace M .

Original: As a consequence, our results provide a plausible explanation for the empirical observation that dense fine-tuning can be effective and that fine-tuning a larger model under DP can generally be more advantageous in terms of utility than fine-tuning a smaller model [LTLH21, YNB + 21].
Revised: As a consequence, our results provide a plausible explanation for the empirical observation that dense fine-tuning can be effective and that fine-tuning a larger model under DP can generally be more advantageous in terms of utility than fine-tuning a smaller model [LTLH21, YNB + 21].

Original: A concurrent work shows that the standard dimension dependence of DP-SGD can be replaced by a dependence on the trace of the Hessian assuming the latter quantity is uniformly bounded [MMZ22].
Revised: A concurrent work shows that the standard dimension dependence of DP-SGD can be replaced by a dependence on the trace of the Hessian assuming the latter quantity is uniformly bounded [MMZ22].

Original: A complementary line of work designed variants of DP-SGD that either explicitly or implicitly control the subspace in which gradients are allowed to reside [AGM + 21, LVS + 21, ADF + 21, KDRT21,
Revised: A complementary line of work designed variants of DP-SGD that either explicitly or implicitly control the subspace in which gradients are allowed to reside [AGM + 21, LVS + 21, ADF + 21, KDRT21,

Original: YZCL21b].
Revised: YZCL21b].

Original: They demonstrated improved dependence of the error on the dimension if the true gradients lie in a “near” low-rank subspace.
Revised: They demonstrated improved dependence of the error on the dimension if the true gradients lie in a “near” low-rank subspace.

Original: Our results are incomparable to this line of work because of two reasons: (i) Our algorithm is vanilla DP-SGD and does not track the gradient subspace either explicitly or implicitly, and hence does not change the optimization landscape.
Revised: Our results are incomparable to this line of work because of two reasons: (i) Our algorithm is vanilla DP-SGD and does not track the gradient subspace either explicitly or implicitly, and hence does not change the optimization landscape.

Original: Our improved dependence on dimensions is an artifact of the analysis.
Revised: Our improved dependence on dimensions is an artifact of the analysis.

Original: (ii) Our analytical results do not need the existence of any public data to obtain tighter dependence on dimensions.
Revised: (ii) Our analytical results do not need the existence of any public data to obtain tighter dependence on dimensions.

Original: All prior works mentioned above need the existence of public data to demonstrate any improvement.
Revised: All prior works mentioned above need the existence of public data to demonstrate any improvement.

Original: On the empirical front, past works have observed that for image classification tasks, gradients of
Revised: On the empirical front, past works have observed that for image classification tasks, gradients of

Original: ConvNets converge to a small subspace spanned by the top directions of the Hessian.
Revised: ConvNets converge to a small subspace spanned by the top directions of the Hessian.

Original: In addition, this span remains stable for long periods of time during training [GARD18].
Revised: In addition, this span remains stable for long periods of time during training [GARD18].

Original: While insightful, this line of work does not look at language model fine-tuning.
Revised: While insightful, this line of work does not look at language model fine-tuning.

Original: Another line of work measures for language model fine-tuning the intrinsic dimension —the minimum dimension such that optimizing in a randomly sampled subspace of such dimension approximately recovers the original performance [LFLY18,
Revised: Another line of work measures for language model fine-tuning the intrinsic dimension —the minimum dimension such that optimizing in a randomly sampled subspace of such dimension approximately recovers the original performance [LFLY18,

Original: AZG20].
Revised: AZG20].

Original: We note that a small intrinsic dimension likely suggests that gradients are approximately low rank.
Revised: We note that a small intrinsic dimension likely suggests that gradients are approximately low rank.

Original: Yet, this statement should not be interpreted as a strict implication, since the notion of intrinsic dimension is at best vaguely defined (e.g., there’s no explicit failure probability threshold over the randomly sampled subspace in the original statement), and the definition involves not a fixed subspace but rather a randomly sampled one.
Revised: Yet, this statement should not be interpreted as a strict implication, since the notion of intrinsic dimension is at best vaguely defined (e.g., there’s no explicit failure probability threshold over the randomly sampled subspace in the original statement), and the definition involves not a fixed subspace but rather a randomly sampled one.

Original:  Conclusion
Revised:  Conclusion

Original: We made an attempt to reconcile two seemingly conflicting results: (i) in private convex optimization, errors are predicted to scale proportionally with the dimension of the learning problem; while (ii) in empirical works on large-scale private fine-tuning through DP-SGD, privacy-utility tradeoffs become better with increasing model size.
Revised: We made an attempt to reconcile two seemingly conflicting results: (i) in private convex optimization, errors are predicted to scale proportionally with the dimension of the learning problem; while (ii) in empirical works on large-scale private fine-tuning through DP-SGD, privacy-utility tradeoffs become better with increasing model size.

Original: We introduced the notion of restricted Lipschitz continuity, with which we gave refined analyses of DP-SGD for DP-ERM and DP-SCO.
Revised: We introduced the notion of restricted Lipschitz continuity, with which we gave refined analyses of DP-SGD for DP-ERM and DP-SCO.

Original: When the magnitudes of gradients projected onto diminishing subspaces decay rapidly, our analysis showed that excess empirical and population losses of DP-SGD are independent of the model dimension.
Revised: When the magnitudes of gradients projected onto diminishing subspaces decay rapidly, our analysis showed that excess empirical and population losses of DP-SGD are independent of the model dimension.

Original: Through preliminary experiments, we gave empirical evidence that gradients of large pretrained language models obtained through fine-tuning mostly lie in the subspace spanned by a few principal components.
Revised: Through preliminary experiments, we gave empirical evidence that gradients of large pretrained language models obtained through fine-tuning mostly lie in the subspace spanned by a few principal components.

Original: Our theoretical and empirical results together give a possible explanation for recent successes in large-scale differentially private fine-tuning.
Revised: Our theoretical and empirical results together give a possible explanation for recent successes in large-scale differentially private fine-tuning.

Original: Given our improved upper bounds on the excess empirical and population risks for differentially private convex learning, it is instructive to ask if such bounds are tight in the mini-max sense.
Revised: Given our improved upper bounds on the excess empirical and population risks for differentially private convex learning, it is instructive to ask if such bounds are tight in the mini-max sense.

Original: We leave answering this inquiry to future work.
Revised: We leave answering this inquiry to future work.

Original: In addition, while we have presented encouraging empirical evidence that fine-tuning gradients mostly lie in a small subspace, more work is required to study the robustness of this phenomenon with respect to the model class and fine-tuning problem.
Revised: In addition, while we have presented encouraging empirical evidence that fine-tuning gradients mostly lie in a small subspace, more work is required to study the robustness of this phenomenon with respect to the model class and fine-tuning problem.

Original: Overall, we hope that our work leads to more research on understanding conditions under which DP learning does not degrade with increasing problem size, and more generally, how theory can inform and explain the practical successes of differentially private deep learning.
Revised: Overall, we hope that our work leads to more research on understanding conditions under which DP learning does not degrade with increasing problem size, and more generally, how theory can inform and explain the practical successes of differentially private deep learning.

Original: Acknowledgements
Revised: Acknowledgements

Original: XL is supported by a Stanford Graduate Fellowship.
Revised: XL is supported by a Stanford Graduate Fellowship.

Original: Appendix A Proof of Theorem 3. 
Revised: Appendix A Proof of Theorem 3.

Original: Before bounding the utility of DP-SGD, we first bound x ( t ) − x (0) in expectation.
Revised: Before bounding the utility of DP-SGD, we first bound x ( t ) − x (0) in expectation.

Original: For simplicity, we write f j ( · ) for f ( · ; s j ) and F ( · ) for F ( · ; D ) when there is no ambiguity.
Revised: For simplicity, we write f j ( · ) for f ( · ; s j ) and F ( · ) for F ( · ; D ) when there is no ambiguity.

Original: Lemma A.1.
Revised: Lemma A.1.

Original: Suppose Assumption 3.2 holds.
Revised: Suppose Assumption 3.2 holds.

Original: Let Q be any orthogonal projection matrix with rank r and suppose that ∥ Q ∇ f ( x ; s ) ∥ 2 ≤ G Q for all x ∈ R d and s ∈ S .
Revised: Let Q be any orthogonal projection matrix with rank r and suppose that ∥ Q ∇ f ( x ; s ) ∥ 2 ≤ G Q for all x ∈ R d and s ∈ S .

Original: If we set η ≤ 12 α in DP-SGD, then for all t > 0 , we have
Revised: If we set η ≤ 12 α in DP-SGD, then for all t > 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma A.1.
Revised: Proof of Lemma A.1.

Original: By the assumption, we know ∥ Q ∇ F ( x ) ∥ 2 ≤ G Q and ∥∇ F ( x ) ∥ 2 ≤ G 0 .
Revised: By the assumption, we know ∥ Q ∇ F ( x ) ∥ 2 ≤ G Q and ∥∇ F ( x ) ∥ 2 ≤ G 0 .

Original: Let z ( t ) =
Revised: Let z ( t ) =

Original: x ( t ) − x (0) .
Revised: x ( t ) − x (0) .

Original: Note that
Revised: Note that

Original: [Equation]
Revised: [Equation]

Original: where ζ ∼ N (0 , σ 2 I d ) is the isotropic Gaussian noise drawn in ( t + 1) th step.
Revised: where ζ ∼ N (0 , σ 2 I d ) is the isotropic Gaussian noise drawn in ( t + 1) th step.

Original: For simplicity, we use ˜ ∇ f j ( x ( t ) ) to denote the noisy subgradient ∇ f j ( x ( t ) )
Revised: For simplicity, we use ˜ ∇ f j ( x ( t ) ) to denote the noisy subgradient ∇ f j ( x ( t ) )

Original: + G 0 · ζ .
Revised: + G 0 · ζ .

Original: Hence, we have
Revised: Hence, we have

Original: [Equation]
Revised: [Equation]

Original: Taking expectation over the random sample f j and random Gaussian noise ζ , we have
Revised: Taking expectation over the random sample f j and random Gaussian noise ζ , we have

Original: [Equation]
Revised: [Equation]

Original: where we used the fact that ζ has zero mean, ∥∇ f j ( x ( t ) )
Revised: where we used the fact that ζ has zero mean, ∥∇ f j ( x ( t ) )

Original: ∥ 2 ≤ G 0 , ∥ Q ∇ F ( x ( t ) )
Revised: ∥ 2 ≤ G 0 , ∥ Q ∇ F ( x ( t ) )

Original: ∥ 2 ≤ G Q and η ≤ 12 α .
Revised: ∥ 2 ≤ G Q and η ≤ 12 α .

Original: Further simplifying and taking expectation over all iterations, we have
Revised: Further simplifying and taking expectation over all iterations, we have

Original: [Equation]
Revised: [Equation]

Original: Using that z (0)
Revised: Using that z (0)

Original: = 

Revised: =

Original: we know E ∥
Revised: we know E ∥

Original: Qz (0) ∥
Revised: Qz (0) ∥

Original: 22 = 0 .
Revised: 22 = 0 .

Original: Solving the recursion (Equation (5)) gives
Revised: Solving the recursion (Equation (5)) gives

Original: [Equation]
Revised: [Equation]

Original: for all t .
Revised: for all t .

Original: This proves the result.
Revised: This proves the result.

Original: Now, we are ready to bound the utility.
Revised: Now, we are ready to bound the utility.

Original: The proof builds off the standard mirror descent proof.
Revised: The proof builds off the standard mirror descent proof.

Original: Lemma A.2.
Revised: Lemma A.2.

Original: Let δ ∈ (0 , 12 ] , and ε ∈ (0 , 10] .
Revised: Let δ ∈ (0 , 12 ] , and ε ∈ (0 , 10] .

Original: Under Assumption 3.2, let x (0) be the initial iterate and x ∗ =
Revised: Under Assumption 3.2, let x (0) be the initial iterate and x ∗ =

Original: argmin x F ( x ) .
Revised: argmin x F ( x ) .

Original: Suppose ∥ x (0) − x ∗ ∥ 2 ≤ D .
Revised: Suppose ∥ x (0) − x ∗ ∥ 2 ≤ D .

Original: For any positive integer k ≤ d , setting
Revised: For any positive integer k ≤ d , setting

Original: T =
Revised: T =

Original: Θ( n 2 + d log 2 d ) , σ = Θ (cid:18) √
Revised: Θ( n 2 + d log 2 d ) , σ = Θ (cid:18) √

Original: [Equation]
Revised: [Equation]

Original: δ
Revised: δ

Original: [Equation]
Revised: [Equation]

Original: T · G 20 · kσ
Revised: T · G 20 · kσ

Original: [Equation]
Revised: [Equation]

Original: and α
Revised: and α

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where S =
Revised: where S =

Original: ⌊ log( d/k ) ⌋ + 1 , x is the output of DP-SGD, and the expectation is under the randomness of DP-SGD.
Revised: ⌊ log( d/k ) ⌋ + 1 , x is the output of DP-SGD, and the expectation is under the randomness of DP-SGD.

Original: Moreover, if G k ≤ G 0 k − c for each k for some c > 1 / 2 , and in addition n > ε − 1 (cid:112) log(1 /δ ) , picking the best k ∈ [ d ] for the bound above gives
Revised: Moreover, if G k ≤ G 0 k − c for each k for some c > 1 / 2 , and in addition n > ε − 1 (cid:112) log(1 /δ ) , picking the best k ∈ [ d ] for the bound above gives

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma A.2.
Revised: Proof of Lemma A.2.

Original: Fix a positive integer 1 ≤ k ≤ d .
Revised: Fix a positive integer 1 ≤ k ≤ d .

Original: Our key idea is to split the whole space R d into different subspaces.
Revised: Our key idea is to split the whole space R d into different subspaces.

Original: We define the following set of subspaces:
Revised: We define the following set of subspaces:

Original: • U 

Revised: • U

Original: =
Revised: =

Original: range ( P k ) .
Revised: range ( P k ) .

Original: • For s = 1 , 2 , . . .
Revised: • For s = 1 , 2 , . . .

Original: , ⌊ log( d/k ) ⌋ , let U s
Revised: , ⌊ log( d/k ) ⌋ , let U s

Original: ⊆ range ( P 2 s k ) be a subspace with maximal dimension such that U s ⊥
Revised: ⊆ range ( P 2 s k ) be a subspace with maximal dimension such that U s ⊥

Original: U i for all
Revised: U i for all

Original: i = 0 , . .
Revised: i = 0 , . .

Original: , s − 1 .
Revised: , s − 1 .

Original: • For S =
Revised: • For S =

Original: ⌊ log( d/k ) ⌋ + 1 , let U S ⊆
Revised: ⌊ log( d/k ) ⌋ + 1 , let U S ⊆

Original: R d be the subspace such that (cid:76) Si
Revised: R d be the subspace such that (cid:76) Si

Original: =
Revised: =

Original: U i
Revised: U i

Original: =
Revised: =

Original: R d , and U S ⊥
Revised: R d , and U S ⊥

Original: U i for all i = 0 , . .
Revised: U i for all i = 0 , . .

Original: , S − 1 .
Revised: , S − 1 .

Original: Recall P i is the orthogonal projection matrix with rank i that gives rise to G i in Assumption 3.2.
Revised: Recall P i is the orthogonal projection matrix with rank i that gives rise to G i in Assumption 3.2.

Original: In the above, we have assumed that the base of log is 2.
Revised: In the above, we have assumed that the base of log is 2.

Original: Let Q s be the orthogonal projection matrix that projects vectors onto the subspace U s .
Revised: Let Q s be the orthogonal projection matrix that projects vectors onto the subspace U s .

Original: Note that rank( Q s ) ≤ 2 s k since U s ⊆ range ( P 2 s k ) .
Revised: Note that rank( Q s ) ≤ 2 s k since U s ⊆ range ( P 2 s k ) .

Original: Moreover, it’s clear that U s ⊥ range ( P 2 s − 1 k ) for all s ∈ { 1 , . . .
Revised: Moreover, it’s clear that U s ⊥ range ( P 2 s − 1 k ) for all s ∈ { 1 , . . .

Original: , S } .
Revised: , S } .

Original: This is because by construction (cid:76) s −i =

Revised: This is because by construction (cid:76) s −i =

Original: U i ⊇ range ( P 2 s − 1 k ) , and that U s ⊥ (cid:76) s − 1 i
Revised: U i ⊇ range ( P 2 s − 1 k ) , and that U s ⊥ (cid:76) s − 1 i

Original: =

Revised: =

Original: U i .
Revised: U i .

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: for all x ∈ R d and all s ∈ { 1 , . . .
Revised: for all x ∈ R d and all s ∈ { 1 , . . .

Original: , S } .
Revised: , S } .

Original: Let j ∈
Revised: Let j ∈

Original: [ n ] be the (uniformly random) index sampled in iteration t of DP-SGD.
Revised: [ n ] be the (uniformly random) index sampled in iteration t of DP-SGD.

Original: By convexity of the individual loss f j ,
Revised: By convexity of the individual loss f j ,

Original: [Equation]
Revised: [Equation]

Original: By construction, R d is the orthogonal direct sum of the subspaces { U j } Sj =0 , and thus any vector v ∈ R
Revised: By construction, R d is the orthogonal direct sum of the subspaces { U j } Sj =0 , and thus any vector v ∈ R

Original: d can be rewritten as the sum (cid:80) Si =

Revised: d can be rewritten as the sum (cid:80) Si =

Original: Q i v .
Revised: Q i v .

Original: We thus split the right hand side of the above as follows
Revised: We thus split the right hand side of the above as follows

Original: [Equation]
Revised: [Equation]

Original: We use different approaches to bound ( Q 0 ∇ f j ( x ( t ) ))
Revised: We use different approaches to bound ( Q 0 ∇ f j ( x ( t ) ))

Original: ⊤ ( x ( t ) − x ∗ ) and ( Q s ∇ f j ( x ( t ) ))
Revised: ⊤ ( x ( t ) − x ∗ ) and ( Q s ∇ f j ( x ( t ) ))

Original: ⊤ ( x ( t ) − x ∗ ) when s ≥ 1 , and we discuss them separately in the following.
Revised: ⊤ ( x ( t ) − x ∗ ) when s ≥ 1 , and we discuss them separately in the following.

Original: Term one: Bound ( Q 0 ∇ f j ( x ( t ) ))
Revised: Term one: Bound ( Q 0 ∇ f j ( x ( t ) ))

Original: ⊤ ( x ( t ) − x ∗ ) .
Revised: ⊤ ( x ( t ) − x ∗ ) .

Original: Recall that
Revised: Recall that

Original: [Equation]
Revised: [Equation]

Original: for some Gaussian ζ ∼ N (0 , σ 2 I d ) .
Revised: for some Gaussian ζ ∼ N (0 , σ 2 I d ) .

Original: Hence, we have
Revised: Hence, we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where we used the fact that Q 20 v = Q 0 v for any v ∈ R d (since Q 0 is a projection matrix)
Revised: where we used the fact that Q 20 v = Q 0 v for any v ∈ R d (since Q 0 is a projection matrix)

Original: [Equation]
Revised: [Equation]

Original: and the last equality follows from that
Revised: and the last equality follows from that

Original: [Equation]
Revised: [Equation]

Original: Taking expectation on ζ over both sides of Equation (8) and making use of the fact that ζ has mean
Revised: Taking expectation on ζ over both sides of Equation (8) and making use of the fact that ζ has mean

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Recalling the definition of Q 0 and that Q 0 has rank at most k
Revised: Recalling the definition of Q 0 and that Q 0 has rank at most k

Original: [Equation]
Revised: [Equation]

Original: one has
Revised: one has

Original: [Equation]
Revised: [Equation]

Original: Moreover, one has
Revised: Moreover, one has

Original: [Equation]
Revised: [Equation]

Original: Hence
Revised: Hence

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where we used η ≤ 12 α at the end.
Revised: where we used η ≤ 12 α at the end.

Original: Term two: Bound ( Q s ∇ f j ( x ( t ) ))
Revised: Term two: Bound ( Q s ∇ f j ( x ( t ) ))

Original: ⊤ ( x ( t ) − x ∗ ) .
Revised: ⊤ ( x ( t ) − x ∗ ) .

Original: We bound the objective above for each s separately.
Revised: We bound the objective above for each s separately.

Original: By taking expectation over the random f j , we have
Revised: By taking expectation over the random f j , we have

Original: [Equation]
Revised: [Equation]

Original: t ) ))
Revised: t ) ))

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: ∗
Revised: ∗

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: t )
Revised: t )

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: where we chose α s = αs − 2 2 − s and used the bound (6) and Young’s inequality at the end.
Revised: where we chose α s = αs − 2 2 − s and used the bound (6) and Young’s inequality at the end.

Original: Combination:
Revised: Combination:

Original: Combining both terms into (7) and taking expectation over all randomness, we have
Revised: Combining both terms into (7) and taking expectation over all randomness, we have

Original: [Equation]
Revised: [Equation]

Original: Recall α · η ≤ 1 / 2 .
Revised: Recall α · η ≤ 1 / 2 .

Original: Under the other assumptions, by Lemma A.1, one can show
Revised: Under the other assumptions, by Lemma A.1, one can show

Original: [Equation]
Revised: [Equation]

Original: Using (cid:80) ∞ s =1 s − 2 ≤ 2 , we have
Revised: Using (cid:80) ∞ s =1 s − 2 ≤ 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Summing up over t = 1 , 2 , · · · , T , by the assumption that ∥ x (0) − x ∗ ∥ 2 ≤ D and convexity of the function
Revised: Summing up over t = 1 , 2 , · · · , T , by the assumption that ∥ x (0) − x ∗ ∥ 2 ≤ D and convexity of the function

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Set the parameters T
Revised: Set the parameters T

Original: = c 1 ( n 2 + d log 2 d ) , σ = c√ T log(1 /δ )
Revised: = c 1 ( n 2 + d log 2 d ) , σ = c√ T log(1 /δ )

Original: nε , η = (cid:113)
Revised: nε , η = (cid:113)

Original: D 2 T · G 20 · kσ 2 and α = 1 D (cid:113)(cid:80) Ss =1 s 2 

Revised: D 2 T · G 20 · kσ 2 and α = 1 D (cid:113)(cid:80) Ss =1 s 2

Original: s G 22 s − 1 k for some large constants c 1 , c 2 .
Revised: s G 22 s − 1 k for some large constants c 1 , c 2 .

Original: Note that this choice of parameters satisfies the condition of
Revised: Note that this choice of parameters satisfies the condition of

Original: [Equation]
Revised: [Equation]

Original: where we used the fact that G k ≤ G 0 , s ≤ S ≤ log(2 d ) , T ≥ n 2 + d log 2 d , and c 2 is large enough.
Revised: where we used the fact that G k ≤ G 0 , s ≤ S ≤ log(2 d ) , T ≥ n 2 + d log 2 d , and c 2 is large enough.

Original: Using the parameters we pick, we have
Revised: Using the parameters we pick, we have

Original: [Equation]
Revised: [Equation]

Original: Moreover, assuming G k ≤ G 0 k − c for some c > 1 / 2 , we have (cid:113)(cid:80) s s 2 

Revised: Moreover, assuming G k ≤ G 0 k − c for some c > 1 / 2 , we have (cid:113)(cid:80) s s 2

Original: s G 22 s − 1 k ≲ G 0 /k c .
Revised: s G 22 s − 1 k ≲ G 0 /k c .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: Since the above bound holds for all k ∈ { 1 , . . .
Revised: Since the above bound holds for all k ∈ { 1 , . . .

Original: , d } , we may optimize it with respect to k .
Revised: , d } , we may optimize it with respect to k .

Original: Recall by assumption that n ≥ ε − 1 (cid:112) log(1 /δ ) .
Revised: Recall by assumption that n ≥ ε − 1 (cid:112) log(1 /δ ) .

Original: Letting
Revised: Letting

Original: [Equation]
Revised: [Equation]

Original: yields the bound
Revised: yields the bound

Original: [Equation]
Revised: [Equation]

Original: Combining the privacy guarantee in Lemma 2.4 and Lemma A.2 directly results in Theorem 3.3.
Revised: Combining the privacy guarantee in Lemma 2.4 and Lemma A.2 directly results in Theorem 3.3.

Original: B Proof of Theorem 3. 
Revised: B Proof of Theorem 3.

Original: We study the generalization error of DP-SGD.
Revised: We study the generalization error of DP-SGD.

Original: Similar to previous works, we make use of the stability of DP-SGD to bound its generalization error.
Revised: Similar to previous works, we make use of the stability of DP-SGD to bound its generalization error.

Original: The bound on the excess population loss follows from combining bounds on the excess empirical loss and the generalization error.
Revised: The bound on the excess population loss follows from combining bounds on the excess empirical loss and the generalization error.

Original: Before stating the proof, we first recall two results in the literature.
Revised: Before stating the proof, we first recall two results in the literature.

Original: Lemma B.1 ([BE02, Lemma 7]) .
Revised: Lemma B.1 ([BE02, Lemma 7]) .

Original: Given a learning algorithm A , suppose the dataset D = { s 1 , · · · , s n } is made up of n i.i.d.
Revised: Given a learning algorithm A , suppose the dataset D = { s 1 , · · · , s n } is made up of n i.i.d.

Original: samples drawn from the underlying distribution P , and we replace one random sample in D with a freshly sampled s ′ ∼ P and get a new neighboring dataset
Revised: samples drawn from the underlying distribution P , and we replace one random sample in D with a freshly sampled s ′ ∼ P and get a new neighboring dataset

Original: D ′ .
Revised: D ′ .

Original: One has
Revised: One has

Original: [Equation]
Revised: [Equation]

Original: where A ( D ) is the output of A with input D .
Revised: where A ( D ) is the output of A with input D .

Original: Lemma B.2 ([BFGT20, Theorem 3.3]) .
Revised: Lemma B.2 ([BFGT20, Theorem 3.3]) .

Original: Suppose Assumption 3.2 holds, running DP-SGD with step size η on any two neighboring datasets D and D ′ for T steps satisfies that
Revised: Suppose Assumption 3.2 holds, running DP-SGD with step size η on any two neighboring datasets D and D ′ for T steps satisfies that

Original: [Equation]
Revised: [Equation]

Original: where x and x ′ are the outputs of DP-SGD with datasets D and D ′ , respectively.
Revised: where x and x ′ are the outputs of DP-SGD with datasets D and D ′ , respectively.

Original: Proof of Theorem 3.5.
Revised: Proof of Theorem 3.5.

Original: Let x and x ′ be the outputs of DP-SGD when applied to the datasets D and
Revised: Let x and x ′ be the outputs of DP-SGD when applied to the datasets D and

Original: D ′ , respectively.
Revised: D ′ , respectively.

Original: D ′ is a neighbor of D with one example replaced by s ′ ∼ P that is independently sampled.
Revised: D ′ is a neighbor of D with one example replaced by s ′ ∼ P that is independently sampled.

Original: Combining Lemma B.1 and Lemma B.2 yields
Revised: Combining Lemma B.1 and Lemma B.2 yields

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Similar to the DP-ERM case, by setting T
Revised: Similar to the DP-ERM case, by setting T

Original: =
Revised: =

Original: c 1 ( n 2 + d log 2 d ) , σ =
Revised: c 1 ( n 2 + d log 2 d ) , σ =

Original: [Equation]
Revised: [Equation]

Original: (cid:113)
Revised: (cid:113)

Original: D 2 T · G 20 ( T/n + kσ 2 ) and α = 1 D (cid:113)(cid:80)
Revised: D 2 T · G 20 ( T/n + kσ 2 ) and α = 1 D (cid:113)(cid:80)

Original: Ss =1 s 2 

Revised: Ss =1 s 2

Original: s G 22 s − 1 k for some large positive constants c 1 and c 2 , we conclude that η · α ≤ 1 / 2 .
Revised: s G 22 s − 1 k for some large positive constants c 1 and c 2 , we conclude that η · α ≤ 1 / 2 .

Original: Hence, Equation (9) shows that, for any fixed dataset D and any x ∗ such that (cid:13)(cid:13) x
Revised: Hence, Equation (9) shows that, for any fixed dataset D and any x ∗ such that (cid:13)(cid:13) x

Original: (0) − x ∗ (cid:13)(cid:13) 2 ≤ D , we have
Revised: (0) − x ∗ (cid:13)(cid:13) 2 ≤ D , we have

Original: [Equation]
Revised: [Equation]

Original: We can therefore rewrite the population loss as follows
Revised: We can therefore rewrite the population loss as follows

Original: [Equation]
Revised: [Equation]

Original: Substituting in the values for parameters T , σ , η , and α yields
Revised: Substituting in the values for parameters T , σ , η , and α yields

Original: [Equation]
Revised: [Equation]

Original: 1 k for all positive integers k ≤ d .
Revised: 1 k for all positive integers k ≤ d .

Original: Similarly, if we have G k ≤ G 0 k − c for some c > 1 / 2 , and in addition n > ε − 1 log(1 /δ ) , it immediately follows that
Revised: Similarly, if we have G k ≤ G 0 k − c for some c > 1 / 2 , and in addition n > ε − 1 log(1 /δ ) , it immediately follows that

Original: [Equation]
Revised: [Equation]

Original: This completes the proof.
Revised: This completes the proof.

Original: C Protocol for Synthetic Example Experiments in Section 4. 
Revised: C Protocol for Synthetic Example Experiments in Section 4.

Original: We detail the construction of the synthetic example in Section 4.1.
Revised: We detail the construction of the synthetic example in Section 4.1.

Original: The training and test sets of this example both have n train = n test = 10000 instances.
Revised: The training and test sets of this example both have n train = n test = 10000 instances.

Original: Each instance x i is sampled from a distribution where the first d min = 10 coordinates are all multi-variate normal distributions with mean and standard deviation both being 1 .
Revised: Each instance x i is sampled from a distribution where the first d min = 10 coordinates are all multi-variate normal distributions with mean and standard deviation both being 1 .

Original: All remaining coordinates are constantly 0 .
Revised: All remaining coordinates are constantly 0 .

Original: This ensures the optimal non-private training losses for problems of different dimensions are the same.
Revised: This ensures the optimal non-private training losses for problems of different dimensions are the same.

Original: D Protocol and Additional Fine-Tuning Experiments for Section 4. D.1 Experimental Protocol
Revised: D Protocol and Additional Fine-Tuning Experiments for Section 4. D.1 Experimental Protocol

Original: For experiments in Section 4.2, we fine-tuned the DistilRoBERTa model with (8 , 1 / n 1 . 1 ) -DP on the
Revised: For experiments in Section 4.2, we fine-tuned the DistilRoBERTa model with (8 , 1 / n 1 . 1 ) -DP on the

Original: SST-2 training set with n ≥ 60000 examples and measured performance on the companion dev set.
Revised: SST-2 training set with n ≥ 60000 examples and measured performance on the companion dev set.

Original: We used the exact set of hyperparameters presented in [LTLH21] for this task.
Revised: We used the exact set of hyperparameters presented in [LTLH21] for this task.

Original: We repeated our re-training experiments over five independent random seeds.
Revised: We repeated our re-training experiments over five independent random seeds.

Original: Fine-tuning for 3 epochs on this task takes 10 minutes on an A100-powered machine with sufficient RAM and CPU cores.
Revised: Fine-tuning for 3 epochs on this task takes 10 minutes on an A100-powered machine with sufficient RAM and CPU cores.

Original: Our spectral analysis relies on running the orthogonal iteration algorithm on the set of collected gradients along the private fine-tuning trajectory [Dem97].
Revised: Our spectral analysis relies on running the orthogonal iteration algorithm on the set of collected gradients along the private fine-tuning trajectory [Dem97].

Original: 3 Unlike other numerical algorithms for estimating eigenvalues of a matrix, orthogonal iteration provably produces the set of eigenvalues that are largest in absolute value (and corresponding eigenvectors, if the underlying matrix is normal) in the limit.
Revised: 4 Unlike other numerical algorithms for estimating eigenvalues of a matrix, orthogonal iteration provably produces the set of eigenvalues that are largest in absolute value (and corresponding eigenvectors, if the underlying matrix is normal) in the limit.

Original: 4 Recall that we needed the top eigenvectors for projection in our re-training experiment.
Revised: 5 Recall that we needed the top eigenvectors for projection in our re-training experiment.

Original: By default, we run orthogonal iteration for ten iterations.
Revised: By default, we run orthogonal iteration for ten iterations.

Original: We show in the following that our results are insensitivity to the number of iterations used in the orthogonal iteration algorithm.
Revised: We show in the following that our results are insensitivity to the number of iterations used in the orthogonal iteration algorithm.

Original: Each orthogonal iteration takes 10 minutes on an A100-powered machine with r = 4000 gradient samples and k = 1000 principal components for the DistilRoBERTa experiments.
Revised: Each orthogonal iteration takes 10 minutes on an A100-powered machine with r = 4000 gradient samples and k = 1000 principal components for the DistilRoBERTa experiments.

Original: D.2 Robustness of Results
Revised: D.2 Robustness of Results

Original: Robustness to the number of orthogonal iterations.
Revised: Robustness to the number of orthogonal iterations.

Original: The orthogonal iteration algorithm is a generalization of the power method that simultaneously produces estimates of multiple eigenvalues.
Revised: The orthogonal iteration algorithm is a generalization of the power method that simultaneously produces estimates of multiple eigenvalues.

Original: Its convergence is known to be sensitivity to the gap between successive eigenvalues.
Revised: Its convergence is known to be sensitivity to the gap between successive eigenvalues.

Original: The algorithm converges slowly if consecutive eigenvalues (with the largest absolute values) are close in absolute value.
Revised: The algorithm converges slowly if consecutive eigenvalues (with the largest absolute values) are close in absolute value.

Original: To confirm that our results aren’t sensitivity to the choice of the number of iterations, we visualize the top 500 eigenvalues for the orthogonal iteration algorithm is run for different number of updates.
Revised: To confirm that our results aren’t sensitivity to the choice of the number of iterations, we visualize the top 500 eigenvalues for the orthogonal iteration algorithm is run for different number of updates.

Original: Figure 3 shows that the linear fit to the top 500 eigenvalues remains stable across different number of orthogonal iterations T .
Revised: Figure 3 shows that the linear fit to the top 500 eigenvalues remains stable across different number of orthogonal iterations T .

Original: Notably, T = 10 produces similar results as T = 100 .
Revised: Notably, T = 10 produces similar results as T = 100 .

Original: These results were obtained with r = 4000 gradients.
Revised: These results were obtained with r = 4000 gradients.

Original: [Figure]
Revised: [Figure]

Original: Robustness to the number of gradient samples.
Revised: Robustness to the number of gradient samples.

Original: We further ran experiments with different numbers of gradient samples r collected along the fine-tuning trajectory, and plot the topeigenvalues.
Revised: We further ran experiments with different numbers of gradient samples r collected along the fine-tuning trajectory, and plot the topeigenvalues.

Original: Figure 4 shows that while the slope and intercept of the fitted line in log-log space changes, the change is moderate.
Revised: Figure 4 shows that while the slope and intercept of the fitted line in log-log space changes, the change is moderate.

Original: Notably, the decaying trend of the top eigenvalues remains stable.
Revised: Notably, the decaying trend of the top eigenvalues remains stable.

Original: [Figure]
Revised: [Figure]

Original: Robustness to the gradient sampling strategy.
Revised: Robustness to the gradient sampling strategy.

Original: We observe that gradients at the beginning of fine-tuning tend to be larger in magnitude than gradients collected later on along the optimization trajectory.
Revised: We observe that gradients at the beginning of fine-tuning tend to be larger in magnitude than gradients collected later on along the optimization trajectory.

Original: To eliminate the potential confounder that the top principal components are solely formed by the initial few gradients evaluated during fine-tuning, we re-ran the spectral analysis experiment without the initial gradients.
Revised: To eliminate the potential confounder that the top principal components are solely formed by the initial few gradients evaluated during fine-tuning, we re-ran the spectral analysis experiment without the initial gradients.

Original: Specifically, we performed PCA for the gradients evaluated from step 300 to step 1300 during private fine-tuning, and compared the distribution of top eigenvalues returned from this setup to when we used the first 1000 gradients.
Revised: Specifically, we performed PCA for the gradients evaluated from step 300 to step 1300 during private fine-tuning, and compared the distribution of top eigenvalues returned from this setup to when we used the first 1000 gradients.

Original: Note the dev set accuracy converged to ≈ 90% by step 200.
Revised: Note the dev set accuracy converged to ≈ 90% by step 200.

Original: Figure 5 shows that while the slope and intercept of linear fits are slightly different in the new setup compared to the old setup (when all gradients along the fine-tuning trajectory were used for PCA), that the eigenvalues follow a rapidly decaying trend remains true under both setups.
Revised: Figure 5 shows that while the slope and intercept of linear fits are slightly different in the new setup compared to the old setup (when all gradients along the fine-tuning trajectory were used for PCA), that the eigenvalues follow a rapidly decaying trend remains true under both setups.

Original: [Figure]
Revised: [Figure]

Original: Robustness to model size.
Revised: Robustness to model size.

Original: In previous experiments, we empirically verified that gradients for fine-tuning DistilRoBERTa are near low rank.
Revised: In previous experiments, we empirically verified that gradients for fine-tuning DistilRoBERTa are near low rank.

Original: Here, we show that similar observations also hold for Roberta-base and Roberta-large when fine-tuning only the attention layers.
Revised: Here, we show that similar observations also hold for Roberta-base and Roberta-large when fine-tuning only the attention layers.

Original: The former setup has approximately 14 million trainable parameters, while the latter has approximately 50 million.
Revised: The former setup has approximately 14 million trainable parameters, while the latter has approximately 50 million.

Original: Figures 6 and 7 illustrate these results.
Revised: Figures 6 and 7 illustrate these results.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Non
Revised: Non

Original: -robustness to fine-tuning strategy.
Revised: -robustness to fine-tuning strategy.

Original: Recall our fine-tuning experiments for classification was based on the template-completion formulation detailed in [LTLH21].
Revised: Recall our fine-tuning experiments for classification was based on the template-completion formulation detailed in [LTLH21].

Original: As opposed to framing the task as integer label prediction
Revised: As opposed to framing the task as integer label prediction

Original: [Figure]
Revised: [Figure]

Original: this formulation requires the model to predict one of K candidate tokens to fill in a templated prompt for a K -way classification problem.
Revised: this formulation requires the model to predict one of K candidate tokens to fill in a templated prompt for a K -way classification problem.

Original: While we have also performed the experiment where we re-train in the subspace of top principal components under the usual fine-tuning setup (stack a randomly initialized prediction head on top of the embedding of the [CLS] token), we found it difficult to recover the original fine-tuning performance when gradients are projected onto the top eigen-subspace with d = 100 dimensions.
Revised: While we have also performed the experiment where we re-train in the subspace of top principal components under the usual fine-tuning setup (stack a randomly initialized prediction head on top of the embedding of the [CLS] token), we found it difficult to recover the original fine-tuning performance when gradients are projected onto the top eigen-subspace with d = 100 dimensions.

Original: Retraining performance exhibited high variance and the final dev accuracy was bi-modal over random seeds with near guess accuracy ( ≈ 50% ) and original accuracy ( ≈ 90% ) being the two modes.
Revised: Retraining performance exhibited high variance and the final dev accuracy was bi-modal over random seeds with near guess accuracy ( ≈ 50% ) and original accuracy ( ≈ 90% ) being the two modes.

Original: We suspect this to be caused by the linear prediction head being randomly initialized.
Revised: We suspect this to be caused by the linear prediction head being randomly initialized.

Original: D.3 Additional Fine-Tuning Experiments with DistilGPT-2 for Generation Tasks
Revised: D.3 Additional Fine-Tuning Experiments with DistilGPT-2 for Generation Tasks

Original: Experiments in Section 4.2 demonstrated that gradients from fine-tuning for classification are mostly controlled by a few principal components.
Revised: Experiments in Section 4.2 demonstrated that gradients from fine-tuning for classification are mostly controlled by a few principal components.

Original: In this section, we show that similar observations hold for fine-tuning on a text generation task.
Revised: In this section, we show that similar observations hold for fine-tuning on a text generation task.

Original: We follow the setup and hyperparameters in [LTLH21] for privately fine-tuning DistilGPT-2 on the E2E dataset [NDR17] under (8 , 1 / n 1 . 1 ) -DP.
Revised: We follow the setup and hyperparameters in [LTLH21] for privately fine-tuning DistilGPT-2 on the E2E dataset [NDR17] under (8 , 1 / n 1 . 1 ) -DP.

Original: We fine-tune all weight matrices in attention layers that produce the queries, values, and keys.
Revised: We fine-tune all weight matrices in attention layers that produce the queries, values, and keys.

Original: This amounts to finetuning approximately 10 .
Revised: This amounts to finetuning approximately 10 .

Original: 6 million parameters of a model with a total parameter count of more than 100 million.
Revised: 6 million parameters of a model with a total parameter count of more than 100 million.

Original: We again collected r = 4000 gradients evaluated during private fine-tuning, performed
Revised: We again collected r = 4000 gradients evaluated during private fine-tuning, performed

Original: PCA, and conducted the eigenspectrum analysis.
Revised: PCA, and conducted the eigenspectrum analysis.

Original: Figure 8 shows that the top eigenspectrum decays rapidly with a rate similar to what is observed in fine-tuning for the classification problem we studied.
Revised: Figure 8 shows that the top eigenspectrum decays rapidly with a rate similar to what is observed in fine-tuning for the classification problem we studied.

Original: [Figure]
Revised: [Figure]