Original: Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss
Revised: Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: A central issue in machine learning is how to train models on sensitive user data.
Revised: A central issue in machine learning is how to train models on sensitive user data.

Original: Industry has widely adopted a simple algorithm:
Revised: Industry has widely adopted a simple algorithm:

Original: Stochastic Gradient Descent withnoise (a.k.a.
Revised: Stochastic Gradient Descent with noise (a.k.a.

Original: Stochastic Gradient Langevin Dynamics).
Revised: Stochastic Gradient Langevin Dynamics).

Original: However, foundationaltheoretical questions about this algorithm’s privacy loss remain open—even inthe seemingly simple setting of smooth convex losses over a bounded domain.
Revised: However, foundational theoretical questions about this algorithm’s privacy loss remain open—even in the seemingly simple setting of smooth convex losses over a bounded domain.

Original: Our main result resolves these questions: for a large range of parameters, wecharacterize the differential privacy up to a constant.
Revised: Our main result resolves these questions: for a large range of parameters, we characterize the differential privacy up to a constant.

Original: This result reveals that allprevious analyses for this setting have the wrong qualitative behavior.
Revised: This result reveals that all previous analyses for this setting have the wrong qualitative behavior.

Original: Speciﬁcally,while previous privacy analyses increase ad inﬁnitum in the number of iterations,we show that after a small burn-in period, running SGD longer leaks no furtherprivacy.
Revised: Specifically, while previous privacy analyses increase ad infinitum in the number of iterations, we show that after a small burn-in period, running SGD longer leaks no further privacy.

Original: Our analysis departscompletely from previous approaches based on fastmixing, instead using techniques based on optimal transport (namely, Privacy Ampliﬁcation by Iteration) and the sampled Gaussian mechanism (namely, Privacy 
Revised: Our analysis departs from previous approaches based on fast mixing, instead using techniques based on optimal transport (namely, Privacy Amplification by Iteration) and the Sampled Gaussian Mechanism (namely, Privacy Amplification by Sampling).

Original: Ampliﬁcation by Sampling). Our techniques readily extend to other settings, e.g.,strongly convex losses, non-uniform stepsizes, arbitrary batch sizes, and random orcyclic batches.
Revised: Our techniques readily extend to other settings.

Original:  Introduction
Revised:  Introduction

Original: Convex optimization is a fundamental task in machine learning.
Revised: Convex optimization is a fundamental task in machine learning.

Original: When models are learnt on sensitivedata, privacy becomes a major concern—motivating a large body of work on differentially privateconvex optimization [8, 3, 4,  13, 20, 18, 27, 5].
Revised: When models are learnt on sensitive data, privacy becomes a major concern—motivating a large body of work on differentially private convex optimization [3, 4, 5, 8, 13, 19, 21, 30].

Original: In practice, the most common approach for trainingprivate models is Noisy-SGD , i e., Stochastic Gradient Descent with noise added each iteration.
Revised: In practice, the most common approach for training private models is N OISY -SGD, i e., Stochastic Gradient Descent with noise added each iteration.

Original: Thisalgorithm is simple and natural, has optimal utility bounds [3, 4] and is implemented in mainstreammachine learning platforms such as Tensorﬂow (TF Privacy) [1], PyTorch (Opacus) [32] and JAX [6].
Revised: This algorithm is simple and natural, has optimal utility bounds [3, 4], and is implemented in mainstream machine learning platforms such as Tensorflow (TF Privacy) [1], PyTorch (Opacus) [35] and JAX [6].

Original: Yet, despite the simplicity and ubiquity of this Noisy-SGD algorithm, we do not understand basicquestions about its privacy loss 1 —i.e., how sensitive the output of Noisy-SGD is with respect to thetraining data.
Revised: Yet, despite the simplicity and ubiquity of this N OISY -SGD algorithm, we do not understand basic questions about its privacy loss 1 —i.e., how sensitive the output of N OISY -SGD is with respect to the training data.

Original: Speciﬁcally:
Revised: Specifically:

Original: Question 1.1.
Revised: Question 1.1.

Original: What is the privacy loss of Noisy-SGD as a function of the number of iterations?
Revised: What is the privacy loss of N OISY -SGD as a function of the number of iterations?

Original: Even in the seemingly simple setting of smooth convex losses over a bounded domain, this fundamental question has remained wide open.
Revised: Even in the seemingly simple setting of smooth convex losses over a bounded domain, this fundamental question has remained wide open.

Original: In fact, even more basic questions are open:
Revised: In fact, even more basic questions are open:

Original: Question 1.2.
Revised: Question 1.2.

Original: Does the privacy loss of Noisy-SGD increase ad inﬁnitum in the number of iterations?
Revised: Does the privacy loss of N OISY -SGD increase ad infinitum in the number of iterations?

Original: The purpose of this paper is to understand these fundamental theoretical questions.
Revised: The purpose of this paper is to understand these fundamental theoretical questions.

Original: Speciﬁcally, weresolve Questions 1.1 and 1.2 by characterizing the privacy loss of Noisy-SGD up to a constantfactor in this (and other) settings for a large range of parameters.
Revised: Specifically, we resolve Questions 1.1 and 1.2 by characterizing the privacy loss of N OISY -SGD up to a constant factor in this (and other) settings for a large range of parameters.

Original: Below, we ﬁrst provide context bydescribing previous analyses in §1.1, and then describe our result in §1.2 and techniques in §1.3.
Revised: Below, we first provide context by describing previous analyses in §1.1, and then describe our result in §1.2 and techniques in §1.3.

Revised: The Supplementary Material contains a full version of this paper with proofs and more discussion.

Revised: Throughout, we follow the literature by writing “privacy loss” to refer to the differential privacy parameters.

Original: 1.1 Previous approaches and limitations
Revised: 1.1 Previous approaches and limitations

Original: Although there is a large body of research devoted to understanding the privacy loss of Noisy-SGD ,all existing analyses have (at least) one of the following two drawbacks.
Revised: Although there is a large body of research devoted to understanding the privacy loss of N OISY -SGD, all existing analyses have (at least) one of the following two drawbacks.

Original: Privacy bounds that increase ad inﬁnitum.
Revised: Privacy bounds that increase ad infinitum.

Original: One type of analysis approach yields upper boundson the DP (resp., Rényi DP) that scale as √ T (resp., T ).
Revised: One type of analysis approach yields upper bounds on the DP (resp., Rényi DP) that scale as √ T (resp., T ).

Original: This includes the original analysesof Noisy-SGD , which were based on the techniques of Privacy Ampliﬁcation by Sampling and
Revised: This includes the original analyses of N OISY -SGD, which were based on the techniques of Privacy Amplification by Sampling and

Original: Advanced Composition [3, 2], as well as alternative analyses based on the technique of Privacy
Revised: Advanced Composition [2, 3], as well as alternative analyses based on the technique of Privacy

Original: Ampliﬁcation by Iteration [12].
Revised: Amplification by Iteration [12].

Original: A key issue with these analyses is that they increase unboundedly inthe number of iterations T .
Revised: A key issue with these analyses is that they increase unboundedly in the number of iterations T .

Original: This limits the number of iterations that Noisy-SGD can be run given areasonable privacy budget, typically leading to suboptimal optimization error in practice.
Revised: This limits the number of iterations that N OISY -SGD can be run given a reasonable privacy budget, typically leading to suboptimal optimization error in practice.

Original: Is this afailure of existing analysis techniques or an inherent fact about the privacy loss of Noisy-SGD ?
Revised: Is this a failure of existing analysis techniques or an inherent fact about the privacy loss of N OISY -SGD?

Original: Privacy bounds that apply for large T and require strong additional assumptions.
Revised: Privacy bounds that apply for large T and require strong additional assumptions.

Original: The secondtype of analysis approach yields convergent upper bounds on the privacy loss, but requires strongadditional assumptions.
Revised: The second type of analysis approach yields convergent upper bounds on the privacy loss, but requires strong additional assumptions.

Original: This approach is based on connections to sampling.
Revised: This approach is based on connections to sampling.

Original: The high-level intuitionis that Noisy-SGD is a discretization of a continuous-time algorithm with bounded privacy loss.
Revised: The high-level intuition is that N OISY -SGD is a discretization of a continuous-time algorithm with bounded privacy loss.

Original: Speciﬁcally, Noisy-SGD can be interpreted as the Stochastic Gradient Langevin Dynamics ( SGLD )algorithm [30], which is a discretization of a continuous-time Markov process that has a simple stationary distribution—and this stationary distribution is equivalent to the exponential mechanism [21],and thus isitself differentially private under certain assumptions.
Revised: Specifically, N OISY -SGD can be interpreted as the Stochastic Gradient Langevin Dynamics (SGLD) algorithm [33], which is a discretization of a continuous-time Markov process whose stationary distribution is equivalent to the exponential mechanism [23] and thus is differentially private under certain assumptions.

Original: However, making this argument rigorous requires strong additional assumptions and/or the resolutionof long-standing open questions aboutboth thediscretization error and mixing time of SGLD (inmetrics relevant to privacy analysis like Rényi divergence).
Revised: However, making this connection precise requires strong additional assumptions and/or the resolution of longstanding open questions about the mixing time of SGLD (see §1.4 for details).

Original: A large effort in this direction recentlyculminated in the breakthrough work by Chourasia et al.
Revised: Only recently did a large effort in this direction culminate in the breakthrough work by Chourasia et al.

Original: [9] which proves that full batch 2 Langevindynamics (a.k.a., Noisy-GD rather than Noisy-SGD ) has a privacy loss that converges as T → ∞ inthis setting where the smooth losses are additionally assumed to be strongly convex .
Revised: [9] which proves that full batch 2 Langevin dynamics (a.k.a., N OISY -GD rather than N OISY -SGD) has a privacy loss that converges as T → ∞ in this setting where the smooth losses are additionally assumed to be strongly convex .

Original: Unfortunately, the assumption of strong convexity seems unavoidable with these techniques.
Revised: Unfortunately, the assumption of strong convexity seems unavoidable with current techniques.

Original: Indeed,in the absence of strong convexity, it is not even known if Noisy-SGD converges to a privatestationary distribution, let alone if this convergence occurs in a reasonable amount of time.
Revised: Indeed, in the absence of strong convexity, it is not even known if N OISY -SGD converges to a private stationary distribution, let alone if this convergence occurs in a reasonable amount of time.

Original: (Thetour-de-force work [7] shows mixing in (large) polynomial time, but only in total variation distancewhich does not have implications for privacy.)
Revised: (The tour-de-force work [7] shows mixing in (large) polynomial time, but only in total variation distance which does not have implications for privacy.)

Original: There are fundamental challenges for proving such aresult.
Revised: There are fundamental challenges for proving such a result.

Original: From a sampling perspective, fast mixing is typically proven via expansion properties suchas log Sobolev inequalities, and these break down without strong convexity.

Original: From an optimizationperspective, SGD is only a weakly contractive process without strong convexity, which means thatits instability increases with the number of iterations [17]—or in other words, it is plausible that
Revised: In short, SGD is only a weakly contractive process without strong convexity, which means that its instability increases with the number of iterations [18]—or in other words, it is plausible that

Original: Noisy-SGD could run for a long time while memorizing training data, which would of course meanit is not a privacy-preserving algorithm.
Revised: N OISY -SGD could run for a long time while memorizing training data, which would of course mean it is not a privacy-preserving algorithm.

Original: As such, given state-of-the-art analyses in both sampling andoptimization, it is unclear if the privacy loss of Noisy-SGD should even remain bounded; i.e., it isunclear what answer one should even expect for Question 1.2, let alone Question 1.1.
Revised: As such, given state-of-the-art analyses in both sampling and optimization, it is unclear if the privacy loss of N OISY -SGD should even remain bounded; i.e., it is unclear what answer one should even expect for Question 1.2, let alone Question 1.1.

Original: 1.2 Contributions
Revised: 1.2 Contributions

Original: The purpose of this paper is to resolve Questions 1.1 and 1.2.
Revised: The purpose of this paper is to resolve Questions 1.1 and 1.2.

Original: To state our result requires ﬁrstrecalling the parameters of the problem.
Revised: To state our result requires first recalling the parameters of the problem.

Original: Throughout, we prefer to state our results in terms of Rényi
Revised: Throughout, we prefer to state our results in terms of Rényi

Original: Differential Privacy (RDP); these RDP bounds are easily translated to DP bounds, as mentionedbelow in Remark 1.4.
Revised: Differential Privacy (RDP); these RDP bounds are easily translated to DP bounds, as mentioned below in Remark 1.4.

Original: See the preliminaries section §2 for deﬁnitions and background on privacy.
Revised: See the preliminaries section §2 for definitions and background on privacy.

Original: Known privacy analyses of Noisy-SGD give an ( ε ) -RDP bound of
Revised: We consider the basic N OISY -SGD algorithm run on a dataset X

Revised: = { x 1 ,...,x n } , where each x i defines a convex, L -Lipschitz, and M -smooth loss function f i (⋅) on a convex set K of diameter D .

Revised: For any step size η ⩽ 2 / M , batch size b , and initialization ω 0 ∈ K , we iterate T times the update ω

Original: [Equation]
Revised: [Equation]

Original: which increases ad inﬁnitum as the number of iterations T → ∞ [3, 2, 12]
Revised: ← Π [Equation]

Original: Our is a characterization of the privacy loss, answering Question This result also answers Question and shows that previous analyses have the wrong qualitative behavior: after a small burn- period
Revised: G

Original: ¯ T =
Revised: [Equation]

Revised: Z t )] , where G t denotes the average gradient vector on a random batch of size b , Z t ∼ N( 0 ,σ 2 I d ) is an isotropic Gaussian, and Π K denotes the Euclidean projection onto K .

Original: Θ ( nDLη ) iterations, Noisy-SGD leaks no further privacy.
Revised: Known privacy analyses of N OISY -SGD give an ( α,ε ) -RDP upper bound of

Original: See Figure
Revised: [Equation] [Equation] which answering Question 1. result also

Original: Theorem 1.3 (Informal statement of main result: tight characterization of the privacy of Noisy-SGD ) .
Revised: Theorem 1.3 (Informal statement of main result: tight characterization of the privacy of N OISY -SGD) .

Original: For all small enough α , Noisy-SGD satisﬁes ( α,ε ) -RDP forε
Revised: For a large range of parameters N OISY -SGD satisfies ( α ε ) -RDP for

Original: [Equation]
Revised: [Equation]

Original: Moreover, for a large range of parameters, this bound is tight.
Revised: and moreover this bound is tight up to a constant factor.

Original: Observe that the privacy bound in Theorem 1.3 is identical to the previous bound (1.1) when thenumber of iterations T is small, but stops increasing when T ⩾ ¯ T .
Revised: Observe that the privacy bound in Theorem 1.3 is identical to the previous bound (1.1) when the number of iterations T is small but stops increasing when T ⩾ ¯ T .

Original: Intuitively, ¯ T can be interpreted asthe smallest number of iterations required for two SGD processes run on adjacent datasets to, withreasonable probability, reach opposite ends of the constraint set.
Revised: Intuitively, ¯ T can be interpreted as the smallest number of iterations required for two N OISY -SGD processes run on adjacent datasets to, with reasonable probability, reach opposite ends of the constraint set.

Original: In other words, ¯ T is effectively thesmallest number of iterations before the ﬁnal iterates could be maximally distinguishable.
Revised: In other words, ¯ T is effectively the smallest number of iterations before the final iterates could be maximally distinguishable.

Original: To prove Theorem 1.3, we show a matching upper bound (in §3) and lower bound (in §4).
Revised: To prove Theorem 1.3, we show a matching upper bound (in §3) and lower bound (in §4).

Original: These twobounds are formally stated as Theorems 3.1 and 4.1, respectively.
Revised: These two bounds are formally stated as Theorems 3.1 and 4.1, respectively.

Original: See §1.3 for an overview of ourtechniques and how they depart from previous approaches.
Revised: See §1.3 for an overview of our techniques and how they depart from previous approaches.

Original: We conclude this section with several remarks about Theorem 1.3.
Revised: We conclude this section with several remarks about Theorem 1.3.

Original: Remark 1.4 (Tight DP characterization) .
Revised: Remark 1.4 (Tight DP characterization) .

Original: While Theorem 1.3 characterizes the privacy loss of
Revised: While Theorem 1.3 characterizes the privacy loss of

Original: Noisy-SGD in terms of RDP, our results can be restated in terms of standard DP bounds.
Revised: N OISY -SGD in terms of RDP, our results can be restated in terms of standard DP bounds.

Original: Speciﬁcally,by a standard conversion from RDP to DP [23, Proposition 1], if δ is not smaller than exponentiallysmall in − bσ / n and if the resulting bound is ε ≲ 1 , it follows that Noisy-SGD is ( ε,δ ) -DP for
Revised: Specifically, by a standard conversion from RDP to DP [25, Proposition 1], if δ is not smaller than exponentially small in − bσ / n and if the resulting bound is ε ≲ 1 , it follows that N OISY -SGD is ( ε,δ ) -DP for

Original: [Equation]
Revised: [Equation]

Original: A matching DP lower bound, for the same wide range of parameters as in Theorem 1.3, is provedalong the way when we establish our RDP lower bound in §4. 
Revised: A matching DP lower bound is proved along the way when we establish our RDP lower bound in §4. Discussion of assumptions in Theorem 1.3.

Original: Discussion of extensions. Our analysis techniques immediately extend to other settings; we highlightseveral below and provide full details in the Supplementary Materials. Strongly convex losses. If the convexity assumption on the losses is replaced by strong convexity,then Noisy-SGD enjoys better privacy.
Revised: Boundedness.

Original: Speciﬁcally, the bound in Theorem 1.3 extends identicallyexcept that the threshold ¯ T for no further privacy loss improves from linear to logarithmic in n ,namely ˜ O ( κ ) where κ denotes the condition number of the losses. This matches the independentresults of [25, 31] (see Footnote 2), and uses completely different techniques from them.
Revised: All previous privacy analyses are oblivious to any sort of diameter bound and therefore unavoidably increase ad infinitum 4 , c.f.

Original: Cyclic batch updates. Noisy-SGD is sometimes run with batches that are chosen cyclically ratherthan randomly. The bound in Theorem 1.3 extends identically except that there is an initial transientphase in which there is unavoidably a large privacy loss since the algorithm may choose to updateusing a sensitive datapoint in an early batch. Non-uniform stepsizes.
Revised: our lower bound in Theorem 1.3.

Original: Noisy-SGD issometimes run with decaying stepsizes η t = t − c for c ∈ ( 0 , 1 ) . The bound in Theorem 1.3 extends identically, with η replaced by η t . This means that the privacyincreases after the threshold, but at the slower rate of T c ≪ T for RDP (resp., T c / 2 ≪ T 1 / 2 for DP).
Revised: Our analysis is the first to exploit boundedness: whereas previous analyses can only argue that having a smaller constraint set does not worsen the privacy loss, we show that this strictly improves the privacy loss, in fact making it convergent as T → ∞ .

Original: Discussion of assumptions in Theorem 1.3.
Revised: See the techniques section §1.3.

Original: Boundedness. While all utility analyses for Noisy-SGD necessarily exploit some form of diameterbound on the iterates, all previous privacy analyses are oblivious to any type of diameterbound. This is why all previous privacy analyses unavoidably increase ad inﬁnitum (c.f., our lower bound in
Revised: We emphasize that this diameter bound is a mild constraint. Indeed, every utility/optimization guarantee for (non-strongly) convex losses inevitably has a similar dependence, simply due to the difference between initialization and optimum.

Original: Theorem 1.3). Our privacy analysis is the ﬁrst to exploit boundedness of  theconstraint set, which iscrucial for obtaining privacy bounds that converge as T → ∞ . (We remark that in  the strongly convexsetting, boundedness is unnecessary; see the Supplementary Materials.)
Revised: We also mention that one can solve an unconstrained problem by solving constrained problems with norm bounds, paying only a small logarithmic overhead on the number of solves and a constant overhead in the privacy loss using known techniques [22]. Moreover, in many optimization problems, the solution set is naturally constrained either from the problem formulation or application.

Original: Smoothness.
Revised: Smoothness.

Original: The smoothness assumption on the losses can be relaxed by running Noisy-SGD ona smoothed version of the objective.
Revised: The smoothness assumption on the losses can be relaxed by running N OISY -SGD on a smoothed version of the objective.

Original: This can be done using standard techniques, e.g., Gaussianconvolution smoothing [12, §5.5], or Moreau-Yousida smoothing by replacing gradient steps withproximal steps [4, §4].
Revised: This can be done using standard techniques, e.g., Gaussian convolution smoothing [12, §5.5], or Moreau-Yousida smoothing by replacing gradient steps with proximal steps [4, §4].

Original: Convexity.
Revised: Convexity.

Original: Convexity appears to be essential for privacy bounds that do not increase ad inﬁnitumin T .
Revised: Convexity appears to be essential for privacy bounds that do not increase ad infinitum in T .

Original: Our analysis uses convexity in an essential way to ensure that Noisy-SGD is a contractiveprocess.
Revised: Our analysis uses convexity in an essential way to ensure that N OISY -SGD is a contractive process.

Original: However, convexity is too restrictive when training deep networks, and it is an interestingopen question if the assumption of convexity be relaxed.
Revised: However, convexity is too restrictive when training deep networks, and it is an interesting open question if the assumption of convexity be relaxed.

Original: Any such result appears to require entirelynew techniques—if even true.
Revised: Any such result appears to require entirely new techniques—if even true.

Original: The key technical challenge is that for any iterative process whoseset of reachable ﬁxed points is non-convex, there will be non-contractivity at the boundary betweenbasins of attraction—and this precludes arguments based on Privacy Ampliﬁcation by Iteration. 
Revised: The key technical challenge is that for any iterative process whose set of reachable fixed points is non-convex, there will be non-contractivity at the boundary between basins of attraction—and this precludes arguments based on Privacy Amplification by Iteration. Lipschitzness. This assumption can be safely removed since smoothness and boundedness imply Lipschitzness for L = MD .

Revised: We write our result in this way in order to clearly isolate where this dependence comes from, and also because the Lipschitz parameter L may be much better than MD .

Original: Mild assumptions on parameters.
Revised: Mild assumptions on parameters.

Original: The lower bound makes the mild assumption that the diameter Dof the decision set is neither asymptotically smaller than the movement size from one gradient step,nor asymptotically smaller than the standard deviation from ¯ T random increments of Gaussian noise
Revised: The lower bound makes the mild assumption that the diameter D of the decision set is neither asymptotically smaller than the movement size from one gradient step, nor asymptotically smaller than the standard deviation from ¯ T random increments of Gaussian noise

Original: N( 0 ,η 2 σ 2 ) so that the noise does not overwhelm the learning.
Revised: N( 0 ,η 2 σ 2 ) so that the noise does not overwhelm the learning.

Original: The upper bound uses the techniqueof Privacy Ampliﬁcation by Sampling, and thus inherits an upper bound assumption on α from theanalysis of the Sampled Gaussian mechanism.
Revised: The upper bound uses the technique of Privacy Amplification by Sampling, and thus inherits an upper bound assumption on α from the analysis of the Sampled Gaussian mechanism, as well as mild bounds on the batch size b that only affect the complete range { 1 ,...,n } up to a constant factor assuming that the noise σ is not so small that it is asymptotically overwhelmed by one gradient step.

Original: This neither affects numerical bounds which can becomputed for any α  (see §2.3), nor does it affect the asymptotic ( ε,δ ) -DP bounds in most parameterregimes of interest. 
Revised: These restrictions neither affect numerical bounds which can be computed for any α and b (see §2.3), nor do they affect the asymptotic ( ε,δ ) -DP bounds in most parameter regimes of interest. Extensions.

Original:  We also remark thatour results require no restriction on α if the batches arechosen cyclically, and thus also if full-batch gradients are used (see the Supplementary Materials).
Revised: Our analysis techniques extend to related settings, as will be investigated in an extended journal version of this paper. We mention here that if the convexity assumption on the losses is replaced by strong convexity, then N OISY -SGD enjoys better privacy. Specifically, Theorem 1. extends identically except that the threshold ¯ T for no further privacy loss improves from linear to logarithmic in n , namely ˜ O

Revised: ( κ ) where κ denotes the condition number of the losses.

Revised: This matches the independent results of [27, 34] (see Footnote 2), and uses completely different techniques from them.

Original: 1.3 Techniques
Revised: 1.3 Techniques

Original: Upper bound on privacy.
Revised: Upper bound on privacy.

Original: Our analysis technique departscompletely from recent approaches basedon fast mixing.
Revised: Our analysis technique departs from recent approaches based on fast mixing.

Original: This allows us to bypass the many long-standing technical challenges for samplingfrom Gibbs distributions in the absence of strong convexity (details in §1.1).
Revised: This allows us to bypass the many longstanding technical challenges discussed in §1.1.

Original: Instead, our analysis combines the techniques of Privacy Ampliﬁcation by Iteration and Privacy
Revised: Instead, our analysis combines the techniques of Privacy Amplification by Iteration and Privacy

Original: Ampliﬁcation by Sampling.
Revised: Amplification by Sampling.

Original: As discussed in §1.1, previous analyses based on these techniques yieldloose privacy bounds that increase ad inﬁnitum in T .
Revised: As discussed in §1.1, previous analyses based on these techniques yield loose privacy bounds that increase ad infinitum in T .

Original: Indeed, bounds based on Privacy Ampliﬁcationby Sampling inevitably diverge since they “pay” for releasing the entire sequence of T iterates, eachof which leaks more information about the private data [3, 2].
Revised: Indeed, bounds based on Privacy Amplification by Sampling inevitably diverge since they “pay” for releasing the entire sequence of T iterates, each of which leaks more information about the private data [2, 3].

Original: Privacy Ampliﬁcation by Iterationavoids releasing the entire sequence by directly arguing about the ﬁnal iterate; however, previousarguments based on this technique are unable to exploit a diameter bound on the constraint set, andtherefore inevitably lead to privacy bounds which grow unboundedly in T [12].
Revised: Privacy Amplification by Iteration avoids releasing the entire sequence by directly arguing about the final iterate; however, previous arguments based on this technique are unable to exploit a diameter bound on the constraint set, and therefore inevitably lead to privacy bounds which grow unboundedly in T [12].

Original: Our analysis is based on the observation that when the iterates are constrained to a bounded domain,we can combine the techniques of Privacy Ampliﬁcation by Iteration and Privacy Ampliﬁcationby Sampling in order to only pay for the privacy loss from the ﬁnal ¯ T =
Revised: Our analysis is based on the observation that when the iterates are constrained to a bounded domain, we can combine the techniques of Privacy Amplification by Iteration and Privacy Amplification by Sampling in order to only pay for the privacy loss from the final ¯ T =

Original: ( Dn )/
Revised: ( Dn )/

Original: ( Lη ) iterations.
Revised: ( Lη ) iterations.

Original: Speciﬁcally, we partition the execution of Noisy-SGD into the initial T − ¯ T iterations and the ﬁnal¯ T iterations.
Revised: Specifically, we partition the execution of N OISY -SGD into the initial T − ¯ T iterations and the final ¯ T iterations.

Original: Recall that to show privacy, we must show that Noisy-SGD produces a similar outputif run on a given dataset or another datast that is changed slightly.
Revised: Recall that to show privacy, we must show that N OISY -SGD produces a similar output if run on a given dataset or another dataset that is changed slightly.

Original: On one hand, if the partiallylearned model ω T − ¯ T were ﬁxed, then the privacy loss from the last ¯ T iterations could be boundedby the standard analysis of Noisy-SGD based on Privacy Ampliﬁcation by Sampling and Advanced
Revised: On one hand, if the partially learned model ω T − ¯ T were fixed, then the privacy loss from the last ¯ T iterations could be bounded by the standard analysis of N OISY -SGD based on Privacy Amplification by Sampling and Advanced

Original: Composition.
Revised: Composition.

Original: On the other hand, irrespective of the input dataset, ω T − ¯ T lies in a set of diameter D .
Revised: On the other hand, irrespective of the input dataset, ω T − ¯ T lies in a set of diameter D .

Original: If the remaining iterations were now run on the same dataset, the Privacy Ampliﬁcation by Iteration
Revised: If the remaining iterations were now run on the same dataset, the Privacy Amplification by Iteration

Original: Argument can be used to show that the algorithm “forgets” this initial displacement by up to D .
Revised: Argument can be used to show that the algorithm “forgets” this initial displacement by up to D .

Original: (Thisis formalized in Proposition 3.2, which is the ﬁrst Privacy Ampliﬁcation by Iteration argument thatis not vacuous as T → ∞ ).
Revised: (This is formalized in Proposition 3.2, which is the first Privacy Amplification by Iteration argument that is not vacuous as T → ∞ ).

Original: In other words, we can control the privacy loss if we change the dataseteither in the ﬁrst T − ¯ T iterations or in the ﬁnal ¯ T iterations.
Revised: In other words, we can control the privacy loss if we change the dataset either in the first T − ¯ T iterations or in the final ¯ T iterations.

Original: At a high-level, our analysis proceeds bycarefully deﬁning sequences of iterates that allow us to run these arguments in parallel, letting uschange the dataset in all iterations.
Revised: At a high-level, our analysis proceeds by carefully running these arguments in parallel.

Original: Lower bound on privacy.
Revised: Lower bound on privacy.

Original: We construct two adjacent datasets on which the Noisy-SGD iteratesare generated by random walks—one symmetric, one biased—constrained within an interval oflength D .
Revised: We construct two adjacent datasets on which the N OISY -SGD iterates are generated by random walks—one symmetric, one biased—constrained within an interval of length D .

Original: In this way, we reduce the question of how large is the privacy loss of Noisy-SGD , to thequestion of how distinguishable is a constrained symmetric random walk from a constrained biasedrandom walk.
Revised: In this way, we reduce the question of how large is the privacy loss of N OISY -SGD, to the question of how distinguishable is a constrained symmetric random walk from a constrained biased random walk.

Original: The core technical challenge is that the distributions of the iterates of these randomwalks are intractable to reason about explicitly—due to the highly non-linear interactions betweenthe projections and random increments.
Revised: The core technical challenge is that the distributions of the iterates of these random walks are intractable to reason about explicitly—due to the highly non-linear interactions between the projections and random increments.

Original: Brieﬂy, our key technique here is a way to modify theseprocesses so that on one hand, their distinguishability is essentially the same, and the other hand, noprojections occur with high probability—allowing us to explicitly compute their distributions andthus also their distinguishability.
Revised: Briefly, our key technique here is a way to modify these processes so that on one hand, their distinguishability is essentially the same, and the other hand, no projections occur with high probability—allowing us to explicitly compute their distributions and thus also their distinguishability.

Original: Details in §4.
Revised: Details in §4.

Original: 1.4 Other related work
Revised: 1.4 Other related work

Original: Private sampling.
Revised: Private sampling.

Original: The mixing time of (stochastic) Langevin dynamics has been extensively studiedin recent years starting with [10, 11], and other more recent works have analyzed mixing in othernotions of distance.
Revised: The mixing time of (stochastic) Langevin Dynamics has been extensively studied in recent years starting with [10, 11], and other more recent works have analyzed mixing in other notions of distance.

Original: From the point of view of privacy, more stringent notions of mixing areneeded, and mixing in Rènyi divergence was studied by Vempala and Wibisono [29] and Ganesh and Talwar [14].
Revised: From the point of view of privacy, more stringent notions of mixing are needed, and mixing in Rènyi divergence was studied by Vempala and Wibisono [32] and Ganesh and Talwar [15].

Original: In addition to the aforementioned works, several recent works have studied algorithmsfor sampling from the distribution exp ( ε ∑ i f( ω,x i  )) or its regularized versions from a privacyviewpoint.
Revised: In addition to the aforementioned works, several recent works have studied algorithms for sampling from the distribution exp ( ε ∑ i f i ( ω )) or its regularized versions from a privacy viewpoint.

Original: [22] proposed a mechanism of this kind that works when f may be unbounded andshowed that it satisﬁes ( ε,δ ) -DP.
Revised: [24] proposed a mechanism of this kind that works when f may be unbounded and showed that it satisfies ( ε,δ ) -DP.

Original: Recently, [16] gave better privacy bounds for such a regularizedexponential mechanism, and designed an efﬁcient sampler based only on function evaluation.
Revised: Recently, [17] gave better privacy bounds for such a regularized exponential mechanism, and designed an efficient sampler based only on function evaluation.

Original: [15]showed a number of results about the continuous Langevin diffusion, showing that it gives optimalutility bounds for different variants of private optimization problems with different parameters.
Revised: [16] showed a number of results about the continuous Langevin Diffusion, showing that it gives optimal utility bounds for different variants of private optimization problems with different parameters.

Revised: As alluded to in §1.1, there are several core issues with trying to prove DP bounds for N OISY -SGD by directly combining “fast mixing” bounds with “private once mixed” bounds.

Revised: First, mixing results typically do not apply, e.g., since DP requires mixing in stringent divergences like Rényi, or because realistic settings with constraints, stochasticity, and lack of strong convexity are difficult to analyze— indeed, understanding the mixing time for such settings is a challenging open problem.

Revised: Second, even when fast mixing bounds do apply, directly combining them with “private once mixed” bounds unavoidably leads to DP bounds that are loose to the point of being essentially useless (e.g., the inevitable dimension dependence in mixing bounds to the stationary distribution of the continuoustime Langevin Diffusion would lead to dimension dependence in DP bounds, which should not occur—as we show).

Revised: Third, even if a Markov chain were private after mixing, one cannot conclude from this that it is private beforehand—indeed, there are simple Markov chains which are private after mixing, yet are exponentially non-private beforehand [14].

Original: Utility bounds.
Revised: Utility bounds.

Original: In the ﬁeld of private optimization, one separately analyzes two properties ofalgorithms: their privacy  and their utility (a.k.a., optimization error).
Revised: In the field of private optimization, one separately analyzes two properties of algorithms: (i) the privacy loss as a function of the number of iterations, and (ii) the utility (a.k.a., optimization error) as a function of the number of iterations.

Original: These two properties can thenbe combined to obtain privacy-utility tradeoffs.
Revised: These two properties can then be combined to obtain privacy-utility tradeoffs.

Revised: The purpose of this paper is to completely resolve (i); this result can then be combined with any bound on (ii).

Original: Utility bounds for SGD are very well-understood [26]and these analyses have allowed understandingthe utility of Noisy-SGD for  empirical [3] and population [4] losses.
Revised: Utility bounds for SGD are well understood [28], and these analyses have enabled understanding utility bounds for N OISY -SGD in empirical [3] and population [4] settings.

Original: While in theory a single pass of SGD achieves the optimal population risk, inpractice one  typically beneﬁts from multi-pass algorithms.
Revised: However, there is a big difference between the minimax-optimal utility bounds in theory versus what is desired in practice. Indeed, while in theory a single pass of N OISY -SGD achieves the minimax-optimal population risk [13], in practice N OISY -SGD benefits from running longer to get more accurate training. In fact, this divergence is even true and well-documented for non-private SGD as well, where one epoch is minimax-optimal in theory, but in practice more epochs help. Said simply, this is because typical problems are not worst-case problems (i.e., minimax-optimal theoretical bounds are typically not representative of practice).

Original: The goal of the present paper is to developtools to analyze the privacy loss of multi-epoch SGD.
Revised: For these practical settings, in order to run N OISY -SGD longer, it is essential to have privacy bounds which do not increase ad infinitum. Our paper resolves this.

Original:  Preliminaries
Revised:  Preliminaries

Original: In this section, we recall relevant preliminaries about convex optimization (§2.1), differential privacy(§2.2), and two by-now-standard techniques for analyzing the privacy of optimization algorithms—namely, Privacy Ampliﬁcation by Sampling (§2.3) and Privacy Ampliﬁcation by Iteration (§2.4).
Revised: In this section, we recall relevant preliminaries about convex optimization (§2.1), differential privacy (§2.2), and two by-now-standard techniques for analyzing the privacy of optimization algorithms— namely, Privacy Amplification by Sampling (§2.3) and Privacy Amplification by Iteration (§2.4).

Original: Notation.
Revised: Notation.

Original: We write P X to denote the law of a random variable X , and P X ∣ Y =
Revised: We write P X to denote the law of a random variable X , and P X ∣ Y =

Original: y to denote the law of
Revised: y to denote the law of

Original: X given the event Y =
Revised: X given the event Y =

Original: y .
Revised: y .

Original: We write Z S ∶ T as shorthand for the vector concatenating Z S ,...,Z T .
Revised: We write Z S ∶ T as shorthand for the vector concatenating Z S ,...,Z T .

Original: Wewrite f # µ to denote the push-forward of a distribution µ under a (possibly random) function f , i.e.,the law of f ( X ) where X ∼ µ .
Revised: We write f # µ to denote the pushforward of a distribution µ under a (possibly random) function f , i.e., the law of f ( X ) where X ∼ µ .

Original: We write λµ + ( 1 − λ )
Revised: We write λµ + ( 1 − λ )

Original: ν to denote the mixture distribution that is µwith probability λ
Revised: ν to denote the mixture distribution that is µ with probability λ

Original: and ν with probability 1 − λ .
Revised: and ν with probability 1 − λ .

Original: We write A(X) to denote the output of an algorithm
Revised: We write A(X) to denote the output of an algorithm

Original: A run on input X ; this is a probability distribution if A is a randomized algorithm.
Revised: A run on input X ; this is a probability distribution if A is a randomized algorithm.

Original: 2.1 Convex optimization
Revised: 2.1 Convex optimization

Original: Throughout, we minimize loss functions f ∶ K × X → R , which are convex in the ﬁrst parameter(a.k.a.
Revised: Throughout, the loss function corresponding to a data point x i or x ′ i is denoted by f i or f ′ i , respectively.

Original: , the machine learning model to train) and arbitrary in the second parameter (a.k.a., the datapoint).
Revised: While the dependence on the data point is arbitrary, the loss functions are assumed to be convex in the argument ω ∈ K (a.k.a., the machine learning model we seek to train).

Original: Throughout, the set K of possible models is a convex subset of R d .
Revised: Throughout, the set K of possible models is a convex subset of R d .

Original: In order to establish bothoptimization and privacy guarantees, two additional assumptions are required on the loss functions.
Revised: In order to establish both optimization and privacy guarantees, two additional assumptions are required on the loss functions.

Original: Recall that a differentiable function g ∶ K → R d is: L -Lipschitz if ∥ g ( ω ) − g ( ω ′ )
Revised: Recall that a differentiable function g ∶ K → R d is: L -Lipschitz if ∥ g ( ω ) − g ( ω ′ )

Original: ∥ ⩽ L∥ ω − ω ′
Revised: ∥ ⩽ L

Original: ∥ forall ω,ω ′ ∈ K ; and M -smooth if ∥∇ g ( ω ) − ∇ g ( ω ′ )
Revised: ∥ ω − ω ′ ∥ for all ω,ω ′ ∈ K ; and M -smooth if ∥∇ g ( ω ) − ∇ g ( ω ′ )

Original: ∥
Revised: ∥

Original: ⩽ M ∥ ω − ω ′ ∥ for all ω,ω ′ ∈ K .
Revised: ⩽ M ∥ ω − ω ′ ∥ for all ω,ω ′ ∈ K .

Original: 2.2 Differential privacy and Rényi differential privacy
Revised: 2.2 Differential privacy and Rényi differential privacy

Original: In order to prove DP guarantees, we work with the related notion of Rényi Differential Privacy (RDP)introduced by [23], since RDP is more amenable to our analysis techniques and is readily convertedto DP guarantees [23, Proposition 1].
Revised: In order to prove DP guarantees, we work with the related notion of Rényi Differential Privacy (RDP) introduced by [25], since RDP is more amenable to our analysis techniques and is readily converted to DP guarantees [25, Proposition 1].

Original: RDP measures how distinguishable the output of an algorithmis when run on two “adjacent” datasets—i.e., two datasets which differ on at most one datapoint—in
Revised: RDP measures how distinguishable the output of an algorithm is when run on two “adjacent” datasets—i.e., two datasets which differ on at most one datapoint—in

Original: Rényi divergence.
Revised: Rényi divergence.

Original: We therefore ﬁrst recall the deﬁnition of Rényi divergence.
Revised: We therefore first recall the definition of Rényi divergence.

Original: Deﬁnition 2.1 (Rényi divergence) .
Revised: Definition 2.1 (Rényi divergence) .

Original: The Rényi divergence between probability measures µ and ν oforder α ∈ ( 1 , ∞) is D α ( µ ∥ ν )
Revised: The Rényi divergence between probability measures µ and ν of order α ∈ ( 1 , ∞) is D α ( µ ∥ ν )

Original: = 1 α − 1 log ∫ ( µ ( x )/ ν ( x ))
Revised: = 1 α − 1 log ∫ ( µ ( x )/ ν ( x ))

Original: α ν ( x ) dx, if µ ≪ ν , and ∞ otherwise.
Revised: α ν ( x ) dx, if µ ≪ ν , and ∞ otherwise.

Original: Here0 / 

Revised: Here 0 /

Original: = 0 and x / 0 = ∞ for x > 0 .
Revised: = 0 and x / 0 = ∞ for x > 0 .

Original: The Rényi divergences of order α ∈ { 1 , ∞} are deﬁned by continuity.
Revised: The Rényi divergences of order α ∈ { 1 , ∞} are defined by continuity.

Original: Deﬁnition 2.2 (Rényi Differential Privacy) .
Revised: Definition 2.2 (Rényi Differential Privacy) .

Original: A randomized algorithm A satisﬁes ( α,ε ) -RDP if
Revised: A randomized algorithm A satisfies ( α,ε ) -RDP if

Original: D α (A(X)
Revised: D α (A(X)

Original: ∥ A(X ′))
Revised: ∥ A(X ′

Original:  ⩽ ε for any two adjacent datasets X , X ′ .
Revised: )) ⩽ ε for any two adjacent datasets X , X ′ .

Original: Following, we recall two basic properties of RDP that we userepeatedly in our analysis.
Revised: Following, we recall two basic properties of RDP that we use in our analysis.

Original: Proofs canbe found, e.g., in [28, Theorem 9] and [23, Proposition 3], respectively.
Revised: Proofs can be found, e.g., in [31, Theorem 9] and [25, Proposition 3], respectively.

Original: Lemma 2.3 (Post-processing property of Rényi divergence) .
Revised: Lemma 2.3 (Post-processing property of Rényi divergence) .

Original: For any Rényi parameter α ⩾ 1 , any(possibly random) function h , and any probability distributions µ,ν ,
Revised: For any Rényi parameter α ⩾ 1 , any (possibly random) function h , and any probability distributions µ,ν ,

Original: [Equation]
Revised: [Equation]

Original: Lemma 2.4 (Strong composition for RDP) .
Revised: Lemma 2.4 (Strong composition for RDP) .

Original: For any Rényi parameter α ⩾ 1 , and any two sequencesof random variables X 1 ,...,X k and Y 1 ,...,Y k ,
Revised: For any Rényi parameter α ⩾ 1 , and any two sequences of random variables X 1 ,...,X k and Y 1 ,...,Y k ,

Original: [Equation]
Revised: [Equation]

Original: 2.3 Privacy Ampliﬁcation by Sampling
Revised: 2.3 Privacy Amplification by Sampling

Original: A core technique in the DP literature is Privacy Ampliﬁcation by Sampling [19], which quantiﬁes theidea that a private algorithm run on a small random sample of the input become more private.
Revised: A core technique in the DP literature is Privacy Amplification by Sampling [20], which quantifies the idea that a private algorithm run on a small random sample of the input become more private.

Original: Werecall a convenient statement of this from [24, Theorem 11] in terms of the RDP of the “Sampled
Revised: We recall a convenient statement of this from [26, Theorem 11] in terms of the RDP of the “Sampled

Original: Gaussian Mechanism” (which is a composition of subsampling and additive Gaussian noise).
Revised: Gaussian Mechanism” (which is a composition of subsampling and additive Gaussian noise).

Original: Deﬁnition 2.5 (Rényi Divergence of the Sampled Gaussian Mechanism) .
Revised: Definition 2.5 (Rényi Divergence of the Sampled Gaussian Mechanism) .

Original: For Rényi parameter α ⩾ 1 ,mixing probability q ∈ ( 0 , 1 ) , and noise parameter σ > 0 , deﬁne
Revised: For Rényi parameter α ⩾ 1 , mixing probability q ∈ ( 0 , 1 ) , and noise parameter σ > 0 , define

Original: [Equation]
Revised: [Equation]

Original: Lemma 2.6 (Bound on Rényi Divergence of the Sampled Gaussian Mechanism) .
Revised: Lemma 2.6 (Bound on Rényi Divergence of the Sampled Gaussian Mechanism) .

Original: Consider Rényiparameter α > 1 , mixing probability q ∈ ( 0 , 1 / 5 ) , and noise level σ ⩾ 4 .
Revised: Consider Rényi parameter α > 1 , mixing probability q ∈ ( 0 , 1 / 5 ) , and noise level σ ⩾ 4 .

Original: If α ⩽ α ∗ ( q,σ ) , then
Revised: If α ⩽ α ∗ ( q,σ ) , then

Original: S α ( q,σ ) ⩽ 2 αq 2 / σ 2 .
Revised: S α ( q, [Equation] ⩽ [Equation]

Original:  This bound restricts to Rényi parameter at most α ∗ ( q,σ ) , which is deﬁned to be the largest αsatisfying α ⩽
Revised: [Equation] This bound restricts to Rényi parameter at most α ∗ ( q,σ ) , which is defined to be the largest α satisfying α ⩽

Original: Mσ 2 / 2 − log ( σ 2 ) and α ⩽ ( M 2 σ 2 / 2 − log ( 5 σ 2 ))/
Revised: Mσ 2 / 2 − log ( σ 2 ) and α ⩽ ( M 2 σ 2 / 2 − log ( 5 σ 2 ))/

Original: ( M + log ( qα ) + 1 /( 2 σ 2 )) , where
Revised: ( M + log ( qα ) + 1 /( 2 σ 2 )) , where

Original: M ∶= log ( 1 + 1 /( q ( α − 1 )) .
Revised: M ∶= log ( 1 + 1 /( q ( α − 1 )) .

Original: While we use Lemma 2.6 to prove the asymptotic bounds in Theorem 1.3,we emphasize that i) our bounds can be computed numerically for any α ⩾ 1 (see the Supplementary Materials), and ii) this upper bound does not preclude α from the typical parameter regime of interest,see the discussion in §1.2.
Revised: While we use Lemma 2.6 to prove the asymptotic bounds in Theorem 1.3, we emphasize that i) our bounds can be computed numerically for any α ⩾ 1 , and ii) this upper bound does not preclude α from the typical parameter regime of interest, see the discussion in §1.2.

Original: 2.4 Privacy Ampliﬁcation by Iteration
Revised: 2.4 Privacy Amplification by Iteration

Original: The Privacy Ampliﬁcation by Iteration technique of [12] bounds the privacy loss of an iterativealgorithm without “releasing” the entire sequence of iterates—unlike arguments based on Privacy
Revised: The Privacy Amplification by Iteration technique of [12] bounds the privacy loss of an iterative algorithm without “releasing” the entire sequence of iterates—unlike arguments based on Privacy

Original: Ampliﬁcation by Sampling, c.f., §1.3.
Revised: Amplification by Sampling, c.f., §1.3.

Original: This technique applies to processes generated by a Contractive
Revised: This technique applies to processes generated by a Contractive

Original: Noisy Iteration (CNI).
Revised: Noisy Iteration (CNI).

Original: We begin by recalling this deﬁnition, albeit in a slightly more general form thatallows for random contractions, which is critical for analyzing the random updates of Noisy-SGD .
Revised: We begin by recalling this definition, albeit in a slightly more general form that allows for two differences. The first difference is allowing the contractions to be random; albeit simple, this generalization is critical for analyzing N OISY -SGD because a stochastic gradient update is random.

Revised: The second difference is that we project each iterate; this generalization is solely for convenience as it simplifies the exposition.

Original: Deﬁnition 2.7 (Contractive Noisy Iteration) .
Revised: Definition 2.7 (Contractive Noisy Iteration) .

Original: Consider a (random) initial state X 0 ∈ R d , a sequence of(random) contractions φ t ∶ R d
Revised: Consider a (random) initial state X 0 ∈ R d , a sequence of (random) contractions ϕ t ∶ R d

Original: → R d ,and a sequence of noise distributions ξ t  .
Revised: → R d , a sequence of noise distributions ξ t , and a convex set K .

Original: TheContractive Noisy
Revised: The

Original:  Iteration CNI ( X 0 , { φ t } , { ξ t }) is the ﬁnal iterate X T produced by the processX t + 1 ∶= φ t
Revised: Projected Contractive Noisy Iteration CNI ( X 0 , { ϕ t } , { ξ t } , K) is the final iterate X T of the process

Original: + 1 ( X t )
Revised: [Equation]

Original: + 261 Z t + 1 , where Z t + 1 is drawn independently from ξ t + 1 .
Revised: where Z t + 1 is drawn independently from ξ t + 1 .

Original: Proposition 2.8 (Original 4 PABI bound) .
Revised: Proposition 2.8 (Original 6 PABI bound) .

Original: Let X T , X ′ T denote the outputs of CNI ( X 0 , { φ t } , { ξ t }) ,  CNI ( X 0 , { φ ′ t } , { ξ t }) where ξ t = N( 0 ,σ 2 t I d ) .
Revised: Let X T and X ′ T denote the outputs of CNI ( X 0 , { ϕ t } , { ξ t } , K) and CNI ( X 0 , { ϕ ′ t } , { ξ t } , K) where ξ t = N( 0 ,σ 2 t I d ) .

Original: Denote s t ∶=sup x ∥ φ t( x ) − φ ′ t ( x )∥ , and consider anya 1 ,...,a T such that z t ∶= ∑ ti = 

Revised: Let s t ∶= ∥ ϕ t − ϕ ′ t ∥ ∞ , and consider any sequence a 1 ,...,a T such that z t ∶= ∑ ti =

Original: ( s i − a i ) is non-negative for all t and satisﬁes z T = 0 .
Revised: ( s i − a i ) is non-negative for all t and satisfies z T = 0 .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original:  Upper bound on privacy
Revised:  Upper bound on privacy

Original: In this section, we prove the upper bound in Theorem 1.3.
Revised: In this section, we prove the upper bound in Theorem 1.3.

Original: The formal statement of this result is asfollows; see §1.2and §2.3 for a discussion of the bound on α .
Revised: The formal statement of this result is as follows; see §1.2 for a discussion of the mild assumptions on the parameters.

Original: Theorem 3.1 (Privacy upper bound for Noisy-SGD ) .
Revised: Theorem 3.1 (Privacy upper bound for N OISY -SGD) .

Original: Let
Revised: Let

Original: K ⊆ R
Revised: K ⊆ R

Original: d be a convex set of diameter D ,and let { f (⋅ ,x )} x ∈X be a family of convex functions over K that are L -Lipschitz and M -smooth.
Revised: d be a convex set of diameter D , and consider optimizing convex losses over K that are L -Lipschitz and M -smooth.

Original: Forany number of iterations T , dataset size n ∈ N , batch size b ⩽ n , stepsize η ⩽ 2 / M , noise parameterσ > 0 , and initialization ω 0 ∈ K , Noisy-SGD satisﬁes ( α,ε ) -RDP if 1 < α ⩽ α ∗ ( bn , bσ 2 √ 2 L ) andε ≲ αLn 2 σ 2 min { T,
Revised: Consider any number of iterations T , dataset size n ∈ N , batch size b ⩽ n , stepsize η ⩽ 2 / M , noise parameter σ > 0 , and initialization ω 0 ∈ K such that b < n / 5 and bσ > 8 √ 2 L . Then N OISY -SGD satisfies ( α,ε ) -RDP if 1 < α ⩽ α ∗ ( bn , bσ 2 √ 2 L ) and

Original: [Equation]
Revised: [Equation]

Original: 3.1 Privacy Ampliﬁcation by Iteration bounds that are not vacuous as T → ∞
Revised: 3.1 Privacy Amplification by Iteration bounds that are not vacuous as T → ∞

Original: Recall from the preliminaries section §2.4 that PABI arguments, while tight for a small number ofiterations T , provide vacuous bounds as T → ∞ (c.f., Proposition 2.8).
Revised: Recall from the preliminaries section §2.4 that PABI arguments, while tight for a small number of iterations T , provide vacuous bounds as T → ∞ (c.f., Proposition 2.8).

Original: The following propositionovercomes this by establishing privacy bounds which are independent of the number of iterations T .
Revised: The following proposition overcomes this by establishing privacy bounds which are independent of the number of iterations T .

Original: This result only  the mild assumption that the CNI iterates are bounded.
Revised: This result only requires additionally making the mild assumption that the CNI iterates are bounded.

Original: Proposition 3.2 (New PABI bound that is not vacuous as T → ∞ ) .
Revised: Proposition 3.2 (New PABI bound that is not vacuous as T → ∞ ) .

Original: Let X T , X ′ T , and s t be as in Proposition 2.8.
Revised: Let X T , X ′ T , and s t be as in Proposition 2.8.

Original: Suppose that all contractions φ t ,φ ′ t have ranges lying in some set K of diameter D .

Original: Consider any τ ∈ { 0 ,...,T − 1 } and  a τ + 1 ,...,a T such that z t ∶= D
Revised: Consider any τ ∈ { 0 ,...,T − 1 } and sequence a τ + 1 ,...,a T such that z t ∶= D

Original: +∑ ti = τ + 1 ( s i − a i ) is non-negative280 for all t and satisﬁes
Revised: + ∑ ti = τ + 1 ( s i − a i ) is non-negative for all t and satisfies z T = 0 .

Original: z T = 0 . Then:
Revised: If K has diameter D , then:

Original: [Equation]
Revised: [Equation]

Original: Proof idea.
Revised: Proof idea.

Original: We ﬁrst recall the basic idea of the proof of the original PABI bound (Proposition 2.8).
Revised: We first recall the basic idea of the proof of the original PABI bound (Proposition 2.8).

Original: Brieﬂy, that proof uses as a potential a certain shifted Rényi divergence , which allows for uncertaintyin one of the two distributions as measured in optimal transport distance (making it geometricallyaware).
Revised: Briefly, that proof uses as a potential a certain shifted Rényi divergence , which allows for uncertainty in one of the two distributions as measured in optimal transport distance (making it geometrically aware).

Original: The original argument bounds the shifted divergence at iteration T (which is precisely the
Revised: The original argument bounds the shifted divergence at iteration T (which is precisely the

Original: RDP in question), by the shifted divergence at iteration T − 1 , and so on to the shifted divergenceat iteration 0 (which vanishes since Noisy-SGD always has the same initialization).
Revised: RDP in question), by the shifted divergence at iteration T − 1 , and so on to the shifted divergence at iteration 0 (which vanishes since N OISY -SGD always has the same initialization).

Original: In this way,by keeping track of the privacy loss at each iteration, PABI bounds the RDP.
Revised: In this way, by keeping track of the privacy loss at each iteration, PABI bounds the RDP.

Original: The main idea behind
Revised: The main idea behind

Original: Proposition 3.2 is to change this argument by simply stopping the induction earlier.
Revised: Proposition 3.2 is to change this argument by simply stopping the induction earlier.

Original: Speciﬁcally,unroll to iteration τ , and then use boundedness of the iterates to control the shifted divergence at thatintermediate time τ .
Revised: Specifically, unroll to iteration τ , and then use boundedness of the iterates to control the shifted divergence at that intermediate time τ .

Original: For brevity, the proof is deferred to the Supplementary Materials.
Revised: For brevity, the proof is deferred to the Supplementary Materials.

Revised: We remark that this new version of PABI uses the shift in the shifted Rényi divergence for a different purpose than previous work: rather than just using the shift to bound the bias incurred from updating on two different losses, here we also use the shift to exploit the boundedness of the constraint set.

Original: 3.2 Proof sketch of Theorem 
Revised: 3.2 Proof sketch of Theorem 3.1 (full details in the Supplementary Materials)

Original: full details in the Supplementary Materials)

Original: Step 1: Coupling the iterates.
Revised: Step 1: Coupling the iterates.

Original: Suppose X = { x 1 ,...,x n } and X ′ =
Revised: Suppose X = { x 1 ,...,x n } and X ′ =

Original: { x ′ 1 ,...,x ′ n } are adjacentdatasets; that is, they agree x i = x
Revised: { x ′ 1 ,...,x ′ n } are adjacent datasets; that is, they agree x i = x

Original: ′ i on all indices i ∈ [ n ] ∖ { i ∗ } except for at most one indexi ∗ ∈ [ n ] .
Revised: ′ i on all indices i ∈ [ n ] ∖ { i ∗ } except for at most one index i ∗ ∈ [ n ] .

Revised: Denote the corresponding loss functions by f i and f ′ i , where f i = f

Original:  Consider running Noisy-SGD on either X or X ′ for T iterations—call the resulting iterates{ W t } Tt = 0 and { W ′ t } Tt = 0 , respectively—where we start from the same point w 0 ∈ K and couple thesequence of random batches { B t } T − 1 t = [-
-] and the random noise in each iteration.
Revised: ′ i except possibly f i ∗ ≠ f ′ i ∗ . Consider running N OISY -SGD on either X or X ′ for T iterations—call the resulting iterates { W t } Tt = 0 and { W ′ t } Tt = 0 , respectively—where we start from the same point w 0 ∈ K and couple the sequence of random batches { B t } T − 1 t = [-
-]0 and the random noise in each iteration.

Original: That is,W t + 1 =
Revised: That is,

Original: Π K [ W t − ηb ∑ i ∈ B t ∇ ω f ( W t ; x i ) + Y t + Z t ] and W ′ t + 1 = Π K [ W ′ t − ηb ∑ i ∈ B t ∇ ω f ( W ′ t ; x i )
Revised: [Equation]

Original: + Y t + Z ′ t ]for all t ⩽ T − 1 , where we have split the noise into terms Y t ∼ N( 0 ,η 2 σ 21 ) , Z t ∼ N( 0 ,η 2 σ 22 ) ,
Revised: for all t ⩽ T − 1 , where we have split the noise into terms Y t ∼ N( 0 ,η 2 σ 21 ) , Z t ∼ N( 0 ,η 2 σ 22 ) ,

Original: Z ′ t ∼ 
Revised: Z ′ t ∼ N( 0 ,η 2 σ 22 ) + ηb [∇ f i ∗ ( W ′ t ) − ∇ f

Original: N( 0 ,η 2 σ 22 )+ ηb [∇ ω f ( W ′ t ; x i ∗) − ∇ ω f ( W ′ t ; x ′ i ∗ )]⋅ 1 i ∗ ∈ S t , for any σ 1 ,σ 2 > 0 satisfying σ 21 + σ 22 = σ 2(we set σ 1 = σ 2 = σ /√ 2 later for asymptotics). 
Revised: i ∗ ( W ′ t )] ⋅ 1 i ∗ ∈ B t , for any σ 1 ,σ 2 > 0 satisfying σ 21 + σ 22 = σ(we set σ 1 = σ 2 = σ /√ 2 later for asymptotics). In words, this noise-splitting lets us use the noise for both the Privacy Amplification by Sampling and Privacy Amplification by Iteration arguments below.

Original: Importantly, notice that in the deﬁnition of W ′ t + 1 , the gradient is taken w.r.t.
Revised: Importantly, notice that in the definition of W ′ t + 1 , the gradient is taken w.r.t.

Original: points in X rather than X ′ ; this is corrected via the bias in the noise term Z ′ t .
Revised: loss functions corresponding to data set X rather than X ′ ; this is corrected via the bias in the noise term Z ′ t .

Original: Notice also that this bias term is only realizedwith probability 1 − b / n because the probability that i ∗ is in a random size- b subset of [ n ] is b / n .
Revised: Notice also that this bias term is only realized (i.e., Z ′ t is possibly non-centered) with probability 1 − b / n because the probability that i ∗ is in a random size- b subset of [ n ] is b / n .

Original: Step 2: Construction of auxiliary CNI sequences.
Revised: Step 2: Interpretation as conditional CNI sequences.

Original: The processes { W t } Tt = 0 and { W ′ t } Tt = 0 are not Contractive Noisy Iterations (c.f., Deﬁnition 2.7) and thus we cannot directly apply the new PABIargument (Proposition 3.2). In this step, we modify these processes to construct CNI thatareamenable to this analysis. For a parameter τ ∈ { 0 ,...,T − 1 } to be chosen shortly, deﬁne { V t } Tt = 0 and{ V ′ t } Tt = 0 by V t ∶= W t − ∑ t − 1 s =
Revised: Observe that conditional on the event that

Original: τ Zs + Y tand V ′ t ∶= W ′ t − ∑ t − 1 s =τ Z ′ s + Y t . Deﬁne also the random functionsφ t ,φ ′ t ∶ R d → R d by φ t ( v ) ∶=
Revised: Z t =

Original: Π K [ v − ηb ∑ i ∈ B t ∇ ω f ( v +∑ t − 1 s = τ Zs − Y t ; x i ) +( Z t +∑ t − 1 s = τ Z s )]−∑ ts = τ Z sand φ ′ t ( v ′ )
Revised: Z ′ t are equal (call their value z t ), then

Original: ∶= Π K [ v ′ − ηb
Revised: [Equation]

Original: ∑ i ∈ B t ∇ ω f ( v ′ + ∑ t − 1 s = τ Z ′ s − Y t ; x i ) + ( Z ′ t
Revised: where

Original: + ∑ t − 1 s = τ Z ′ s )] − ∑ ts = τ Z ′ s for allt ⩽ T − 1 . Note that Z t + ∑ t − 1 s =
Revised: [Equation]

Original: τ Z s is different from ∑ ts = τ Z s since it contains Z t even if t < τ ; thisnuance ensures that Observation 3.3 holds for all t ⩾ 0 , not just t ⩾ τ . We now make two observations that, together, establish that { V t [-} Tt = 0 and { V ′ t } Tt = 
 are CNI.-]
Revised: This gives us a CNI since ϕ t [-} Tt = 0 and { V ′ t } Tt = 
 are CNI.-]is contractive.

Original: The formeris a straightforward calculation; the latter uses the fact that a stochastic gradient update on a smoothconvex function is contractive. For brevity, proofs are deferred to the Supplementary Materials. Observation 3.3.
Revised: (This contractivity follows from the fact that a stochastic gradient update on a smooth convex function is contractive; details in the Supplementary Materials.)

Original: For all t , it holds that V t + 1 =

Original: φ t ( V t )

Original: + Y t + 1 and V ′ t + 1 = φ ′ t

Original: ( V ′ t )

Original: + Y t + 1 .

Original: Observation 3.4.

Original: For all t , the maps φ t ,φ ′ t are contractive.

Original: Step 3:
Revised: Step 3:

Original: Bounding the privacy loss of the auxiliary CNI sequences.
Revised: Bounding the privacy loss.

Original: Recall that we seek to upperbound D α( P W T ∥
Revised: Recall that we seek to upper bound D α

Original:  P W ′ T ) .
Revised: ( P W T ∥ P W ′ T ) .

Original: We argue that:
Revised: Let τ ∈ { 0 ,...,T − 1 } be a threshold parameter to be chosen shortly. Then

Original: [Equation]
Revised: [Equation]

Original: Above, the ﬁrst step is by the post-processing inequality for the Rényi divergence (Lemma 2.3).
Revised: +

Original: Thesecond step is by the strong composition rule for the Rényi divergence (Lemma 2.4). The third step isalso by the post-processing inequality (Lemma 2.3) because, conditional on Z τ ∶ T − 1 =
Revised: [Equation]

Original: Z ′ τ
Revised: ∶

Original: ∶ T − 1 =
Revised: [Equation]

Original: z , itholds that W T
Revised: ∶

Original: = h ( V T − 1 ) and W ′ T = h ( V ′ T − 1 ) for the same random function
Revised: [Equation] Above, the first step is by the post-processing inequality for the Rényi divergence (Lemma 2.3), and the second step is by the strong composition rule for the Rényi divergence (Lemma 2.4).

Original: h .325 Step 3a:
Revised: Step 3a:

Original: Bounding the ﬁrst term in (3.1), using Privacy Ampliﬁcation by Sampling.
Revised: Bounding the first term in (3.2), using Privacy Amplification by Sampling.

Original: In words,this is the privacy loss from releasing T − τ noisy gradients.
Revised: In words, this is the privacy loss from releasing T − τ noisy gradients.

Original: This can be bounded by Privacy
Revised: This can be bounded by Privacy

Original: Ampliﬁcation by Sampling (Lemma 2.6) and Advanced Composition (Lemma 2.4) via a standardargument [2].
Revised: Amplification by Sampling (Lemma 2.6) and Advanced Composition (Lemma 2.4) via a standard argument [2].

Original: This yields the bound ( T − τ ) S α ( bn , bσ 2 2 L ) ; full details in the Supplementary Materials.330 Step 3b: Bounding the second term in (3.1), using Privacy Ampliﬁcation by Iteration.
Revised: This yields the bound ( T − τ ) S α ( bn , bσ 2 2 L ) ; full details in the Supplementary Materials. Step 3b: Bounding the second term in (3.2), using Privacy Amplification by Iteration.

Original: Observethat conditional on  Z τ ∶ T − 1 =
Revised: From step 2, we know that conditional on the event that Z t =

Original: Z ′τ ∶ T − 1 , the maps φ t ≡ φ ′ tare identical for all t ⩾ τ . Thus, by Observations 3.3 and 3.4, the iterates V T − 1 ,V ′ T − 1 are produced by CNI  with  the same contractionmappings φ t ≡ φ ′ t for all t ⩾ τ .
Revised: Z ′ t for all t ⩾ τ , then { W t } t ⩾ τ and { W ′ t } t ⩾ τ are projected CNI (c.f., Definition 2.7) with respect to the same update functions.

Original: Thus we may apply the new Privacy Ampliﬁcation by Iteration bound(Proposition 3.2) with s t ≡ 0 and a t ≡ D /( T − τ− 1 ) to obtain the bound αD 2 /( 2 η 2 σ 21 ( T − τ− 1 )) .
Revised: Thus we may apply the new Privacy Amplification by Iteration bound (Proposition 3.2) with s t ≡ 0 and a t ≡ D /( T − τ ) to obtain the bound αD 2 /( 2 η 2 σ 21 ( T − τ )) .

Original: Step 4: Putting the bounds together.
Revised: Step 4: Putting the bounds together.

Original: By plugging into (3.1) the bounds in Steps 3a-3b, we concludethat Noisy-SGD is ( α,ε ) -RDP for
Revised: By plugging into (3.2) the bounds in Steps 3a-3b, we conclude that N OISY -SGD is ( α,ε ) -RDP for

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] By bounding S α ( bn , bσ 2 2 L ) using Lemma 2.6, setting σ 1 = σ 2 = σ √ 2 , setting τ = T − Θ ( DnLη ) if DηL ≲ Tn ,and otherwise using the simple bound (1.1) which scales linearly in T , and simplifying, we obtainthe desired privacy bound on ε , proving Theorem 3.1; full details in the Supplementary Materials.
Revised: By bounding S α ( bn , bσ 2 2 L ) using Lemma 2.6, setting σ 1 = σ 2 = σ √ 2 , setting τ = T − Θ ( DnLη ) if DηL ≲ Tn , and otherwise using the simple bound (1.1) which scales linearly in T , and simplifying, we obtain the desired privacy bound on ε , proving Theorem 3.1; full details in the Supplementary Materials.

Original:  Lower bound on privacy
Revised:  Lower bound on privacy

Original: In this section, we prove the lower bound in Theorem 1.3.
Revised: In this section, we prove the lower bound in Theorem 1.3.

Original: This is stated formally below and holdseven for linear loss functions in one dimension, in fact even when all but one of the loss functions arezero.
Revised: This is stated formally below and holds even for linear loss functions in one dimension, in fact even when all but one of the loss functions are zero.

Original: See §1.2 for a discussion of the mild assumptions on the diameter.
Revised: See §1.2 for a discussion of the mild assumptions on the diameter.

Original: Below let ¯ T ∶= 0 .
Revised: Below let ¯ T ∶= 0 .

Original: 75 DnLη .
Revised: 75 DnLη .

Original: Theorem 4.1 (Privacy lower bound for Noisy-SGD ) .
Revised: Theorem 4.1 (Privacy lower bound for N OISY -SGD) .

Original: There exist universal constants c D , c σ , c α , ¯ αand a family{ f (⋅ ,x )} x ∈X of L -Lipschitz linear functions over the interval K = [− D / 2 ,D / 2 ] such346 that the following holds.
Revised: There exist universal constants c D , c σ , c α , ¯ α and a family of L -Lipschitz linear functions over the interval K = [− D / 2 ,D / 2 ] such that the following holds.

Original: Consider running Noisy-SGD from arbitrary initialization ω 0 with anyparameters satisfying D ⩾ c D ηL and σ 2 ⩽ c σ D 2 /( η 2 ¯ T ) .
Revised: Consider running N OISY -SGD from arbitrary initialization ω 0 with any parameters satisfying D ⩾ c D ηL and σ 2 ⩽ c σ D 2 /( η 2 ¯ T ) .

Original: Then Noisy-SGD is not ( ¯ α,ε ) -RDP for
Revised: Then N OISY -SGD is not ( ¯ α,ε ) -RDP for

Original: [Equation]
Revised: [Equation]

Original: Construction.
Revised: Construction.

Original: Consider datasets X
Revised: Consider datasets X

Original: = { x 1 ,...,x n − 1 ,x n } and X ′ = { x 1 ,...,x n − 1 ,x ′ n } which differonly on x ′ n , and corresponding functions which are all zero f(⋅ ; x 1 ) =
Revised: = { x 1 ,...,x n − 1 ,x n } and X ′ = { x 1 ,...,x n − 1 ,x ′ n } which differ only on x ′ n , and corresponding functions which are all zero f 1 (⋅)

Original:  ⋅⋅⋅ =
Revised: = ⋅⋅⋅ =

Original: f(⋅ ; x n ) = 0 , exceptfor f( ω ; x ′ n  ) =
Revised: f n (⋅) = 0 , except for f ′ n ( ω ) =

Original: L ( D − ω ) .
Revised: L ( D − ω ) .

Original: Clearly these functions are linear and L -Lipschitz.
Revised: Clearly these functions are linear and L -Lipschitz.

Original: The intuition behindthis construction is that running Noisy-SGD on X or X ′ generates a random walk that is clampedto stay within the interval K —but with the key difference that running Noisy-SGD on dataset Xgenerates a symmetric random walk { ω t } , whereas running Noisy-SGD on dataset X ′ generates abiased random walk { ω ′ t } that biases right with probability b / n each step.
Revised: The intuition behind this construction is that running N OISY -SGD on X or X ′ generates a random walk that is clamped to stay within the interval K —but with the key difference that running N OISY -SGD on dataset X generates a symmetric random walk { ω t } , whereas running N OISY -SGD on dataset X ′ generates a biased random walk { ω ′ t } that biases right with probability b / n each step.

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where the processes are initialized at ω 0 = ω ′ 

Revised: where the processes are initialized at ω 0 = ω ′

Original: = 0 , each random increment Z t ∼ N( 0 ,η 2 σ 2 ) is anindependent Gaussian, and each bias Y t is ηL / b with probability b / n and otherwise is 0 .
Revised: = 0 , each random increment Z t ∼ N( 0 ,η 2 σ 2 ) is an independent Gaussian, and each bias Y t is ηL / b with probability b / n and otherwise is 0 .

Original: Key obstacle for analysis: intractability of reasoning about the iterates’ distributions.
Revised: Key obstacle for analysis: intractability of reasoning about the iterates’ distributions.

Original: Thehigh-level intuition behind this construction is simple to state: the bias of the random walk { ω ′ t }makes it distinguishable (to the minimax optimal extent, as  show) from the symmetric random walk{ ω t } .
Revised: The high-level intuition behind this construction is simple to state: the bias of the random walk { ω ′ t } makes it distinguishable (to the minimax optimal extent, as we show) from the symmetric random walk { ω t } .

Original: However, making this intution precise is challenging because the distributions of the iteratesω t ,ω ′ t are intractable to reason about explicitly—due to the highly non-linear interactions betweenthe projections and the random increments.
Revised: However, making this intution precise is challenging because the distributions of the iterates ω t ,ω ′ t are intractable to reason about explicitly—due to the highly non-linear interactions between the projections and the random increments.

Original: Thus we must establish the distinguishability of ω T ,ω ′ Tin a way that avoids reasoning explicitly about their distributions.
Revised: Thus we must establish the distinguishability of ω T ,ω ′ T in a way that avoids reasoning explicitly about their distributions.

Original: We show how to do this in the Supplementary Materials and give a full proof of Theorem 4.1 there.  
Revised: We show how to do this in the Supplementary Materials and give a full proof of Theorem 4.1 there.  Discussion

Revised: The results of this paper suggest several natural directions for future work:

Revised: Clipped gradients?

Revised: N OISY -SGD implementations sometimes clip gradients to ensure small norms [2].

Revised: For generalized linear models, the clipped gradients are gradients of an auxiliary convex loss [29], so our results can be applied directly.

Original: Checklist
Revised: However, in general, clipped gradients do not correspond to gradients of a convex loss, in which case our results (as well as all other works in the literature that aim at proving convergent privacy bounds) do not apply. Can this be remedied? Average iterate? Can similar privacy guarantees be established for the average iterate rather than the last iterate?

Revised: There are fundamental difficulties with trying to proving this: indeed, the average iterate is provably not as private for N OISY -CSGD [4].

Revised: Adaptive stepsizes?

Revised: Can similar privacy guarantees be established for optimization algorithms with adaptive stepsizes?

Revised: The main technical obstacle is how to control the privacy loss from how past iterates affect the adaptivity in later iterates.

Revised: This appears to preclude using our analysis techniques, at least in their current form.

Revised: Beyond convexity?

Revised: Convergent privacy bounds break down without convexity.

Revised: This precludes applicability to deep neural networks.

Revised: Is there any hope of establishing similar results under some sort of mild non-convexity?

Revised: Due to simple non-convex counterexamples where the privacy of N OISY -SGD diverges, any such extension would have to make additional structural assumptions on the nonconvexity (and also possibly change the N OISY -SGD algorithm), although it is unclear how this would even look.

Revised: Moreover, this appears to require significant new machinery as our techniques are the only known way to solve the convex problem, and they break down in the non-convex setting (see also the discussion in §1.2).

Revised: General techniques?

Revised: Can the analysis techniques developed in this paper be used in other settings?

Original: For all authors...
Revised: Our techniques readily generalize to any iterative algorithm which interleaves contractive steps and noise convolutions. Such algorithms are common in differentially private optimization, and it would be interesting to apply them to other variants of N OISY -SGD. Acknowledgments and Disclosure of Funding We are grateful to Hristo Paskov for many insightful conversations.

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and 
Revised: This work was done in part while JA was an intern at Apple during Summer 2021. JA was also supported in part by NSF Graduate Research Fellowship 1122374, a TwoSigma PhD Fellowship, and an NYU Faculty Fellowship.

Original: scope?
Revised: Checklist

Original: Yes
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contribu- tions and scope?

Original:  (b) Did you describe the limitations of your work?[Yes]
Revised: [Yes] (b) Did you describe the limitations of your work?

Original:  (c) Did you discuss any potential negative societal impacts of your work?[N/A]
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original:  On the contrary, the main results of this paper have positive societal impacts: we show that machine learning models can be trained for longer while still protecting the privacy of sensitive user data.
Revised: [N/A] On the contrary, the main results of this paper have positive societal impacts: we show that machine learning models can be trained for longer while still protecting the privacy of sensitive user data.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] 
Revised: [Yes] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A] (b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
Revised: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
Revised: [N/A] (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?

Original: [N/A] .
Revised: [N/A] .

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs,
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original:  internal cluster, or cloud provider)?
Revised: g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A] .
Revised: [N/A] .

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?[N/A]
Revised: [N/A] (b) Did you mention the license of the assets?

Original:  (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re us- ing/curating?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: (e) Did you discuss whether the data you are using/curating
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: IRB) c)
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original:  the estimated hourly  paid to  and the total amount spent on  compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A
Revised: [N/A]