Title: On the Generalization Error of Stochastic Mirror Descent for Quadratically-Bounded Losses: an Improved Analysis
Venue: NeurIPS 2023
Accepted: yes
Keywords: high probability

generalization

convex optimization

nonconvex optimization
Abstract: In this work, we revisit the generalization error of stochastic mirror descent for quadratically bounded losses studied in Telgarsky (2022). Quadratically bounded losses is a broad class of loss functions, capturing both Lipschitz and smooth functions, for both regression and classification problems. We study the high probability generalization for this class of losses on linear predictors in both realizable and non-realizable cases when the data are sampled IID or from a Markov chain. The prior work relies on an intricate coupling argument between the iterates of the original problem and those projected onto a bounded domain. This approach enables blackbox application of concentration inequalities, but also leads to suboptimal guarantees due in part to the use of a union bound across all iterations. In this work, we depart significantly from the prior work of Telgarsky (2022), and introduce a novel approach for establishing high probability generalization guarantees. In contrast to the prior work, our work directly analyzes the moment generating function of a novel supermartingale sequence and leverages the structure of stochastic mirror descent. As a result, we obtain improved bounds in all aforementioned settings. Specifically, in the realizable case and non-realizable case with light-tailed sub-Gaussian data, we improve the bounds by a $\log T$ factor, matching the correct rates of $1/T$ and $1/\sqrt{T}$, respectively. In the more challenging case of heavy-tailed polynomial data, we improve the existing bound by a $\mathrm{poly}\ T$ factor.

On the Generalization Error of Stochastic Mirror Descent for Quadratically-Bounded Losses: an Improved Analysis

Abstract

In this work, we revisit the generalization error of stochastic mirror descent for quadratically bounded losses studied in Telgarsky (2022). Quadratically bounded losses is a broad class of loss functions, capturing both Lipschitz and smooth functions, for both regression and classification problems. We study the high probability generalization for this class of losses on linear predictors in both realizable and non-realizable cases when the data are sampled IID or from a Markov chain. The prior work relies on an intricate coupling argument between the iterates of the original problem and those projected onto a bounded domain. This approach enables blackbox application of concentration inequalities, but also leads to suboptimal guarantees due in part to the use of a union bound across all iterations. In this work, we depart significantly from the prior work of Telgarsky (2022), and introduce a novel approach for establishing high probability generalization guarantees. In contrast to the prior work, our work directly analyzes the moment generating function of a novel supermartingale sequence and leverages the structure of stochastic mirror descent. As a result, we obtain improved bounds in all aforementioned settings. Specifically, in the realizable case and non-realizable case with light-tailed sub-Gaussian data, we improve the bounds by a log T factor, matching the correct rates of 1 /T and 1 / √ T , respectively. In the more challenging case of heavy-tailed polynomial data, we improve the existing bound by a poly T factor.

1 Introduction

Along with convergence analysis of optimization methods, understanding the generalization of models trained by these methods on unseen data is an important question in machine learning. However, despite the number of works attempting to answer it, the problem has not been fully understood, even in the simplest setting of linear predictors constructed with the standard stochastic gradient/mirror descent. A great part of prior works [33, 15, 30, 31, 32] focus only on the generalization on linearly separable data and/or of models trained with specific losses with exponentially decaying tails such as logistic loss. The question of what we can guarantee beyond these settings remains open.

Recently, [35] proposes a new approach to analyze the generalization error with high probability of stochastic mirror descent for a broad class of quadratically bounded losses, beyond the realizable setting. This class of losses encapsulates both Lipschitz and smooth functions, for both regression and classification problems. The obtained bounds complement existing in-expectation bounds [12] and nearly match the counterpart of convergence rates in optimization. While this result pushes forward the state of the art, the obtained guarantees do not completely resolve the problem. The central piece of the proposed approach is a 'coupling' technique between the iterates of the original problem and those projected onto a bounded domain. In this technique, one first constrains the problem in a bounded domain with a well chosen diameter. The bounded domain diameter allows to apply concentration inequalities as a blackbox and obtain bounds in high probability. Then using an inductive argument and a union bound across all iterations, one can show that the iterates in the original problem coincide with the ones in the constrained problem. Due to the union bound, the success probability decreases from 1 -δ to 1 -Tδ , where T is the number of iterations in the algorithm. This loss translates to a milder log T factor loss in the guarantee in the case of realizable data , and a more significant poly T factor loss in the non-realizable setting when the data has polynomial tails. Thus a natural question arises of whether we can obtain a stronger analysis that closes these remaining gaps.

In this paper, we revisit these generalization bounds for quadratically bounded losses by [35]. We introduce a novel approach to analyze the generalization errors of stochastic mirror descent in both realizable and non-realizable cases when the data are sampled IID or from a Markov chain. In all these cases, we remove the need to use the union bound argument, thus preventing the loss in the success probability. This translates to the following improvements:

-In the realizable, and the non-realizable cases with sub-gaussian tailed data and Markovian data, we improve the bounds by a log T factor. This improvement comes from analyzing the moment generating function of a martingale difference sequence with well-chosen coefficients. In these cases, we also remove the necessity of using the coupling-based argument used in the same work by [35]. Instead, by solely making use of the problem structure, we arrive at the same conclusion that with high probability, the iterates of stochastic mirror descent for quadratically bounded losses behave as if the problem domain is bounded.

-In the non-realizable case with polynomial tailed data, we improve the existing bound by a poly T factor. Due to the polynomial dependency on 1 δ , being able to maintain the same success probability through all iterations is crucial in this case. Unlike the previous work, we rely on a truncation technique. Using a more refined analysis of the truncated random variables, in combination with suitable concentration inequalities and the coupling technique, we improve the existing bounds significantly.

1.1 Related Work

Broadly speaking, there is a rich body of works in optimization and generalization that provide convergence guarantees and generalization bounds for stochastic methods. Earlier works often focus on in-expectation bounds [8, 24, 26, 18, 12, 3], and bounds in high probability [16, 28, 14, 13] for problems with bounded domains or under various additional assumptions such as strong convexity, noise with light tails. Recent developments for optimization [25, 10, 20, 23, 11, 17, 9, 19, 29, 22, 21] are able to handle unconstrained problems and relax these assumptions, but also require changes to the algorithm such as gradient clipping.

In generalization error analysis, specifically, a number of prior works, including [33, 15, 30, 31, 32], focus only on linearly separable data. Among these, [33, 15, 32] only deal with exponentially tailed losses while [30, 31] show generalization bounds for general smooth convex losses. Our work, similarly to [35], goes beyond the realizable setting and specific losses. We show high probability generalization bounds in both realizable and non-realizable settings for the broad class of quadratically bounded losses, for both regression and classification problems.

Other related works include the line of works that examine generalization errors via algorithmic stability. The works by [12, 5, 6, 2, 1] show the generalization error of an arbitrary algorithm via a quantity called uniform stability. By bounding this quantity for specific algorithms on a fixed training dataset, they derive generalization bounds. Our work focuses on a different setting where we assume the algorithm has access to a fresh data sample in each iteration. In this regard, the setting in our work,

as well as the prior work by [35], stays closer to the world of optimization. However, in contrast to the optimization world in which we commonly impose assumptions on the stochastic gradients (such as having bounded variance or sub-gaussian noise), we make assumptions on the data (such as sub-gaussian or polynomial tailed data). This difference introduces several challenges which we overcome in our work.

The main point of reference for this paper is the work by [35]. This work develops a 'coupling' technique to bound the generalization error of stochastic mirror descent for quadratically bounded losses. This technique has been employed in prior works [10, 11, 9, 29, 27, 22] to obtain high probability convergence bounds of stochastic methods in optimization. Our work improves their results by using a different approach that takes a closer look at the mechanism of the concentration inequalities and leverages the problem structure. When the data are bounded or have sub-gaussian tails, analyzing the moment generating function of a novel martingale difference sequence allows us to maintain the same success probability, without using either the coupling technique or the union bound. This new analysis, however, does not change the observation by [35] that the iterates of the unconstrained and the constrained problems coincide with high probability. When the data have a polynomial tail, we rely on a truncation technique. In this case, the coupling technique is necessary but not the union bound, and we are still able to significantly improve the success rate.

In terms of techniques, the work by [21] for optimization is the closest to ours. In this work, the authors develop the whitebox approach to analyzing stochastic methods for optimization with lighttailed noise. In this work, we study generalization errors. Moreover, in all settings, our choice of martingale difference sequences and coefficients are a significant departure from the prior work. In particular, in [21] the choice of coefficients only depends on the problem parameters whereas in the realizable case, our coefficients depend also on the historical data. Our approach also allows for a flexible use of an induction argument without decreasing the success probability, while in [21] the bounds are simpler and can be easily achieved in a single step.

2 Preliminaries

In this section, we provide the general set up and necessary notations before analyzing stochastic mirror descent in the subsequent sections. Overall, we closely follow notations used in [35].

Domain and norms . In this work, we consider X -the domain of the problem-to be a closed convex set or R d . We will use ∥·∥ to denote an arbitrary norm on X and let ∥·∥ ∗ be its dual norm. We define the Bregman divergence as D ψ ( w ; v ) = ψ ( w ) -ψ ( v ) -⟨∇ ψ ( v ) , w -v ⟩ where ψ : R d → R is a differentiable function that is 1 -strongly convex with respect to the norm ∥·∥ .

Loss functions . Each loss function ℓ : R × R → R ≥ 0 in our consideration can be written using a convex scalar function ˜ ℓ in one of the two following forms: 1) ℓ ( y, ̂ y ) = ˜ ℓ (sign( y ) ̂ y ) where sign( y ) = 1 if y ≥ 0 and = -1 otherwise; and 2) ℓ ( y, ̂ y ) = ˜ ℓ ( y -̂ y ) . The first form captures classification losses and the second regression losses. We will assume that subgradients ∂ℓ of ℓ in the second argument always exist, and let ℓ ′ denote a subgradient in ∂ℓ . For a function f , we also use ∥ ∂f ( w ) ∥ := sup {∥ g ∥ : g ∈ ∂f ( w ) } . We further make the following assumptions, introduced in [35] as quadratic boundedness and self-boundedness.

Assumption 1 . We assume that ℓ is ( C 1 , C 2 ) -quadratically-bounded, for some constants C 1 , C 2 ≥ 0 , i.e., for all y, ̂ y

This condition captures both classes of Lipschitz and smooth functions. Indeed, Lemma 1.2 from [35] shows that α -Lipschitz functions are ( α, 0) -quadratically-bounded while β -smooth functions are ( ∣ ∣ ∣ ∂ ˜ ℓ (0) ∣ ∣ ∣ , β ) -quadratically-bounded.

Assumption 2 . In the realizable setting, we assume that ℓ is ρ -self-bounding, i.e., ˜ ℓ satisfies ˜ ℓ ′ ( z ) 2 ≤ 2 ρ ˜ ℓ ( z ) for all z ∈ R .

The second assumption is a generalization of smoothness. This assumption is satisfied by smooth losses but also certain non-smooth losses such as the exponential loss. This condition is necessary in the current analysis to prove 1 /T rates in the realizable setting. The readers can refer to [34, 35] for more detailed discussion on this assumption.

Assumptions 1 and 2 are satisfied by commonly used loss functions in machine learning. These include the logistic loss ℓ ( y, ̂ y ) = ln(1 + exp( -y ̂ y )) and the squared loss ℓ ( y, ̂ y ) = 1 2 ( y -̂ y ) 2 (see Lemma 1.4 in [35]).

For the loss function ℓ and the configuration w , and sample ( x, y ) where x denotes the attribute and y the label, we will write ℓ x,y = ℓ ( y, w T x ) . We state the following crucial lemma which is the same as Lemma A.1 in [35], whose proof will be omitted.

Lemma 1 (Lemma A.1 in ([35])) . Suppose ℓ is ( C 1 , C 2 ) -quadratically-bounded and B x ≥ 0 is given. Given ( x, y ) such that max {∥ x ∥ ∗ , | y |} ≤ B x and any u, v ,

Risk, IID and Markovian data . When sample ( x i , y i ) arrives in iteration i of an algorithm, we will use the notation ℓ i ( w ) = ℓ ( y i , w T x i ) . For an algorithm of T iterations, we use F t = σ (( x 1 , y 1 ) , . . . , ( x t , y t )) to denote the natural filtration up to and including time t . When the data are IID and generated from a distribution π , we define the risk

In contrast to IID data, Markovian data come from a stochastic process. This setting has also been considered in [4]. We let P t s be the distribution of ( x t , y t ) at iteration t conditioned on F s . We make the following assumption regarding the uniform mixing time of the stochastic process. Note that similar assumptions have also appeared in [35, 4].

Assumption 3 . We assume that for some ϵ, τ ≥ 0 of our choice, there is a distribution π such that

Werefer to the triple ( π, τ, ϵ ) as an approximate stationarity witness. We then define the risk according to the approximate stationary distribution π : R ( w ) = E ( x,y ) ∼ π [ ℓ ( y, w T x ) ] .

Algorithm . Stochastic Mirror Descent is given in Algorithm 1. In this algorithm, for the simplicity of the analysis, we consider a fixed step size η . In each iteration, we pick a subgradient g t ∈ ∂ℓ t ( w t -1 ) and perform the update step.

We finally introduce a standard lemma used in the analysis of Stochastic Mirror Descent.

Lemma 2. For t ≥ 0 and w ref ∈ X , we have

Other notations . We will use w ref to refer to a comparator of interest. For the simplicity of the exposition, we let D 0 = D ψ ( w ref ; w 0 ) , and R ∗ = inf v ∈X R ( v ) . For a loss function ℓ that is ( C 1 , C 2 ) -quadratically-bounded, we let C 4 = C 1 + C 2 (1 + ∥ w ref ∥ ) .

3 Generalization bounds of SMD for IID data

In this section, we distinguish between two cases: the realizable case and the non-realizable case. In the realizable case, there exists an optimal solution w ∗ ∈ X such that R ( w ∗ ) = 0 . We will show that under mild assumptions, the risks of the solutions output by Algorithm 1 are bounded by O (1 /T ) . In the non-realizable case, we will show, on the other hand, a weaker statement that the excess risks of the solutions are bounded by O (1 / √ T ) .

3.1 Realizable case

In the realizable case, the comparator w ref is not necessarily the global minimizer. To show the 1 /T rate, we will assume w ref satisfies R ( w ref ) ≤ ρ D ψ ( w ref ; w 0 ) /T and that the loss at w ref is bounded. The guarantee for the iterates of Algorithm 1 is provided in Theorem 3.

Theorem 3. Suppose ℓ is convex, ( C 1 , C 2 ) -quadratically-bounded, and ρ -self-bounding. Given T , (( x t , y t )) t ≤ T are IID samples with max {∥ x t ∥ ∗ , | y t |} ≤ 1 almost surely, w ref satisfies R ( w ref ) ≤ ρ D ψ ( w ref ; w 0 ) /T , and max t<T ℓ t +1 ( w ref ) ≤ C 3 almost surely. Then for η ≤ 1 2 ρ , with probability at least 1 -2 δ , for every 0 ≤ k ≤ T -1

The analysis of Theorem 3 relies on the use of concentration inequalities. In contrast to existing works that utilize concentration inequalities as a blackbox, we will make use of the mechanism for proving concentration inequalities in order to obtain stronger guarantees. The type of concentration inequalities we consider are shown by analyzing the moment generating function of suitably chosen martingale sequences. We will use Lemma 14 (Appendix) which gives a basic inequality that bounds the moment generating function of a bounded random variable. To start the analysis, we use Lemma 2 and Assumption 2 to obtain

Lemma 4. For all t ≥ 0 , we have

First, let us pay attention to the term ∑ t i =1 ℓ i ( w ref ) . Recall that the terms ℓ i ( w ref ) are non-negative and bounded by a constant C 3 almost surely. We can analyze the term ∑ T i =1 ℓ i ( w ref ) which upper bounds all sums ∑ t i =1 ℓ i ( w ref ) by studying its moment generating function (or via a concentration inequality). We state this bound in the next lemma and defer the proof to the appendix.

Lemma 4 and lemma 5 and the assumption that R ( w ref ) = O (1 /T ) imply that with probability at least 1 -δ , D ψ ( w ref ; w t ) is bounded. In other words, with probability at least 1 -δ , the iterates w t all lie in a bounded region. One could therefore proceed to assume that the problem domain is simply this bounded ball around w ref . This is the basic idea behind the 'coupling' technique demonstrated in [35]. However, the important question is how to obtain a bound for the risk of all iterates even when we are working with a problem with unbounded domain. Here, not paying close attention to the structure of the problem and the blackbox use of concentration inequalities lead to suboptimal bounds. On the other hand, as discussed above, a crucial novelty in our analysis is the choice of a supermartingale difference sequence, defined in the proof below. By working from first principles using moment generating function of this sequence, we derive two conclusions: 1) an improved risk bound can be obtained, and 2) the coupling technique is not necessary.

Proof Sketch. Towards bounding the risk ∑ k t =0 R ( w t ) , we define random variables

and we let S t = ∑ t i =0 Z i ; ∀ 0 ≤ t ≤ T -1 . The reason to define these variables is because from Lemma 4, we can bound

where now inside the expectation, we have the term R ( w t ) - R ( w ref ) + ℓ t +1 ( w ref ) -ℓ t +1 ( w t ) which has expectation 0 . Let C 4 = C 1 + C 2 (1 + ∥ w ref ∥ ) , we can bound

and z t ≤ 1 ηC 4 ∥ w ref -w t ∥ . Now we can apply Lemma 14 (Appendix) to bound

Therefore E [exp ( Z t ) | F t ] ≤ 1 and hence (exp ( S t )) t ≥ 0 is a supermartingale. By Ville's inequality, we have with probability at least 1 -δ , for all 0 ≤ k ≤ T -1

Expanding this inequality, in combination with Lemma 5, we obtain the conclusion.

Remark 6 . The new analysis does not change the conclusion observed in [35]-that is, with high probability, the iterate sequence ( w t ) t ≥ 0 behaves as if the domain of the problem is bounded. We improve the probability that this event happens.

3.2 Non-realizable case

In the non-realizable case, we do not aim for 1 /T but only 1 / √ T rates. Hence we do not assume that the comparator w ref satisfies R ( w ref ) ≤ ρ D ψ ( w ref ; w 0 ) /T but rather the following assumption on the excess risk:

We also relax the assumption on the data samples. In the previous case, the data are bounded, i.e {∥ x ∥ ∗ , | y |} ≤ 1 a.s. We will consider in this section two settings, one when the data come from a sub-Gaussian distribution and one when the data distribution has a polynomial tail.

Remark on Theorem 10 in [35] . There seems to be an issue with the proof of Theorem 10 in [35]. The proof uses a variant of Azuma's inequality ([37], Problem 3.11) which allows the ranges of the random variables to not be specified up front. However, when bounding the range of relevant random variables, the proof uses Z t +1 , which is correlated with the data ∥ x t +1 ∥ ∗ and y t +1 . Thus the condition of Azuma's inequality is not satisfied. We do not see an immediate way to resolve this problem. In the following, we consider separately the two cases of sub-Gaussian and polynomial

tailed data for which we use different proof techniques to show the error bounds. In the first case of IID data with sub-Gaussian tails, we proceed by bounding the moment generating function of a well-chosen martingale sequence. In the second case of IID data with polynomial tails, we introduce a truncation technique. In both cases, we are able to obtained better bounds compared with [35].

3.2.1 IID data with sub-Gaussian tails

We will show the following guarantee:

Theorem 7. Suppose ℓ is convex, ( C 1 , C 2 ) -quadratically-bounded. Given T , (( x t , y t )) t ≤ T are IID samples with Q t = max { 1 , ∥ x t ∥ 2 ∗ , | y t | 2 } and there exists σ ≥ 0 such that for all λ

Let µ 1 = E [ Q t ] and µ 2 = E [ Q 2 t ] . Suppose that w ref satisfies Assumption 4. Then for η ≤ 1 4 C 2 √ Tµ 2 +2 σ √ T log 1 δ , with probability at least 1 -2 δ , for every 0 ≤ k ≤ T -1

Remark 8 . For zero-mean sub-Gaussian variable X , the definition E [exp ( λX )] ≤ exp ( λ 2 σ 2 ) for all λ is equivalent to E [ exp ( λ 2 X 2 )] ≤ exp ( λ 2 σ 2 ) for all 0 ≤ λ ≤ 1 σ (see [38]). The lemma below shows a property of sub-Gaussian variables under scaling and translating. First let us consider ∑ T t =1 Q 2 t . Similar to Lemma 5, by bounding the moment generating function of this term, we have the following (see also Section B4 in [35]).

Proof of Theorem 7. The proof of this Theorem uses the technique developed in [21]. We will also analyze the moment generating function of a suitable martingale sequence. However, the choice of the coefficients will differ significantly from the previous proof. In this case the structure of the problem is deeply integrated into the analysis of the martingale. We define

where z t = 1 4 η 2 C 2 4 ( σ 2 +4 µ 2 1 ) ( T + t +1)

and let S t = ∑ t i =0 Z i ; ∀ 0 ≤ t ≤ T -1 . By Lemma 2, we have

where we have E [( R ( w t ) -R ( w ref ) + ℓ t +1 ( w ref ) -ℓ t +1 ( w t ))] = 0 , and using the same notation C 4 = C 1 + C 2 (1 + ∥ w ref ∥ ) , by Lemma 1,

Therefore E [exp ( Z t ) | F t ] ≤ 1 and hence (exp ( S t )) t ≥ 0 is a supermartingale. By Ville's inequality, we have with probability at least 1 -δ , for all 0 ≤ k ≤ T -1

Expanding this inequality we have

where for ( a ) , by the choice of z t = 1 4 η 2 C 2 4 ( σ 2 +4 µ 2 1 ) ( T +1+ t ) we have z t -1 -z t ≥ 4 z 2 t η 2 C 2 4 ( σ 2 +4 µ 2 1 ) . We highlight that this is where the structure of the problem comes into play. That is, by setting appropriate coefficients, we can leverage gain in the distance in the martingale difference sequence ( ( z t -z t -1 ) D ψ ( w ref ; w t ) ) to cancel out the loss from bounding the moment generating function ( 4 z 2 t η 2 C 2 4 ( σ 2 +4 µ 2 1 ) D ψ ( w ref ; w t ) ). Another important property of the sequence ( z t ) is that it is a decreasing sequence and z t z k ≤ 2 for all t, k . Hence we have

Combined with Lemma 9, with probability at least 1 -2 δ , for all 0 ≤ k ≤ T -1

Conditioned on this event, we will prove by induction that

For the base case k = 0 , it is trivial. Suppose for all t ≤ k we have D ψ ( w ref ; w t ) ≤ R 2 , now we prove for t = k +1 . By Lemma 1,

Therefore

Finally we obtain, η ∑ k t =0 ( R ( w t ) -R ∗ ) + D ψ ( w ref ; w k +1 ) ≤ R 2 , as needed.

3.2.2 IID data with polynomial tails

Theorem 10. Suppose ℓ is convex, ( C 1 , C 2 ) -quadratically bounded. Given T , (( x t , y t )) t ≤ T are IID samples with Q t = max { 1 , ∥ x t ∥ 2 ∗ , | y t | 2 } and for some p ≥ 2 there exists M ≥ p e such that for all λ

Let µ 1 = E [ Q t ] and µ 2 = E [ Q 2 t ] . Suppose that w ref satisfies Assumption 4. Then for η ≤ 1 C 2 √ 6 ( Tµ 2 +2 M √ T ( 2 δ ) 1 p ) , with probability at least 1 -3 δ , for every 0 ≤ k ≤ T -1

where R = max { √ 6 ( D 0 ( 1 + η √ T ) + η 2 C 2 4 ( Tµ 2 +2 M √ T ( 2 δ ) 1 p )) , 6 ( 2 3 γ ( 7 ( MT δ ) 1 / 2 p + 2 µ 1 ) + √ log 2 δ Tµ 2 ηC 4 )} = O (1) , γ = max { 1 , log 2 δ } .

Remark 11 . Since p ≥ 2 , the rate is O ( 1 T 1 / 2 log 1 δ + 1 T 3 / 4 ( 1 δ ) 1 2 p ) . This rate improves over the O (( 1 T 1 / 2 + 1 T 3 / 4 ( T δ ) 1 2 p ) log T δ ) rate by [35] by a polynomial factor T 1 2 p log T δ in the high probability regime where δ = 1 poly( T ) .

We will give a proof sketch for this theorem. The full proof is deferred to the appendix.

Proof Sketch. The heavy tailed distribution of the data does not allow us to analyze the moment generating function. In this case, we rely on the coupling technique as in [35]. Since it is not possible to apply Azuma's inequality due to the bounds on the variables being not measurable, and the variables are heavy tailed, we use truncation technique. We define,

where we use g t ( v t -1 ) to denote the gradient at v t -1 using the same data point ( x t , y t ) when computing w t and we define

We can write

We bound ∑ k t =0 ( P t -E [ P t | F t ]) by applying Freedman's inequality. The terms ∑ k t =0 E [ P t | F t ] and ∑ k t =0 B t are both the bias terms can be bounded by analyzing the tail of the distribution and Markov's inequality. We also use Lemma 12 to bound ∑ k t =0 ∥ g t +1 ( v t ) ∥ 2 ∗ . Finally, using the induction technique, we can prove that w t = v t with high probability and obtain the desired result.

Lemma 12 (Lemma A.5 from [35]) . With probability ≥ 1 -δ , ∑ T t =1 Q 2 t ≤ Tµ 2 +2 M √ T ( 2 δ ) 1 p .

4 Generalization bounds of SMD for Markovian data

The final result we present in this paper is the following theorem for the case when the data are sampled from a Markov chain.

Theorem 13. Suppose ℓ is convex, ( C 1 , C 2 ) -quadratically bounded. Given T , (( x t , y t )) t ≤ T are sampled from a Markov chain with max { ∥ x t ∥ 2 ∗ , | y t | 2 } ≤ 1 and ( π, τ, ϵ = 1 √ T ) is an approximate stationarity witness. Suppose that w ref satisfies Assumption 4. Then for η ≤ 1 2 C 2 √ T (1+2 τ ) , with probability at least 1 -τδ , for every 0 ≤ k ≤ T -1

We will give a proof sketch for this theorem.

Proof Sketch. The proof of this Theorem follow similarly to that of Theorem 7. The difference here is we need to bound τ different martingale difference sequences in the form of

for 0 ≤ j ≤ τ -1 , 0 ≤ i ≤ T -1 -j τ . We also need the assumption on the approximate stationarity witness to see that

Now we only need the union bound over τ sequences, instead of all iterations. The success probability will decrease from 1 -δ to 1 -τδ .

5 Conclusion

In this paper, we show a new approach to analyze the generalization error of SMD for quadratically bounded losses. Our approach improves a logarithmic factor for the realizable setting and nonrealizable setting with light tailed data and a poly T factor for the non-realizable setting with polynomial tailed data from the prior work by [35]. An inherent limitation of the current approach is the assumption that we can obtain a fresh sample in each iteration, whereas the setting with finite training data is still not well understood. In the realizable setting, we require that the data is bounded, as opposed to more relaxed assumptions in the non-realizable settings. We leave the question of resolving these issues for future works.

References

Raef Bassily, Vitaly Feldman, Cristóbal Guzmán, and Kunal Talwar. Stability of stochastic gradient descent on nonsmooth convex losses. Advances in Neural Information Processing Systems , 33:4381-4391, 2020.

Olivier Bousquet, Yegor Klochkov, and Nikita Zhivotovskiy. Sharper bounds for uniformly stable algorithms. In Conference on Learning Theory , pages 610-626. PMLR, 2020.

Radu Alexandru Dragomir, Mathieu Even, and Hadrien Hendrikx. Fast stochastic bregman gradient methods: Sharp analysis and variance reduction. In International Conference on Machine Learning , pages 2815-2825. PMLR, 2021.

John C Duchi, Alekh Agarwal, Mikael Johansson, and Michael I Jordan. Ergodic mirror descent. SIAM Journal on Optimization , 22(4):1549-1578, 2012.

Vitaly Feldman and Jan Vondrak. Generalization bounds for uniformly stable algorithms. Advances in Neural Information Processing Systems , 31, 2018.

Vitaly Feldman and Jan Vondrak. High probability generalization bounds for uniformly stable algorithms with nearly optimal rate. In Conference on Learning Theory , pages 1270-1279. PMLR, 2019.

David A Freedman. On tail probabilities for martingales. the Annals of Probability , pages 100-118, 1975.

Saeed Ghadimi and Guanghui Lan. Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization i: A generic algorithmic framework. SIAM Journal on Optimization , 22(4):1469-1492, 2012.

Eduard Gorbunov, Marina Danilova, David Dobre, Pavel Dvurechenskii, Alexander Gasnikov, and Gauthier Gidel. Clipped stochastic methods for variational inequalities with heavy-tailed noise. Advances in Neural Information Processing Systems , 35:31319-31332, 2022.

Eduard Gorbunov, Marina Danilova, and Alexander Gasnikov. Stochastic optimization with heavy-tailed noise via accelerated gradient clipping. Advances in Neural Information Processing Systems , 33:15042-15053, 2020.

Eduard Gorbunov, Marina Danilova, Innokentiy Shibaev, Pavel Dvurechensky, and Alexander Gasnikov. Near-optimal high probability complexity bounds for non-smooth stochastic optimization with heavy-tailed noise. arXiv preprint arXiv:2106.05958 , 2021.

Moritz Hardt, Ben Recht, and Yoram Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International conference on machine learning , pages 1225-1234. PMLR, 2016.

Nicholas JA Harvey, Christopher Liaw, Yaniv Plan, and Sikander Randhawa. Tight analyses for non-smooth stochastic gradient descent. In Conference on Learning Theory , pages 1579-1613. PMLR, 2019.

Elad Hazan and Satyen Kale. Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimization. The Journal of Machine Learning Research , 15(1):2489-2512, 2014.

Ziwei Ji and Matus Telgarsky. Risk and parameter convergence of logistic regression. arXiv preprint arXiv:1803.07300 , 2018.

Sham M Kakade and Ambuj Tewari. On the generalization ability of online strongly convex programming algorithms. Advances in Neural Information Processing Systems , 21, 2008.

Ali Kavis, Kfir Yehuda Levy, and Volkan Cevher. High probability bounds for a class of nonconvex algorithms with adagrad stepsize. In International Conference on Learning Representations , 2021.

Ahmed Khaled and Peter Richtárik. Better theory for sgd in the nonconvex world. arXiv preprint arXiv:2002.03329 , 2020.

Shaojie Li and Yong Liu. High probability guarantees for nonconvex stochastic gradient descent with heavy tails. In International Conference on Machine Learning , pages 12931-12963. PMLR, 2022.

Xiaoyu Li and Francesco Orabona. A high probability analysis of adaptive sgd with momentum. arXiv preprint arXiv:2007.14294 , 2020.

Zijian Liu, Ta Duy Nguyen, Thien Hang Nguyen, Alina Ene, and Huy Lê Nguyen. High probability convergence of stochastic gradient methods. arXiv preprint arXiv:2302.14843 , 2023.

Zijian Liu, Jiawei Zhang, and Zhengyuan Zhou. Breaking the lower bound with (little) structure: Acceleration in non-convex stochastic optimization with heavy-tailed noise. arXiv preprint arXiv:2302.06763 , 2023.

Liam Madden, Emiliano Dall'Anese, and Stephen Becker. High probability convergence and uniform stability bounds for nonconvex stochastic gradient descent. arXiv preprint arXiv:2006.05610 , 2020.

Eric Moulines and Francis Bach. Non-asymptotic analysis of stochastic approximation algorithms for machine learning. Advances in neural information processing systems , 24, 2011.

Alexander V Nazin, Arkadi S Nemirovsky, Alexandre B Tsybakov, and Anatoli B Juditsky. Algorithms of robust stochastic optimization based on mirror descent method. Automation and Remote Control , 80(9):1607-1627, 2019.

Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization , 19(4):15741609, 2009.

Ta Duy Nguyen, Thien Hang Nguyen, Alina Ene, and Huy Le Nguyen. High probability convergence of clipped-sgd under heavy-tailed noise. arXiv preprint arXiv:2302.05437 , 2023.

Alexander Rakhlin, Ohad Shamir, and Karthik Sridharan. Making gradient descent optimal for strongly convex stochastic optimization. arXiv preprint arXiv:1109.5647 , 2011.

Abdurakhmon Sadiev, Marina Danilova, Eduard Gorbunov, Samuel Horváth, Gauthier Gidel, Pavel Dvurechensky, Alexander Gasnikov, and Peter Richtárik. High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 , 2023.

Matan Schliserman and Tomer Koren. Stability vs implicit bias of gradient methods on separable data and beyond. In Conference on Learning Theory , pages 3380-3394. PMLR, 2022.

Matan Schliserman and Tomer Koren. Tight risk bounds for gradient descent on separable data. arXiv preprint arXiv:2303.01135 , 2023.

Ohad Shamir. Gradient methods never overfit on separable data. The Journal of Machine Learning Research , 22(1):3847-3866, 2021.

Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent on separable data. The Journal of Machine Learning Research , 19(1):2822-2878, 2018.

Matus Telgarsky. Margins, shrinkage, and boosting. In International Conference on Machine Learning , pages 307-315. PMLR, 2013.

Matus Telgarsky. Stochastic linear optimization never overfits with quadratically-bounded losses on general data. In Conference on Learning Theory , pages 5453-5488. PMLR, 2022.

Joel Tropp. Freedman's inequality for matrix martingales. 2011.

R van Handel. Apc 550: Probability in high dimension. Lecture Notes. Princeton University. Retrieved from https://web. math. princeton. edu/rvan/APC550. pdf on December , 21:2016, 2016.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018.

A Concentration Inequalities

Lemma 14. Let X be a random variable such that E [ X ] = 0 and | X | ≤ R almost surely. Then for 0 ≤ λ ≤ 1 R

The following lemma is similar to Lemma 2.2 in [21].

Lemma 15. Suppose that Q satisfies for all 0 ≤ λ ≤ 1 σ , E [ exp ( λ 2 Q 2 )] ≤ exp ( λ 2 σ 2 ) . Then for variable X such that E [ X ] = 0 and | X | ≤ a ( Q + b ) for some a ≥ 0 then for all λ ≥ 0

In particular, if b = 0 we can have a tighter constant: E [exp ( λX )] ≤ exp ( λ 2 a 2 σ 2 ) .

Proof. We consider E [exp ( λX )]

Otherwise 1 σ ≤ λ √ 2 a

Theorem 16 (Freedman's inequality [7, 36]) . Let ( X t ) t ≥ 1 be a martingale difference sequence. Assume that there exists a constant c such that | X t | ≤ c almost surely for all t ≥ 1 and define σ 2 t = E [ X 2 t | X t -1 , . . . , X 1 ] . Then for all b > 0 , F > 0 and T ≥ 1

B

Missing Proofs

Proof of Lemma 2. Using the optimality condition

we have

Hence

as needed.

Proof of Lemma 4. We have

Summing up, we have, for any 0 ≤ t ≤ T

Proof of Lemma 5. We have | ℓ i ( w ref ) -R ( w ref ) | ≤ max { ℓ i ( w ref ) , R ( w ref ) } ≤ C 3 thus by lemma 14, for λ ≤ 1 C 3

where for ( a ) we use E [ ( X -E [ X ]) 2 ] ≤ E [ X 2 ] and for ( b ) we use ℓ i ( w ref ) ≤ C 3 . Since ℓ i ( w ref ) are independent random variables, we have

Hence by Markov's inequality

Choose λ = 1 C 3 we have with probability at least 1 -δ

Proof of Theorem 3. Towards bounding the risk ∑ k t =0 R ( w t ) , we define random variables

The reason to define these variables is because from Lemma 4, we can bound

where now inside the expectation, we have the term R ( w t ) - R ( w ref ) + ℓ t +1 ( w ref ) -ℓ t +1 ( w t ) which has expectation 0 . This reminds us of Lemma 14. To use this lemma, we notice that, by the assumption that the samples are IID with max {∥ x ∥ ∗ , | y |} ≤ 1 and Lemma 1,

We also have

Therefore

By the choice of z t we have

Now we can apply Lemma 14 to bound

Therefore E [exp ( Z t ) | F t ] ≤ 1 and hence (exp ( S t )) t ≥ 0 is a supermartingale. By Ville's inequality, we have with probability at least 1 -δ , for all 0 ≤ k ≤ T -1

Expanding this inequality, we obtain

For ( a ) we use the fact that ( z t ) is a decreasing sequence and R ( w ref ) ≤ ρD 0 T . For ( b ) we use the assumption ℓ t +1 ( w ref ) ≤ C 3 . Now notice that we can write ηℓ t +1 ( w ref ) √ ∑ t +1 ≤

2 D 0 +2 η i =1 ℓ i ( w ref ) √ we obtain

Hence

By Lemma 5, with probability at least 1 -δ we have

Therefore with probability at least 1 -2 δ

which gives us the conclusion.

Proof of Theorem 10. First we consider the bounded domain case. Let

where we use g t ( v t -1 ) to denote the gradient at v t -1 using the same data point ( x t , y t ) when computing w t and we choose

We have

Let us define the following variables

In words, U t is the variable of our interest and P t is the truncated version of U t and B t is the bias. We would want to control these terms in order to bound ∑ k t =0 U t . We start with the following decomposition

First, we consider the term ∑ k t =0 E [ P t | F t ] .

where the last inequality is because p ≥ 2 . We obtain

The term ∑ k t =0 B t ≤ ∑ k t =0 | B t | ≤ ∑ T -1 t =0 | B t | will be bounded by Markov inequality. From the above deduction,

With probability at least 1 -δ ,

Finally, we will use Freedman's inequality to bound the remaining term ∑ k t =0 ( P t -E [ P t | F t ]) . First, notice that

We have ( P t -E [ P t | F t ]) is a martingale difference sequence with | P t -E [ P t | F t ] | ≤ 2 ( A +2 µ 1 ) RC 4 . We can apply Freedman's inequality,

If we select

we obtain with probability at least 1 -δ , for all k ≥ 0

Therefore with probability at least 1 -3 δ we have the following event E : for all k ≥ 0

where we denote γ = max { 1 , log 2 δ } . Furthermore

Now we will proceed by induction to show that conditioned on the event E , w t = v t . For the base case, we have w 0 = v 0 . Suppose that we have w t = v t for all t ≤ k . We will show that w k +1 = v k +1 . From Lemma 2, we have

Thus ∥ w k +1 -w ref ∥ ≤ R . And thus w k +1 = v k +1 . Finally, we can conclude that with probability at least 1 -3 δ , for all 0 ≤ k ≤ T -1

Proof of Theorem 13. For 0 ≤ j ≤ τ -1 , we define

where

We bound

Therefore E [ exp ( Z j i ) | F τi + j ] ≤ 1 and hence ( exp ( S j k )) k ≥ 0 is a supermartingale. By Ville's inequality, we have with probability at least 1 -δ , for all 0 ≤ k ≤ κ

By union bound over j = 0 , . . . , τ -1 , and with probability at least 1 -τδ we have for all 0 ≤ k ≤ T -τ -1

We will proceed to prove by induction that D ψ ( w ref ; w t ) ≤ 1 2 R 2

For the base case t = 0 , this holds trivially. Suppose that this is true for all 0 ≤ t ≤ k , we now show for t = k +1 .

We have and

Therefore

If k ≥ τ ,

where in the last inequality we use z t = 1 8 η 2 C 2 4 ( T +1+ t ) to see that z t +8( z t η ) 2 C 2 4 ≤ z t -1 . Notice that, z t z k ≤ 2 , and ( R ( w ref ) -R ∗ ) ≤ D 0 √ T we have

Now we bound each term. For A

For B , by Assumption 3, sup t ∈ Z ≥ 0 sup F t TV ( P t + τ t , π ) ≤ ϵ ,

Thus

For C , similarly to before

For D , we have

We obtain

For E , since

Hence

Sum up we have

as needed. Finally we have

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes the generalization bound for stochastic mirror descent applied to linear models and a general class of quadratically-bounded losses. In addition to the more common IID sampled data, this paper also considers where the data is sampled from a Markov chain. This paper follows up from the work of [Telgarsky 2022] and improved the bounds, particularly in the case of non-realizable setting. The improvements in this paper come from a more clever application of concentration inequalities.\n\nSTRENGTHS:\nSince this paper is mainly a follow up to [Telgarsky, 2022], I would refer that paper as [T22] for simplicity.\n\nOne major strength of this paper compare to [T22] is that the content are better organized and the notations are also slightly better. I found this paper to be generally easy to follow.\n\nAlso, the theoretical analysis seem to be in good shape. While I did not have the time to read through the proofs line-by-line, I have reasonable faith that there are not major technical issues buried in the proof.\n\nWEAKNESSES:\nWhile the overall quality of this paper is good, I feel that the contribution over [T22] is not sufficient to warrant a new paper. In particular, compare to [T22]: Theorems 3 and 13 in this paper have the same guarantee as Theorems 5 and 8 in [T22], respectively. While Theorem 7 of this paper is an improvement over Theorem 10 of [T22], a log factor is not particularly interesting. Also, the problem setting of this paper is almost entirely the same as [T22] and I would love to see more novelty on this front.\n\nQUESTIONS:\nI would like to hear from the author on how this paper presents a significant delta over [T22] and in what ways their improved bounds can be useful. Given what I currently gathered from the paper, I am inclined to say that the contribution is insufficient, but I would love to be proven wrong.\n\nLIMITATIONS:\nN/A. This paper is purely theoretical and I do not believe there would be any negative society impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study Stochastic Mirror Descent, and analysis its generalization error under a quadratic loss assumptions, self boundedness of the loss (a relaxation of smoothness), iid or Markovian data, and realizable or non-realizable cases, which amount to assume that the error of the reference point is of different orders.\nThe approach taken consists in deriving concentration bounds taylored to the problem.\nThe results improve previous ones by some log(T) (resp. polylog(T) ) factor in the realizatble (resp. non-realizable) setting.\n\nSTRENGTHS:\n**Disclaimer.** I acknowledge not being familiar with the generalization error literature cited in the paper (excpet the Hardt et al. paper on algorithmic stability), although I am quite familiar with the SGD literature and especially Mirror descent and its stochastic counterparts.  \n\nStochastic Mirror Descent is a widely used and studied algorithm in many domains of optimization. Since existing generalization error analyses (the SOTA rates being Telgarsky [30]), as cited in the paper) appear to be suboptimal in light of the rates obtained in the present submission, I think this is a contribution worth acceptance at Neurips. The whole paper being organized around the generalization bounds for SMD under the various data assumptions (realizable or not cases, Markovian or iid), it is quite technical, but the authors make an effort to put as much detail as possible of the analysis in the main part of the paper\n\nWEAKNESSES:\nI think that the results should be put in light of optimization error analyses of SMD, the current best rates being in the following paper [A]:\n\n**[A]** *Fast Stochastic Bregman Gradient Methods: Sharp Analysis and Variance Reduction*, Dragomir et al., Icml 2021.\n\nThe following work also studies SMD convergence rates.\n\n**[B]** *Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize*, Loizou et al. 2022.\n\n\nIn particular, the rates in [A] being for SMD with iid data and fresh samples at each iteration (and thus the loss being optimized in the generalization error), can these rates be considered, in the context of the present submission, as in-expectation gen error ?\n\nAlso, in many applications, MD or SMD is used when (S)GD cannot converge (or is too slow) due to non-smoothness and degenerate frontiers (e.g, minimization of KL distances). It seems to me that the assumptions on the loss in the present submission are too restrictive to allow this. This, in addition to some strong assumptions on the data lead to somewhat quite restrictive assumptions; this is however fine as it appears that previous works also make similar assumptions. To what extent the gap in the theory between SMD convergence rates in expectation (as in [A]) and the high probability generalization error of this submission seems hard to fill ?\n\nQUESTIONS:\nSee above. \n\nI suggest acceptance for this paper for the reasons noted above. I suggest adding references such as [A,B] in order to add discussions for readers who are more familiar with such works.\n\nLIMITATIONS:\nNo such limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

comments: SUMMARY:
The authors propose a novel approach to analyze the generalization error of Stochastic Mirror Descent (SMD) for quadratically bounded losses. Compared to previous methods, their approach demonstrates improved accuracy by a logarithmic factor when the data is realizable. Additionally, in scenarios where the data is not realizable and has a polynomial tail, their method achieves enhanced accuracy by a polynomial factor of T.

STRENGTHS:
1. The paper provides a clear and well-explained improvement over previous work.
2. The approach presented in the paper, which involves analyzing the moment generating function of a martingale difference sequence with well-chosen coefficients is interesting as it avoids the need for the coupling technique used in previous works and might be used also in future works.

WEAKNESSES:
1.A potential weakness of the paper is that the improvement over previous work in the realizable case is relatively modest, as it is limited to logarithmic factors in the bound. 
2. Although the paper suggests improvements in the bound, these may not result in entirely new conclusions or fundamentally alter our understanding of the generalization behavior of Stochastic Mirror Descent (SMD) with quadratically bounded losses. The incremental nature of the improvement might not have a significant impact on the practical performance or broader understanding of the algorithm.

QUESTIONS:
See weaknesses.

LIMITATIONS:
yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-05

comments: SUMMARY:
This paper revisits the generalization error of stochastic mirror descent for quadratically bounded losses studied in Telgarsky (2022). The authors depart significantly from the prior work of Telgarsky (2022), and introduce a novel approach for establishing high probability generalization guarantees. In contrast to the prior work, this work directly analyzes the moment generating function of a novel supermartingale sequence and leverages the structure of stochastic mirror descent. As a result, this paper obtains improved bounds.

STRENGTHS:
This paper revisits the generalization error of stochastic mirror descent for quadratically bounded losses studied in Telgarsky (2022). The authors depart significantly from the prior work of Telgarsky (2022), and introduce a novel approach for establishing high probability generalization guarantees. In contrast to the prior work, this work directly analyzes the moment generating function of a novel supermartingale sequence and leverages the structure of stochastic mirror descent. As a result, this paper obtains improved bounds.

WEAKNESSES:
No

QUESTIONS:
In the proof of  Lemma 4, line391-line392,  the last inequality uses assumption 2. 
However, Assumption assumes that $\ell'(z)^2 \le 2 \ell(z)$ for $z\in \R$ but $w_t$ in the proof is a vector. 
Could you fix it?

LIMITATIONS:
No.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09