Paper: High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise (ICLR-2024)

Reviewer comment (Theory):
The significance can be enhanced by clarifying the motivation and theoretical improvements of the new clipping method, such as providing theoretical analysis to illustrate the efficiency of this gradient update. The contribution can be strengthened by including a discussion that compares the derived bounds in this paper with existing gradient clipping results.

Author response:
We provided the motivation behind the construction of the gradient updates in our methods in Section 2 (before Theorem 2.2). We also compared our results with the existing ones in Tables 1 and 2, and also provide discussions after each theorem that relate our results to the existing ones. We have expanded the discussions in the revised version.

Reviewer comment (Novelty):
The motivation for considering both composite optimization and variational inequality problems is unclear. These two problems appear to be independent and weakly connected. The contribution of addressing two problems appears incremental.

Author response:
Minimization problems and variational inequalities are quite strongly related, though have important differences. For example, convex minimization of differentiable function $f(x)$ is equivalent to variational inequality with $F(x) = \nabla f(x)$. Therefore, one cannot say that these problems are unrelated. Moreover, under (star-)cocoercivity the analysis of SGDA-type methods share a lot of similarities with SGD-type methods, motivating to consider both problems together.

However, minimization problems have a lot of differences with variational inequalities. For example, simple gradient method is not necessary convergent for monotone Lipschitz variational inequalities. This motivates us to consider Extragradient-type methods that are specific methods for variational inequalities (though can be applied to minimization problems as well). Next, acceleration is not possible in general for (strongly-) monotone variational inequalities. Therefore, to get a complete study in the case of minimization, it is not sufficient to consider only variational inequalities formulation.

Finally, both problem types are important and appear in various applications. Therefore, we find it crucial to consider both minimization and variational inequalities problems. Such consideration provides a wider picture and more complete study of high-probability convergence of stochastic methods.

Reviewer comment (Writing):
Regarding Assumption 1, would it be better to clarify that equations (4) and (5) apply to all i in [n]?

Author response:
We thank the reviewer for the suggestion. We clarified this assumption in the revised version.

Reviewer comment (Theory):
It would be helpful to include a discussion after each theorem to compare the bound with existing clipping method bounds.

Author response:
We had such discussions in the original version of the paper and expanded them in the revised version.

Reviewer comment (Theory):
How would the proposed algorithm perform in the centralized composite optimization problem? Would it yield better convergence bounds compared to existing bounds?

Author response:
To the best of our knowledge, the results for the distributed methods proposed in our work are the only existing ones under Assumption 2 (even if we take into account the in-expectation convergence results). In the special case of $\alpha = 2$, our results match the SOTA ones from [1,2] since parallelization with linear speed-up follows for free under the bounded variance assumption (if the clipping is applied after averaging as it should be in the parallelized version of methods from [1,2] to keep the analysis from [1,2] unchanged).

---

References:

[1] E. Gorbunov, M. Danilova, I. Shibaev, P. Dvurechensky, and A. Gasnikov. Near-optimal high probability complexity bounds for non-smooth stochastic optimization with heavy-tailed noise. arXiv preprint arXiv:2106.05958, 2021.

[2] E. Gorbunov, M. Danilova, D. Dobre, P. Dvurechenskii, A. Gasnikov, and G. Gidel. Clipped stochastic methods for variational inequalities with heavy-tailed noise. NeurIPS 2022.

Reviewer comment (Novelty):
High probability bounds do not bring some technical challenges than the in-expectation ones. After establishing the martingale difference sequences, the Freedman’s inequality is enough to give a high probability bound. In this line, there are numerous papers in recent years.

Author response:
Although the reviewer silently replaced their review just two days before the deadline, we addressed all the comments by this reviewer. **Our general response and responses below justify that Reviewer LB8P provided a review of extremely low quality.**

>**High probability bounds do not bring some technical challenges than the in-expectation ones. After establishing the martingale difference sequences, the Freedman’s inequality is enough to give a high probability bound. In this line, there are numerous papers in recent years.**

We politely disagree with the reviewer. We have multiple solid arguments justifying that deriving high-probability bounds is, in general, more challenging than obtaining in-expectation ones.

First of all, a large body of the optimization community has acknowledged the importance, interest, and difficulty to get high-probability convergence guarantees [1,2,3]. For instance, we can explicitly quote [1]:

> *high probability guarantees are significantly harder to obtain and they hold in more limited settings with stronger assumptions on the problem settings and the stochastic noise distribution.*

to illustrate our point that **high-probability convergence guarantees bring new challenges in contrast to in expectation proofs**. We now develop the reasons behind this statement.

**Argument 1: proofs in expectation are usually simpler.** To prove in-expectation results, it is not needed to recursively upper-bound the distance to the solution/norm of the gradient of the objective: one can just take expectations and get rid of sums of inner product terms using unbiasedness (e.g., in the case of SGD / Accelerated SGD under bounded variance assumption, see [9]). Conversely, to apply Bernstein-Freedman, one needs either to assume that the problem is defined on a compact, which does not allow to consider an important class of unconstrained minimization problems, or to recursively bound the distance to the solution with high probability.

**Argument 2: necessary additional assumption for convergence with high probability for standard methods.** In many previous works on SGD, the high-probability analysis requires an additional assumption in comparison to in-expectation analysis: sub-Gaussian noise assumption [5,6]. This is not an artifact of the analysis – **it is, in general, impossible to show good high-probability convergence for SGD; see Theorem 2.1 in [4]** 

**Argument 3: the large gap in time between in-expectation results and high-probability ones is evidence of non-triviality.** If high-probability analysis was that trivial to obtain, it would be done much earlier since Assumption 1.1 was known 40 years ago – in 1983, the first in-expectation results were already obtained in [7]. Next, in the non-convex case, the first in-expectation analysis appeared in December 2019 (the first version of [7]), but the first high-probability analysis appeared only in June 2021 [3] (then appeared in NeurIPS 2021; this fact indicates the importance of the problem for the community) and this analysis relies on the additional assumption and is also given for much more sophisticated method than clipped-SGD. If the extension was trivial, it would not require 1.5 years and many changes in the algorithm.

Reviewer comment (Novelty):
The technical contribution is limited. The author uses gradient clipping to handle the $\alpha$-th moment, which is frequently used in recent papers. The $\alpha$-th moment condition is also widely studied. Combining the technique in gradient clipping and the technique in distributed composite minimization/distributed composite VIPs, most of the results in this paper are easily obtained.

Author response:
>**The technical contribution is limited. The author uses gradient clipping to handle the $\alpha$-th moment, which is frequently used in recent papers. The $\alpha$-th moment condition is also widely studied. Combining the technique in gradient clipping and the technique in distributed composite minimization/distributed composite VIPs, most of the results in this paper are easily obtained.**

>**What are the technical innovations of this paper compared to existing works? This paper seems to be a simple combination of existing techniques in related works**

We politely disagree with the reviewer. Our contributions go far from just combining known techniques, though even a proper combination of existing techniques is not always obvious beforehand while the reviewer is trying to claim the opposite.


Before we elaborate on the technical difficulties in the extensions, we would like to discuss the algorithms’ design. Having a goal in mind of proper extension of the results from [11] to the composite and distributed problems, we were trying to use “Occam’s razor” principle: we did not want to change the existing technique without the necessity of doing it. In particular, we did not want to change the existing algorithms just for the sake of developing new methods and having different analyses. For example, in our opinion, Prox-clipped-SGD-star has a minor but extremely important difference from just Prox-clipped-SGD. Due to this visually tiny algorithmic difference, the analysis of Prox-clipped-SGD-star in the *composite* case is almost the same as the analysis of clipped-SGD in the *unconstrained (non-composite)* case. We believe that this idea is non-trivial beforehand. Our claim is justified by the fact the previous works (like [12]) also study constrained problems but do not achieve similar results in the strongly convex case and do not get the acceleration without an extra restrictive assumption that $\nabla f(x^\ast) = 0$.

Next, to get practical methods, we made an extra step forward and replaced $\nabla f(x^\ast)$ with a learnable shift $h^k$ and also generalized the method to the distributed setup. This is the moment when noticeable technical difficulties arise. First of all, we introduce a Lyapunov function to control not only the proximity of $x^k$ to $x^\ast$ but also the quality of shifts (see line 240). In the (quasi-) strongly (monotone) convex case, this leads to the need to estimate $\|\| h_i^{k+1} - h_i^\ast \|\|^2$ and also to apply Lemma B.2 we have to estimate $\|\| F_i(x^k) - h_i^k \|\|$ and connect it to the Lyapunov function and the clipping level (see inequality (200)). This leads to the non-trivial interplay between the coefficient in the second term in the Lyapunov function, the number of workers $n$, and the clipping level $\lambda_t$ that we also need to change (increase $n$ times) compared to the non-distributed case, see the discussion after Lemma B.3. Moreover, since the clipping level is $n$ times larger, we need to be very careful in the analysis of the sums like the ones appearing in (96), especially with sum (6) from inequality (96). We emphasize that they do not appear in the non-distributed case.

Nevertheless, we circumvented all of these difficulties and managed to obtain a linear speed-up in the distributed case, and in the composite non-distributed case, we managed to obtain direct analogs of the results from [11] obtained for the unconstrained problems without any sacrifice in the complexities. That is, for the first time in the literature, we achieve $1/n$ dependence in the statistical term (the one depending on $\sigma$) of our complexity bounds under bounded central $\alpha$-th moment assumption. Let us elaborate on the non-triviality of achieving $1/n$ factor. When $\alpha = 2$, the result is standard since the variance of the average of independent random variables scales as $1/n$, and one can simply replace $\sigma^2$ with $\sigma^2/n$. This is a key property of mini-batching in stochastic optimization. However, there are no *dimension-independent* analogs of this result in the literature for the case of $\alpha < 2$. The only result we are aware of is given in Lemma 7 from [13], and it has an extra factor of $d^{1-\frac{\alpha}{2}}$, where $d$ is the dimension of the problem. For huge-scale problems, this factor can be large even for $\alpha \approx 3/2$. In contrast, our rates are dimension-independent.