Title: Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance
Venue: NeurIPS 2023
Accepted: yes
Keywords: stochastic optimization

gradient-free optimization

zero-order oracle

gradient clipping

infinite variance
Abstract: In this paper, we consider non-smooth stochastic convex optimization with two function evaluations per round under infinite noise variance. In the classical setting when noise has finite variance, an optimal algorithm, built upon the batched accelerated gradient method, was proposed in (Gasnikov et. al., 2022). This optimality is defined in terms of iteration and oracle complexity, as well as the maximal admissible level of adversarial noise. However, the assumption of finite variance is burdensome and it might not hold in many practical scenarios. To address this, we demonstrate how to adapt a refined clipped version of the accelerated gradient (Stochastic Similar Triangles) method from (Sadiev et al., 2023) for a two-point zero-order oracle. This adaptation entails extending the batching technique to accommodate infinite variance — a non-trivial task that stands as a distinct contribution of this paper.

Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance

Abstract

In this paper, we consider non-smooth stochastic convex optimization with two function evaluations per round under infinite noise variance. In the classical setting when noise has finite variance, an optimal algorithm, built upon the batched accelerated gradient method, was proposed in [17]. This optimality is defined in terms of iteration and oracle complexity, as well as the maximal admissible level of adversarial noise. However, the assumption of finite variance is burdensome and it might not hold in many practical scenarios. To address this, we demonstrate how to adapt a refined clipped version of the accelerated gradient (Stochastic Similar Triangles) method from [35] for a two-point zero-order oracle. This adaptation entails extending the batching technique to accommodate infinite variance - a non-trivial task that stands as a distinct contribution of this paper.

1 Introduction

In this paper, we consider stochastic non-smooth convex optimization problem

where f ( x, ξ ) is M 2 ( ξ ) -Lipschitz continuous in x w.r.t. the Euclidean norm, and the expectation E ξ ∼D [ f ( x, ξ )] is with respect to a random variable ξ with unknown distribution D . The optimization is performed only by accessing two function evaluations per round rather than sub-gradients, i.e., for any pair of points x, y ∈ R d , an oracle returns f ( x, ξ ) and f ( y, ξ ) with the same ξ . The primary motivation for this gradient-free oracle arises from different applications where calculating gradients is computationally infeasible or even impossible. For instance, in medicine, biology, and physics, the objective function can only be computed through numerical simulation or as the result of a real experiment, i.e., automatic differentiation cannot be employed to calculate function derivatives. Usually, a black-box function we are optimizing is affected by stochastic or computational noise. This noise can arise naturally from modeling randomness within a simulation or by computer discretization.

In classical setting, this noise has light tails. However, usually in black-box optimization, we know nothing about the function, only its values at the requested points are available/computable, so light tails assumption may be violated. In this case, gradient-free algorithms may diverge. We aim to develop an algorithm that is robust even to heavy-tailed noise, i.e., noise with infinite variance. In theory, one can consider heavy-tailed noise to simulate situations where noticeable outliers may occur (even if the nature of these outliers is non-stochastic). Therefore we relax classical finite variance assumption and consider less burdensome assumption of finite α -th moment, where α ∈ (1 , 2] .

In machine learning, interest in gradient-free methods is mainly driven by the bandit optimization problem [14, 2, 5], where a learner engages in a game with an adversary: the learner selects a point x , and the adversary chooses a point ξ . The learner's goal is to minimize the average regret based solely on observations of function values (losses) f ( x, ξ ) . As feedback, at each round, the learner receives losses at two points. This corresponds to a zero-order oracle in stochastic convex optimization with two function evaluations per round. The vast majority of researches assume sub-Gaussian distribution of rewards. However, in some practical cases (e.g., in finance [33]) reward distribution has heavy-tails or can be adversarial. For the heavy-tailed bandit optimization, we refer to [9].

Two-point gradient-free optimization for non-smooth (strongly) convex objective is a well-researched area. Numerous algorithms have been proposed which are optimal with respect to two criteria: oracle and iteration complexity. For a detailed overview, see the recent survey [15] and the references therein. Optimal algorithms, in terms of oracle call complexity, are presented in [10, 36, 3]. The distinction between the number of successive iterations (which cannot be executed in parallel) and the number of oracle calls was initiated with the lower bound obtained in [6]. It culminated with the optimal results from [17], which provides an algorithm which is optimal in both criteria. Specifically, the algorithm produces ˆ x , an ε -solution of (1), such that we can guarantee E [ f (ˆ x )] -min x ∈ R d f ( x ) after

∼ d 1 4 ε -1 successive iterations,

∼ dε -2 oracle calls (number of f ( x, ξ ) calculations).

The convergence guarantee for this optimal algorithm from [17] was established in the classical setting of light-tailed noise, i.e., when noise has finite variance: E ξ [ M 2 ( ξ ) 2 ] < ∞ . However, in many modern learning problems the variance might not be finite, leading the aforementioned algorithms to potentially diverge. Indeed, heavy-tailed noise is prevalent in contemporary applications of statistics and deep learning. For example, heavy-tailed behavior can be observed in training attention models [41] and convolutional neural networks [37, 20]. Consequently, our goal is to develop an optimal algorithm whose convergence is not hampered by this restrictive assumption. To the best of our knowledge, no existing literature on gradient-free optimization allows for E ξ [ M 2 ( ξ ) 2 ] to be infinite. Furthermore, convergence results for all these gradient-free methods were provided in expectation, that is, without (non-trivial) high-probability bounds. Although the authors of [17] mentioned (without proof) that their results can be formulated in high probability using [19], this aspect notably affects the oracle calls complexity bound and complicates the analysis.

A common technique to relax finite variance assumption is gradient clipping [31]. Starting from the work of [26] (see also [8, 19]), there has been increased interest in algorithms employing gradient clipping to achieve high-probability convergence guarantees for stochastic optimization problems with heavy-tailed noise. Particularly, in just the last two years there have been proposed

an optimal algorithm with a general proximal setup for non-smooth stochastic convex optimization problems with infinite variance [39] that converges in expectation (also referenced in [27]),

an optimal adaptive algorithm with a general proximal setup for non-smooth online stochastic convex optimization problems with infinite variance [42] that converges with high probability,

optimal algorithms using the Euclidean proximal setup for both smooth and non-smooth stochastic convex optimization problems and variational inequalities with infinite variance [35, 30, 29] that converge with high probability,

an optimal variance-adaptive algorithm with the Euclidean proximal setup for non-smooth stochastic (strongly) convex optimization problems with infinite variance [24] that converges with high probability.

None of these papers discuss a gradient-free oracle. Moreover, they do not incorporate acceleration (given the non-smooth nature of the problems) with the exception of [35]. Acceleration is a crucial

component to achieving optimal bounds on the number of successive iterations. However, the approach in [35] presumes smoothness and does not utilize batching. To apply the convergence results from [35] to our problem, we need to adjust our problem formulation to be smooth. This is achieved by using the Smoothing technique [27, 36, 17]. In the work [22] authors proposed an algorithm based on Smoothing technique and non-accelerated Stochastic Mirror Descent with clipping. However, this work also does not support acceleration, minimization on the whole space and batching. Adapting the technique from [35] to incorporate batching necessitates a substantial generalization. We regard this aspect of our work as being of primary interest.

Heavy-tailed noise can also be handled without explicit gradient clipping, e.g., by using Stochastic Mirror Descent algorithm with a particular class of uniformly convex mirror maps [39]. However, the convergence guarantee for this algorithm is given in expectation. Moreover, applying batching and acceleration is a non-trivial task. Without this, we are not able to get the optimal method in terms of the number of iterations and not only in terms of oracle complexity. There are also some studies on the alternatives to gradient clipping [21] but the results for these alternatives are given in expectation and are weaker than the state-of-the-art results for the methods with clipping. This is another reason why we have chosen gradient clipping to handle the heavy-tailed noise.

1.1 Contributions

We generalize the optimal result from [17] to accommodate a weaker assumption that allows the noise to exhibit heavy tails . Instead of the classical assumption of finite variance, we require the boundedness of the α -moment: there exists α ∈ (1 , 2] such that E ξ [ M 2 ( ξ ) α ] < ∞ . Notably, when α < 2 , this assumption is less restrictive than the assumption of a finite variance and thus it has garnered considerable interest recently, see [41, 35, 29] and the references therein. Under this assumption we prove that for convex f , an ε -solution can be found with high probability after

and for µ -strongly convex f , the ε -solution can be found with high probability after

In both instances, the number of oracle calls is optimal in terms of ε -dependency within the non-smooth setting [27, 39]. For first-order optimization under heavy-tailed noise, the optimal ε -dependency remains consistent, as shown in [35, Table 1].

In what follows, we highlight the following several important aspects of our results

High-probability guarantees. We provide upper-bounds on the number of iterations/oracle calls needed to find a point ˆ x such that f (ˆ x ) -min x ∈ R d f ( x ) ≤ ε with probability at least 1 -β . The derived bounds have a poly-logarithmic dependence on 1 / β . To the best of our knowledge, there are no analogous high-probability results, even for noise with bounded variance.

Generality of the setup. Our results are derived under the assumption that gradient-free oracle returns values of stochastic realizations f ( x, ξ ) subject to (potentially adversarial) bounded noise. We further provide upper bounds for the magnitude of this noise, contingent upon the target accuracy ε and confidence level β . Notably, our assumptions about the objective and noise are confined to a compact subset of R d . This approach, which differs from standard ones in derivativefree optimization, allows us to encompass a wide range of problems. This approach differs from standard ones in derivative-free optimization

Batching without bounded variance. To establish the aforementioned upper bounds, we obtain the following: given X 1 , . . . , X B as independent random vectors in R d where E [ X i ] = x ∈ R d and E ∥ X i -x ∥ α 2 ≤ σ α for some σ ≥ 0 and α ∈ (1 , 2] , then

When α = 2 , this result aligns with the conventional case of bounded variance (accounting for a numerical factor of 2). Unlike existing findings, such as [40, Lemma 7] where α < 2 , the relation (2) does not exhibit a dependency on the dimension d . Moreover, (2) offers a theoretical basis to highlight the benefits of mini-batching, applicable to methods highlighted in this paper as well as first-order methods presented in [35, 29].

Dependency on d . As far as we are aware, an open question remains: is the bound ( √ d/ε ) α α -1 optimal regarding its dependence on d ? For smooth stochastic convex optimization problems using a ( d +1) -points stochastic zeroth-order oracle, the answer is negative. The optimal bound is proportional to dε -α α -1 . Consequently, for α ∈ (1 , 2) , our results are intriguing because the dependence on d in our bound differs from known results in the classical case where α = 2 .

1.2 Paper organization

The paper is organized as follows. In Section 2, we give some preliminaries, such as smoothing technique and gradient estimation, that are workhorse of our algorithms. Section 3 is the main section presenting two novel gradient-free algorithms along with their convergence results in high probability. These algorithms solve non-smooth stochastic optimization under heavy-tailed noise, and they will be reffered to as ZO-clipped-SSTM and R-ZO-clipped-SSTM (see Algorithms 1 and 2 respectively). In Section 4, we extend our results to gradient-free oracle corrupted by additive deterministic adversarial noise. In Section 5, we describe the main ideas behind the proof and emphasize key lemmas. In Section 6, we provide numerical experiments on the synthetic task that demonstrate the robustness of the proposed algorithms towards heavy-tailed noise.

2 Preliminaries

Assumptions on a subset. Although we consider an unconstrained optimization problem, our analysis does not require any assumptions to hold on the entire space. For our purposes, it is sufficient to introduce all assumptions only on some convex set Q ∈ R d since we prove that the considered methods do not leave some ball around the solution or some level-set of the objective function with high probability. This allows us to consider fairly large classes of problems.

Assumption 1 (Strong convexity) There exist a convex set Q ⊂ R d and µ ≥ 0 such that function f ( x, ξ ) is µ -strongly convex on Q for any fixed ξ , i.e.

for all x 1 , x 2 ∈ Q,λ ∈ [0 , 1] .

This assumption implies that f ( x ) is µ -strongly convex as well.

For a small constant τ > 0 , let us define an expansion of set Q namely Q τ = Q + B d 2 , where + stands for Minkowski addition. Using this expansion we make the following assumption.

Assumption 2 (Lipschitz continuity and boundedness of α -moment) There exist a convex set Q ⊂ R d and τ > 0 such that function f ( x, ξ ) is M 2 ( ξ ) -Lipschitz continuous w.r.t. the Euclidean norm on Q τ , i.e., for all x 1 , x 2 ∈ Q τ

Moreover, there exist α ∈ (1 , 2] and M 2 > 0 such that E ξ [ M 2 ( ξ ) α ] ≤ M α 2 .

If α < 2 , we say that noise is heavy-tailed . When α = 2 , the above assumption recovers the standard uniformly bounded variance assumption.

Lemma 1 Assumption 2 implies that f ( x ) is M 2 -Lipschitz on Q .

Randomized smoothing. The main scheme that allows us to develop batch-parallel gradient-free methods for non-smooth convex problems is randomized smoothing [13, 17, 27, 28, 38] of a nonsmooth function f ( x, ξ ) . The smooth approximation to a non-smooth function f ( x, ξ ) is defined as

where u ∼ U ( B d 2 ) is a random vector uniformly distributed on the Euclidean unit ball B d 2 def = { x ∈ R d : ∥ x ∥ 2 ≤ 1 } . In this approximation, a new type of randomization appears in addition to stochastic variable ξ .

The next lemma gives estimates for the quality of this approximation. In contrast to f ( x ) , function ˆ f τ ( x ) is smooth and has several useful properties.

Lemma 2 [17, Theorem 2.1.] Let there exist a subset Q ⊂ R d and τ > 0 such that Assumptions 1 and 2 hold on Q τ . Then,

Function ˆ f τ ( x ) is convex, Lipschitz with constant M 2 on Q , and satisfies

Function ˆ f τ ( x ) is differentiable on Q with the following gradient

where e ∼ U ( S d 2 ) is a random vector uniformly distributed on unit Eucledian sphere S d 2 def = { x ∈ R d : ∥ x ∥ 2 = 1 } .

Function ˆ f τ ( x ) is L -smooth with L = √ dM 2 / τ on Q .

Our algorithms will aim at minimizing the smooth approximation ˆ f τ ( x ) . Given Lemma 2, the output of the algorithm will also be a good approximate minimizer of f ( x ) when τ is sufficiently small.

Gradient estimation. Our algorithms will based on randomized gradient estimate, which will then be used in a first order algorithm. Following [36], the gradient can be estimated by the following vector:

where τ > 0 and e ∼ U ( S d 2 ) . We also use batching technique in order to allow parallel calculation of gradient estimation and acceleration. Let B be a batch size, we sample { e i } B i =1 and { ξ i } B i =1 independently, then

The next lemma states that g B ( x, { ξ } , { e } ) from (5) is an unbiased estimate of the gradient of ˆ f τ ( x ) (3). Moreover, under heavy-tailed noise (Assumption 2) with bounded α -moment g B ( x, { ξ } , { e } ) has also bounded α -moment.

Lemma 3 Under Assumptions 1 and 2, it holds

and

3 Main Results

In this section, we present two our new zero-order algorithms, which we will refer to as ZO-clippedSSTM and R-ZO-clipped-SSTM , to solve problem (1) under heavy-tailed noise assumption. To deal with heavy-tailed noise, we use clipping technique which clips heavy tails. Let λ > 0 be clipping constant and g ∈ R d , then clipping operator clip is defined as

̸

We apply clipping operator for batched gradient estimate g B ( x, { ξ } , { e } ) from (5) and then feed it into first-order Clipped Stochastic Similar Triangles Method ( clipped-SSTM ) from [18]. We will refer to our zero-order versions of clipped-SSTM as ZO-clipped-SSTM and R-ZO-clipped-SSTM for the convex case and strongly convex case respectively.

3.1 Convex case

Let us suppose that Assumption 1 is satisfied with µ = 0 .

Algorithm 1 ZO-clipped-SSTM ( x 0 , K, B, a, τ, { λ k } K -1 k =0 )

Input: starting point x 0 , number of iterations K , batch size B , stepsize a > 0 , smoothing parameter τ , clipping levels { λ k } K -1 k =0 . √

1: Set L = dM 2 / τ , A 0 = α 0 = 0 , y 0 = z 0 = x 0

2: for k = 0 , . . . , K -1 do

3: Set α k +1 = k +2 / 2 aL , A k +1 = A k + α k +1 .

4: x k +1 = A k y k + α k +1 z k A k +1 .

5: Sample { ξ k i } B i =1 ∼ D and { e k i } B i =1 ∼ S d 2 independently.

6: Compute gradient estimation g B ( x k +1 , { ξ k } , { e k } ) as defined in (5).

7: Compute clipped ˜ g k +1 = clip ( g B ( x k +1 , { ξ k } , { e k } ) , λ k ) as defined in (7).

8: z k +1 = z k -α k +1 ˜ g k +1 .

9: y k +1 = A k y k + α k +1 z k +1 A k +1 .

10: end for

Output:

y K

Theorem 1 (Convergence of ZO-clipped-SSTM ) Let Assumptions 1 and 2 hold with µ = 0 and Q = R d . Let ∥ x 0 -x ∗ ∥ 2 ≤ R 2 , where x 0 is a starting point and x ∗ is an optimal solution to (1) . Then for the output y K of ZO-clipped-SSTM , run with batchsize B , A = ln 4 K / β ≥ 1 , a = Θ(max { A 2 , √ dM 2 K ( α +1) α A ( α -1) α / LRB ( α -1) α } ) , τ = ε / 4 M 2 and λ k = Θ( R / ( α k +1 A ) ) , we can guarantee f ( y K ) -f ( x ∗ ) ≤ ε with probability at least 1 -β ( for any β ∈ (0 , 1] ) after

successive iterations and K · B oracle calls. Moreover, with probability at least 1 -β the iterates of ZO-clipped-SSTM remain in the ball B 2 R ( x ∗ ) , i.e., { x k } K +1 k =0 , { y k } K k =0 , { z k } K k =0 ⊆ B 2 R ( x ∗ ) .

Here and below notation ˜ O means an upper bound on the growth rate hiding logarithms. The first term in bound (8) is optimal for the deterministic case for non-smooth problem (see [6]) and the second term in bound (8) is optimal in ε for α ∈ (1 , 2] and zero-point oracle (see [27]).

We notice that increasing the batch size B to reduce the number of successive iterations makes sense only as long as the first term of (8) lower than the second one, i.e. there exists optimal value of batchsize

3.2 Strongly-convex case

Now we suppose that Assumption 1 is satisfied with µ > 0 . For this case we employ ZO-clippedSSTM with restarts technique. We will call this algorithm as R-ZO-clipped-SSTM (see Algorithm 2). At each round R-ZO-clipped-SSTM call ZO-clipped-SSTM with starting point ˆ x t , which is the output from the previous round, and for K t iterations.

Algorithm 2 R-ZO-clipped-SSTM

Input: starting point x 0 , number of restarts N , number of steps { K t } N t =1 , batchsizes { B t } N t =1 , stepsizes { a t } N t =1 , smoothing parameters { τ t } N t =1 , clipping levels { λ 1 k } K 1 -1 k =0 , ..., { λ N k } K N -1 k =0

Theorem 2 (Convergence of R-ZO-clipped-SSTM ) Let Assumptions 1, 2 hold with µ > 0 and Q = R d . Let ∥ x 0 -x ∗ ∥ 2 ≤ R 2 , where x 0 is a starting point and x ∗ is the optimal solution to (1) . Let also N = ⌈ log 2 ( µR 2 / 2 ε ) ⌉ be the number of restarts. Let at each stage t = 1 , ..., N of R-ZOclipped-SSTM , ZO-clipped-SSTM is run with batchsize B t , τ t = ε t / 4 M 2 , L t = M 2 √ d / τ t , K t = ˜ Θ(max { √ L t R 2 t -1 / ε t , ( σR t -1 / ε t ) α ( α -1) /B t } ) , a t = ˜ Θ(max { 1 , σK α +1 α t / L t R t } ) and λ t k = ˜ Θ( R / α t k +1 ) , where R t -1 = 2 -( t -1) 2 R , ε t = µR 2 t -1 / 4 , ln 4 NK t / β ≥ 1 , β ∈ (0 , 1] . Then to guarantee f (ˆ x N ) -f ( x ∗ ) ≤ ε with probability at least 1 -β , R-ZO-clipped-SSTM requires

oracle calls. Moreover, with probability at least 1 -β the iterates of R-ZO-clipped-SSTM at stage t = 1 , . . . , N stay in the ball B 2 R t -1 ( x ∗ ) .

The obtained complexity bound (see the proof in Appendix C.2) is the first optimal (up to logarithms) high-probability complexity bound under Assumption 2 for the smooth strongly convex problems. Indeed, the first term cannot be improved in view of the deterministic lower bound [27], and the second term is optimal [41].

4 Setting with Adversarial Noise

Often, black-box access of f ( x, ξ ) are affected by some deterministic noise δ ( x ) . Thus, now we suppose that a zero-order oracle instead of objective values f ( x, ξ ) returns its noisy approximation

This noise δ ( x ) can be interpreted, e.g., as a computer discretization when calculating f ( x, ξ ) . For our analysis, we need this noise to be uniformly bounded. Recently, noisy «black-box» optimization with bounded noise has been actively studied [11, 25]. The authors of [11] consider deterministic adversarial noise, while in [25] stochastic adversarial noise was explored.

Assumption 3 (Boundedness of noise) There exists a constant ∆ > 0 such that | δ ( x ) | ≤ ∆ for all x ∈ Q .

This is a standard assumption often used in the literature (e.g., [17]). Moreover, in some applications [4] the bigger the noise the cheaper the zero-order oracle. Thus, it is important to understand the maximum allowable level of adversarial noise at which the convergence of the gradient-free algorithm is unaffected.

4.1 Non-smooth setting

In noisy setup, gradient estimate from (4) is replaced by

Then (6) from Lemma 3 will have an extra factor driven by noise (see Lemma 2 . 3 from [22])

To guarantee the same convergence of the algorithm as in Theorem 1 (see (8)) for the adversarial deterministic noise case, the variance term must dominate the noise term, i.e. d ∆ τ -1 ≲ √ dM 2 . Note that if the term with noise dominates the term with variance, it does not mean that the gradient-free algorithm will not converge. In contrast, algorithm will still converge, only slower (oracle complexity will be ∼ ε -2 times higher). Thus, if we were considering the zero-order oracle concept with adversarial stochastic noise, it would be enough to express the noise level ∆ , and substitute the value of the smoothing parameter τ to obtain the maximum allowable noise level. However, since we are considering the concept of adversarial noise in a deterministic setting, following previous work [11, 1] we can say that adversarial noise accumulates not only in the variance, but also in the bias:

This bias can be controlled by the noise level ∆ , i.e., in order to achieve the ε -accuracy algorithm considered in this paper, the noise condition must be satisfied:

As we can see, we have a more restrictive condition on the noise level in the bias (12) than in the variance ( ∆ ≲ γM 2 / √ d ). Thus, the maximum allowable level of adversarial deterministic noise, which guarantees the same convergence of ZO-clipped-SSTM , as in Theorem 1 (see (8)) is as follows

where τ = ε / 2 M 2 the smoothing parameter from Lemma 2.

Remark 1 ( µ -strongly convex case) Let us assume that f ( x ) is also µ -strongly convex (see Assumption 1). Then, following the works [11, 22], we can conclude that the R-ZO-clipped-SSTM has the same oracle and iteration complexity as in Theorem 2 at the following maximum allowable level of adversarial noise: ∆ ≲ µ 1 / 2 ε 3 / 2 / √ dM 2 .

4.2 Smooth setting

Now we examine the maximum allowable level of noise at which we can solve optimization problem (1) with ε -precision under the following additional assumption

Assumption 4 (Smoothness) The function f is L -smooth, i.e., it is differentiable on Q and for all x, y ∈ Q with L > 0 :

If Assumption 4 holds, then Lemma 2 can be rewritten as

Thus, we can now present the convergence results of the gradient-free algorithm in the smooth setting. Specifically, if the Assumptions 2-4 are satisfied, then ZO-clipped-SSTM converges to ε -accuracy after K = ˜ O ( √ LR 2 ε -1 ) iterations with probability at least 1 -β . It is easy to see that the iteration complexity improves in the smooth setting (since the Lipschitz gradient constant L already exists,

i.e., no smoothing is needed), but oracle complexity remained unchanged (since we are still using the gradient approximation via l 2 randomization (11) instead of the true gradient ∇ f ( x ) ), consistent with the already optimal estimate on oracle calls: ˜ O ( ( √ dM 2 Rε -1 ) α α -1 ) . And to obtain the maximum allowable level of adversarial noise ∆ in the smooth setting, which guarantees such convergence, it is sufficient to substitute the smoothing parameter τ = √ ε/L in the inequality (12):

Thus, we can conclude that smooth setting improves iteration complexity and the maximum allowable noise level for the gradient-free algorithm, but the oracle complexity remains unchanged.

Remark 2 ( µ -strongly convex case) Suppose that f ( x ) is also µ -strongly convex (see Assumption 1). Then we can conclude that R-ZO-clipped-SSTM has the oracle and iteration complexity just mentioned above at the following maximum allowable level of adversarial noise: ∆ ≲ µ 1 / 2 ε / √ dL .

Remark 3 (Upper bounds optimality) The upper bounds on maximum allowable level of adversarial noise obtained in this section in both non-smooth and smooth settings are optimal in terms of dependencies on ε and d according to the works [32, 34].

Remark 4 (Better oracle complexity) In the aforementioned approach in the case when f ( x, ξ ) has Lipschitz gradient in x (for all ξ ) one can improve oracle complexity from ˜ O ( ( √ dM 2 Rε -1 ) α α -1 ) to ˜ O ( d ( M 2 Rε -1 ) α α -1 ) . This can be done by using component-wise finite-difference stochastic gradient approximation [15]. Iteration complexity remains ˜ O ( √ LR 2 ε -1 ) . The same can be done for µ -strongly convex case: from ˜ O ( ( dM 2 2 ( µε ) -1 ) α 2( α -1) ) to ˜ O ( d ( M 2 2 ( µε ) -1 ) α 2( α -1) ) .

5 Details of the proof

The proof is built upon a combination of two techniques. The first one is the Smoothing technique from [17] that is used to develop a gradient-free method for convex non-smooth problems based on full-gradient methods. The second technique is the Accelerated Clipping technique that has been recently developed for smooth problems with the noise having infinite variance and first-order oracle [35]. The authors of [35] propose clipped-SSTM method that we develop in our paper. We modify clipped-SSTM by introducing batching into it. Note that due to the infinite variance, such a modification is interesting in itself. Then we run batched clipped-SSTM with gradient estimations of function f obtained via Smoothing technique and two-point zeroth-order oracle. To do this, we need to estimate the variance of the clipped version of the batched gradient estimation.

In more detail, we replace the initial problem of minimizing f by minimizing its smoothed approximation ˆ f τ , see Lemma 2 In order to use estimated gradient of ˆ f τ defined in (4) or (5), we make sure that it has bounded α -th moment. For these purposes we proof Lemma 3. First part shows boundness of unbatched estimated gradient g defined in (4). It follows from measure concentration phenomenon for the Euclidean sphere for M 2 τ -Lipschitz function f ( x + e τ ) . According to this phenomenon probability of the functions deviation from its math expectation becomes exponentially small and α -th moment of this deviation becomes bounded. Furthermore, the second part of Lemma 3 shows that batching helps to bound α -th moment of batched gradient g B defined in (5) even more. Also batching allows parallel calculations reducing number of necessary iteration with the same number of oracle calls. All this is possible thanks to the result, interesting in itself, presented in the following Lemma.

Lemma 4 Let X 1 , . . . , X B be d -dimensional martingale difference sequence (i.e. E [ X i | X i -1 , . . . , X 1 ] = 0 for 1 < i ≤ B ) satisfying for 1 ≤ α ≤ 2 E [ ∥ X i ∥ α | X i -1 , . . . , X 1 ] ≤ σ α .

Then we have

Next, we use the clipped-SSTM for function ˆ f τ with heavy-tailed gradient estimates. This algorithm was initially proposed for smooth functions in the work [35]. The scheme for proving convergence with high probability is also taken from it, the only difference is additional randomization caused by smoothing scheme.

6 Numerical experiments

We tested ZO-clipped-SSTM on the following problem

where ξ is a random vector with independent components sampled from the symmetric Levy α -stable distribution with α = 3 / 2 , A ∈ R m × d , b ∈ R m (we used d = 16 and m = 500 ). For this problem, Assumption 1 holds with µ = 0 and Assumption 2 holds with α = 3 / 2 and M 2 ( ξ ) = ∥ A ∥ 2 + ∥ ξ ∥ 2 .

We compared ZO-clipped-SSTM , proposed in this paper, with ZOSGD and ZO-SSTM . For these algorithms, we gridsearched batchsize B : 5 , 10 , 50 , 100 , 500 and stepsize γ : 1 e -3 , 1 e -4 , 1 e -5 , 1 e -6 . The best convergence was achieved with the following parameters:

ZO-clipped-SSTM : γ = 1 e -3 , B = 10 , λ = 0 . 01 ,

ZO-SSTM : γ = 1 e -5 , B = 500 ,

ZO-SGD : γ = 1 e -4 , B = 100 , ω = 0 . 9 , where ω is a heavy-ball momentum parameter.

ZO-clipped-SSTM

ZO-SSTM

ZO-SGD

101

（x）-（x）

100

10-1

0

1000000

2000000

3000000

4000000

5000000

samples

The code is written on Pythone and is publicly available at https://github.com/ClippedStochasticMethods/ZO-clipped-SSTM . Figure 1 presents the comparison of convergences averaged over 15 launches with different noise. In contrast to ZO-clipped-SSTM , the last two methods are unclipped and therefore failed to converge under haivy-tailed noise.

7 Conclusion and future directions

In this paper, we propose a first gradient-free algorithm ZO-clipped-SSTM and to solve problem (1) under heavy-tailed noise assumption. By using the restart technique we extend this algorithm for strongly convex objective, we refer to this algorithm as R-ZO-clipped-SSTM . The proposed algorithms are optimal with respect to oracle complexity (in terms of the dependence on ε ), iteration complexity and the maximal level of noise (possibly adversarial). The algorithms can be adapted to composite and distributed minimization problems, saddle-point problems, and variational inequalities. Despite the fact that the algorithms utilize the two-point feedback, they can be modified to the one-point feedback. We leave it for future work.

Moreover, we provide theoretical basis to demonstrate benefits of batching technique in case of heavy-tailed stochastic noise and apply it to methods from this paper. Thanks to this basis, it is possible to use batching in other methods with heavy-tiled noise, e.g. first-order methods presented in [35, 29].

8 Acknowledgment

The work of Alexander Gasnikov, Aleksandr Lobanov, and Darina Dvinskikh was supported by a grant for research centers in the field of artificial intelligence, provided by the Analytical Center for the Government of the Russian Federation in accordance with the subsidy agreement (agreement identifier 000000D730321P5Q0002) and the agreement with the Ivannikov Institute for System Programming of dated November 2, 2021 No. 70-2021-00142.

References

BA Alashkar, Alexander Vladimirovich Gasnikov, Darina Mikhailovna Dvinskikh, and Aleksandr Vladimirovich Lobanov. Gradient-free federated learning methods with l_1 and l_2-randomization for non-smooth convex stochastic optimization problems. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki , 63(9):1458-1512, 2023.

Peter Bartlett, Varsha Dani, Thomas Hayes, Sham Kakade, Alexander Rakhlin, and Ambuj Tewari. High-probability regret bounds for bandit online linear optimization. In Proceedings of the 21st Annual Conference on Learning Theory-COLT 2008 , pages 335-342. Omnipress, 2008.

Anastasia Sergeevna Bayandina, Alexander V Gasnikov, and Anastasia A Lagunovskaya. Gradient-free two-point methods for solving stochastic nonsmooth convex optimization problems with small non-random noises. Automation and Remote Control , 79:1399-1408, 2018.

Lev Bogolubsky, Pavel Dvurechenskii, Alexander Gasnikov, Gleb Gusev, Yurii Nesterov, Andrei M Raigorodskii, Aleksey Tikhonov, and Maksim Zhukovskii. Learning supervised pagerank with gradient-based and gradient-free optimization methods. Advances in neural information processing systems , 29, 2016.

Sébastien Bubeck and Nicoló Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning , 5(1):1-122, 2012.

Sébastien Bubeck, Qijia Jiang, Yin-Tat Lee, Yuanzhi Li, and Aaron Sidford. Complexity of highly parallel non-smooth convex optimization. Advances in neural information processing systems , 32, 2019.

Yeshwanth Cherapanamjeri, Nilesh Tripuraneni, Peter Bartlett, and Michael Jordan. Optimal mean estimation without a variance. In Conference on Learning Theory , pages 356-357. PMLR, 2022.

Damek Davis, Dmitriy Drusvyatskiy, Lin Xiao, and Junyu Zhang. From low probability to high confidence in stochastic convex optimization. Journal of Machine Learning Research , 22(49):1-38, 2021.

Yuriy Dorn, Kornilov Nikita, Nikolay Kutuzov, Alexander Nazin, Eduard Gorbunov, and Alexander Gasnikov. Implicitly normalized forecaster with clipping for linear and non-linear heavy-tailed multi-armed bandits. arXiv preprint arXiv:2305.06743 , 2023.

John C Duchi, Michael I Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory , 61(5):2788-2806, 2015.

Darina Dvinskikh, Vladislav Tominin, Iaroslav Tominin, and Alexander Gasnikov. Noisy zeroth-order optimization for non-smooth saddle point problems. In Mathematical Optimization Theory and Operations Research: 21st International Conference, MOTOR 2022, Petrozavodsk, Russia, July 2-6, 2022, Proceedings , pages 18-33. Springer, 2022.

Pavel Dvurechenskii, Darina Dvinskikh, Alexander Gasnikov, Cesar Uribe, and Angelia Nedich. Decentralize and randomize: Faster algorithm for wasserstein barycenters. Advances in Neural Information Processing Systems , 31, 2018.

Yu Ermoliev. Stochastic programming methods, 1976.

Abraham D Flaxman, Adam Tauman Kalai, and H Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 , 2004.

Alexander Gasnikov, Darina Dvinskikh, Pavel Dvurechensky, Eduard Gorbunov, Aleksander Beznosikov, and Alexander Lobanov. Randomized gradient-free methods in convex optimization. arXiv preprint arXiv:2211.13566 , 2022.

Alexander Gasnikov and Yurii Nesterov. Universal fast gradient method for stochastic composit optimization problems. arXiv preprint arXiv:1604.05275 , 2016.

Alexander Gasnikov, Anton Novitskii, Vasilii Novitskii, Farshed Abdukhakimov, Dmitry Kamzolov, Aleksandr Beznosikov, Martin Takac, Pavel Dvurechensky, and Bin Gu. The power of first-order smooth optimization for black-box non-smooth problems. In International Conference on Machine Learning , pages 7241-7265. PMLR, 2022.

Eduard Gorbunov, Marina Danilova, and Alexander Gasnikov. Stochastic optimization with heavy-tailed noise via accelerated gradient clipping. Advances in Neural Information Processing Systems , 33:15042-15053, 2020.

Eduard Gorbunov, Marina Danilova, Innokentiy Shibaev, Pavel Dvurechensky, and Alexander Gasnikov. Near-optimal high probability complexity bounds for non-smooth stochastic optimization with heavy-tailed noise. arXiv preprint arXiv:2106.05958 , 2021.

Mert Gurbuzbalaban and Yuanhan Hu. Fractional moment-preserving initialization schemes for training deep neural networks. In International Conference on Artificial Intelligence and Statistics , pages 2233-2241. PMLR, 2021.

Dusan Jakovetic, Dragana Bajovic, Anit Kumar Sahu, Soummya Kar, Nemanja Milosevich, and Dusan Stamenkovic. Nonlinear gradient mappings and stochastic optimization: A general framework with applications to heavy-tail noise. SIAM Journal on Optimization , 33(2):394-423, 2023.

Nikita Kornilov, Alexander Gasnikov, Pavel Dvurechensky, and Darina Dvinskikh. Gradientfree methods for non-smooth convex stochastic optimization with heavy-tailed noise on convex compact. Computational Management Science , 20(1):37, 2023.

Michel Ledoux. The concentration of measure phenomenon. ed. by peter landweber et al. vol. 89. Mathematical Surveys and Monographs. Providence, Rhode Island: American Mathematical Society , page 181, 2005.

Zijian Liu and Zhengyuan Zhou. Stochastic nonsmooth convex optimization with heavy-tailed noises. arXiv preprint arXiv:2303.12277 , 2023.

Aleksandr Lobanov. Stochastic adversarial noise in the" black box" optimization problem. arXiv preprint arXiv:2304.07861 , 2023.

Aleksandr Viktorovich Nazin, AS Nemirovsky, Aleksandr Borisovich Tsybakov, and AB Juditsky. Algorithms of robust stochastic optimization based on mirror descent method. Automation and Remote Control , 80(9):1607-1627, 2019.

Arkadij Semenoviˇ c Nemirovskij and David Borisovich Yudin. Problem complexity and method efficiency in optimization. M.: Nauka , 1983.

Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics , 17:527-566, 2017.

Ta Duy Nguyen, Alina Ene, and Huy L Nguyen. Improved convergence in high probability of clipped gradient methods with heavy tails. arXiv preprint arXiv:2304.01119 , 2023.

Ta Duy Nguyen, Thien Hang Nguyen, Alina Ene, and Huy Le Nguyen. High probability convergence of clipped-sgd under heavy-tailed noise. arXiv preprint arXiv:2302.05437 , 2023.

Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International conference on machine learning , pages 1310-1318, 2013.

Dmitry A Pasechnyuk, Aleksandr Lobanov, and Alexander Gasnikov. Upper bounds on maximum admissible noise in zeroth-order optimisation. arXiv preprint arXiv:2306.16371 , 2023.

Svetlozar Todorov Rachev. Handbook of heavy tailed distributions in finance: Handbooks in finance, Book 1 . Elsevier, 2003.

Andrej Risteski and Yuanzhi Li. Algorithms and matching lower bounds for approximatelyconvex optimization. Advances in Neural Information Processing Systems , 29, 2016.

Abdurakhmon Sadiev, Marina Danilova, Eduard Gorbunov, Samuel Horváth, Gauthier Gidel, Pavel Dvurechensky, Alexander Gasnikov, and Peter Richtárik. High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 , 2023.

Ohad Shamir. An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research , 18(1):1703-1713, 2017.

Umut Simsekli, Levent Sagun, and Mert Gurbuzbalaban. A tail-index analysis of stochastic gradient noise in deep neural networks. In International Conference on Machine Learning , pages 5827-5837. PMLR, 2019.

James C Spall. Introduction to stochastic search and optimization: estimation, simulation, and control . John Wiley & Sons, 2005.

Nuri Mert Vural, Lu Yu, Krishna Balasubramanian, Stanislav Volgushev, and Murat A Erdogdu. Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In Conference on Learning Theory , pages 65-102. PMLR, 2022.

Hongjian Wang, Mert Gurbuzbalaban, Lingjiong Zhu, Umut Simsekli, and Murat A Erdogdu. Convergence rates of stochastic gradient descent under infinite noise variance. Advances in Neural Information Processing Systems , 34:18866-18877, 2021.

Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J Reddi, Sanjiv Kumar, and Suvrit Sra. Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems , 33, 2020.

Jiujia Zhang and Ashok Cutkosky. Parameter-free regret in high probability with heavy tails. Advances in Neural Information Processing Systems , 35:8000-8012, 2022.

A Batching with unbounded variance

To prove batching Lemma 4 we generalize Lemma 4 . 2 from [7] not only for i.i.d. random variables with zero mean but for martingale difference sequence.

̸

Proof. Consider l ( x ) = g ( x + h ) -g ( x ) . We see that l is differentiable everywhere except x = 0 and x = -h . As long as x = 0 , -h , we have

Since, we have α > 1 , x = -h 2 is local maxima for l ( x ) . Furthermore, note that l ′ ( x ) ≥ 0 for x ∈ ( -∞ , -h 2 ) \ {-h } and l ′ ( x ) ≤ 0 for x ∈ ( -h 2 , ∞ ) \ { 0 } . Therefore, -h 2 is global maxima.

Lemma 6 Let x 1 , . . . , x B be one-dimensional martingale difference sequence, i.e. E [ x i | x i -1 , . . . , x 1 ] = 0 for 1 < i ≤ B , satisfying for 1 ≤ α ≤ 2

We have:

Proof. Everywhere below we will use the following notations.

For α = 1 proof follows from triangle inequality for | · | . When α > 1 , we start by defining

Then we can calculate desired expectation as

While { x i } is martingale difference sequence then E [ x i f ′ ( S i -1 )] = E <i [ E | <i [ x i f ′ ( S i -1 )]] = 0 . From (13) and Lemma 5 ( g ( x ) = f ′ ( x ) /α ) we obtain

Now we are ready to prove batching lemma for random variables with infinite variance.

Lemma 7 Let X 1 , . . . , X B be d -dimensional martingale difference sequence, i.e., E [ X i | X i -1 , . . . , X 1 ] = 0 for 1 < i ≤ B , satisfying for 1 ≤ α ≤ 2

We have

Proof.

Let g ∼ N (0 , I ) and y i def = X ⊤ i g . Firstly, we prove that E [ y α i ] ≤ E [ ∥ X i ∥ α ] . Indeed, using conditional expectation we get

Next, considering X ⊤ i g ∼ N (0 , ∥ X ∥ 2 ) and, thus, E g | X ⊤ i g | = ∥ X ∥ , we bound desired expectation as

Finally, we apply Lemma 6 for y i sequence with bounded α -th moment from (15) and get

B Smoothing scheme

Lemma 8 Assumption 2 implies that f ( x ) is M 2 Lipschitz on Q .

Proof. For all x 1 , x 2 ∈ Q

The following lemma gives some facts about the measure concentration on the Euclidean unit sphere for next proof.

Lemma 9 Let f ( x ) be M 2 Lipschitz continuous function w.r.t ∥ · ∥ . If e is random and uniformly distributed on the Euclidean sphere and α ∈ (1 , 2] , then

Proof. A standard result of the measure concentration on the Euclidean unit sphere implies that ∀ t > 0

(see the proof of Proposition 2.10 and Corollary 2.6 in [23]). Therefore,

Finally, below we prove Lemma 3 which states that batched gradient estimation from (5) has bounded α -th moment.

Proof of Lemma 3.

We will prove equality immediately for g B . Firstly, we notice that distribution of e is symmetric and by definition (5) we get

Using ∇ ˆ f τ ( x ) = d τ E e [ f ( x + τ e ) e ] from Lemma 2 we obtain necessary result.

By definition (4) of estimated gradient g we bound α -th moment as

Considering ∥ e ∥ 2 = 1 we can omit it. Next we add ± δ ( ξ ) in (23) for all δ ( ξ ) and get

Using Jensen's inequality for | · | α we bound

We note that distribution of e is symmetric and add two terms together

Let δ ( ξ ) = E e [ f ( x + τ e , ξ )] , then because of Cauchy-Schwartz inequality and conditional expectation properties we obtain

Next, we use Lemma 9 for f ( x + τ e , ξ ) with fixed ξ and Lipschitz constant M 2 ( ξ ) τ

Finally, we get desired bound of estimated gradient

Now we apply Jensen inequality to

And get necessary result.

For batched gradient g B we use Batching Lemma 7 and estimation (25).

C Missing Proofs for ZO-clipped-SSTM and R-ZO-clipped-SSTM

In this section, we provide the complete formulation of the main results for clipped-SSTM and R-clipped-SSTM and the missing proofs.

Minimization on the subset Q In order to work with subsets of Q ⊆ R d we must assume one more condition on ˆ f τ ( x ) .

Assumption 5 We assume that there exist some convex set Q ⊆ R d , constants τ, L > 0 such that for all x, y ∈ Q

When Q = R d (26) follows from Lemma 2 as well. But in general case this is not true. In work [35] in Section 'Useful facts' authors show that, in the worst case, to have (26) on a set Q one needs to assume smoothness on a slightly larger set.

Thus, in the full version of the theorems in which we can require much smaller Q , we will also require satisfying all three Assumptions 1, 2, 5.

C.1 Convex Functions

We start with the following lemma, which is a special case of Lemma 6 from [19]. This result can be seen the 'optimization' part of the analysis of clipped-SSTM : the proof follows the same steps as the analysis of deterministic Similar Triangles Method [16], [12] and separates stochasticity from the deterministic part of the method.

Pay attention that in full version of Theorem 1 we require Assumptions 1, 2 to hold only on Q = B 3 R ( x ∗ ) , however we need to require one more smoothness Assumption 5 as well.

Theorem 3 (Full version of Theorem 1) Let Assumptions 1,2, 5 with µ = 0 hold on Q = B 3 R ( x ∗ ) , where R ≥ ∥ x 0 -x ∗ ∥ , and

for some K > 0 and β ∈ (0 , 1] such that ln 4 K β ≥ 1 . Then, after K iterations of ZO-clipped-SSTM the iterates with probability at least 1 -β satisfy

In particular, when parameter a equals the maximum from (27) , then the iterates produced by ZO-clipped-SSTM after K iterations with probability at least 1 -β satisfy

meaning that to achieve f ( y K ) -f ( x ∗ ) ≤ ε with probability at least 1 -β with τ = ε 4 M 2 ZOclipped-SSTM requires

In case when second term in max in (31) is greater total number of oracle calls is

Proof. The proof is based on the proof of Theorem F.2 from [35]. We apply first-order algorithm clipped-SSTM for M 2 √ d τ -smooth function ˆ f τ with unbiased gradient estimation g B , that has α -th moment bounded with 2 σ α B α -1 . Additional randomization caused by smoothing doesn't affect the proof of the original Theorem.

According to it after K iterations we have that with probability at least 1 -β

and { x k } K +1 k =0 , { z k } K k =0 , { y k } K k =0 ⊆ B 2 R ( x ∗ ) .

Considering approximation properties of ˆ f τ from Lemma 2

Finally, if

then with probability at least 1 -β

where L = M 2 √ d τ by Lemma 2.

To get f ( y K ) -f ( x ∗ ) ≤ ε with probability at least 1 -β it is sufficient to choose τ = ε 4 M 2 and K such that both terms in the maximum above are O ( ε ) . This leads to

that concludes the proof.

C.2 Strongly Convex Functions

In the strongly convex case, we consider the restarted version of ZO-clipped-SSTM ( R-ZO-clippedSSTM ). The main result is summarized below.

Pay attention that in full version of Theorem 2 we require Assumptions 1, 2 to hold only on Q = B 3 R ( x ∗ ) , however we need to require one more smoothness Assumption 5 as well.

Theorem 4 (Full version of Theorem 2) Let Assumptions 1, 2, 5 with µ > 0 hold for Q = B 3 R ( x ∗ ) , where R ≥ ∥ x 0 -x ∗ ∥ 2 and R-ZO-clipped-SSTM runs ZO-clipped-SSTM N times. Let

for t = 1 , . . . , τ . Then to guarantee f (ˆ x τ ) -f ( x ∗ ) ≤ ε with probability ≥ 1 -β R-clipped-SSTM requires

oracle calls. Moreover, with probability ≥ 1 -β the iterates of R-clipped-SSTM at stage t stay in the ball B 2 R t -1 ( x ∗ ) .

Proof. The proof itself repeats of the proof Theorem F.3 from [35]. In this theorem authors prove convergence of restarted clipped-SSTM . In our case it is sufficient to change clipped-SSTM to ZO- clipped-SSTM and put results of Theorem 3 in order to guarantee ε -solution after N ∑ t =1 K t successive iterations.

It remains to calculate the overall number of oracle calls during all runs of clipped-SSTM . We have

which concludes the proof.

Peer reviews:
comments: SUMMARY:
This paper proposes a novel gradient-free (zeroth-order) clipped version of stochastic similar triangles method for solving non-smooth stochastic convex optimization problem under a much weaker infinite variance assumption. The derived iteration and oracle complexity bounds are optimal in both convex and strongly convex case.

STRENGTHS:
The problem addressed in this paper is well-motivated, the write-up is easy to follow, and the novelty and contribution are easy to identify.

WEAKNESSES:
1. While the paper is a valuable theoretical contribution, the addition of experimental results would enhance the overall work by demonstrating the feasibility and effectiveness of this method in practical applications.

2. The absence of a Conclusion section or any definitive end to the work makes it feel like a work in progress.

QUESTIONS:
Line 118: Should the upper bound of the second inequality be a notation $\sigma_B^{\alpha}$?

LIMITATIONS:
None
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this papers, the authors build upon the work that has been done in [28] and adjust the proposed algorithms there for zero-order oracles rather than the gradient oracles. The goal is to optimize non-smooth stochastic convex optimization problems with infinite variance.\n\nSTRENGTHS:\nThe paper does a good job when it comes to introducing the notions they have used with clarity. The organization of the paper helps the reader to understand the concepts. The quality of the write up and the technical contributions look solid.\n\nWEAKNESSES:\n- Minor typo in the abstract: ajust --> adjust.\n\n- \"We emphasis (--> emphasize) that this generalization requires an extension of the batching technique to (--> for) infinite variance.\" The emphasis should be emphasize in this sentence and I believe for is more suitable than to.\n\n- The motivation about why we should be interested in such problems is only explained in citation numbers ([30, 6]) with no mention of what these motivating examples actually are. You have 9 pages of space without the references and you are not utilizing all of it. I suggest using the remaining space to attract more attention to your paper by mentioning concrete examples. This would help with the visibility of your paper as well.\n\n- \"• can be generalized for saddle-point problems (based on [28]) and one-point feedback [13]. We leave it for future work.\" This should not been mentioned in the contributions since this paper has not actually made this contributions yet. You can add a conclusion and/or future work paragraph or section in the end mention it there.\n\n- Similarly, related work could have a section of its own.\n\n- My main concern about this paper is that, even though adjusting the method from [28] for two-point zero-order oracle is nontrivial, it feels a bit too incremental for a venue like NeurIPS. Also, the trade-off between using weaker or stronger assumptions can be discussed further in order to convince the reader on the advantages and the disadvantages of the zeroth-order methods and where to use which.\n\nQUESTIONS:\n- In which cases we should utilize the proposed accelaerated zeroth-order method for non-smooth stochastic convex optimization problems with infinite variance?\n\n- It is not very clear to me why the paper considers the case when \\delta = 0 on page 6. Could you please elaborate more on that?\n\n- Could you please list some instances where non-smooth stochastic convex optimization problems show up and why we should be interested in them?\n\nLIMITATIONS:\nThe advantages and the disadvantages of the proposed method is not discussed in detail except for the advantage of enabling weaker assumptions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper proposed and analyzed a zeroth-order method for non-smooth stochastic optimization under heavy-tailed noise and adversarial noise, by combining ball-averaging-based smoothing technique (to tackle non-smoothness) and gradient clipping technique (to tackle heavy-tailed/adversarial noise). This generalizes previous results where $L_\infty$ or $L_2$ boundness of noise is assumed. There are also several technical improvements over previous results, including proving a high-probability bound instead of a bound in expectation.

STRENGTHS:
1. The topic is important. Gradient free method for stochastic optimization is a popular field of research, and it certainly helps to handle the heavy-tailed/adversarial noisy setting which is common in practice. 
2. I have briefly gone through the proof and have no doubt on its correctness. 
3. Though not proved in the paper, the theoretical bound seems close to being tight.

WEAKNESSES:
1. The presentation is concise, but maybe at the cost of some necessary clarity. In Eqn. (2) it's not specified what distribution should $\bf e$ follow (should be the uniform distribution in the unit _ball_), which leaves the entire equation undefined. It becomes even more confusing that later in (3) $\bf e$ is used with a different meaning, denoting a random vector uniformly distributed on the unit _sphere_!
2. It may also improve clarity to discuss the order of the theoretical bounds and compare them with previous results. In particular, it would be helpful to discuss whether the dependence of $\alpha$ is optimal.

QUESTIONS:
I have no additional questions other than the ones discussed in `Weaknesses`.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-07

comments: SUMMARY:
This paper presented derivative-free methods for the optimization of stochastic convex functions with a potentially infinite variance noise. Here the level of noise is defined in terms of the boundedness of modulus of a Hölder-type continuous condition. The main technique is to adopt a gradient clipping to the two-point estimation of the gradient of the randomized smoothed function. For some of their results, they also claim the attained bounds are rate optimal. The presentation and organization is of very clear and relatively easy to follow.

STRENGTHS:
Overall the contribution is well-motivated, and fits into the flurry of recent development of methods for problem with infinite noise variance. The paper is well written, and the proofs are intuitive and relatively easy to follow.

WEAKNESSES:
I only have some minor comments and questions:

* L113: it might be better to use {\xi_i}_i and {e_i}_i as the input of the function g^B.
* L119: where is the \sigma_B used in the statement of Lemma 3?
* the second line of Section 2: you might need to assume g \neq 0.
* L126: where is the distribution D_k defined?
* L139: could you elaborate on why the first term is optimal using the lower bound from [4]? Do you need some assumption on batch-size B here to illustrate rate optimality?
* L152: it might be better to emphasize that w.p. 1-\beta, *for any* 1 \leq t \leq N, ..., if the boundedness of iteration holds uniformly.
* Question (out of curiosity): is the bound in L190 optimal? Or, is there any lower bound for such a so-called maximum allowable noise level?
* L224: proof -> prove.

QUESTIONS:
See above.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides high probability bounds for the convergence of gradient-free methods on convex and strongly-convex functions when the noise in the gradient oracle has infinite variance. An oracle provides $f(x,\\xi)$, a noisy evaluation of the function $f$ at point $x\\in \\mathbb{R}^d$ by the oracle, where $\\xi$ is the noise variable. For the same noise $\\xi$, the function is $M_2(\\xi)$-Lipschitz, where $M_2(\\xi)$ quantifies the noise level.\n\nIf the noise variance is finite, ($\\mathbb{E}[M_2(\\xi)^2] < \\infty$), [1] provides optimal iteration and oracle complexity for convergence of in expectation. The primary technique used in [1] is a batched accelerated gradient method which uses smoothing. For a fixed constant $\\tau>0$ which defines the smoothing level, smoothing computes an approximate gradient of $f$ at point $x\\in \\mathbb{R}^d$ as, \n\n$$\ng^B(x) = \\frac{d}{2B \\tau }\\sum_{i=1}^B (f(x + \\tau e_i, \\xi_i) - f(x - \\tau e_i, \\xi_i))e_i\n$$\nHere, $e_i$ are sampled uniformly from the unit sphere, $\\xi_i$ are the noise variables of the oracle and $B$ is the batch size. On expectation, the smoothed gradient is the gradient of $ \\mathbb{E}_{e,\\xi}[f(x + \\tau e, \\xi)] $. For small value of $\\tau$, this approximation is close to $f$. Further, even if the function $f$ is non-smooth but Lipschitz, smoothing makes  $\\frac{\\sqrt{d}M_2}{\\tau}$ smooth.\n\n\n\nThis paper extends this technique to the infinite noise variance setting, $\\mathbb{E}[M_2(\\xi)^\\alpha] < M_2^\\alpha$ for some $\\alpha \\in (1,2]$ by applying clipping. Specifically, the technique of clipped Stochastic Similar Triangles used for handling heavy tailed noise in smooth optimization is extended to the non-smooth case by the above smoothing procedure.\n\nFor convex Lipschitz functions, the iteration complexity and oracle complexity is $\\frac{\\sqrt{d}^{1/4}}{\\epsilon}$ and $\\left(\\frac{\\sqrt{d}}{\\epsilon}\\right)^\\frac{\\alpha}{\\alpha-1}$. For $\\mu$-strongly convex and lipshcitz functions, the corresponding bounds are $\\frac{d^{1/4}}{(\\mu\\epsilon)^{-1/2}}$ and $\\left(\\frac{d}{\\mu\\epsilon}\\right)^{\\frac{\\alpha}{\\alpha-1}}$. These rates are shown to be optimal in $\\epsilon$.\n\n\nFurther, when if the oracle provides a corrupted value of $f$ with an additive corruption of $\\lvert\\delta(x)\\rvert \\leq \\Delta$, the authors derive the maximum possible values of $\\Delta$ such that the convergence rates for both smooth and non-smooth settings are unaffected by the corruption.\n\n\n**References**\n1. Gasnikov et al. The power of first-order smooth optimization for black-box non-smooth problems. ICML 2022.\n2. Sadiev et al. High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance. ICML 2023.\n\nSTRENGTHS:\n- **Interesting Problem Setting**:  Heavy-tailed noise is a significant problem which violates the commonly used bounded variance assumption in stochastic optimization.  The authors extend the solution of clipping to handle it for derivative-free methods.\n- **Convergence Rates** : These are the first convergence rates for derivate-free optimization under heavy-tailed noise. Further, the rates are high probability bounds instead of in expectation. Additionally, for both cases, convex and strongly convex,  the obtained rates are optimal in terms of error $\\epsilon$.\n- **Thorough literature review** : The authors thoroughly review existing results in derivative-free methods and clipping.\n\nWEAKNESSES:\n- **Presentation** : The paper seems to be missing an introduction and experiment section. Although the paper is theoretical, the proposed algorithm, clipped-SSTM with two-point feedback, is new and should have been tested on at least synthetic problems.\n\n- **Lack of a motivating example** : The authors do not provide a motivating example which justifies the heavy-tailed noise in gradient-free settings.\n\nQUESTIONS:\n- Are there any lower bounds for oracle and iteration complexity in the adversarial corruption case in terms of $\\Delta$?\n- Are there methods other than clipping to handle unbounded variance in stochastic optimization?\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}