Title: Stochastic Zeroth-Order Optimization under Strongly Convexity and Lipschitz Hessian: Minimax Sample Complexity
Venue: NeurIPS 2024
Accepted: yes
Keywords: Stochastic Optimization

Sample Complexity

Optimality
Abstract: Optimization of convex functions under stochastic zeroth-order feedback has been a major and challenging question in online learning. In this work, we consider the problem of optimizing second-order smooth and strongly convex functions where the algorithm is only accessible to noisy evaluations of the objective function it queries. 
We provide the first tight characterization for the rate of the minimax simple regret by developing matching upper and lower bounds. 
We propose an algorithm that features a combination of a bootstrapping stage and a mirror-descent stage. 
Our main technical innovation consists of a sharp characterization for the spherical-sampling gradient estimator under higher-order smoothness conditions, which allows the algorithm to optimally balance the bias-variance tradeoff, 
and a new iterative method for the bootstrapping stage, which maintains the performance for unbounded Hessian.

Stochastic Zeroth-Order Optimization under Strongly Convexity and Lipschitz Hessian: Minimax Sample Complexity

Abstract

Optimization of convex functions under stochastic zeroth-order feedback has been a major and challenging question in online learning. In this work, we consider the problem of optimizing second-order smooth and strongly convex functions where the algorithm is only accessible to noisy evaluations of the objective function it queries. We provide the first tight characterization for the rate of the minimax simple regret by developing matching upper and lower bounds. We propose an algorithm that features a combination of a bootstrapping stage and a mirror-descent stage. Our main technical innovation consists of a sharp characterization for the spherical-sampling gradient estimator under higher-order smoothness conditions, which allows the algorithm to optimally balance the bias-variance tradeoff, and a new iterative method for the bootstrapping stage, which maintains the performance for unbounded Hessian.

1 Introduction

Stochastic optimization of an unknown function with access to only noisy function evaluations is a fundamental problem in operations research, optimization, simulation and bandit optimization research, commonly known as zeroth-order optimization (Chen et al., 2017), derivative-free optimization (Conn et al., 2009; Rios & Sahinidis, 2013) or bandit optimization (Bubeck et al., 2021). In this problem, an optimization algorithm interacts sequentially with an oracle and obtains noisy function evaluations at queried points every time. The algorithm produces an approximately optimal solution after T such evaluations, with its performance evaluated by the expected difference between the function values at the approximate optimal solution produced and the optimal solution. A more rigorous formulation of the problem is given in Sec. 2 below.

Existing works and results on stochastic zeroth-order optimization could be broadly categorized into two classes:

Convex functions . In the first thread of research, the unknown objective function to be optimized is assumed to be concave (for maximization problems) or convex (for minimization problems). For these problems, with minimal smoothness (e.g. objective function being Lipschitz continuous) it is possible to achieve a sample complexity of ˜ O ( ε -2 ) for an expected optimization error or ε , which is also a polynomial function of domain dimension

d ; see for example the works of Agarwal et al. (2013); Lattimore & Gyorgy (2021); Bubeck et al. (2021);

Smooth functions . In the second thread of research, the unknown objective function to be optimized is assumed to be highly smooth , but not necessary concave/convex. Typical results assume the objective function is Hölder smooth of order k ≥ 1 , meaning that the ( k -1) -th derivative of the objective function is Lipschitz continuous. Without additional conditions, the optimal sample complexity with such smoothness assumptions is ˜ O ( ε -(2+ d/k ) ) (Wang et al., 2019), which scales exponentially with the domain dimension d .

In this paper, we study the optimal sample complexity of stochastic zeroth-order optimization when the objective function exhibits both (strong) convexity and a high degree of smoothness. As we have remarked in the first bullet point above, with convexity and Hölder smoothness of order k = 1 (equivalent to the objective function being Lipschitz continuous), the works of Agarwal et al. (2013); Lattimore & Gyorgy (2021); Bubeck et al. (2021) established an ˜ O ( ε -2 ) upper bound. With higher order of Hölder smoothness, i.e., k = 2 (equivalent to the gradient of the objective being Lipschitz continuous), it is shown that simpler algorithms exist but the sample complexity remains ˜ O ( ε -2 ) (Besbes et al., 2015; Agarwal et al., 2010; Hazan & Levy, 2014), which seemingly suggests the relatively smaller role smoothness plays in the presence of convexity. In this paper we show that with even higher order of Hölder smoothness, i.e., k = 3 (specifically, the Hessian of the objective being Lipschitz continuous), the optimal sample complexity is improved to O ( ε -1 . 5 ) , which is significantly smaller than the sample complexity of the convex-without-smoothness setting ˜ O ( ε -2 ) , or the smooth-without-convexity setting ˜ O ( ε -(2+ d/ 3) ) . More importantly, when the Lipschitzness of Hessian is defined in Frobenius norm (see condition A1), we propose an algorithm that also achieves the optimal dimension dependency, which fully characterizes the optimal sample complexity.

Summary of technical contributions. We developed several important techniques in this paper to achieve the optimal sample complexity when the objective function is strongly convex and has Lipschitz Hessian. First, we show that when estimating the gradient under a stochastic environment, even with an unbounded action space, it could be beneficial to sample with non-isotropic distributions (as opposed to conventional standard Gaussian, or uniform distributions on hyperspheres). Second, we present a new approach to analyze the bias and variance of the hyperellipsoid-sampling-based gradient estimators, which enables obtaining sharp bounds with tight constants and strengthens the best-known results in the higher-order smoothness case. Third, we present a two-stage bootstrap-type framework for the algorithmic design, which extends the perturbative analysis in the final stage to the full regime. This extension relies on a non-trivial modification of Newton's method, and we proved its robustness under stochastic observation. We complete the characterization of the minimax regret by deriving a lower bound using the KL-divergence-based approach.

Additional related works on higher-order smoothness. Recent years have seen increasing attention on exploiting higher order smoothness in bandit optimization. Remarkably, it was shown that when the Hölder smoothness condition holds simultaneously for both k = 2 and k = 3 , the optimal sample complexity can be improved to O ( ε -1 . 5 ) . (Akhavan et al., 2020; Novitskii & Gasnikov, 2021). We list our results together with the most relevant work in Table 1. While this line of work also demonstrates the benefit of higher-order smoothness in improving the sample complexity, their setting is related but slightly different from what we considered in this work. (See reference therein: Bach & Perchet (2016); Akhavan et al. (2020); Novitskii & Gasnikov (2021)). On one hand, the prior work concentrates on projected gradient-descent-like algorithms, which require a Lipschitz gradient (i.e., the k = 2 requirement, and we do not). This additional requirement can not be removed by

simply replacing the gradient steps with Newton's methods, which can lead to unbounded expectation in simple regret in the stochastic case. 1 On the other hand, their results are based on the generalized Hölder condition, which is different from our assumption that the Hessian is Lipschitz in Frobenius norm. Therefore we only emphasize the dependence of d, T and M in Table 1 and omit other parameters. We provide a detailed comparison on the implication of these results in Appendix A.

Our results are also related to a special case discussed in (Shamir, 2013), which shows that for quadratic functions it is possible to achieve a sample complexity of ˜ O ( ε -1 ) . As quadratic functions are infinitely differentiable with bounded derivatives on orders, they are Hölder smooth of any arbitrary order k →∞ , which could be regarded as an extreme of the results established in this paper which only require k = 3 .

Related works on gradient estimators. Gradient estimation serves as a key building block for stochastic zeroth-order optimization algorithms. For instance, a classical one-point estimator was proposed as early as in Flaxman et al. (2005); Blair (1985), where the gradient ∇ f ( x ) is estimated based on empirical measures of f ( x + r u ) for some fixed r and i.i.d. uniformly random u on the unit hypersphere. This was later refined to be two-point estimators, and the sampling distribution of u was generalized to isotropic distributions such as standard Gaussian (e.g., see Agarwal et al. (2010); Bach & Perchet (2016); Zhang et al. (2020)). A majority of prior work focused on the analysis for such estimators under the Lipschitz gradient assumption, where the best guaranteed bound for the bias is at the order of Θ( r ) , with a polynomial factor dependent on d . The line of works by Bach & Perchet (2016); Akhavan et al. (2020); Novitskii & Gasnikov (2021) also adopted isotropic sampling, and it was shown that with higher-order smoothness of k = 3 , this bound can be improved to Θ( r 2 ) . The improvement of sample complexity in our work is mainly due to the tight characterization of our gradient estimator, which covers the special case of isotropic sampling and provides a bound of r 2 ρ √ d 2( d +2) in the estimation bias. This strengthens or improves the bounds presented in prior works, and a detailed comparison can be found in Appendix A.

On the other hand, non-isotropic sampling was used as early as in Abernethy et al. (2008), then extended in Saha & Tewari (2011); Hazan & Levy (2014). Primarily, they were used to ensure that the sampling points are contained within a bounded action set. (Suggala et al., 2021) showed the necessity of non-isotropic sampling over quadratic loss function in the adversarial setting. In this work, we essentially demonstrated that non-isotropic sampling can be used to refine a preliminary algorithm by adding a mirror-descent-like final stage. More recently, non-isotropic sampling was also adopted in Lattimore & György (2023) to optimize convex and global Lipschitz functions.

Notations. We follow the convention of machine learning theory where ∇ 2 f ( x ) denotes the Hessian of f at point x , while the trace of Hessian is denoted by Tr ( ∇ 2 f ( x ) ) . This should not be confused with the notation in classical field theory, where ∇ 2 f ( x ) instead denotes the trace of the Hessian. We use ∥ · ∥ 2 to denote vector ℓ 2 norms, and ∥ · ∥ F to denote matrix Frobenius norms. We use I d to denote the identity matrix, and S d -1 to denote the unit hypersphere centered at the origin, both for the d -dimensional Euclidean space R d . We adopt the conventional notations (i.e., O , Ω , o , and ω ) to describe regret bounds in the asymptotic sense with respect to the total number of samples (denoted by T ).

2 Problem Formulation

We consider the stochastic optimization problem under the class of functions that are strongly convex and have Lipschitz Hessian. The goal in this setting is to design learning algorithms to achieve approximately the global minimum of an unknown objective function f : R d → R .

A learning algorithm A can interact with the function by adaptively sampling their value for T times, and receive noisy observations. At each time t ∈ [ T ] , the algorithm selects x t ∈ R d , and receives the

following observation,

where { w t } T t =1 are independent random variables with zero mean and bounded variance. Formally, the algorithm can be described by a list of conditional distributions where each x t is selected based on all historical data { x τ , y τ } τ<t and the corresponding distribution. Then for any t , we assume that E [ w t |{ x τ , y τ } τ<t , x t ] = 0 and Var [ w t |{ x τ , y τ } τ<t , x t ] ≤ 1 for any t . 2 For simplicity, we also adopt a common assumption that the additive noises are subgaussian, particularly, P [ | w t | > s |{ x τ , y τ } τ<t , x t ] ≤ 2 e -s 2 for all s > 0 and t ∈ [ T ] . However, the subgaussian assumption can be removed by adopting more sophisticated mean-estimation methods (e.g., see Nemirovskii & Yuom (1983); Jerrum et al. (1986); Alon et al. (1999); Lee & Valiant (2022); Yu et al. (2023a)).

We assume that the objective function f is second-order differentiable. Furthermore, we impose the following conditions.

(A1) (Lipschitz Hessian). There exist a constant ρ ∈ (0 , + ∞ ) such that for all x , x ′ ∈ R d , it holds that ∥∇ 2 f ( x ) -∇ 2 f ( x ′ ) ∥ F ≤ ρ ∥ x ′ -x ∥ 2 , where ∥ · ∥ F denotes the Frobenius norm;

(A2) (Strong Convexity). There exists a constant M ∈ (0 , + ∞ ) such that for any x ∈ R d , the minimum eigenvalue of the Hessian ∇ 2 f ( x ) is greater than M .

(A3) (Bounded Distance from Initialization to Optimum Point). There exists a constant R ∈ (0 , + ∞ ) such that the infimum of f ( x ) within the hyperball ∥ x ∥ 2 ≤ R is identical to the infimum of f ( x ) over the entire R d .

In the rest of this paper, we let F ( ρ, M, R ) denote the set of all second-order differentiable functions that satisfy the above conditions, with corresponding constants given by ρ, M , and R . We aim to find algorithms to achieve asymptotically the following minimax simple regret, which measures the expected difference of the objective function on x T and the optimum.

where x ∗ denotes the global minimum point of f .

3 Main Results

Theorem 3.1. For any dimension d and constants ρ, M, R , the minimax simple regrets are upper bounded by lim sup T →∞ R ( T ; ρ, M, R ) · T 2 3 ≤ C · ( ρ 2 3 M d ) , where C is a universal constant.

Theorem 3.2. For any fixed dimension d and constants ρ, M, R , the minimax simple regrets are lower bounded by lim inf T →∞ R ( T ; ρ, M, R ) · T 2 3 ≥ C · ( ρ 2 3 M d ) when the additive noises w 1 , ..., w T are standard Gaussian, where C is a universal constant.

4 Proof Ideas for Theorem 3.1

The proposed algorithm operates in two stages (see Algorithm 4). In the first stage, the algorithm uses a small fraction of samples to obtain a rough estimation of the global minimum point. We ensure that the estimation in the first stage is sufficiently accurate with high probability, so that in the following final stage, the objective function can be approximated by a quadratic function and the resulting approximation error can be bounded using tensor analysis.

4.1 Key Techniques and The Final Stage

We first present the key steps of our algorithm, which relies on the subroutines presented in Algorithm 1-3, i.e., GradientEst, BootstrappingEst, and HessianEst. These subroutines estimate the (linearly

transformed) gradients and Hessian functions of f at any given point by sampling the values of f on hyperellipsoids. The key ingredient of our proof is the sharp characterizations for the biases and variances of the GradientEst estimator, stated in Theorem 4.1.

Algorithm 2 BootstrappingEst

where λ Z is the largest singular value of Z .

Remark 4.2. Inequality (2) provides a sharp characterization for the bias of the gradient estimator, as it can be matched for any λ Z and d with a cubic polynomial f . Inequality (3) is sharp in the asymptotic regime when both ∇ f and λ Z approaches zero.

We also provide rough estimates on the high-probability bounds for the BootstrappingEst and the HessianEst functions. Specifically, we show that their errors have sub-Gaussain tails in distribution, as stated in the following theorem.

Theorem 4.3. For any fixed inputs x , r , n , any function f satisfying the Lipschitz Hessian condition with parameter ρ , and any variable K > 0 , the outputs ˆ m and ˆ H returned by the BootstrappingEst and the HessianEst subroutine satisfy the following conditions.

We postpone the proof of the above theorems to Section 4.2 and Appendix C and proceed to describe how these results are used in the algorithm.

For brevity, let ϵ ≜ ρ 2 3 M dT -2 3 be the minimax regret we aim to achieve, and let x B denote the estimator x stored at the end of the first stage. The role of the final stage is to ensure that if f ( x B ) -f ( x ∗ ) is sufficiently small with high probability, the final result of the proposed algorithm achieves the stated simple regret guarantees. Formally, we require the following achievability result from the Bootstrapping stage.

Theorem 4.4. For any fixed ρ, M and R , the result returned by the first stage of Algorithm 4 satisfies

Note that the above condition implies that f ( x B ) -f ( x ∗ ) concentrates below o ( ϵ 2 3 ) , which is weaker than the O ( ϵ ) rates stated in our main theorems. 3 The bottleneck of the overall algorithm is on the final stage, and one can achieve equation (6) using any suboptimal algorithm with an expected simple regret of o ( T -4 9 ) . For example, one can run the suboptimal algorithm twice, estimate their achieved function values by averaging over o ( T ) samples, and then choose the outcome with the smaller estimated function value as x B. In the rest of this section, we prove Theorem 3.1 assuming the correctness of the above theorem. A self-contained proof for Theorem 4.4 is provided in Appendix E.

Before proceeding with the proof, we provide a high-level description of the algorithm in the final stage. At the beginning, we perform a Hessian estimation near x B using the HessianEst subroutine with O ( T ) samples. From Theorem 4.3, our choice of parameters results in an expected estimation error of o (1) for sufficiently large T .

The algorithm proceeds to find a real matrix Z H , which essentially serves as a linear transformation on the action domain such that the Hessian of the transformed function is approximately the identity matrix. Note that the projection step in the HessianEst function ensures the eigenvalues of the estimator are no less than M . There is always a valid solution of Z H .

Then, we estimate the gradient at x B using the GradientEst subroutine, which samples on a hyperellipsoid with a shape characterized by Z H . We chose the hyperellipsoid sampling in the final stage due to its superior performance in the small-gradient regime compared to coordinate-wise sampling. In contrast, the coordinate-wise estimator is used in the bootstrapping stage to eliminate the dependency of the local gradient on its bias-variance tradeoff, which is beneficial for the non-asymptotic analysis. Particularly, we scale the hyperellipsoid with a carefully designed factor (see the definition of variable r g) to minimize the estimation error. Then, the remaining steps can be interpreted as a modified Newton step, which essentially approximates the global minimum point with a quadratic approximation.

The analysis in our proof relies on the following proposition, which is proved in Appendix D.

Proposition 4.5. For any given point x B and any function f that satisfies strong convexity and Lipschitz Hessian, let ˜ x ≜ x B -( ∇ 2 f ( x B )) -1 ∇ f ( x B ) and ˜ f ( x ) denote the quadratic approximation

Algorithm 4 An Example Algorithm to Achieve the Minimax Rates

Input T, ρ, M

The First Stage:

Let ˆ m = BootstrappingEst ( x , r m , n m ) , ˆ H = HessianEst ( x , r H , n H )

Let H m ∗ denote the matrix with the same eigenvectors of ˆ H but each eigenvalue λ replaced by max { λ, m ∗ } , choose m ∗ to be the smallest value such that || H -1 m ∗ ˆ m || 2 ≤ M ρ .

end for

The Final Stage:

Let ˆ H = HessianEst ( x , r H , n H ) , Z H be any symmetric matrix such that Z 2 H = ˆ H -1 , and λ Z H be the largest eigenvalue of Z H

Return x = x + r

1 2 ( x -˜ x ) ⊺ ∇ 2 f ( x B )( x -˜ x ) , we have the following inequality for all x with || x -x B || 2 ≤ M ρ .

Furthermore, if x is generated by the final stage of Algorithm 4 with any parameter values that satisfy n g ≥ d 3 , n H ≥ 64 ρ 4 d 6 M 6 and the first-stage output is set to x B , then

Now, we use Proposition 4.5 to prove the achievability result.

Proof of Theorem 3.1 given Theorem 4.4. First, recall our construction ensures that || x T -x B || 2 ≤ M ρ . Inequality (7) can always be applied and we have

Then, when T is sufficiently large, the conditions of (8) holds and we have

Note that inequality (6) implies that E [ f ( x B ) -f ( x ∗ )] = o ( ϵ 2 3 ) = o ( T -4 9 ) and we have that n -1 3 H + n -1 g = o ( T -2 9 ) . The RHS of the above inequality is dominated by the last term. Hence,

To complete the proof, we show that the proposed algorithm samples the function values of f at most T -1 times. In the first stage, both BootstrappingEst and HessianEst are executed once per loop, with BootstrappingEst requiring at most 2 dn m ≤ T 0 . 9 5 samples and HessianEst requiring at most 2 d 2 n H ≤ T 0 . 9 5 samples each time. Therefore, the total number of samples used in the first stage is bounded by ⌊ T ⌋ ( T 0 . 9 5 + T 0 . 9 5 ) ≤ 2 T 5 . In the final stage, we make one call to both HessianEst and GradientEst, which together require n H (2 d 2 ) + 2 n g ≤ 2 T 5 samples. Thus, the overall number of samples is bounded by 4 T 5 , which ensures that it is no greater than T -1 .

4.2 Proof of Theorem 4.1

To prove inequality (2), we investigate the following function

where Unif ( S d -1 ) denotes the uniform distribution on S d -1 . Recall that in our algorithm we have E [ˆ g ] = r G ( r ; x ) if Z = rI d for some r ∈ (0 , + ∞ ) , and by differentiability we have ∇ f ( x ) = lim z → 0 + G ( z ; x ) . Under this condition, we can bound || E [ˆ g ] -r ∇ f ( x ) || 2 by integration, i.e.,

Note that G ( z ; x ) can be written into the following equivalent form.

where the integration is with respect to u over the surface S d -1 , and dA is the vector surface element, i.e., with the magnitude being the infinitesimally small surface area and the direction perpendicular to the surface (pointing outward). The differential of G ( z ; x ) over z can be written as

The gist of this proof is to note that for any u ∈ S we have u and dA are parallel (i.e., u is parallel to the normal vector of the hypersphere at the same point), so the second term in the integral above on the numerator can be written as

Hence, by divergence theorem, we have

where B d denotes the standard hyperball.

Now consider any unit vector e . Let u e denote the reflection of u with respect to the hyperplane orthogonal to e , i.e., u e ≜ u -2( u · e ) e . Because the hyperball B is invariant under the reflection u → u e , equation (10) can also be written as

Hence, by averaging equation (10) and (11), we have

By the Lipschitz Hessian condition and Cauchy's inequality, the difference between the differential terms above can be bounded as follows.

Consequently,

Note that e can be any unit vector. We have essentially bounded the ℓ 2 norm of d dz G ( z ; x ) , i.e.,

As mentioned earlier, when Z = rI d inequality (2) is obtained by applying this gradient-norm bound to inequality (9) .

For general input matrix Z , we can view GradientEst as a subroutine that operates on the same function f but with a linear transformation applied to the input domain. Formally, let f ′ ( y ) ≜ f ( x + Z λ Z ( y -x )) . We have that f ′ satisfies the Lipschitz Hessian condition with parameter ρ as well. Therefore, inequality (2) can be obtained following the same analysis by replacing f with f ′ and Z with λ Z I d .

Now we present the proof for inequality (3). Formally, let w + , w -be two independent samples of additive noises. Then the trace of covariance matrix of ˆ g can upper bounded using the second moments of single measurements.

The identity above uses the fact that additive noises are unbiased and have bounded variances.

Note that from the Lipschitz Hessian condition, we have that

where f 2 is the Taylor polynomial of f expanded at x up to the quadratic terms. Consequently, inequality (14) implies

where the expectations are taken of u ∼ Unif ( S d -1 ) , and the last equality is due to the well-known fact that E [ uu ⊺ ] = 1 d I d .

5 Conclusion and Future Work

In this work, we achieve the first minimax simple regret for bandit optimization of second-order smooth and strongly convex functions. We derived the matching upper and lower bounds and proposed an algorithm that integrates a bootstrapping stage with a mirror-descent stage. Our key technical innovations include a sharp characterization of the spherical-sampling gradient estimator under higher-order smoothness conditions and a novel iterative method for the bootstrapping stage that remains effective with unbounded Hessians.

While these advancements settle the fundamental problem of optimizing second-order smooth and strongly convex functions with zeroth-order feedback, the techniques and insights presented in this paper also pave the way for further research in this domain. One interesting follow-up direction is to generalize our analysis to the online setting for the average regret metric. Additionally, investigating the fundamental tradeoff between simple regret and average regret could yield valuable insights for task-specific algorithmic designs.

References

Jacob Abernethy, Elad Hazan, and Alexander Rakhlin. Competing in the dark: An efficient algorithm for bandit linear optimization. pp. 263-273, 2008. 21st Annual Conference on Learning Theory, COLT 2008 ; Conference date: 09-07-2008 Through 12-07-2008.

Alekh Agarwal, Ofer Dekel, and Lin Xiao. Optimal algorithms for online convex optimization with multi-point bandit feedback. In Colt , pp. 28-40. Citeseer, 2010.

Alekh Agarwal, Dean P. Foster, Daniel Hsu, Sham M. Kakade, and Alexander Rakhlin. Stochastic convex optimization with bandit feedback. SIAM Journal on Optimization , 23(1):213-240, 2013.

Arya Akhavan, Massimiliano Pontil, and Alexandre Tsybakov. Exploiting higher order smoothness in derivative-free optimization and continuous bandits. Advances in Neural Information Processing Systems , 33:9017-9027, 2020.

Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. Journal of Computer and System Sciences , 58(1):137-147, 1999. ISSN 0022-0000. doi: https://doi.org/10.1006/jcss.1997.1545. URL https://www.sciencedirect.com/ science/article/pii/S0022000097915452 .

Francis Bach and Vianney Perchet. Highly-smooth zero-th order online optimization. In Conference on Learning Theory , pp. 257-283. PMLR, 2016.

Omar Besbes, Yonatan Gur, and Assaf Zeevi. Non-stationary stochastic optimization. Operations research , 63(5):1227-1244, 2015.

Charles Blair. Problem complexity and method efficiency in optimization (as nemirovsky and db yudin). Siam Review , 27(2):264, 1985.

Stephen P Boyd and Lieven Vandenberghe. Convex optimization . Cambridge university press, 2004.

Sébastien Bubeck, Ronen Eldan, and Yin Tat Lee. Kernel-based methods for bandit convex optimization. Journal of the ACM (JACM) , 68(4):1-35, 2021.

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security , pp. 15-26, 2017.

Andrew R Conn, Katya Scheinberg, and Luis N Vicente. Introduction to derivative-free optimization . SIAM, 2009.

Abraham D. Flaxman, Adam Ta uman Kalai, and H. Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms , SODA '05, pp. 385-394, USA, 2005. Society for Industrial and Applied Mathematics. ISBN 0898715857.

Elad Hazan and Kfir Levy. Bandit convex optimization: Towards tight bounds. Advances in Neural Information Processing Systems , 27, 2014.

Mark R. Jerrum, Leslie G. Valiant, and Vijay V. Vazirani. Random generation of combinatorial structures from a uniform distribution. Theoretical Computer Science , 43:169-188, 1986. ISSN 0304-3975. doi: https://doi.org/10.1016/0304-3975(86)90174-X. URL https: //www.sciencedirect.com/science/article/pii/030439758690174X .

Tor Lattimore and Andras Gyorgy. Improved regret for zeroth-order stochastic convex bandits. In Conference on Learning Theory , pp. 2938-2964. PMLR, 2021.

Tor Lattimore and András György. A second-order method for stochastic bandit convex optimisation. In Gergely Neu and Lorenzo Rosasco (eds.), Proceedings of Thirty Sixth Conference on Learning Theory , volume 195 of Proceedings of Machine Learning Research , pp. 2067-2094. PMLR, 12-15 Jul 2023. URL https://proceedings.mlr.press/v195/lattimore23a.html .

Jasper C.H. Lee and Paul Valiant. Optimal sub-gaussian mean estimation in R . In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS) , pp. 672-683, 2022. doi: 10.1109/FOCS52979.2021.00071.

Nemirovskii and D. Yuom. Problem Complexity and Method Efficiency in Optimization," . Wiley, 1983.

Vasilii Novitskii and Alexander Gasnikov. Improved exploiting higher order smoothness in derivativefree optimization and continuous bandit. arXiv preprint arXiv:2101.03821 , 2021.

Luis Miguel Rios and Nikolaos V Sahinidis. Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization , 56(3):1247-1293, 2013.

Ankan Saha and Ambuj Tewari. Improved regret guarantees for online smooth convex optimization with bandit feedback. In Geoffrey Gordon, David Dunson, and Miroslav Dudík (eds.), Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , volume 15 of Proceedings of Machine Learning Research , pp. 636-642, Fort Lauderdale, FL, USA, 11-13 Apr 2011. PMLR. URL https://proceedings.mlr.press/v15/saha11a.html .

Ohad Shamir. On the complexity of bandit and derivative-free stochastic convex optimization. In Shai Shalev-Shwartz and Ingo Steinwart (eds.), Proceedings of the 26th Annual Conference on Learning Theory , volume 30 of Proceedings of Machine Learning Research , pp. 3-24, Princeton, NJ, USA, 12-14 Jun 2013. PMLR. URL https://proceedings.mlr.press/v30/Shamir13. html .

Arun Sai Suggala, Pradeep Ravikumar, and Praneeth Netrapalli. Efficient bandit convex optimization: Beyond linear losses. In Mikhail Belkin and Samory Kpotufe (eds.), Proceedings of Thirty Fourth Conference on Learning Theory , volume 134 of Proceedings of Machine Learning Research , pp. 4008-4067. PMLR, 15-19 Aug 2021. URL https://proceedings.mlr.press/v134/ suggala21a.html .

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018.

Yining Wang, Sivaraman Balakrishnan, and Aarti Singh. Optimization of smooth functions with noisy observations: Local minimax rates. IEEE Transactions on Information Theory , 65(11): 7350-7366, 2019.

Qian Yu, Yining Wang, Baihe Huang, Qi Lei, and Jason D Lee. Sample complexity for quadratic bandits: Hessian dependent bounds and optimal algorithms. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems , volume 36, pp. 35121-35138. Curran Associates, Inc., 2023a. URL https://proceedings.neurips.cc/paper_files/paper/2023/ file/6e60a9023d2c63f7f0856910129ae753-Paper-Conference.pdf .

Qian Yu, Yining Wang, Baihe Huang, Qi Lei, and Jason D. Lee. Optimal sample complexity bounds for non-convex optimization under kurdyka-lojasiewicz condition. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206 of Proceedings of Machine Learning Research , pp. 6806-6821. PMLR, 25-27 Apr 2023b. URL https:// proceedings.mlr.press/v206/yu23a.html .

Yan Zhang, Yi Zhou, Kaiyi Ji, and Michael M Zavlanos. Boosting one-point derivative-free online optimization via residual feedback. arXiv preprint arXiv:2010.07378 , 2020.

A Detailed Comparison on Different Smoothness Conditions

In a relevant line of work, the higher-order smoothness of the objective function in the k = 3 case is characterized by the following generalized Hölder condition.

Note that the ρ -Lipschitz Hessian condition in our work implies an Hölder condition with parameter L = ρ 6 . A direct application of the works in Table 1 requires order-wise larger sample complexities in our setting on top of the additional k = 2 smoothness condition. On the other hand, the L -Holder condition implies ρ = O ( √ dL ) -Lipschitz Hessian. Hence, a direct application of our algorithm order-wise improves the sample complexity in the setting of generalized Hölder condition in a polynomial factor of d as well.

In terms of the characterization of gradient estimators, the prior works of Bach & Perchet (2016); Akhavan et al. (2020); Novitskii & Gasnikov (2021) used isotropic sampling over a bounded set of radius r and presented upper bounds on the estimation bias of O ( Lr 2 ) , O ( Ldr 2 ) , and O ( L √ dr 2 ) , respectively. In this special case, our Theorem 4.1 implies an upper bound of O ( ρr 2 / √ d ) , and similar to the above analysis, this bound strengths the bounds in prior works.

B Proof of Theorem 3.2

To illustrate the proof idea, we start with the case of d = 1 .

B.1 Illustrating example: 1D case

The gist of our proof is to construct a pair of hard-instance functions that need to sufficiently distant from each other to avoid trivial optimizers with low simple regret. We also require them to be sufficiently close to each other so that they are indistinguishable without sufficiently many samples. These requirements are captured quantitatively in the following result, which is proved using an analysis of KL divergence. Here we assume their correctness and focus on the construction.

Definition B.1. For any (Borel measurable) function class F H and any distribution p defined on F H, we define the uniform sampling error to be

We also define the maximum local variance to be

Lemma B.2 (Restatement of Proposition 7 in Yu et al. (2023b)) . For any sampling algorithm to achieve an expected simple regret of ϵ > 0 over a function class F H , if P 2 ϵ/c ≥ c for some universal constant c ∈ (0 , 1) , and the observation noises are standard Gaussian, then the required sample complexity to achieve a minimax regret of ϵ is at least Ω(1 /V ) .

We construct our hard instances using the following function

Some key properties of g ( x ) to be used are that its differential g ′ ( x ) is 1 -Lipschitz, and we have | g ′ ( x ) | ≤ 1 for all x . Our hard instances consist of two functions. We define

where y 0 , x 0 are normalization factors given by y 0 = 1 π √ T , x 0 = ( y 0 ρ ) 1 3 . The normalization factors are chosen to satisfy the Lipschitz Hessian condition and a maximum local variance bound required for a KL-divergence based approach presented in Lemma B.2.

Specifically, the choice of x 0 and the fact that g ′ ( x ) is 1 -Lipschitz imply that both f 1 and f 2 satisfy the Lipschitz Hessian condition. Then because the absolute value of integration of g ( x ) is bounded by 2 π , one can show that the maximum local variance for the function class { f 1 , f 2 } is no greater than π 2 y 2 0 = 1 T for the uniform prior distribution, which is to be used to show the sample complexity lower bound.

We first check that both f 1 and f 2 are within our function class of interests. Note that both f ′′ 1 ( x ) and f ′′ 2 ( x ) belong to the interval [ 2 M -5 4 y 0 x 2 0 , 2 M -3 4 y 0 x 2 0 ] . From the fact that lim T →∞ y 0 x 2 0 = 0 and M > 0 , we have both f ′′ 1 ( x ) > M and f ′′ 2 ( x ) > M for all x for sufficiently large T . So the strong convexity requirement is satisfied. On the other hand, consider any global minimum point x ∗ of either f 1 or f 2 . Because of their differentiability, we must have f ′ 1 ( x ) = 0 or f ′ 2 ( x ) = 0 . Note that for all x , we have | g ( x ) | ≤ 2 , and

We must have | x ∗ | ≤ y 0 x 0 /M , where the RHS is o (1) for large T . Combined with strong convexity, this inequality implies that assumption A3 holds for both functions. To conclude, we have proved that f 1 , f 2 ∈ F ( ρ, M, R ) for sufficiently large T .

Now we let ϵ = 1 128 M ( y 0 x 0 ) 2 and c = 1 2 to apply Lemma B.2. Note that

The quantity ϵ exactly matches the lower bounds we aim to prove. Therefore, it remains to check that the required condition on uniform sampling errors in Definition B.1 are satisfied.

Formally, we need to show that f k (0) -inf x f k ( x ) ≥ 4 ϵ for k ∈ { 1 , 2 } , so that the uniform sampling error P 4 ϵ under uniform distribution over F H is lower bounded by 1 2 and Lemma B.2 can be applied. Without loss of generality, we focus on the case of k = 1 . Note that f ′′ 1 ( x ) ≤ 2 M + y 0 4 x 2 0 for all x ∈ [ -πx 0 , 0] . Therefore, we have

for x ∈ [ -πx 0 , 0] , and lim T →∞ x 0 = 0 . Consider any sufficiently large T such that y 0 4 x 2 0 ≤ 2 M , we can choose x = -y 0 2 x 0 1 2 M + y 0 4 x 2 0 for the above bound, which falls into the interval of [ -πx 0 , 0] . Then we have

We use this inequality to lower bound the minimum sampling error. Note that f 1 is an increasing function for x ≥ 0 and inf x f 1 ( x ) = inf x f 2 ( x ) . We have f 1 ( x ) ≥ inf x f 2 ( x ) + 4 ϵ for x ≥ 0 .

Following the same arguments, we also have f 2 ( x ) ≥ inf x f 1 ( x ) + 4 ϵ for x ≤ 0 . Recall the definition of uniform sampling error in Definition B.1. We have essentially proved that P 4 ϵ ≥ 1 2 . According to earlier discussions, this implies that the minimax simple regret is lower bounded by

B.2 Proof for the General Case

The generalization of the earlier 1D lower bound is obtained by constructing a set of hard-instance functions where the optimization problem over this subset consists of d binary hypothesis estimation problems, each identical to a 1D construction. Formally, for any s = ( s 1 , s 2 , ..., s d ) ∈ { 1 , 2 } d and any input x = ( x (1) , x (2) , ..., x ( d ) ) , we let

One can verify that f s ∈ F ( ρ, M, R ) for all s for sufficiently large T .

Note that the simple regret for the above function class can be written as the sum of d individual terms ∑ d j =1 ( f s j ( x ( j ) ) -inf x f s j ( x ) ) . As proved earlier, the expectation of each term associated with any index j is at least Ω ( ρ 2 3 T -2 3 M ) even if all entries of s except s j is known. Therefore, the total expected regret is lower bounded by Ω ( dρ 2 3 T -2 3 M ) .

C Proof of Theorem 4.3

We use the following elementary facts, which are versions of well-known properties of subgaussian and subexponential distributions in Vershynin (2018), but with explicit and possibly improved constant factors. For completeness, we provide their proofs in Appendix F.

Proposition C.1. For any real-valued zero-mean independent random variables z 1 , ..., z k , if

for some σ 1 , ..., σ k , then

Proposition C.2. For any real-valued independent random variables z 1 , ..., z k , if

for some positive σ 1 , ..., σ k , then

Proof of equation (4) . We first prove the bound entry-vise. Consider any m k , which contains a summation of 2 n independent subgaussian variables. By Prop. C.1 we have that

Then, by the Lipschitz Hessian condition, the bias for each entry is bounded as follows.

where x k denotes the k th entry of x . Hence, each ∣ ∣ ∣ m k -∂ ∂x k f ( x ) ∣ ∣ ∣ 2 is subexpoential, i.e.,

By the independence of m k 's, we can apply Prop. C.2 to the inequality. Therefore,

Proof of equation (5) . We first provide the entry-wise bounds for the intermediate estimator ˆ H 0 . Each diagonal entry H kk contains the weighted average of 3 n subgaussian variables. Conditioned on any realization of y , which is shared among all diagonal elements, Prop. C.1 can be applied for the rest of the 2 n terms, and provides the following bounds.

̸

Then, because the off-diagonal entries are independent, we have the following bounds for any j = k .

Hence, similar to the earlier proof steps, Prop. C.2 implies that

Now, we take into account the estimation bias and the error of y . By Lipschitz Hessian, it is clear that

Hence,

Furthermore, note that E [ ˆ H 0 ] -E [ ˆ H 0 | y ] = 2( y -f ( x )) I d /r 2 , where I d denotes the identity matrix. The subgaussian condition and Prop. C.1 imply that

We can combine the above bounds using triangle inequality and the union bound. Specifically, from inequalities (25), (26), and (27), we have the following bound for any K ≥ √ 2 dρr 3 .

Utilize the fact that any probability measure is no greater than 1 , the above inequality implies that

Finally, the needed bound for ˆ H is due to the projection to a convex set where the target ∇ 2 f ( x ) belongs. Hence, the distance is not increased w.p.1, i.e., we always have || ˆ H - ∇ 2 f ( x ) || F ≤ || ˆ H 0 -∇ 2 f ( x ) || F.

D Proof of Proposition 4.5

Inequality (7) is derived from the following approximations, which are due to the Lipschitz Hessian condition at x B.

Noting that ˜ f ( x ∗ ) ≥ 0 , the above inequalities imply that

By strong convexity, we have || x ∗ -x B || 2 2 ≤ 2( f ( x B ) -f ( x ∗ )) M . Hence, it remains to provide an upper bound for 1 6 ρ || x -x B || 3 2 .

When || x -x B || 2 ≤ √ 3 || x -˜ x || 2 , we apply the condition || x -x B || 2 ≤ M ρ to obtain that

where the last step is due to the strong convexity of ˜ f . This implies inequality (7).

For the other case, we have || x -x B || 2 ≥ √ 3 || x -˜ x || 2 . We replace the variable x in inequality (29) with ˜ x to obtain that

and by triangle inequality, we have that

Hence, to summarize,

By strong convexity,

and the needed result is obtained by applying the above to inequality (31).

Now we prove inequality (8). The proof consists of three steps. For brevity, let H ≜ ∇ 2 f ( x B ) and x + = x B -ˆ H -1 Z -1 ˆ g . We first prove that

As the second step, we prove that

We shall repetitively use the fact that || z -˜ x || 2 ≤ √ 2 ˜ f ( z ) /M for any z ∈ R d , which is due to strong convexity. When both ˜ f ( x + ) and ˜ f ( x B ) are no greater than M 3 8 ρ 2 , both || x + -˜ x || 2 and || x B -˜ x || 2 are no greater than M 2 ρ . By triangle inequality, we have || x + -x B || 2 ≤ M ρ . Recall the construction of x , which is identical to x + in this case, inequality (34) clearly holds. Otherwise, note that x belongs to the line segment between x B and x + . By convexity, we always have ˜ f ( x ) ≤ max { ˜ f ( x + ) , ˜ f ( x B ) } . Recall that in this case, ˜ f ( x B ) ≥ ˜ f ( x + ) can only hold when ˜ f ( x B ) ≥ M 3 8 ρ 2 , we have ˜ f ( x ) ≤ ✶ ( ˜ f ( x B ) ≤ M 3 8 ρ 2 ) ˜ f ( x + )+ ✶ ( ˜ f ( x B ) ≥ M 3 8 ρ 2 ) max { ˜ f ( x + ) , ˜ f ( x B ) } , which implies inequality (34).

Note that the estimation error of ˜ x can be decomposed into two terms, i.e.,

where the first term is due to the error of the Hessian estimator, and the second is mostly contributed by the GradientEst estimator. We apply the AM-QM inequality to their quadratic forms, i.e.,

where λ Z H , r g are defined in Algorithm 4 and || · || denotes the spectrum norm. By theorem 4.1, we can first take the expectation of the above bound conditioned on any realization of ˆ H . Specifically,

Recall the definition of Z , we have

Hence, by our choice of r g in Algorithm 4 and note that λ Z H ≤ M -1 2 is implied by strong convexity,

To characterize the above bound, we first note that the singular values of H 1 2 ˆ H -1 2 equals the eigenvalues of ˆ H -1 2 H ˆ H -1 2 = I d + ˆ H -1 2 ( H -ˆ H ) ˆ H -1 2 . As the eigenvalues of ˆ H are no less than M , by triangle inequality, all eigenvalues of ( I d + ˆ H -1 2 ( H -ˆ H ) ˆ H -1 2 ) are bounded within [1 -|| H -ˆ H || F M , 1 + || H -ˆ H || F M ] . Hence, we have

Similarly, bounds on the singular values of H 1 2 ˆ H -1 2 imply bounds on the eigenvalues of H 1 2 ˆ H -1 H 1 2 , i.e.,

Therefore, we have

Now that the above bound is simply a polynomial of || H -ˆ H || F. We can use Theorem 4.3 to obtain P [∣ ∣ ∣ ∣ ∣ ∣ ˆ H -H ∣ ∣ ∣ ∣ ∣ ∣ F ≥ K ] ≤ 2 exp ( -K 2 16 d 2 ρ 4 3 /n 1 3 H ) then apply a direct integration. We utilize the assumptions in the statement of proposition to obtain a simpler estimate, expressed as follows.

Then, inequality (8) is implied by the definition of ˜ f .

For the third step, we observe that the earlier proof steps imply that

and it remains characterize ˜ f ( x B ) . To that end, we reuse inequality (32) and (33), which implies that ˜ f ( x B ) ≤ 2( f ( x B ) -f ( x ∗ )) when || ˜ x -x B || 2 ≤ 3 M 2 ρ . For the other case, we have || ˜ x -x B || 2 ≥ 3 M 2 ρ . We instead let x r ≜ x B + √ 3 M 2 ρ || ˜ x -x B || 2 (˜ x -x B ) and the Lipschitz Hessian condition implies that

By convexity, we have ˜ f ( x B ) -˜ f ( x r ) ≥ √ 3 M 2 ρ || ˜ x -x B || 2 ˜ f ( x B ) , which can be applied to the above bound. Then, together with inequality (33) and the condition of || ˜ x -x B || 2 , we have

To summarize, the following inequality holds in both cases.

To apply inequality (36), we use the following implications.

Then, the derived inequality can be simplified using our assumptions on n g and n H.

E Remaining details for Theorem 3.1

To complete the proof, we essentially need to prove Theorem 4.4. To illustrate the main ideas, we start with an analysis in a simplified setting where estimation errors for the BootstrapingEst and HessianEst functions are zero. Then, we show how the proof steps can be modified to have the errors and uncertainties incorporated.

E.1 Analysis for the zero-error case

We prove that when the estimation errors are set to zero, the first stage of Algorithm 4 reduces ∇ f ( z t ) to a vector of bounded length in boundedly many iterations. This is summarized in the following proposition.

Proposition E.1. For any fixed parameter values ρ, M , R , let { z t } t ∈ N + be sequences defined for any f ∈ F ( ρ, M , R ) , such that z 1 = 0 and ( z t +1 -z t ) equals -˜ H -1 t ∇ f ( z t ) , where ˜ H t is a matrix that has the same eigenvectors of ∇ 2 f ( z t ) , with each eigenvalue λ replaced by max { λ, m t } , and m t being the smallest value for || ˜ H -1 t ∇ f ( z t ) || 2 ≤ M ρ . There exists an explicit function T ( ρ, M, R ) ≤ 5 R 2 ρ 2 /M 2 +1 such that ∇ f ( z t ) ≤ M 2 2 ρ holds for any f and any t ≥ T ( ρ, M, R ) .

Proof. For convenience, let ˜ r t ≜ -( ∇ 2 f ( z t )) -1 ∇ f ( z t ) and r t ≜ -˜ H -1 t ∇ f ( z t ) . To investigate the evolution of gradients, we integrate the Lipschitz Hessian condition and obtain that

From the definition of r t , if || ˜ r t || 2 ≤ M/ρ , we have

Note that by strong convexity, when the above bound holds,

Hence, once || ˜ r t || 2 reaches below M/ρ for some t 0 , our desired bound on ||∇ f ( z t +1 ) || 2 remain hold for any t > t 0 . Therefore, for the purpose of our proof, we can focus on the case where || ˜ r t || 2 > M/ρ and show that this condition can only hold for boundedly many iterations.

Consider any fixed function f ∈ F ( ρ, M, R ) , let x ∗ denote its global minimum point. A crucial step in our proof is to show that

For brevity, let β denote the minimum eigenvalue of ˜ H t , and ˜ f denote the following quadratic approximation.

We apply the strong convexity condition at point y ≜ z t +1 -0 . 4 β ˜ H -1 t r t . Recall that f is minimized at x ∗ , we have

By integrating the Lipschitz Hessian, similar to inequality (39), ∇ f ( y ) can be approximated with ∇ f ( z t ) + ∇ 2 f ( z t )( y -z t ) = ∇ ˜ f ( y ) . Formally,

Hence, inequality (43) implies that

To characterize the terms in the above inequality, we first note that

For convenience, we denote c ≜ || ˜ H -1 t β r t || 2 / || r t || 2 . We have that c ∈ (0 , 1] and

We also consider the following vector,

of which the L2 norm is no greater than 0 . 6 c || r t || 2 ≤ 0 . 6 cM/ρ , which can be proved in the eigenbasis of ∇ 2 f ( z t ) . By Cauchy's inequality, we have that

Note that x ∗ -y = ( ∇ 2 f ( z t )) -1 ( ∇ ˜ f ( x ∗ ) -∇ ˜ f ( y ) ) . The LHS of the above inequality can be written as ( x ∗ -y ) · ( ∇ 2 f ( z t )) · q + q · ∇ ˜ f ( y ) , where the first term contains ∇ ˜ f ( y ) · ( x ∗ -y ) , and the second term is bounded as follows.

On the other hand, observe that inequality (44) holds for any generic y ∈ R d . The RHS of inequality (47) can be characterized as follows.

Therefore, by combining inequalities (45), (47), and (49), we have that

where the second line is obtained by taking the infimum w.r.t. || x ∗ -y || 2 . Then, we apply inequalities (46), (48), and || r t || 2 ≤ M/ρ to obtain the following bound.

Note that the above bound is non-decreasing w.r.t. β , and our construction implies β ≥ M . We can substitute β in the above inequality with M . Further, note that

We have obtained a lower bound of ( x ∗ -z t ) · r t as a function of c . This dependency is removed by taking the infimum, i.e.,

then inequality (42) is obtained.

We use this key inequality to obtain the following recursion rule.

Recall that for f ∈ F ( ρ, M, R ) , we assumed that || x ∗ || 2 ≤ R . Therefore, for z 1 = 0 , the above recursion implies that the inequality || ˜ r t || 2 > M/ρ can hold for no greater than 5 R 2 ρ 2 /M 2 iterations as || x ∗ -z t || 2 2 has to be non-negative for any t . Hence, based on the earlier discussion, we have proved that either || ˜ r t || 2 ≤ M/ρ or ∇ f ( z t +1 ) = 0 for all t ≥ 5 R 2 ρ 2 /M 2 and all f ∈ F ( ρ, M, R ) . Recall inequality (40), this implies that ||∇ f ( z t ) || 2 ≤ M 2 2 ρ for all t ≥ 5 R 2 ρ 2 /M 2 +1 .

Remark E.2. Recall the recursion provided by inequality (40) and (41) . The gradients for the sequence z t decay double-exponentially once they are sufficiently close to zero. Hence, Proposition E.1 proves that it takes finitely many iterations for the bootstrapping stage of Algorithm 4 to get arbitrarily close to x ∗ in the zero-error case.

E.2 Generalization to the noisy case

Now we prove that, given a bounded number of iterations, the bootstrapping stage in Algorithm 4 provides an x B that is sufficiently close to x ∗ with high probability even in the presence of noise. Similar to the zero-error case, we provide the following guarantee.

Theorem E.3. For any fixed ρ, M and R , the result returned by the first stage of Algorithm 4 satisfies

For convenience, let x ( B ) k denote the realization of vector x at the end of the k th iteration in the bootstrapping stage and N ≜ ⌊ T 0 . 1 ⌋ denote the number of iterations. Therefore, we have x B = x ( B ) N . Further, we define x ( B ) 0 ≜ 0 . We let m k , H k denote the realization of ˆ m , H m ∗ in the ( k + 1) th iteration of the bootstrapping stage. Therefore, we have x ( B ) k +1 = x ( B ) k -H -1 k m k . As a Benchmark for our analysis, we use ˜ H k to denote the value of H m ∗ in the zero-error case, i.e., they denotes the value of H m ∗ under the special case of ˆ m = ∇ f ( x ( B ) k ) and ˆ H = ∇ 2 f ( x ( B ) k ) . Hence, the update in the zero-error case can be denoted as r k ≜ -˜ H -1 k ∇ f ( x ( B ) k ) .

We let E k be the indicator function of the event where there exists an j < k such that ∣ ∣ ∣ ∣ x ( B ) j +1 -x ( B ) j -r j ∣ ∣ ∣ ∣ 2 ≥ MT -0 . 2 /ρ . Intuitively, E k = 0 describes the event that the optimization steps can be characterized similar to the zero-error case. Notice that E k is non-decreasing. We have either E N = 0 , or E k 0 = 1 for some k 0 ∈ { 1 , 2 , ..., N } . We provide the analysis of Theorem E.3 separately for each of these two cases.

For the first case, i.e., when E N = 0 , we can follow the earlier arguments and prove the following proposition (see Appendix F.3 for details).

Proposition E.4. For any function f ∈ F ( ρ, M, R ) and any sequence x ( B ) 0 , x ( B ) 1 , ..., x ( B ) N -1 ∈ R d that satisfies x ( B ) 0 = 0 and E N -1 = 0 , we have || r N -1 || 2 ≤ 2 MT -0 . 2 /ρ when T is sufficiently large.

Recall the definition of E N = 0 . The above proposition immediately implies that

when T is large. Hence, in such cases, x ( B ) N is obtained by the Newton update. Formally, if ˆ H denotes the estimator returned by the HessianEst function in the N th iteration, we have

Therefore, by applying the above results to the Lipschitz Hessian condition, we have

Hence, by direct integration of the tail bounds in Theorem 4.3, we can conclude that

Now we consider the second case, i.e., when E N = 1 . By its definition, we must have the event of E k = 0 to E k +1 = 1 for a unique k ∈ { 0 , 1 , ..., N -1 } , which implies that ∣ ∣ ∣ ∣ x ( B ) k +1 -x ( B ) k -r k ∣ ∣ ∣ ∣ 2 ≥ MT -0 . 2 /ρ . We prove that conditioned on any of these events, the random variable ||∇ f ( x ( B ) k +1 ) || 2 has a super-polynomial tail, which contributes vanishingly to their moments in the asymptotic sense. Formally, let

We aim to prove that

Consider any fixed k ∈ { 0 , 1 , ..., N -1 } and conditioned on any realization of x ( B ) k , we characterize the distribution of x ( B ) k +1 by providing the following proposition, which is proved in Appendix F.4.

Proposition E.5. Consider any vectors m , m ′ ∈ R n , any positive definite matrices H,H ′ ∈ R n with all eigenvalues lower bounded by M , and any fixed parameter R 0 ∈ N + . Let H m ∗ be the symmetric matrix sharing the same eigenbasis of H but with each eigenvalue λ replaced with max { λ, m ∗ } , where m ∗ is chosen to be the smallest value such that || H -1 m ∗ m || 2 ≤ R 0 . Let H ′ m ′∗ be defined correspondingly for m ′ and H ′ . We have that

Furthermore, when ∣ ∣ ∣ ∣ H -1 m ∗ m -H ′-1 m ′∗ m ′ ∣ ∣ ∣ ∣ 2 > 0 , we have

By choosing R 0 = M/ρ , H m ∗ = H N -1 , H ′ m ′∗ = ∇ 2 f ( x ( B ) N -1 ) , m = m N -1 , and m ′ = ∇ f ( x ( B ) N -1 ) for Proposition E.5, the condition of E k +1 can be characterized by the estimation errors of the gradient and Hessian. For brevity, we define

We also let β denote the minimum eigenvalue of H k . The condition of E k +1 = 1 and E k = 0 implies that || H -1 m ∗ m -H ′-1 m ′∗ m ′ || 2 ≥ R 0 T -0 . 2 , which implies that Ψ ≥ βR 0 T -0 . 2 3+2 T 0 . 2 according to inequality (56). Hence, M k can be bounded as follows.

On the other hand, by generalizing inequality (52), we have

Therefore,

Since the above bounds are uniform over the index k , equation (54) is implied. The above arguments also show that

So far, we have proved that the moments of the gradient norm ∣ ∣ ∣ ∣ ∣ ∣ ∇ f ( x ( B ) k ) ∣ ∣ ∣ ∣ ∣ ∣ 2 is bounded after entering the E k = 1 phase. We proceed to bound their contribution to the N th iteration. To that end, we denote

This sequence is initialized with G 0 = 0 by definition. We establish the following recursion for sufficiently large T .

We note that conditioned on any fixed x ( B ) k the gradient norm function ||∇ f ( x ( B ) k +1 ) || 2 can be approximated with its linear expansion. Formally, let ˜ g ( x ) ≜ ∇ f ( x ( B ) k ) + ( x -x ( B ) k ) · ∇ 2 f ( x ( B ) k ) , we have

Then, in the eigenbasis of ˆ H , it is clear that

Recall that by Theorem 4.3, when T is sufficiently large, the moments of ∣ ∣ ∣ ∣ ∣ ∣ m k -∇ f ( x ( B ) k ) ∣ ∣ ∣ ∣ ∣ ∣ 2 +

R 0 ∣ ∣ ∣ ∣ ∣ ∣ ˆ H -∇ 2 f ( x ( B ) k ) ∣ ∣ ∣ ∣ ∣ ∣ F is upper bounded by any fixed quantity. Therefore, as a rough estimate, we have

when T is sufficiently large. Consequently, our needed recursion is implied by the monotonicity of E k , and we have

Finally, Theorem E.3 is proved by noting that

Hence, equation (50) is implied by equation (53) and inequality (60).

E.3 Proof of Theorem 4.4

Proof. Given Theorem E.3, our needed inequality (6) is implied by the strong convexity assumption. Particularly, the implication is due to the fact that ||∇ f ( x ) || 2 2 2 M ≥ f ( x ) -f ∗ for any x ∈ R d .

Remark E.6. Note that compared to the simple regret guarantee stated in inequality (6) , we have essentially proved a stronger statement that the moments of the gradient at the outcome of the bootstrapping stage follow similar power decay laws. Therefore, while we presented a final stage algorithm that uses non-isotropic sampling to be compatible with general bootstrapping stages, our specific bootstrapping stage actually allows for the use of isotropic (hyperspherical) sampling for gradient estimation in the final stage.

F Proofs of some useful propositions

F.1 Proof of Proposition C.1

Proof. Recall that all z j 's have zero expectations. By subgaussianity, we have that all even moments of z j are bounded as follows.

Using AM-GM inequality, the odd moments of z j can then be bounded using the even moments. Specifically,

Therefore, we have obtained the following upper bounds for the moment-generating function.

Applying inequality (62), the expression above can be bounded with a series of ( sσ j ) 2 . The coefficient of each ( sσ j ) 2 ℓ is no greater than 1 ℓ ! , which can be verified numerically for ℓ ≤ 2 and inductively for ℓ ≥ 3 . Hence, we have

Because z j 's are independent,

Inequality (16) is implied by Markov's bound. Specifically, for any K ≥ 0 ,

For the same reason, we also have

Hence, by union bound,

F.2 Proof of Proposition C.2

Proof. By subexponentiality, the moment-generating function of each | z j | is bounded as follows for any s < 1 σ j .

Because z j 's are independent,

We choose s = 1 / (3 ∑ j σ j ) , note that sσ j ≤ 1 / 3 , we have (1 -sσ j ) ≥ ( 2 3 ) 3 sσ j . Hence,

Then, inequality (18) is implied by Markov's bound, i.e.,

F.3 Proof of Proposition E.4

To prove the proposition for sufficiently large T , we focus on the regime where N ≥ 10 R 2 ρ 2 /M 2 +2 . We first use proof by contradiction to show the existence of k 0 ≤ 10 R 2 ρ 2 /M 2 such that || r k 0 || 2 < M/ρ . Assume the contrary, we have || r k || 2 ≥ M/ρ for all k ≤ 10 R 2 ρ 2 /M 2 . Recall we have proved earlier that (see inequality (42))

This assumption implies that R ≥ 0 . 6 M/ρ and ∣ ∣ ∣ ∣ ∣ ∣ x ∗ -x ( B ) k ∣ ∣ ∣ ∣ ∣ ∣ 2 ≥ 0 . 6 M/ρ for all k ≤ 10 R 2 ρ 2 /M 2 .

We characterize the evolution of x ( B ) k . By Cauchy's inequality and inequality (67),

Note that our assumed lower bound on N implies a lower bound on T . Numerically, one can prove that T 0 . 2 ≥ 20 ρR/M . Hence, the above inequality implies that

Then, by following the proof steps in Proposition E.1, we have that

where the second step is due to the construction of x ( B ) k +1 in Algorithm 4. Recall that ∣ ∣ ∣ ∣ ∣ ∣ x ∗ -x ( B ) 0 ∣ ∣ ∣ ∣ ∣ ∣ 2 ≤ R . The above inequality implies that if || r k || 2 ≥ M/ρ for all k ≤ 10 R 2 ρ 2 /M 2 , then ∣ ∣ ∣ ∣ ∣ ∣ x ∗ -x ( B ) k ∣ ∣ ∣ ∣ ∣ ∣ 2 is non-increasing and reaches below 0 at k = ⌊ 10 R 2 ρ 2 /M 2 ⌋ + 1 . However, this contradicts the fact that || r k || 2 is non-negative, and we must conclude the existence of k 0 ≤ 10 R 2 ρ 2 /M 2 such that || r k 0 || 2 < M/ρ .

Now consider any index k with || r k || 2 < M/ρ . By the construction of r k , we have that ∇ f ( x k ) = -r k · ∇ 2 f ( x k ) . Then, by the Lipschitz Hessian condition,

Using the strong convexity assumption and triangle inequality, the above bound implies that

Note that the first term in the bound above is upper bounded by the product of || x k +1 -x k -r k || 2 and the spectral norm of ∇ 2 f ( x k ) · ( ∇ 2 f ( x k +1 )) -1 . By the Lipschitz Hessian condition and strong convexity, this spectrum norm is further bounded by 1 + ρ M || x k +1 -x k || 2 . Therefore,

Weuse inequality (70) to bound || r k || 2 recursively. Assume T is sufficiently large such that T 0 . 2 ≥ 20 . As a rough estimate, we have

Recall we can find k 0 ≤ 10 R 2 ρ 2 /M 2 such that || r k 0 || 2 < M/ρ . By induction, we have || r k || 2 ≤ 0 . 6 M/ρ for all k > k 0 . Hence, when k > k 0 , inequality (70) implies the following relation, where the RHS is obtained by triangle inequality and the definition of E N -1 = 0 .

Therefore, by induction, we have

for any k > k 0 +1 , and numerically, || r N -1 || 2 ≤ 2 MT -0 . 2 /ρ if T 0 . 1 ≥ 2 k 0 +6 .

F.4 Proof of Proposition E.5

Proof of inequality (55) . We prove the inequality by considering two possible cases. In the first case, we assume that the ℓ 2 norms of both H -1 m and H ′-1 m ′ are no greater than R 0 . In this case, we have H m ∗ = H and H ′ m ′∗ = H ′ . Hence,

By the fact that all eigenvalues of H are lower bounded by M and the triangle inequality,

Then, the needed inequality is obtained by ∣ ∣ ∣ ∣ H -1 m ∗ m -H ′-1 m ′∗ m ′ ∣ ∣ ∣ ∣ 2 ≤ 2 R 0 , which follows from the construction of H m ∗ , H ′ m ′∗ and triangle inequality.

For the other case, we have max { || H -1 m || 2 , || H ′-1 m ′ || 2 } > R 0 . Without loss of generality, we assume that m ∗ ≥ m ′∗ . To be rigorous, here we adopted the convention that m ∗ = -∞ if the ℓ 2 norms of H -1 m is no greater than R 0 , and the same for m ′∗ accordingly. Based on this assumption, the condition in this case can be simplified as || H -1 m || 2 > R 0 , and we have that || H -1 m ∗ m || 2 = R 0 . Furthermore, we also have m ∗ > M .

To prove the needed inequality, we introduce an intermediate variable H ′ m ∗ , which is defined as the symmetric matrix sharing the eigenbasis of H ′ , but with each eigenvalue λ replaced with max { λ, m ∗ } . Note that H m ∗ and H ′ m ∗ are obtained by projecting H and H ′ to a convex set of matrices under the Frobenius norm. We have that

Therefore, by following the same steps in the first case and noting that all eigenvalues of H ′ m ∗ are lower bounded by m ∗ , we have that

Compare the above to inequality (55), it remains to prove that

For brevity, we denote that

In the eigenbasis of H ′ , it is clear that

Hence, by Cauchy's inequality,

Recall that || c || 2 ≤ R 0 and in this case we have || a || 2 = R 0 . Therefore, the RHS of the above inequality is upper bounded by (1 -α ) || a || 2 2 , and we have

where the first step above is equivalent to inequality (75), and the second step is due to Cauchy's inequality. Finally, it remains to notice that the LHS of inequality (74) equals || a -c || 2 2 , which is upper bounded by the LHS of the above inequality, and its RHS equals the RHS of the above inequality. Hence, inequality (74) is proved.

Proof of inequality (56) . Firstly, if m ∗ = m ′∗ , we follow similar arguments from equation (71) to inequality (72). I.e., in this case, we have

Hence, by triangle inequality and inequality (73),

Then, for m ∗ > m ′∗ , we define H ′ m ∗ and a , b , c as in the earlier proof steps. We first prove the following key inequality.

Recall the assumption in this case implies that || a || 2 = R 0 . By taking the squares on both sides, the inequality above is equivalent to the following linear inequality of vector a .

By Cauchy's inequality, the LHS of inequality (79) is upper bounded by || a || 2 ·|| 8 R 2 0 b -2 || b -c || 2 2 c || 2 . The coefficient of || a || 2 in this expression can be further characterized as follows.

We prove that the contribution from the second term and the third term in the above expression is non-positive. To that end, note that the definition of H ′ m ∗ , H ′ m ′∗ and the assumption of m ∗ > m ′∗ imply that ( c -b ) · b ≥ 0 . We have the following inequalities.

Therefore, 0 ≤ 4 R 2 0 · ( R 2 0 -|| b || 2 2 ) -|| b -c || 2 2 · ( R 2 0 -|| c || 2 2 ) ≤ 4 R 2 0 · || b -c || 2 2 , and equation (80) implies that

By utilizing the above bound, inequality (79) is proved by noting that its RHS is non-negative, which can be proved using inequality (81). As mentioned earlier, this implies inequality (78).

To proceed further, we note that b -c lies in the eigenspace of H ′ m ∗ associated with eigenvalue m ∗ . Hence,

Therefore, by triangle inequality, we have

Note that by inequality (78) and the fact that all eigenvalues of H ′ m ∗ are lower bounded by m ∗ , we have

Therefore, it remains to upper bound the ℓ 2 norm of H ′ m ∗ ( a -b ) .

By the definition of vectors a , b ,

The triangle inequality implies that

Hence,

where the last step is due to inequality (73). Thus, inequality (56) is implied by the semi-positivedefiniteness of H ′ m ∗ -H ′ m ′∗ .

Finally, when m ∗ < m ′∗ , we let H m ′∗ denote the symmetric matrix sharing the same eigenbasis of H , but with each eigenvalue λ replaced by max { λ, m ′∗ } . Due to the equivalence of H and H ′ , our earlier proof steps imply that

Hence, by triangle inequality, we can use the above bound as follows.

Note that ∣ ∣ ∣ ∣ H -1 m ∗ m -H ′-1 m ′∗ m ′ ∣ ∣ ∣ ∣ 2 ≤ 2 R 0 . By inequality (73), it is clear that

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims are included in our theorem statements.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have included a discussion of constraints of our work.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have provided the full set of assumptions and a complete proof.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: This paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: This paper does not include experiments.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: the paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: This paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: This paper does not include experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: This paper follows the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a theory paper and there is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: This paper does not use existing assets

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper consider the problem of optimizing second-order smooth and strongly convex functions where the algorithm is only accessible to noisy evaluations of the objective function it queries. Authors provide the first tight characterization for the rate of the minimax simple regret by developing matching upper and lower bounds. They propose an algorithm that features a combination of a bootstrapping stage and a mirror-descent stage.\n\nSTRENGTHS:\nThe authors consider a problem formulation where the gradient of the function or higher order derivatives are not available. This problem formulation is of extreme interest in the field of Machine Learning. However, the work has many weaknesses (see below).\n\nWEAKNESSES:\n1. Almost no mention is made of work motivation. This is an important part that **cannot be missed**...\n\n\n\n\n2. Also, this paper gives the impression that **it was produced clearly not in this year**, since the main references are **from 2021** and earlier. But since 2021, the research, in particular in the area of zero-order optimization has a lot of interest in the community and is highly advanced compared to what the authors cited as related works. For example, in addition to a number of cited papers studying the zero-order optimization problem with the assumption of increased smoothness of the function, there is already a more recent paper that has been missed: _Akhavan et al. (2023)_ [1], which proposes an improved analysis of gradient approximation bias estimation as well as second moment estimation. Similarly, there is a weakly described section presenting different gradient approximations, where the main focus fell on the smoothing vector (Gaussian vector or Randomized vector). But, there are already a number of papers, such as _Gasnikov et al. (2024)_ [2], which have shown that the randomized approximation clearly performs better on practical experiments than the Gaussian approximation. It seems better to give an overview of different gradient approximations, including $l_1$ randomization, in this section\n\n\n3. Since I have already mentioned the topic of practical experiments, it is important to note that despite the fact that the authors provide mainly theoretical work, a paper that is intended for presentation at the NeurIPS conference should **show the effectiveness of the proposed results** in real life. This paper lacks any experiments, which is a major weakness of the paper.\n\n4. The structure of the article is poorly chosen... It seems that the sections \"Main contribution\", \"Discussion\" and \"Conclusion\" would **have improved the presentation** of the article's results.\n\n5. Regarding the results themselves, essentially **only one result is presented**, which is a minor contribution. It would be interesting to consider when the function is not only twice differentiable but also has a higher order of smoothness, as well as other settings of the problem: convex, non-convex, maybe the Polyak–Lojasiewicz condition, etc.\n\n[1] Akhavan A. et al. (2023). Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm //arXiv preprint arXiv:2306.02159.\n\n[2] Gasnikov, A. et al. (2024). Highly smooth zeroth-order methods for solving optimization problems under the PL condition. Computational Mathematics and Mathematical Physics.\n\nQUESTIONS:\nI have a few questions and suggestions:\n\n- It seems that Theorem 3.1 can be improved in terms of dependence on the problem dimension $d$ using tricks from the work of _Akhavan et al. (2023)_ [1]\n\n- Can the authors explain what the final result would look like if $w_t$ does not necessarily have zero mean, but is independent of vector $e$ (randomization on the sphere)? Also, if we consider deterministic noise $|\\delta(x)| \\leq \\Delta$ as noise. \n\n[1] Akhavan A. et al. (2023). Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm //arXiv preprint arXiv:2306.02159.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies zero order stochastic optimization (the learner has access to noisy function evaluation only) assuming the objective is $M$-strongly convex and has a $\\rho$-Lipschitz (in forbenius norm) Hessian. Matching upper and lower bounds are presented, which establish a tight result of suboptimality $\\Theta(d T^{-2/3}M^{-1})$ (w.h.p.) after $T$ function evaluations.\n\nSTRENGTHS:\n* A tight result is achieved for the setting studied.\n* On the technical side, the gradient evaluation procedure in the final stage of the algorithm uses samples from the ellipsoid induced by the Hessian approximation rather than opting for isotropic sampling. Further, a novel approach for analyzing the bias and variance of this estimation procedure is presented that yields sharp bounds. This technique has not been used by prior works, and seems crucial for the minimax optimal result.\n* This result improves over the most closely related prior state-of-the-art Novitskii & Gasnikov (2021) by a factor of $d^{1/3}$, who require an additional Lipschitz gradient assumption (see discussion in Appendix A).\n\nWEAKNESSES:\nNothing in particular.\n\nQUESTIONS:\n- Lines 125-126: 1-subgaussianity already implies variance $\\leq 1$.\n- Akhavan et al. (2020) consider the Lipschitz gradient setting and do not require a Lipschitz Hessian. Hence it seems their results and yours are incomparable (correct?). Given this I did not understand lines 315-317: \"On the other hand ... well.\" Do you include Akhavan et al in this statement? It doesn't seem true that a direct application of your algorithm order wise improves the result of Akhavan et al because your result does not apply in their setting.\n- Since we consider a twice differentiable objective over a bounded domain, it follows the gradient is in fact Lipschitz. With this in mind, are the assumptions of Novitskii & Gasnikov (2021) indeed stronger than those made in this work?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the convergence of zeroth order stochastic optimization for a class of strongly convex, second-order smooth objective functions.  The authors assume that the noisy one-point feedback oracle is available, and the additive noise is subgaussian. Both the asymptotic upper bound and the matching lower bound for the minimax simple regret are provided.\n\nSTRENGTHS:\nThe authors propose a novel algorithm that combines gradient estimation, bootstrapping procedures, and a mirror descent procedure. This combination enables accurate estimation of gradients and Hessians under second-order smoothness, thereby achieving the optimal rate.\n\nWEAKNESSES:\nThe idea of this work is interesting, but the presentation needs improvement. Some main results should be explicitly and rigorously stated as theorems, for example, eq (6). Additionally, it is unclear why the two stages in Algorithm 4 are necessary. Providing intuition and explanation for the two stages and the choice of parameters in Algorithm 4 would be beneficial.\n\nFurthermore, the proof of the upper bounds contains gaps. The result obtained in line 601 is $\\lim_{T\\to\\infty} T^{2/3}E[\\|\\nabla f(x_N^B)\\|^3]=0$. It is unclear how $M$ and $\\rho$ enter the final result and match eq(6) with the specific choice of $\\epsilon$ in line 170. More detailed steps in the proof would help clarify this connection.\n\nQUESTIONS:\n1. Remark 4.2: I cannot understand the first sentence. Are there any established results for the case of the cubic polynomial function $f$?\n2. Algorithm 4: Is the $\\hat H$  returned by HessianEst invertible?\n3. Proof of Theorem 3.1: \n1)  In line 200, \"First, recall our construction ensures that,\" could the author provide the proof of this statement? 2) In lines 203-204, the two displays omit the dimension dependence. Will this omission impact the final dimension dependence?\n4. Line 358: I cannot see how to apply Lemma B.2, also known as Proposition 7 in [Yu et. al. 2023]. What is the $Var_{f\\sim p}[f(x)]$?\n\nLIMITATIONS:\nThe paper proposes a new algorithm but does not provide any numerical results to validate it. There are many interesting questions that could be investigated numerically regarding this method, such as its dependence on the length of the first and second stages.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of zero-order optimization of a strongly convex function whose Hessian is Lipschitz continuous. The proposed algorithm exploits zero-order information from the oracle to estimate the Hessian and gradient of the function at each iteration. Using these estimates, the authors employ a second-order method to achieve convergence for the optimization error. The results are asymptotic and valid for a sufficiently large number of function evaluations. The main contribution of the paper is improving the dependency of the achieved optimization error with respect to the dimension. The derived upper bound is minimax optimal with respect to the number of function evaluations, dimension, and the strong convexity parameter.\n\nSTRENGTHS:\nThe result of the paper regarding the upper bound is a significant and surprising contribution. In the literature, there was an optimality gap with respect to the dimension: the existing lower bound was of the order $d$, while the upper bound was of the order $d^{4/3}$. The main anticipation was that the lower bound was not tight enough. However, the result of this paper shows that $d$ is minimax optimal, which I find to be a very valuable observation.\n\nWEAKNESSES:\nThe main issue I encountered while reading the paper is that the authors should provide more motivation for their algorithm. For instance, they use two different gradient estimators, GradientEst and BootstrappingEst, but it is unclear why both are necessary and why one is not sufficient. The assumption on the noise is also more restrictive compared to previous work in the literature, see e.g. [1]. The authors assume that the noise is sub-Gaussian, whereas other papers have only assumed a finite second moment. The authors did not explain why this stronger restriction on the noise is needed.\n\nThe main algorithm of the paper, Algorithm 4, consists of two stages. While the first stage of the algorithm seems natural, the second stage is not well motivated.\n\n\n[1] Arya Akhavan, Massimiliano Pontil, and Alexandre Tsybakov. Exploiting higher order smoothness in derivative-free optimization and continuous bandits. Advances in Neural Information Processing Systems, 2020.\n\nQUESTIONS:\n1. As I mentioned earlier, could the authors explain why there is a need for two different gradient estimators?\n\n2. Why does the noise need to be sub-Gaussian? Specifically, the results of Theorem 4.1 also hold for any noise with a finite second moment.\n\n3. Could the authors motivate the second stage of the algorithm? The authors mentioned, \"The role of the final stage is to ensure that $f(x_B) - f(x^*)$ is sufficiently small with high probability,\" but why you need such a guarantee with high probability, and not just in expectation?\n\n4. Could the authors explain why they need a high probability bound in Theorem 4.3 and why expectation bounds are not sufficient?\n\n5. I am a bit confused about where the authors used Assumption A (3). The parameter $R$ does not appear anywhere in the algorithms or in the final bound.\n\n6. It would be helpful if the authors provided a discussion on the number of function evaluations used overall in the main algorithm.\n\n7. A version of Theorem 3.2 has already appeared in [1] without the dependency on $\\rho$. So, the novel aspect of this lower bound is its dependency on $\\rho$, which I found interesting. Please correct me if I'm wrong, but unfortunately, I cannot see the proof of this theorem in the paper.\n\n8. I would like to know the authors' thoughts on higher-order smoothness: if we assume that the higher derivatives are uniformly bounded, do the authors believe it is possible to achieve the dependency of $d$ with respect to the dimension by estimating higher-order derivatives?\n\n9. Could the authors specify the lower bound for $T$ for all the results in the paper to hold?\n\n\n[1] Arya Akhavan, Massimiliano Pontil, and Alexandre Tsybakov. Exploiting higher order smoothness in derivative-free optimization and continuous bandits. Advances in Neural Information Processing Systems, 2020.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}