Approximate optimization of convex functions with outlier noise

1 Introduction . Unconstrained convex optimization is among the most well-studied problems in mathematical optimization and has extensive applications in machine learning [ 7 ] . In the classic unconstrained convex minimization problem , we are given oracle access to a convex function f : Rd → R , and seek to efficiently find a point x̃ that is close to the minimizer of f ( call it x∗ ) 1 . To obtain meaningful guarantees for approximating the minimizer x∗ , one needs to make certain assumptions on the convexity and smoothness of f . In particular , f is commonly assumed to be α-strongly convex and β-smooth for some β > α > 0 . This means that , when f is twice differentiable , its second derivative in any direction is between α and β . For ease of presentation , we will say a convex function is ( α , β ) -nice if it satisfies these two conditions . It is well known that the minimizer of an ( α , β ) -nice function can be approximated arbitrarily well in polynomial time by the classic gradient descent algorithm , or its accelerated version due to Nesterov [ 18 ] . To formally state the performance of these two algorithms , let us suppose an ( α , β ) -nice function f : Rd → R is given to us as a zeroth order oracle , which returns the function value f ( x ) on any input point x ∈ Rd . Then we have : Theorem 1.1 ( [ 7 ] ) . Given any initial point x0 ∈ Rd and > 0 , the classic gradient descent algorithm outputs a point x̃ s.t . ‖x̃− x∗‖2 ≤ using O ( d ( β/α ) log ‖x0−x∗‖ ) oracle queries . 1Sometimes , instead of finding a point close to x∗ , the goal is to find a point whose function value is close to f ( x∗ ) . However , as we point out later , it is more natural to study approximations in the domain in our setting . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Theorem 1.2 ( [ 18 ] ) . Given any initial point x0 ∈ Rd and > 0 , the accelerated gradient descent algorithm outputs a point x̃ s.t . ‖x̃− x∗‖2 ≤ using O ( d √ β/α log ‖x0−x ∗‖ ) oracle queries . We note that the quantity β/α is called the condition number of the function f . The efficiency of a convex minimization algorithm is usually measured with respect to such a quantity . Given that the complexity of convex minimization with exact oracles has been well understood , a natural question is “ can we still minimize a convex function efficiently if the oracle is to some extent inaccurate ? ” There has in fact been a rich body of work in efforts to answer this question . Roughly , they can be divided into two categories based on their assumptions on the oracle ’ s inaccuracy . The first category studies the case when the oracle is corrupted by stochastic noise2 — namely , the errors of the oracle are assumed to be random and independently drawn from some distribution [ 10 , 20 , 19 ] . Notice that as the oracle is correct in expectation , one can obtain good estimates to the true values efficiently by averaging over sufficiently many points in a small neighborhood . As a result , it is still possible to approximate the minimizer x∗ to within arbitrarily small distance in polynomial time . The main focus of these results is thus to obtain optimal algorithm efficiency , ideally matching that of Nesterov ’ s accelerated gradient descent . The second category , on the other hand , considers the adversarial noise . That is , the errors are no longer drawn from certain distributions , but rather are added adversarially , while obeying certain constraints . A representative such model is what we call pointwise-bounded noise model [ 22 , 5 ] , in which the only assumption on the noise is that it is pointwise bounded in magnitude ; other than that , the specific perturbation on each point can be arbitrary . Formally , it is allowed for the oracle to return , on an input point x , a value in the range [ f ( x ) − , f ( x ) + ] ( absolute errors ) or [ ( 1 − ) f ( x ) , ( 1 + ) f ( x ) ] ( relative errors ) , for some > 0 . [ 5 ] shows that when the errors are absolute and is on the order of 1/d , there is a polynomial-time algorithm that can find a point with function value arbitrarily close to f ( x∗ ) . Whereas [ 22 ] shows that , in sharp contrast , when is about 1/ √ d or larger , no polynomial-time algorithm is able to find a point with function value within some constant error of f ( x∗ ) , for both absolute and relative errors . Our model . Following the second line of research above , in this paper we study another type of adversarial noise model , called outlier noise model , which can be seen as a natural variant of the pointwise-bounded noise model in [ 22 , 5 ] . In this model , the only assumption we make is a bound on the “ number ” of points corrupted by the noise ; apart from that , we do not assume any bounds on the magnitude of the errors on the corrupted points . Formally , we are given access to an exact zeroth order oracle of a function f̂ that differs from the true convex function f only on a set C ⊂ Rd , with the guarantee that the volume of C is at most that of a d-dimensional ball of radius K , for some K > 0 . Notice that although C is bounded in volume , for any x ∈ C , f̂ ( x ) and f ( x ) can differ arbitrarily . Though variants of each other , both the pointwise-bounded noise model in [ 22 , 5 ] and our model may be considered as special cases of a more general type of adversarial noise model , namely the ` p-bounded noise model . To see this , let us write the noise as a function η : Rd → R , such that the noisy zeroth order oracle given to us corresponds to the function f + η . Then , in the ( absolute ) pointwise-bounded noise model , η is bounded in ` ∞-norm ( they assume ‖η‖∞ ≤ ) , whereas in the outlier noise model , η is bounded in ` 0-norm ( we assume ‖η‖0 ≤ vol ( radius-K ball ) ) . It will certainly be an interesting future direction to explore what can be achieved for minimizing convex functions with general ` p-bounded noise . We would also like to point out that in our noise model , it is more natural to consider getting as close as possible to x∗ as opposed to finding a point with function value close to f ( x∗ ) . This is because our only assumption on the noise is that the corrupted region C , which is in the domain of the function f , is bounded in volume , and in particular , it may be located around x∗ . Therefore , we believe it makes the most sense to measure the quality of the solution of a minimization algorithm by its distance to the optimal in the domain ( specifically , how the distance compares with the corruption radius K ) . Our results . Let us first consider what one might expect to achieve in the outlier noise model . By the observation made above , it is not hard to see that we can not hope to always find a point that is within distance K of the minimizer x∗ , as an adversary could potentially corrupt some radius-K ball 2This model more often deals with the first order oracle , which gives the ( noisy ) gradient∇f ( x ) at a point x. around x∗ , making it impossible for us to know where the true minimizer lies . However , is it possible to obtain a distance bound that is close to K ( e.g . O ( K ) ) ? Our first result in this paper is a lower bound showing that , somewhat surprisingly , this goal is in general impossible to achieve , even for algorithms that are allowed an unbounded number of queries to the oracle . In fact , our lower bound indicates that the best distance bound one can hope for is at least Ω ( K √ β/α ) , where we recall that α , β are the “ niceness ” of the function . This result is a consequence of the existence of two ( α , β ) -nice functions that only differ in a small region of the domain , but whose minimizers are sufficiently far apart . We then develop , as our second result , an efficient algorithm that finds a point within distance O ( K √ β/α ) of the minimizer x∗ , thus matching our lower bound up to constant factors . Roughly , our algorithm performs two stages of gradient descent , using approximate gradients computed from the noisy zeroth order oracle . We state our results formally in the theorems below . For simplicity , let us say a zeroth order oracle of a function f is K-corrupted if it is perturbed by an outlier noise of corruption radius K. Theorem 1.3 ( Lower bound ; formal version appears as Theorem 3.1 ) . For α , β , K > 0 and sufficiently large d , there exist two ( α , β ) -nice functions f0 , f1 : Rd → R which satisfy the following conditions : ( i ) The ` 2-distance between the minimizers of f0 , f1 is Ω ( K √ β/α ) ; ( ii ) The total volume of points where f0 and f1 differ is at most that of a d-dimensional ball of radius K. To see the implication of this theorem , consider an adversary that randomly picks an index i ∈ { 0 , 1 } and lets fi be the true convex function , but always uses f0 as the K-corrupted oracle . Then no matter which point our algorithm outputs , with probability at least 1/2 over the randomness of i , it is Ω ( K √ β/α ) away from the minimizer of the true convex function fi . Theorem 1.4 ( Algorithm ; formal version appears as Theorems 4.2 , 5.1 ) . Let α , β , K > 0 and d be sufficiently large . There is an algorithm that , given access to a K-corrupted oracle of an ( α , β ) -nice function f : Rd → R and an initial point x0 ∈ Rd , makes Õ ( d · ( β2/α2 ) · log ‖x0 − x∗‖ ) 3 queries to the oracle and outputs a point x̂ s.t . ‖x̂− x∗‖2 ≤ O ( K √ β/α ) . Here x∗ is the minimizer of f . We note that in both our results , our conditions on d are roughly that d ≥ Ω ( log β/α ) . In other words , we require the function ’ s condition number β/α to be at most 2O ( d ) , a quantity exponential in d. Our techniques . We now briefly describe our techniques used to obtain the above results . Our lower bound proceeds by proving the existence of two ( α , β ) -nice functions f0 and f1 s.t . ( i ) they differ only in an ellipsoid with volume at most that of a ball of radius K ; ( ii ) their minimizers are at ` 2-distance Ω ( K √ β/α ) -away from each other . As mentioned above , this implies that even with infinitely many queries , one can not approximate the minimizer x∗ to within distance o ( K √ β/α ) . More concretely , we start by letting f0 be a simple ( α , β ) -nice quadratic function whose minimizer is at the origin . Then the existence of our desired function f1 will follow by two steps : first we show that there exists another function f ′ with certain good properties , then we obtain f1 by taking a piecewise combination of f0 and f ′ . Specifically , the properties that we need f ′ to satisfy is as follows : ( i ) f ′ is ( α , β ) -nice ; ( ii ) both the function values and the gradients of f0 and f ′ agree on all points on the periphery of an ellipsoid E that is centered at the origin and has bounded volume ; ( iii ) the minimizer of f ′ has coordinate of the form ( Ω ( K √ β/α ) , 0 , . . . , 0 ) . We show that this essentially boils down to a convex function interpolation task where the function values and gradients at some infinitely many points are given . To this end , we adapt an interpolation result from [ 23 ] , which was proposed to accommodate interpolation from finitely many points , to prove the existence of f ′ . Finally , we construct the function f1 by letting f1 = f ′ inside the ellipsoid E , but f1 = f0 outside of E . For the upper bound , our algorithm proceeds in two stages . In the first stage , we come to within distance O ( K ( β/α ) ) of the minimizer x∗ and in the subsequent stage , we improve it to O ( K √ β/α ) . The first algorithm essentially follows a gradient descent , but using an approximate gradient at each step . In order to estimate the gradient at a point x , we set up a system of linear equations , where each equation adds a constraint on the derivative at x along a uniformly random direction . There are however two types of noise in these equations , one from using a zeroth order oracle to compute first 3Here and throughout the paper , we write Õ ( f ) to denote O ( f · poly ( log f ) ) . order information , and the other from the outlier noise added to the oracle . While we can solve this noisy linear system with good enough accuracy via an exhaustive search , we show that using an LP decoding routine in [ 12 ] , we can solve it more efficiently in polynomial time . We note that using the latter is only to improve the running time , as both approaches result in the same query complexity . However , the one bottleneck in this approach is that at any point , the ball of radius < K around it could potentially be ( nearly ) completely corrupted . Thus , to get a meaningful estimate of the gradient , we have to sample points which are more than distance K-far apart . This tradeoff eventually allows us to get O ( K ( β/α ) ) -close to the minimizer . To get to the optimal closeness of O ( K √ β/α ) , we next start at a point which is guaranteed to be O ( K ( β/α ) ) -close to the true minimizer . We now consider the function f̄ which is defined as the average of f in a ball of radius ( say ) 2K . It is not hard to verify that f̄ continues to be ( α , β ) -nice . Moreover , the minimizers of f̄ and f can be shown to be O ( K √ β/α ) -close to each other . Thus , it suffices to get close to the minimizer of f̄ . We will do so by simulating a gradient descent on f̄ . It therefore boils down to how we can efficiently approximate the gradients of f̄ . By the definition of f̄ , the gradient∇f̄ ( x ) is also equal to the average of the gradients∇f ( y ) over all points y within distance 2K of x . This suggests that we can approximate ∇f̄ ( x ) by averaging the gradients ∇f ( y ) at sufficiently many randomly sampled y ’ s , and bounding the error using concentration inequalities for sums of random vectors . Now a key observation is that , since these y ’ s are sampled randomly from a radius 2K-ball , it is highly likely that each sampled y sits in a mostly uncorrupted neighborhood , as long as the dimension is sufficiently high . Consequently , we can use the LP decoding approach above to obtain an accurate estimate of the gradient at each of these points . Prior work on noisy convex optimization . Other than the works mentioned above , noisy convex optimization has also been investigated in the context of multi-armed bandits and regret minimization [ 2 , 8 , 1 ] . In the direction of convex optimization under adversarial noise , the early results in fact date back to the 90s by [ 3 ] . Specifically for the pointwise-bounded noise model , there are subsequent works such as [ 21 , 24 , 17 ] that have improved on the guarantees of [ 5 ] . Due to space limitation , we include some other related work in Appendix A . Organization . In Section 2 , we set up a few notations and give some basic definitions and technical preliminaries . In Section 3 , we prove our lower bound result . In Section 4 , we give a first algorithm that gets us to within distance O ( K ( β/α ) ) of x∗ . In Section 5 , we give a second algorithm that gets us to within distance O ( K √ β/α ) of x∗ . In Section 6 , we propose several future directions .

Summary: The authors consider a convex minimization problem with a certain noisy oracle. Specifically, the function to be minimized is assumed to be alpha-strongly convex, and beta-Lipschitz (referred to as alpha-beta nice). It is also assumed that we have access to the function values everywhere except for an ellipsoid with bounded volume. Within the mentioned ellipsoid, the oracle may select the values "adversarially" so as to confuse the minimization algorithm. In this setting, the authors first show the existence of two alpha-beta nice functions that differ on a bounded ball and such that their minimizers are a certain distance apart. This provides a bound on what one could possibly achieve. Next, they provide two algorithms that guarantee to produce approximate minimizers that are sufficiently close to the minimizer (within the bounds permitted by the first negative result), along with an upper bound on the number of queries that the algorithms make.