Approximate optimization of convex functions with outlier noise

1 Introduction . Unconstrained convex optimization is among the most well-studied problems in mathematical optimization and has extensive applications in machine learning [ 7 ] . In the classic unconstrained convex minimization problem , we are given oracle access to a convex function f : Rd → R , and seek to efficiently find a point x̃ that is close to the minimizer of f ( call it x∗ ) 1 . To obtain meaningful guarantees for approximating the minimizer x∗ , one needs to make certain assumptions on the convexity and smoothness of f . In particular , f is commonly assumed to be α-strongly convex and β-smooth for some β > α > 0 . This means that , when f is twice differentiable , its second derivative in any direction is between α and β . For ease of presentation , we will say a convex function is ( α , β ) -nice if it satisfies these two conditions . It is well known that the minimizer of an ( α , β ) -nice function can be approximated arbitrarily well in polynomial time by the classic gradient descent algorithm , or its accelerated version due to Nesterov [ 18 ] . To formally state the performance of these two algorithms , let us suppose an ( α , β ) -nice function f : Rd → R is given to us as a zeroth order oracle , which returns the function value f ( x ) on any input point x ∈ Rd . Then we have : Theorem 1.1 ( [ 7 ] ) . Given any initial point x0 ∈ Rd and > 0 , the classic gradient descent algorithm outputs a point x̃ s.t . ‖x̃− x∗‖2 ≤ using O ( d ( β/α ) log ‖x0−x∗‖ ) oracle queries . 1Sometimes , instead of finding a point close to x∗ , the goal is to find a point whose function value is close to f ( x∗ ) . However , as we point out later , it is more natural to study approximations in the domain in our setting . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Theorem 1.2 ( [ 18 ] ) . Given any initial point x0 ∈ Rd and > 0 , the accelerated gradient descent algorithm outputs a point x̃ s.t . ‖x̃− x∗‖2 ≤ using O ( d √ β/α log ‖x0−x ∗‖ ) oracle queries . We note that the quantity β/α is called the condition number of the function f . The efficiency of a convex minimization algorithm is usually measured with respect to such a quantity . Given that the complexity of convex minimization with exact oracles has been well understood , a natural question is “ can we still minimize a convex function efficiently if the oracle is to some extent inaccurate ? ” There has in fact been a rich body of work in efforts to answer this question . Roughly , they can be divided into two categories based on their assumptions on the oracle ’ s inaccuracy . The first category studies the case when the oracle is corrupted by stochastic noise2 — namely , the errors of the oracle are assumed to be random and independently drawn from some distribution [ 10 , 20 , 19 ] . Notice that as the oracle is correct in expectation , one can obtain good estimates to the true values efficiently by averaging over sufficiently many points in a small neighborhood . As a result , it is still possible to approximate the minimizer x∗ to within arbitrarily small distance in polynomial time . The main focus of these results is thus to obtain optimal algorithm efficiency , ideally matching that of Nesterov ’ s accelerated gradient descent . The second category , on the other hand , considers the adversarial noise . That is , the errors are no longer drawn from certain distributions , but rather are added adversarially , while obeying certain constraints . A representative such model is what we call pointwise-bounded noise model [ 22 , 5 ] , in which the only assumption on the noise is that it is pointwise bounded in magnitude ; other than that , the specific perturbation on each point can be arbitrary . Formally , it is allowed for the oracle to return , on an input point x , a value in the range [ f ( x ) − , f ( x ) + ] ( absolute errors ) or [ ( 1 − ) f ( x ) , ( 1 + ) f ( x ) ] ( relative errors ) , for some > 0 . [ 5 ] shows that when the errors are absolute and is on the order of 1/d , there is a polynomial-time algorithm that can find a point with function value arbitrarily close to f ( x∗ ) . Whereas [ 22 ] shows that , in sharp contrast , when is about 1/ √ d or larger , no polynomial-time algorithm is able to find a point with function value within some constant error of f ( x∗ ) , for both absolute and relative errors . Our model . Following the second line of research above , in this paper we study another type of adversarial noise model , called outlier noise model , which can be seen as a natural variant of the pointwise-bounded noise model in [ 22 , 5 ] . In this model , the only assumption we make is a bound on the “ number ” of points corrupted by the noise ; apart from that , we do not assume any bounds on the magnitude of the errors on the corrupted points . Formally , we are given access to an exact zeroth order oracle of a function f̂ that differs from the true convex function f only on a set C ⊂ Rd , with the guarantee that the volume of C is at most that of a d-dimensional ball of radius K , for some K > 0 . Notice that although C is bounded in volume , for any x ∈ C , f̂ ( x ) and f ( x ) can differ arbitrarily . Though variants of each other , both the pointwise-bounded noise model in [ 22 , 5 ] and our model may be considered as special cases of a more general type of adversarial noise model , namely the ` p-bounded noise model . To see this , let us write the noise as a function η : Rd → R , such that the noisy zeroth order oracle given to us corresponds to the function f + η . Then , in the ( absolute ) pointwise-bounded noise model , η is bounded in ` ∞-norm ( they assume ‖η‖∞ ≤ ) , whereas in the outlier noise model , η is bounded in ` 0-norm ( we assume ‖η‖0 ≤ vol ( radius-K ball ) ) . It will certainly be an interesting future direction to explore what can be achieved for minimizing convex functions with general ` p-bounded noise . We would also like to point out that in our noise model , it is more natural to consider getting as close as possible to x∗ as opposed to finding a point with function value close to f ( x∗ ) . This is because our only assumption on the noise is that the corrupted region C , which is in the domain of the function f , is bounded in volume , and in particular , it may be located around x∗ . Therefore , we believe it makes the most sense to measure the quality of the solution of a minimization algorithm by its distance to the optimal in the domain ( specifically , how the distance compares with the corruption radius K ) . Our results . Let us first consider what one might expect to achieve in the outlier noise model . By the observation made above , it is not hard to see that we can not hope to always find a point that is within distance K of the minimizer x∗ , as an adversary could potentially corrupt some radius-K ball 2This model more often deals with the first order oracle , which gives the ( noisy ) gradient∇f ( x ) at a point x. around x∗ , making it impossible for us to know where the true minimizer lies . However , is it possible to obtain a distance bound that is close to K ( e.g . O ( K ) ) ? Our first result in this paper is a lower bound showing that , somewhat surprisingly , this goal is in general impossible to achieve , even for algorithms that are allowed an unbounded number of queries to the oracle . In fact , our lower bound indicates that the best distance bound one can hope for is at least Ω ( K √ β/α ) , where we recall that α , β are the “ niceness ” of the function . This result is a consequence of the existence of two ( α , β ) -nice functions that only differ in a small region of the domain , but whose minimizers are sufficiently far apart . We then develop , as our second result , an efficient algorithm that finds a point within distance O ( K √ β/α ) of the minimizer x∗ , thus matching our lower bound up to constant factors . Roughly , our algorithm performs two stages of gradient descent , using approximate gradients computed from the noisy zeroth order oracle . We state our results formally in the theorems below . For simplicity , let us say a zeroth order oracle of a function f is K-corrupted if it is perturbed by an outlier noise of corruption radius K. Theorem 1.3 ( Lower bound ; formal version appears as Theorem 3.1 ) . For α , β , K > 0 and sufficiently large d , there exist two ( α , β ) -nice functions f0 , f1 : Rd → R which satisfy the following conditions : ( i ) The ` 2-distance between the minimizers of f0 , f1 is Ω ( K √ β/α ) ; ( ii ) The total volume of points where f0 and f1 differ is at most that of a d-dimensional ball of radius K. To see the implication of this theorem , consider an adversary that randomly picks an index i ∈ { 0 , 1 } and lets fi be the true convex function , but always uses f0 as the K-corrupted oracle . Then no matter which point our algorithm outputs , with probability at least 1/2 over the randomness of i , it is Ω ( K √ β/α ) away from the minimizer of the true convex function fi . Theorem 1.4 ( Algorithm ; formal version appears as Theorems 4.2 , 5.1 ) . Let α , β , K > 0 and d be sufficiently large . There is an algorithm that , given access to a K-corrupted oracle of an ( α , β ) -nice function f : Rd → R and an initial point x0 ∈ Rd , makes Õ ( d · ( β2/α2 ) · log ‖x0 − x∗‖ ) 3 queries to the oracle and outputs a point x̂ s.t . ‖x̂− x∗‖2 ≤ O ( K √ β/α ) . Here x∗ is the minimizer of f . We note that in both our results , our conditions on d are roughly that d ≥ Ω ( log β/α ) . In other words , we require the function ’ s condition number β/α to be at most 2O ( d ) , a quantity exponential in d. Our techniques . We now briefly describe our techniques used to obtain the above results . Our lower bound proceeds by proving the existence of two ( α , β ) -nice functions f0 and f1 s.t . ( i ) they differ only in an ellipsoid with volume at most that of a ball of radius K ; ( ii ) their minimizers are at ` 2-distance Ω ( K √ β/α ) -away from each other . As mentioned above , this implies that even with infinitely many queries , one can not approximate the minimizer x∗ to within distance o ( K √ β/α ) . More concretely , we start by letting f0 be a simple ( α , β ) -nice quadratic function whose minimizer is at the origin . Then the existence of our desired function f1 will follow by two steps : first we show that there exists another function f ′ with certain good properties , then we obtain f1 by taking a piecewise combination of f0 and f ′ . Specifically , the properties that we need f ′ to satisfy is as follows : ( i ) f ′ is ( α , β ) -nice ; ( ii ) both the function values and the gradients of f0 and f ′ agree on all points on the periphery of an ellipsoid E that is centered at the origin and has bounded volume ; ( iii ) the minimizer of f ′ has coordinate of the form ( Ω ( K √ β/α ) , 0 , . . . , 0 ) . We show that this essentially boils down to a convex function interpolation task where the function values and gradients at some infinitely many points are given . To this end , we adapt an interpolation result from [ 23 ] , which was proposed to accommodate interpolation from finitely many points , to prove the existence of f ′ . Finally , we construct the function f1 by letting f1 = f ′ inside the ellipsoid E , but f1 = f0 outside of E . For the upper bound , our algorithm proceeds in two stages . In the first stage , we come to within distance O ( K ( β/α ) ) of the minimizer x∗ and in the subsequent stage , we improve it to O ( K √ β/α ) . The first algorithm essentially follows a gradient descent , but using an approximate gradient at each step . In order to estimate the gradient at a point x , we set up a system of linear equations , where each equation adds a constraint on the derivative at x along a uniformly random direction . There are however two types of noise in these equations , one from using a zeroth order oracle to compute first 3Here and throughout the paper , we write Õ ( f ) to denote O ( f · poly ( log f ) ) . order information , and the other from the outlier noise added to the oracle . While we can solve this noisy linear system with good enough accuracy via an exhaustive search , we show that using an LP decoding routine in [ 12 ] , we can solve it more efficiently in polynomial time . We note that using the latter is only to improve the running time , as both approaches result in the same query complexity . However , the one bottleneck in this approach is that at any point , the ball of radius < K around it could potentially be ( nearly ) completely corrupted . Thus , to get a meaningful estimate of the gradient , we have to sample points which are more than distance K-far apart . This tradeoff eventually allows us to get O ( K ( β/α ) ) -close to the minimizer . To get to the optimal closeness of O ( K √ β/α ) , we next start at a point which is guaranteed to be O ( K ( β/α ) ) -close to the true minimizer . We now consider the function f̄ which is defined as the average of f in a ball of radius ( say ) 2K . It is not hard to verify that f̄ continues to be ( α , β ) -nice . Moreover , the minimizers of f̄ and f can be shown to be O ( K √ β/α ) -close to each other . Thus , it suffices to get close to the minimizer of f̄ . We will do so by simulating a gradient descent on f̄ . It therefore boils down to how we can efficiently approximate the gradients of f̄ . By the definition of f̄ , the gradient∇f̄ ( x ) is also equal to the average of the gradients∇f ( y ) over all points y within distance 2K of x . This suggests that we can approximate ∇f̄ ( x ) by averaging the gradients ∇f ( y ) at sufficiently many randomly sampled y ’ s , and bounding the error using concentration inequalities for sums of random vectors . Now a key observation is that , since these y ’ s are sampled randomly from a radius 2K-ball , it is highly likely that each sampled y sits in a mostly uncorrupted neighborhood , as long as the dimension is sufficiently high . Consequently , we can use the LP decoding approach above to obtain an accurate estimate of the gradient at each of these points . Prior work on noisy convex optimization . Other than the works mentioned above , noisy convex optimization has also been investigated in the context of multi-armed bandits and regret minimization [ 2 , 8 , 1 ] . In the direction of convex optimization under adversarial noise , the early results in fact date back to the 90s by [ 3 ] . Specifically for the pointwise-bounded noise model , there are subsequent works such as [ 21 , 24 , 17 ] that have improved on the guarantees of [ 5 ] . Due to space limitation , we include some other related work in Appendix A . Organization . In Section 2 , we set up a few notations and give some basic definitions and technical preliminaries . In Section 3 , we prove our lower bound result . In Section 4 , we give a first algorithm that gets us to within distance O ( K ( β/α ) ) of x∗ . In Section 5 , we give a second algorithm that gets us to within distance O ( K √ β/α ) of x∗ . In Section 6 , we propose several future directions .

Summary: The paper studies the problem of convex optimization using a zeroth order oracle with adversarial noise.  In particular, the paper studies the setting where there is some convex function $f: \mathbb{R}^d \rightarrow \mathbb{R}$ and the learner may interact with an oracle that returns the value of $f$ at a given point except the oracle is corrupted on some set of measure equal to $\textsf{Vol}(B(0,K))$ (where $B(0,K)$ is the ball of radius $K$) on which the oracle may return an arbitrary value.  The paper gives an algorithm that outputs a point that is within $K \sqrt{\beta/\alpha}$ of the minimizer of $f$ (where $\beta/\alpha$ is the condition number of $f$).  They also prove a matching information-theoretic lower bound that no algorithm can guarantee to get closer than $K \sqrt{\beta/\alpha}$.  At a high level, their algorithm is based on estimating the gradient at each point using the corrupted zeroth order oracle and then running gradient-descent.