Distributed Zero-Order Optimization under Adversarial Noise

1 Introduction . We study the problem of distributed optimization where each node ( or agent ) has an objective function fi : Rd → R and exchange of information is limited between neighbouring agents within a prescribed network of connections . The goal is to minimize the average of these objectives on a closed bounded convex set Θ ⊂ Rd , min x∈Θ f ( x ) where f ( x ) = 1 n n∑ i=1 fi ( x ) . ( 1 ) Distributed optimization has been widely studied in the literature , we refer to Tsitsiklis et al . [ 1986 ] , Nedic and Ozdaglar [ 2009 ] , Nedic et al . [ 2010 ] , Boyd et al . [ 2011 ] , Duchi et al . [ 2012 ] , Jakovetić et al . [ 2014 ] , Lobel et al . [ 2011 ] , Kia et al . [ 2015 ] , Shi et al . [ 2014 ] , Jakovetić [ 2019 ] , Scaman et al . [ 2019 ] , Pu et al . [ 2021 ] and references therein . This problem has broad applications such as multi-agent target seeking Liu et al . [ 2017 ] , distributed learning Kraska et al . [ 2013 ] , and wireless networks Park et al . [ 2020 ] , among others . We address problem ( 1 ) from the perspective of zero-order distributed optimization . That is we assume that only function values can be queried by the algorithm , subject to measurement noise . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . During the optimization procedure , each agent maintains a local copy of the variables which are sequentially updated based on local and neighboring functions ’ queries . We wish to devise such optimization procedures which are efficient in bounding the average optimization error and cumulative regret in terms of the functions ’ properties and network topology . Contributions Our principal contribution is a distributed zero-order optimization algorithm , introduced in Section 2 , which we show to achieve tight rates of convergence under certain assumptions on the objective functions , outlined in Section 3 . Specifically , we consider that the local objectives fi are β-Hölder and the average objective f is α-strongly convex . The algorithm relies on a novel zero-order gradient estimator , presented in Section 4 . Although conceptually very simple , this estimator , when employed within our algorithm , allows us to obtain an O ( d2 ) computational gain as well as improved error rates than previous state-of-the-art zero-order optimization procedures Akhavan et al . [ 2020 ] , Bach and Perchet [ 2016 ] , in the special case of standard ( undistributed ) setting . Another key advantage of our approach is due to the general noise model presented in Section 5 , under which function values are queried . The noise variables do not need to be zero mean or independently sampled , and thus they include “ adversarial ” noise . In Section 6 , we derive the rates of convergence for the cumulative regret and the optimization error of the proposed algorithm , and in Section 7 we consider the special case of 2-smooth functions . The rates highlight the dependency with respect to the number of variables d , the number of function queries T , the spectral gap of the network matrix 1− ρ , and the parameters n , α and β . The bounds enjoy a better dependency on 1− ρ than previous bounds on zero-order distributed optimization Qu and Li [ 2018 ] , Yu et al . [ 2019 ] , Tang et al . [ 2019 ] . We also compare our bounds to related lower bounds in Akhavan et al . [ 2020 ] for undistributed setting , observing that our rates are optimal either with respect to T and α , or with respect to T and d. Previous Work We briefly comment on previous related work and defer to Section 8 for a more in depth discussion and comparison . For both deterministic and stochastic scenarios of problem ( 1 ) , a large body of literature is devoted to first-order gradient based methods with a consensus scheme ( see the papers cited above and references therein ) . On the other hand , the study of zero-order methods was started only recently Qu and Li [ 2018 ] , Sahu et al . [ 2018b , a ] , Hajinezhad et al . [ 2019 ] , Yu et al . [ 2019 ] , Tang et al . [ 2019 ] . The works Qu and Li [ 2018 ] , Yu et al . [ 2019 ] , Tang et al . [ 2019 ] are dealing with zero-order distributed methods in noise-free settings while the noisy setting is developed in Hajinezhad et al . [ 2019 ] , Sahu et al . [ 2018b , a ] . Namely , Hajinezhad et al . [ 2019 ] considers 2-point zero-order methods with stochastic queries for non-convex optimization but assume that the noise is the same for both queries , which makes the problem analogous to noise-free scenario in terms of optimization rates . Papers Sahu et al . [ 2018b , a ] study zero-order distributed optimization for strongly convex and β-smooth functions fi with β ∈ { 2 , 3 } . They derive bounds on the optimization error , though without providing closed form expressions . Notation Throughout we denote by 〈· , ·〉 and ‖ · ‖ be the standard inner product and Euclidean norm on Rd , respectively , and by ‖ · ‖∗ the spectral norm of a matrix . The notation I is used for the n-dimensional identity matrix and 1 for the vector in Rn with all entries equal to 1 . We denote by ej the j-th canonical basis vector in Rd . For any set A , the number of elements in A is denoted by |A| . For x ∈ R , the value bxc is the maximal integer less than x . For every closed convex set Θ ⊂ Rd and x ∈ Rd we denote by ProjΘ ( x ) = argmin { ‖z − x‖ : z ∈ Θ } the Euclidean projection of x onto Θ . We denote by diam ( Θ ) the Euclidean diameter of Θ . Finally we let U [ −1 , 1 ] be the uniform distribution on [ −1 , 1 ] . 2 The Problem . Let n be the number of agents and let G = ( V , E ) be an undirected graph , where V = { 1 , . . . , n } is the set of nodes and E ⊆ V × V is the set of edges . The adjacency matrix of G is the symmetric matrix ( Aij ) ni , j=1 defined as Aij = 1 , if ( i , j ) ∈ E and zero otherwise . We consider the following sequential learning framework , where each agent i gets values of function fi corrupted by noise and shares information with other agents . At step t , agent i acts as follows : • makes queries and gets noisy values of fi , • provides a local output ui ( t ) based on these queries and on the past information , • broadcasts ui ( t ) to neighboring agents , Algorithm 1 Distributed Zero-Order Gradient Input Communication matrix ( Wij ) ni , j=1 , step sizes ( ηt > 0 ) T0−1 t=1 Initialization Choose initial vectors x1 ( 1 ) = · · · = xn ( 1 ) ∈ Rd For t = 1 , . . . , T0 − 1 For i = 1 , . . . , n 1 . Build an estimate gi ( t ) of the gradient∇fi ( xi ( t ) ) using noisy evaluations of fi 2 . Update xi ( t+1 ) = ∑n k=1WikProjΘ ( x k ( t ) − ηtgk ( t ) ) End End Output Approximate minimizer x̄ ( T0 ) = 1n ∑n i=1 x i ( T0 ) of the average objective f = 1n ∑n i=1 fi • updates its local variable using information from other agents as follows : xi ( t+1 ) = n∑ j=1 Wiju j ( t ) , where W = ( Wij ) ni , j=1 is a given matrix called the consensus matrix . Below we use the following condition on the consensus matrix . Assumption A. Matrix W is symmetric , doubly stochastic , and ρ : = ∥∥W − n−111 > ∥∥∗ < 1 . Matrix W accounts for the connectivity properties of the network . If Wij = 0 the agents i and j are not connected ( do not exchange information ) . Often W is defined as a doubly stochastic matrix function of the adjacency matrix A of the graph . One popular example is as follows : Wij = Aij γmax { d ( i ) , d ( j ) } if i 6= j , 1− ∑ k : k 6=i Aki γmax { d ( i ) , d ( k ) } if i = j , where d ( i ) = ∑n j=1Aij is the degree of node i and γ > 0 is a constant . Then , clearly , W = ( Wij ) is a symmetric and doubly stochastic matrix , andWij = 0 if agents i and j are not connected . Moreover , we have ρ < 1 − c/n2 for a constant c > 0 ( see Qu and Li [ 2018 ] , Olshevsky [ 2014 ] ) . Values of spectral gaps ρ for some other W reflecting different network topologies can be found in Duchi et al . [ 2012 ] . Typically , ρ < 1− an , where an = Ω ( n−1 ) or an = Ω ( n−2 ) . Parameter ρ can be viewed as a measure of difference between the distributed problem and a standard optimization problem . If the graph of communication is a complete graph a natural choice is W = n−11n1 > n and then ρ = 0 . For more examples of consensus matrices W , see Olshevsky and Tsitsiklis [ 2009 ] , Duchi et al . [ 2012 ] and references therein . The local outputs ui can be defined in different ways . Our approach is outlined in Algorithm 1 . At Step 1 , an estimate of the gradient of the local objective fi at xi ( t ) is constructed . This involves a randomized procedure that we describe and justify in Section 4 . The local output ui is defined as an update of the projected gradient algorithm with such an estimated gradient . At Step 2 of the algorithm , each agent computes the next point by a local consensus gradient descent step , which uses local and neighbor information . Step 2 of the algorithm is known as gossip method , see e.g. , Boyd et al . [ 2006 ] ) , which was initially introduced as an approach for the networks with the imposed connection between the nodes changing by time . We also refer to Sayin et al . [ 2017 ] for similar algorithms in the context of distributed stochastic first-order gradient methods . 3 Assumptions on Local Objectives . In this section , we give some definitions and introduce our assumptions on the local objective functions f1 , . . . , fn . Definition 1 . Denote by Fβ ( L ) the set of all functions f : Rd → R that are ` = bβc times differentiable and satisfy , for all x , z ∈ Rd the Hölder-type condition∣∣∣∣f ( z ) − ∑ 0≤|m|≤ ` 1 m ! Dmf ( x ) ( z − x ) m ∣∣∣∣ ≤ L‖z − x‖β , ( 2 ) Algorithm 2 Gradient Estimator with 2d Queries Input Function F : Rd → R and point x ∈ Rd Requires Kernel K : [ −1 , 1 ] → R , parameter h > 0 Initialization Generate random r from uniform distribution on [ −1 , 1 ] For j = 1 , . . . , d 1 . Obtain noisy values yj = F ( x+ hrej ) + ξj and y′j = F ( x− hrej ) + ξ′j 2 . Compute gj = 12h ( yj − y ′ j ) K ( r ) End Output g = ( gj ) dj=1 ∈ Rd estimator of∇F ( x ) where L > 0 , the sum is over the multi-index m = ( m1 , ... , md ) ∈ Nd , we used the notation m ! = m1 ! · · ·md ! , |m| = m1 + · · ·+md , and we defined , for every ν = ( ν1 , . . . , νd ) ∈ Rd , Dmf ( x ) νm = ∂|m|f ( x ) ∂m1x1 · · · ∂mdxd νm11 · · · ν md d . Elements of the class Fβ ( L ) are referred to as β-Hölder functions . Definition 2 . Function f : Rd → R is called 2-smooth if it is differentiable on Rd and there exists L̄ > 0 such that , for every ( x , x′ ) ∈ Rd × Rd , it holds that ‖∇f ( x ) −∇f ( x′ ) ‖ ≤ L̄‖x− x′‖ . Definition 3 . Let α > 0 . Function f : Rd → R is called α-strongly convex if f is differentiable on Rd and f ( x ) − f ( x′ ) ≥ 〈∇f ( x′ ) , x− x′〉+ α 2 ‖x− x′‖2 , ∀x , x′ ∈ Rd . Assumption B . Functions f1 , . . . , fn : ( i ) belong to the class Fβ ( L ) , for some β ≥ 2 , and ( ii ) are 2-smooth . In Section 6 we will analyse the convergence properties of Algorithm 1 when the objective function f in 1 is α-strongly convex . We stress that we do not need the functions f1 , . . . , fn , to be as well α-strongly convex . It is enough to make such an assumption on the compound function f , while the local functions fi only need to satisfy the smoothness conditions stated in Assumption B above .

Summary: This paper studies zero-order optimization for strongly convex functions. The authors propose a novel 2d point zero-order gradient estimator, combing with distributed projected gradient descent, they provide upper bounds that are robust to adversary noises and have better dependence on (1 - \rho). In addition, they provide an improved analysis for 2-Holder functions using 2 point gradient estimator in that the dimension dependence is improved from d^1.5 to d.