Distributed Zero-Order Optimization under Adversarial Noise

1 Introduction . We study the problem of distributed optimization where each node ( or agent ) has an objective function fi : Rd → R and exchange of information is limited between neighbouring agents within a prescribed network of connections . The goal is to minimize the average of these objectives on a closed bounded convex set Θ ⊂ Rd , min x∈Θ f ( x ) where f ( x ) = 1 n n∑ i=1 fi ( x ) . ( 1 ) Distributed optimization has been widely studied in the literature , we refer to Tsitsiklis et al . [ 1986 ] , Nedic and Ozdaglar [ 2009 ] , Nedic et al . [ 2010 ] , Boyd et al . [ 2011 ] , Duchi et al . [ 2012 ] , Jakovetić et al . [ 2014 ] , Lobel et al . [ 2011 ] , Kia et al . [ 2015 ] , Shi et al . [ 2014 ] , Jakovetić [ 2019 ] , Scaman et al . [ 2019 ] , Pu et al . [ 2021 ] and references therein . This problem has broad applications such as multi-agent target seeking Liu et al . [ 2017 ] , distributed learning Kraska et al . [ 2013 ] , and wireless networks Park et al . [ 2020 ] , among others . We address problem ( 1 ) from the perspective of zero-order distributed optimization . That is we assume that only function values can be queried by the algorithm , subject to measurement noise . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . During the optimization procedure , each agent maintains a local copy of the variables which are sequentially updated based on local and neighboring functions ’ queries . We wish to devise such optimization procedures which are efficient in bounding the average optimization error and cumulative regret in terms of the functions ’ properties and network topology . Contributions Our principal contribution is a distributed zero-order optimization algorithm , introduced in Section 2 , which we show to achieve tight rates of convergence under certain assumptions on the objective functions , outlined in Section 3 . Specifically , we consider that the local objectives fi are β-Hölder and the average objective f is α-strongly convex . The algorithm relies on a novel zero-order gradient estimator , presented in Section 4 . Although conceptually very simple , this estimator , when employed within our algorithm , allows us to obtain an O ( d2 ) computational gain as well as improved error rates than previous state-of-the-art zero-order optimization procedures Akhavan et al . [ 2020 ] , Bach and Perchet [ 2016 ] , in the special case of standard ( undistributed ) setting . Another key advantage of our approach is due to the general noise model presented in Section 5 , under which function values are queried . The noise variables do not need to be zero mean or independently sampled , and thus they include “ adversarial ” noise . In Section 6 , we derive the rates of convergence for the cumulative regret and the optimization error of the proposed algorithm , and in Section 7 we consider the special case of 2-smooth functions . The rates highlight the dependency with respect to the number of variables d , the number of function queries T , the spectral gap of the network matrix 1− ρ , and the parameters n , α and β . The bounds enjoy a better dependency on 1− ρ than previous bounds on zero-order distributed optimization Qu and Li [ 2018 ] , Yu et al . [ 2019 ] , Tang et al . [ 2019 ] . We also compare our bounds to related lower bounds in Akhavan et al . [ 2020 ] for undistributed setting , observing that our rates are optimal either with respect to T and α , or with respect to T and d. Previous Work We briefly comment on previous related work and defer to Section 8 for a more in depth discussion and comparison . For both deterministic and stochastic scenarios of problem ( 1 ) , a large body of literature is devoted to first-order gradient based methods with a consensus scheme ( see the papers cited above and references therein ) . On the other hand , the study of zero-order methods was started only recently Qu and Li [ 2018 ] , Sahu et al . [ 2018b , a ] , Hajinezhad et al . [ 2019 ] , Yu et al . [ 2019 ] , Tang et al . [ 2019 ] . The works Qu and Li [ 2018 ] , Yu et al . [ 2019 ] , Tang et al . [ 2019 ] are dealing with zero-order distributed methods in noise-free settings while the noisy setting is developed in Hajinezhad et al . [ 2019 ] , Sahu et al . [ 2018b , a ] . Namely , Hajinezhad et al . [ 2019 ] considers 2-point zero-order methods with stochastic queries for non-convex optimization but assume that the noise is the same for both queries , which makes the problem analogous to noise-free scenario in terms of optimization rates . Papers Sahu et al . [ 2018b , a ] study zero-order distributed optimization for strongly convex and β-smooth functions fi with β ∈ { 2 , 3 } . They derive bounds on the optimization error , though without providing closed form expressions . Notation Throughout we denote by 〈· , ·〉 and ‖ · ‖ be the standard inner product and Euclidean norm on Rd , respectively , and by ‖ · ‖∗ the spectral norm of a matrix . The notation I is used for the n-dimensional identity matrix and 1 for the vector in Rn with all entries equal to 1 . We denote by ej the j-th canonical basis vector in Rd . For any set A , the number of elements in A is denoted by |A| . For x ∈ R , the value bxc is the maximal integer less than x . For every closed convex set Θ ⊂ Rd and x ∈ Rd we denote by ProjΘ ( x ) = argmin { ‖z − x‖ : z ∈ Θ } the Euclidean projection of x onto Θ . We denote by diam ( Θ ) the Euclidean diameter of Θ . Finally we let U [ −1 , 1 ] be the uniform distribution on [ −1 , 1 ] . 2 The Problem . Let n be the number of agents and let G = ( V , E ) be an undirected graph , where V = { 1 , . . . , n } is the set of nodes and E ⊆ V × V is the set of edges . The adjacency matrix of G is the symmetric matrix ( Aij ) ni , j=1 defined as Aij = 1 , if ( i , j ) ∈ E and zero otherwise . We consider the following sequential learning framework , where each agent i gets values of function fi corrupted by noise and shares information with other agents . At step t , agent i acts as follows : • makes queries and gets noisy values of fi , • provides a local output ui ( t ) based on these queries and on the past information , • broadcasts ui ( t ) to neighboring agents , Algorithm 1 Distributed Zero-Order Gradient Input Communication matrix ( Wij ) ni , j=1 , step sizes ( ηt > 0 ) T0−1 t=1 Initialization Choose initial vectors x1 ( 1 ) = · · · = xn ( 1 ) ∈ Rd For t = 1 , . . . , T0 − 1 For i = 1 , . . . , n 1 . Build an estimate gi ( t ) of the gradient∇fi ( xi ( t ) ) using noisy evaluations of fi 2 . Update xi ( t+1 ) = ∑n k=1WikProjΘ ( x k ( t ) − ηtgk ( t ) ) End End Output Approximate minimizer x̄ ( T0 ) = 1n ∑n i=1 x i ( T0 ) of the average objective f = 1n ∑n i=1 fi • updates its local variable using information from other agents as follows : xi ( t+1 ) = n∑ j=1 Wiju j ( t ) , where W = ( Wij ) ni , j=1 is a given matrix called the consensus matrix . Below we use the following condition on the consensus matrix . Assumption A. Matrix W is symmetric , doubly stochastic , and ρ : = ∥∥W − n−111 > ∥∥∗ < 1 . Matrix W accounts for the connectivity properties of the network . If Wij = 0 the agents i and j are not connected ( do not exchange information ) . Often W is defined as a doubly stochastic matrix function of the adjacency matrix A of the graph . One popular example is as follows : Wij = Aij γmax { d ( i ) , d ( j ) } if i 6= j , 1− ∑ k : k 6=i Aki γmax { d ( i ) , d ( k ) } if i = j , where d ( i ) = ∑n j=1Aij is the degree of node i and γ > 0 is a constant . Then , clearly , W = ( Wij ) is a symmetric and doubly stochastic matrix , andWij = 0 if agents i and j are not connected . Moreover , we have ρ < 1 − c/n2 for a constant c > 0 ( see Qu and Li [ 2018 ] , Olshevsky [ 2014 ] ) . Values of spectral gaps ρ for some other W reflecting different network topologies can be found in Duchi et al . [ 2012 ] . Typically , ρ < 1− an , where an = Ω ( n−1 ) or an = Ω ( n−2 ) . Parameter ρ can be viewed as a measure of difference between the distributed problem and a standard optimization problem . If the graph of communication is a complete graph a natural choice is W = n−11n1 > n and then ρ = 0 . For more examples of consensus matrices W , see Olshevsky and Tsitsiklis [ 2009 ] , Duchi et al . [ 2012 ] and references therein . The local outputs ui can be defined in different ways . Our approach is outlined in Algorithm 1 . At Step 1 , an estimate of the gradient of the local objective fi at xi ( t ) is constructed . This involves a randomized procedure that we describe and justify in Section 4 . The local output ui is defined as an update of the projected gradient algorithm with such an estimated gradient . At Step 2 of the algorithm , each agent computes the next point by a local consensus gradient descent step , which uses local and neighbor information . Step 2 of the algorithm is known as gossip method , see e.g. , Boyd et al . [ 2006 ] ) , which was initially introduced as an approach for the networks with the imposed connection between the nodes changing by time . We also refer to Sayin et al . [ 2017 ] for similar algorithms in the context of distributed stochastic first-order gradient methods . 3 Assumptions on Local Objectives . In this section , we give some definitions and introduce our assumptions on the local objective functions f1 , . . . , fn . Definition 1 . Denote by Fβ ( L ) the set of all functions f : Rd → R that are ` = bβc times differentiable and satisfy , for all x , z ∈ Rd the Hölder-type condition∣∣∣∣f ( z ) − ∑ 0≤|m|≤ ` 1 m ! Dmf ( x ) ( z − x ) m ∣∣∣∣ ≤ L‖z − x‖β , ( 2 ) Algorithm 2 Gradient Estimator with 2d Queries Input Function F : Rd → R and point x ∈ Rd Requires Kernel K : [ −1 , 1 ] → R , parameter h > 0 Initialization Generate random r from uniform distribution on [ −1 , 1 ] For j = 1 , . . . , d 1 . Obtain noisy values yj = F ( x+ hrej ) + ξj and y′j = F ( x− hrej ) + ξ′j 2 . Compute gj = 12h ( yj − y ′ j ) K ( r ) End Output g = ( gj ) dj=1 ∈ Rd estimator of∇F ( x ) where L > 0 , the sum is over the multi-index m = ( m1 , ... , md ) ∈ Nd , we used the notation m ! = m1 ! · · ·md ! , |m| = m1 + · · ·+md , and we defined , for every ν = ( ν1 , . . . , νd ) ∈ Rd , Dmf ( x ) νm = ∂|m|f ( x ) ∂m1x1 · · · ∂mdxd νm11 · · · ν md d . Elements of the class Fβ ( L ) are referred to as β-Hölder functions . Definition 2 . Function f : Rd → R is called 2-smooth if it is differentiable on Rd and there exists L̄ > 0 such that , for every ( x , x′ ) ∈ Rd × Rd , it holds that ‖∇f ( x ) −∇f ( x′ ) ‖ ≤ L̄‖x− x′‖ . Definition 3 . Let α > 0 . Function f : Rd → R is called α-strongly convex if f is differentiable on Rd and f ( x ) − f ( x′ ) ≥ 〈∇f ( x′ ) , x− x′〉+ α 2 ‖x− x′‖2 , ∀x , x′ ∈ Rd . Assumption B . Functions f1 , . . . , fn : ( i ) belong to the class Fβ ( L ) , for some β ≥ 2 , and ( ii ) are 2-smooth . In Section 6 we will analyse the convergence properties of Algorithm 1 when the objective function f in 1 is α-strongly convex . We stress that we do not need the functions f1 , . . . , fn , to be as well α-strongly convex . It is enough to make such an assumption on the compound function f , while the local functions fi only need to satisfy the smoothness conditions stated in Assumption B above .

Summary: This paper considers distributed optimization with zero-order information and under rather general setting of noise in the evaluation of function values. The authors propose a zero-order projected gradient descent algorithm, in which a novel zero-order gradient estimation scheme is used. An important contribution of this paper is to provide explicit upper bounds for the average cumulative regret and optimization error of the proposed algorithm. The obtained bound is clearly presented and show the advantage of the proposed algorithm over existing ones.