Information-constrained optimization: can adaptive processing of gradients help?

1 Introduction . Distributed optimization has emerged as a central tool in federated learning for building statistical and machine learning models for data distributed across users . In addition , large scale optimization is typically implemented in a distributed fashion over multiple machines or multiple cores within the same machine . These distributed implementations fit naturally in the oracle framework of first-order optimization ( see [ 22 ] ) where in each iteration a user or machine computes the gradient oracle output . Due to practical local constraints such as communication bandwidth , privacy concerns , or computational issues , the entire gradient can not be directly made available to the optimization algorithm . Instead , the gradients must be passed through a randomized mechanism which , respectively , ensures privacy of user data ( local privacy constraints ) ; or compresses them to a small number of bits ( communication constraints ) ; or only computes a few coordinates of the gradient ( computational constraints ) . Motivated by these applications , we consider first-order optimization under such local information constraints placed on the gradient oracle . When designing a first-order optimization algorithm under local information constraints , one not only needs to design the optimization algorithm itself , but also the algorithm for local processing of the gradient estimates . Many such algorithms have been proposed in recent years ; see , for instance , [ 12 ] , [ 1 ] , [ 6 ] , [ 15 ] , [ 29 ] , [ 16 ] , and the references therein for privacy constraints ; [ 28 ] , [ 7 ] , [ 30 ] , [ 18 ] , ∗Supported by NSF-CCF-1846300 ( CAREER ) , NSF-CCF-1815893 . †Part of this work was done while at IBM Research , supported by a Goldstine Postdoctoral Fellowship . ‡Supported by a Ph.D. fellowship from Wipro Ltd. §Supported by a grant from Robert Bosch Center for Cyber Physical Systems ( RBCCPS ) , Indian Institute of Science . The full version of this paper [ 2 ] contains the proofs of all results discussed here , as well as other additional details . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . [ 14 ] , [ 25 ] , [ 4 ] , [ 11 ] , [ 17 ] , [ 20 ] , [ 19 ] , [ 27 ] , and the references therein for communication constraints ; [ 24 , 26 ] for computational constraints . However , these algorithms primarily consider nonadaptive procedures for gradient processing ( with the exception of [ 14 ] ) : that is , where the scheme used to process the gradients at any iteration can not depend on information gleaned from previous iterations . As a result , the following question remains largely open : Can adaptively processing gradients improve convergence in information-constrained optimization ? In this paper , we study this question for optimization over both convex and strongly convex function families and under the three different local constraints mentioned above : local privacy , communication , and computational . For each of these constraints , we establish lower bounds on convergence rates which hold even when the gradients are adaptively processed . In the next sections , we cover prior related work , before elaborating on our results and techniques . 1.1 Prior work . The model studied here can be viewed as extension of the classical query complexity model in [ 22 ] . Without information constraints , [ 5 ] study this setting and provide a general recipe for proving convex optimization lower bounds for different function families . Specifically , they reduce the problem of optimization with a first-order oracle to a mean estimation problem whose probability of error is lower bounded using Fano ’ s method ( cf . [ 31 ] ) . While our work also relies on a reduction to mean estimation , we deviate from their approach , using instead Assouad ’ s method to prove lower bounds for various function families . This different approach in turn enables us to derive lower bounds for adaptive processing of gradients . In the information-constrained setting , motivated by privacy concerns , [ 12 ] consider the problem where the gradient estimates must pass through a locally differentially private ( LDP ) channel . However , in their setting the LDP channels for all time steps are selected at the start of optimization algorithm – in other words , the channel selection strategy is nonadaptive . Similarly , [ 20 ] and [ 19 ] consider a similar problem and impose the constraint that the gradient estimates be quantized to a fixed number of bits . They , too , fix the quantization channels used at each time step at the start of optimization algorithm . In contrast , in this paper , we allow for adaptive channel selection strategies ; as a result , the lower bounds established in these papers do not apply to our setting , and are more restrictive than our bounds . The results of Duchi and Rogers [ 13 ] for Bernoulli product distributions could be combined with our construction to obtain tight lower bounds for optimization in p ∈ [ 1 , 2 ] under LDP constraints , but would not extend to the entire range of p. The work of Braverman , Garg , Ma , Nguyen , and Woodruff [ 9 ] on communication constraints , also for p ∈ [ 1 , 2 ] , is relevant as well ; however , their bounds on mutual information can not be applied directly , as their setting ( Gaussian distributions ) would not satisfy our almost sure gradient oracle assumption . [ 14 ] provide adaptive quantization schemes for convex and ` 2 Lipschitz function family . While the worst-case convergence guarantees for the quantizers in [ 14 ] are similar to those in [ 7 ] and [ 20 ] , it shows some practical improvements over the state-of-the-art for some specific problem instances . This suggests that while adaptive quantization may not help in the worst case for non-smooth convex optimization , it may be useful for a smaller subclass of convex optimization problems . 1.2 Our contributions . We consider a general formulation for first-order optimization under local information constraints where the output of the gradient oracle must be passed through a channel W selected from a fixed channel family W . This family W captures the information constraints ; see Section 2.3 for a description of the channel families corresponding to our constraints of interest . Specifically , the gradient is sent as input to this channel W , and the algorithm receives the output . In each iteration of the algorithm , the channel to be used in that iteration can be selected adaptively based on previously received channel outputs by the algorithm or channels to be used throughout can be fixed upfront , nonadaptively . The detailed problem setup is given in Section 2.1 . We obtain general lower bounds for optimization of convex and strongly convex functions usingW , when adaptivity is allowed . These bounds are then applied to the specific constraints of interest to obtain our main results . Our first contribution is in showing that adaptive gradient processing does not help for some of the most typical optimization problems . Namely , we prove that for most regimes of local privacy , communication , or computational constraints , adaptive gradient processing has nearly the same convergence rate as nonadaptive gradient processing for both convex and strongly convex function families . As a consequence , this shows that the nondaptive LDP algorithms from [ 12 ] and nonadaptive compression protocols from [ 20 ] , [ 19 ] are optimal for private and communication-constrained optimization , respectively . In another direction , under computational constraints , where we are allowed to compute only one gradient coordinate , we show that standard RCD ( cf . [ 10 , Section 6.4 ] ) , which employs uniform ( nonadaptive ) sampling of gradient coordinates , is optimal for convex and strongly convex function families . This proves that adaptive sampling of gradient coordinates does not improve over nonadaptive sampling strategies . As previously discussed , prior work in both the local privacy and communication-constrained settings concerned itself with the family of convex functions , with no lower bounds known for the more restricted family of strongly convex functions , even for nonadaptive gradient processing protocols . The key obstacle is the fact that during the reduction from optimization to mean estimation , the known difficult case for the strongly convex family , even when analyzed for nonadaptive protocols , leads to an estimation problem using adaptive protocols , and the lack of known lower bounds for adaptive information-constrained estimation prevented this approach from succeeding . Specifically , this difficult case has gradients that can depend on the query point which in turn can be chosen based on previously observed channel outputs , an issue which does not arise in the case of the convex family , where the lower bounds are derived using affine functions for which the gradients do not depend on query point . We manage to circumvent this issue , by relying on a different reduction which lets us capitalize on a recent lower bound for adaptive mean estimation . Crucially , this recent lower bound does apply to adaptive estimation algorithms as well , bringing the last missing piece to the puzzle . This lets us derive lower bounds for both convex Lipschitz and strongly convex functions under adaptive gradient processing . For general function classes , the results discussed above show that adaptive processing of gradients does not help . This begs the question of whether there are natural function families where adaptive gradient processing can lead to significant savings . Our third contribution is to provide an example of such a family : specifically , we exhibit a natural optimization problem ( entailing ` 2 minimization ) under computational constraints for which adaptive gradient processing provides a polynomial factor improvement in convergence rates compared to nonadaptive processing . The key feature of this optimization problem is that the resulting gradients have structured sparsity ; adaptivity then allows for a two-phase optimization procedure , where the algorithm first “ explores ” to find the structure before , in a second phase , “ exploiting ” it to obtain more focused information about the function to minimize . However , nonadaptive gradient processing protocols can not easily exploit this hidden structure , as finding it is now akin to locating a needle in a haystack ; and thus exhibit much slower convergence rates . Notation : Throughout the paper q denotes the Hölder conjugate of p ( that is , q = p/ ( p− 1 ) ) . a ∨ b and a ∧ b denote . . . max { a , b } and min { a , b } , respectively . We denote by log the logarithm to the base 2 and by ln the logarithm to the base e. The information-theoretic quantities , such as mutual information and KL divergence , are defined using ln . ln∗ ( a ) denotes the number of times ln must be iteratively applied to a before the result is at most 1 . { e1 , ... , ed } denotes the standard basis of Rd . 2 Setup and preliminaries . 2.1 Optimization under information constraints . We consider the problem of minimizing an unknown convex function f : X → R over its domain X using oracle access to noisy subgradients of the function ( cf . [ 22 ] ) . In our setup , the gradient estimates supplied by the oracle must pass through a channel W ,5 chosen from a fixed set of channels W , and the optimization algorithm π only has access to the output of this channel . The channel familyW represents our information constraints . 5 A channel W with input alphabetX and output alphabetY , denoted W : X → Y , represents the conditional distribution of the output of a randomized function given its input . In particular , W ( ·|x ) is the conditional distribution of the channel given that the input is x ∈ X . In detail , the framework is as follows : 1 . At iteration t , the optimization algorithm π makes a query for point xt to the oracle O . 2 . Upon receiving the point xt , the oracle outputs ĝ ( xt ) , where E [ ĝ ( xt ) | xt ] ∈ ∂f ( xt ) and ∂f ( xt ) is the subgradient set of function f at xt . 3 . The subgradient estimate ĝ ( xt ) is passed through a channel Wt ∈ W , and the output of the channel Wt along with outputs of channels { Wi } i∈ [ t−1 ] , where [ n ] = { 1 , . . . , n } , can be used by the first order optimization algorithm to further update xt to xt+1 . Denote by π the first-order optimization algorithm which is allowed T queries to the oracle O and , after the tth query , gets back the output Yt with distribution Wt ( · | ĝ ( xt ) ) . Denote the set of all such optimization algorithms by Πt . Our goal is to select channels Wts and an optimization algorithm π to get a small worst-case optimization error . Two classes of channel selection strategies are of interest : adaptive and nonadaptive . Adaptive channel selection strategies can be described as follows . The channel Wt selected at time t may depend on the previous outputs of channels { Wi } i∈ [ t−1 ] . Specifically , denoting by Yt and Yt , respectively , the output alphabet and the output for the channel used at time t , the adaptive channel selection strategy S : = ( S1 , . . . , ST ) over T iterations comprises mappings St : Yt−1 →W taking Y1 , . . . Yt−1 as input and yielding a channel inW as output . We write SW , T for the collection of all such channel selection strategies . For nonadaptive channel selection strategies , we fix the channels { Wt } t∈ [ T ] ∈ Wobl through which all the gradient estimates must pass at the start of the optimization algorithm.6 Denote the class of all nonadaptive strategies by SNAW , T . We measure the performance of an optimization protocol π and a channel selection strategy S for a given function f and oracle O using the metric E ( f , O , π , S ) defined as E ( f , O , π , S ) : = E [ f ( xT ) −min x∈X f ( x ) ] , ( 1 ) where the expectation is over the randomness in xT . For various function and oracle classes , denoted by O , the channel constraint familyW , and the number of iteration T , we will characterize the adaptive minmax optimization error E∗ ( X , O , T , W ) = inf π∈ΠT inf S∈SW , T sup ( f , O ) ∈O E ( f , O , π , S ) ( 2 ) and the corresponding nonadaptive minmax optimization error ENA∗ ( X , O , T , W ) = inf π∈ΠT inf S∈SNAW , T sup ( f , O ) ∈O E ( f , O , π , S ) . ( 3 ) Since nonadaptive channel selection strategies are a subset of adaptive channel selection strategies , we have ENA∗ ( X , O , T , W ) ≥ E∗ ( X , O , T , W ) .

Summary: Here, all the references are to the supplementary material section.  This work studies the problem of first-order optimization under local constraints, where at each step an agent computes a sub-gradient and the sub-gradient should pass through a channel. Then, output of the channel will be available for the optimization algorithm. The act of the channel is to constraint the information of the sub-gradient. Their idea is to construct an adaptive method, where at each step the computed sub-gradient depends on the previous outputs of the channel, in advance. They call this approach adaptive and the main goal of the authors is to investigate whether there is any gain in using adaptive methods compared to nonadaptive ones.   Assumptions on the objective function: Denote the objective function as $f$, which is defined on a compact subset of $\mathbb{R}^d$. They studied the above problem in the case that: 1. $f$ is Lipschitz with respect to $\ell_p$-norm, where $p \geq 1$. 2. $f$ is (strongly) convex.  Different types of information constraints: Performance of an adaptive method has been studied in this paper over three different types of information constraints: 1. Local differential privacy (defined in page 7) 2. Communication constraints (defined in page 8) 3. Computational constraints (defined in page 8)  Contributions: They used a novel technique for obtaining lower bounds by using Asoud's lemma (outlined in the lines 80 to 87, on page 3 and Section 4) for the adaptive methods. By comparing their derived lower bounds with the upper bounds for the non-adaptive scheme, they concluded that there is no gain in using adaptive methods, in general setting.  At the end, they studied adaptivity for the special case of structured $\ell$_2 minimization (defined in Section 5.1, page 32.), for computational constraints (oblivious channels). First they derived a lower bound for non-adaptive setting, then they proposed an adaptive algorithm (Algorithm 1, page 28) and concluded that adaptive methods significantly outperform non-adaptive ones in this specific example.