Learning-to-learn non-convex piecewise-Lipschitz functions

1 Introduction . While learning-to-learn or meta-learning has long been an object of study [ 45 ] , in recent years it has gained significant attention as a multi-task paradigm for developing algorithms for learning in dynamic environments , from multiple data sources , and in federated settings . Such methods focus on using data from multiple tasks to improve performance when facing a new , potentially related task . A popular approach is initialization-based meta-learning , in which the meta-learner uses multi-task data to output an initialization for an iterative algorithm such as stochastic gradient descent ( SGD ) [ 25 ] . The flexibility of this approach has led to its widespread adoption , e.g . in robotics [ 23 ] and federated learning [ 18 ] , and to a growing number of attempts to understand it , both empirically and theoretically [ 20 , 29 , 24 , 40 , 42 ] . However , outside some stylized setups our learning-theoretic understanding of how to meta-learn an initialization is largely restricted to the convex Lipschitz setting . We relax both assumptions to study the meta-learning of online algorithms over piecewise-Lipschitz functions , which can be nonconvex and highly discontinuous . As no-regret online learning over such functions is impossible in-general , we study the case of piecewise-Lipschitz functions with dispersed discontinuities that do not concentrate in any small compact subset of the domain [ 11 ] . Such functions arise frequently in data-driven algorithm design , where the goal is to learn the optimal parameter settings of algorithms for difficult ( often NP-Hard ) problems over a distribution or sequence of instances [ 4 ] ; for example , a small change to the metric in cluster linkage can lead to a discontinuous change in the classification error [ 7 ] . In this paper , we also demonstrate that such losses are relevant in the setting of adversarial robustness , where we introduce a novel online formulation . For both cases , the associated problems are often solved across many time periods or for many different problem domains , resulting in natural multi-task structure that might improve performance . To the best of our knowledge , ours is the first theoretical study of meta-learning in both of these application settings . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In the single-task setting the problem of learning dispersed functions can be solved using simple methods such as the exponentially-weighted forecaster . To design an algorithm for learning to initialize online learners in this setting , we propose a method that optimizes a sequence of datadependent upper-bounds on the within-task regret [ 29 ] . The result is an averaged bound that improves upon the regret of the single-task exponential forecaster so long as there exists an initial distribution that can compactly contain many of the within-task optima of the different tasks . Designing the meta-procedure is especially challenging in our setting because it involves online learning over a set of distributions on the domain . To handle this we study a “ prescient ” form of the classic follow-the-regularized leader ( FTRL ) scheme that is run over an unknown discretization ; we then show the existence of another algorithm that plays the same actions but uses only known information , thus attaining the same regret while being practical to implement . To demonstrate the usefulness of our method , we study this algorithm in two settings . Multi-task data-driven algorithm design . We consider data-driven tuning of the parameters of combinatorial optimization algorithms for hard problems such as knapsack and clustering . The likely intractability of these problems on worst case instances have led to several approaches to study them in more realistic settings , such as smoothed analysis [ 44 ] and data-driven algorithm configuration [ 4 ] . Our setting is more realistic than those considered in prior work . It is more challenging than learning from iid instances [ 27 , 12 ] , but at the same time less pessimistic than online learning over adversarial problem instances [ 11 ] as it allows us to leverage similarity of problem instances coming from different but related distributions . We instantiate our bounds theoretically on several problems where the cost functions are piecewise-constant in the tuned parameters , allowing our meta-procedure to learn the right initial distribution for exponential forecasters . This includes well-known combinatorial optimization problems like finding the maximum weighted independent set ( MWIS ) of vertices on a graph , solving quadratic programs with integer constraints using algorithms based on the celebrated Goemans-Williamson algorithm , and mechanism design for combinatorial auctions . Then we consider experimentally the problem of tuning the right α for the α-Lloyd ’ s family of clustering algorithms [ 15 ] . In experimental evaluations on two datasets—a synthetic Gaussian mixture model and the well-known Omniglot dataset from meta-learning [ 33 ] —our meta-procedure leads to improved clustering accuracy compared to single-task learning to cluster . We also study our results for a family of greedy algorithms for the knapsack problem introduced by [ 27 ] and obtain similar results for a synthetic dataset . The results holds for both one-shot and five-shot tasks . Online robust meta-learning . The second instantiation of our meta-learning procedure is to a new notion of adversarial robustness for the setting of online learning , where our results imply robust meta-learning in the presence of outliers . In this setting , the adversary can make ( typically small ) modifications to some example x ∈ X , which can result in potentially large changes to the corresponding loss value lh ( x ) , where h ∈ H is our hypothesis . For instance , consider the well-studied setting of adversarial examples for classification of images using deep neural networks [ 36 , 17 ] . Given a neural network f , the adversary can perturb a datapoint x to a point x′ , say within a small Lp-ball around x , such that f ( x ) = f ( x′ ) but the true label of x′ does not match x , and therefore lf ( x ) 6= lf ( x′ ) . In general , under the adversarial influence , we observe a perturbed loss function l̃h ( x ) = lh ( x ) + ah ( x ) . Typically we are interested in optimizing both the perturbed loss l̃h ( x ) , i.e . measuring performance relative to optimum for adversarially perturbed losses , and the true loss lh ( x ) ( performance on the unobserved , unperturbed loss ) . For example , in the online learning setting , [ 1 ] consider perturbed loss minimization for linear dynamical systems , while [ 41 ] look at true { 0 , 1 } loss minimization in the presence of adversarial noise . Our approach ensures that regret for both the perturbed and true loss are small , for piecewise-Lipschitz but dispersed adversaries . 2 Preliminaries and initialization-dependent learning of dispersed functions . In this section we introduce our setup and notation for online learning of piecewise-Lipschitz functions in a multi-task environment . We then generalize existing results for the single-task setting in order to obtain within-task regret bounds that depend on both the initialization and the task data . This is critical for both defining a notion of task similarity and devising a meta-learning procedure . Meta-learning setup Following past setups [ 2 , 21 , 29 ] , for some T , m > 0 and all t ∈ [ T ] and i ∈ [ m ] we consider a meta-learner faced with a sequence of Tm loss functions ` t , i : C 7→ [ 0 , 1 ] over a compact subset C ⊂ Rd that lies within a ball B ( ρ , R ) of radius R around some point ρ ∈ Rd . Algorithm 1 Exponential Forecaster 1 : Input : step size parameter λ ∈ ( 0 , 1 ] , initialization w : C → R≥0 . 2 : Initialize w1 = w 3 : for i = 1 , 2 , . . . , m do 4 : Wi : = ∫ C wi ( ρ ) dρ 5 : Sample ρi with probability proportional to wi ( ρi ) , i.e . with probability pi ( ρi ) = wi ( ρi ) Wi 6 : Suffer ` i ( ρi ) and observe ` i ( · ) 7 : For each ρ ∈ C , set wi+1 ( ρ ) = e−λ ` i ( ρ ) wi ( ρ ) Here we used the notation [ n ] = { 1 , . . . , n } . Before each loss function ` t , i the meta-learner must pick an element ρt , i ∈ C before then suffering a loss or cost ` t , i ( ρt , i ) . For a fixed t , the subsequence ` t,1 , . . . , ` t , m defines a task for which we expect a single element ρ∗t ∈ C to do well , and thus we will use the within-task regret on task t to describe the quantity Rt , m = m∑ i=1 ` t , i ( ρt , i ) − ` t , i ( ρ∗t ) where ρ∗t ∈ arg min ρ∈C m∑ i=1 ` t , i ( ρ ) ( 1 ) In the single-task setting the goal is usually to show that Rt , m is sublinear in m , i.e . that the average loss decreases with more rounds . A key point here is that the functions we consider can have numerous global optima . In this work we will assume , after going through the m rounds of task t , that we have oracle access to a single fixed optimum for t , which we will refer to using ρ∗t and use in both our algorithm and to define the task-similarity . Note that in the types of applications we are interested in—piecewise-Lipschitz functions—the complexity of computing optima scales with the number of discontinuities . In the important special case of piecewise-constant functions , this dependency becomes logarithmic [ 19 ] . Thus this assumption does not affect the usefulness of the result . Our goal will be to improve the guarantees for regret in the single-task case by using information obtained from solving multiple tasks . In particular , we expect average performance across tasks to improve as we see more tasks ; to phrase this mathematically we define the task-averaged regret R̄T , m = 1 T T∑ t=1 Rt , m = 1 T T∑ t=1 m∑ i=1 ` t , i ( ρt , i ) − ` t , i ( ρ∗t ) ( 2 ) and claim improvement over single-task learning if in the limit of T →∞ it is smaller than Rt , m . Note that for simplicity in this work we assume all tasks have the same number of rounds within-task , but as with past work our results are straightforward to extend to the more general setting . Learning piecewise-Lipschitz functions We now turn to our target functions and within-task algorithms for learning them : piecewise-Lipschitz losses , i.e . functions that are L-Lipschitz w.r.t . the Euclidean norm everywhere except on measure zero subsets of the space ; here they may have arbitrary jump discontinuities so long they still bounded between [ 0 , 1 ] . Apart from being a natural setting of interest due to its generality compared to past work on meta-learning , this class of functions has also been shown to have important applications in data-driven algorithm configuration [ 11 ] ; there these functions represent the cost , e.g . an objective value or time-complexity , of algorithms for difficult problems such as integer programming , auction design , and clustering . This literature has also shown lower bounds demonstrating that no-regret learning piecewise-Lipschitz function is impossible in general , necessitating assumptions about the sequence . One such condition is dispersion , which requires that the discontinuities are not too concentrated . Definition 2.1 ( [ 11 ] ) . The sequence of random loss functions ` 1 , . . . , ` m is said to be β-dispersed with Lipschitz constant L if , for all m and for all ≥ m−β , we have that , in expectation over the randomness of the functions , at most Õ ( m ) functions ( the soft-O notation suppresses dependence on quantities beside , m and β , as well as logarithmic terms ) are not L-Lipschitz for any pair of points at distance in the domain C. That is , for all m and for all ε ≥ m−β , E max ρ , ρ′∈C ‖ρ−ρ′‖2≤ ‖ ∣∣ { i ∈ [ m ] | ` i ( ρ ) − ` i ( ρ′ ) > L‖ρ− ρ′‖2 } ∣∣ ≤ Õ ( m ) ( 3 ) Assuming a sequence of m β-dispersed loss functions and initial distribution w1 set to the uniform distribution over C and optimize the step size parameter , the exponential forecaster presented in Algorithm 1 achieves sublinear regret Õ ( √ dm log ( Rm ) + ( L+1 ) m1−β ) . While this result achieves a no-regret procedure , its lack of dependence on both the task-data and on the chosen initialization makes it difficult to meta-learn . In the following theorem , we generalize the regret bound for the exponential forecaster to make it data-dependent and hyperparameter-dependent : Theorem 2.1 . Let ` 1 , . . . , ` m : C 7→ [ 0 , 1 ] be any sequence of piecewise L-Lipschitz functions that are β-dispersed . Suppose C ⊂ Rd is contained in a ball of radius R. The exponentially weighted forecaster ( Algorithm 1 ) has expected regret Rm ≤ mλ + log ( 1/Z ) λ + Õ ( ( L + 1 ) m 1−β ) , where Z = ∫ B ( ρ∗ , m−β ) w ( ρ ) dρ∫ C w ( ρ ) dρ for ρ∗ the optimal action in hindsight . The proof of this result adapts past analyses of Algorithm 1 ; setting step-size λ appropriately recovers the previously mentioned bound . The new bound is useful due to its explicit dependence on both the initialization w and the optimum in hindsight via the log ( 1/Z ) term . Assuming w is a ( normalized ) distribution , this effectively measures the overlap between the chosen initialization and a small ball around the optimum ; we thus call − logZ = − log ∫ B ( ρ∗ , m−β ) w ( ρ ) dρ∫ C w ( ρ ) dρ the negative log-overlap of initialization w ( . ) with the optimum ρ∗ . We also obtain an asymptotic lower bound on the expected regret of any algorithm by extending the argument of [ 10 ] to the multi-task setting . We show that for finite D∗ we must suffer Ω̃ ( m1−β ) regret , which limits the improvement we can hope to achieve from task-similarity . Theorem 2.2 . There is a sequence of piecewise L-Lipschitz β-dispersed functions ` i , j : [ 0 , 1 ] 7→ [ 0 , 1 ] , whose optimal actions in hindsight arg minρ ∑m i=1 lt , i ( ρ ) are contained in some fixed ball of diameter D∗ , for which any algorithm has expected regret Rm ≥ Ω̃ ( m1−β ) .

Summary: The paper study the initialization-based meta-learning focus on the piecewise-Lipschitz functions whose discontinuities are dispersed. The authors designed an initialization method that implicitly run on online convex optimization procedure over an adaptive discretization to improve the performance of the exponential forecaster across multi-task. To demonstrate the usefulness of their proposed method, the authors instantiate the method in two settings: online configuration of clustering algorithms and adversarial robustness in online learning.