Learning-to-learn non-convex piecewise-Lipschitz functions

1 Introduction . While learning-to-learn or meta-learning has long been an object of study [ 45 ] , in recent years it has gained significant attention as a multi-task paradigm for developing algorithms for learning in dynamic environments , from multiple data sources , and in federated settings . Such methods focus on using data from multiple tasks to improve performance when facing a new , potentially related task . A popular approach is initialization-based meta-learning , in which the meta-learner uses multi-task data to output an initialization for an iterative algorithm such as stochastic gradient descent ( SGD ) [ 25 ] . The flexibility of this approach has led to its widespread adoption , e.g . in robotics [ 23 ] and federated learning [ 18 ] , and to a growing number of attempts to understand it , both empirically and theoretically [ 20 , 29 , 24 , 40 , 42 ] . However , outside some stylized setups our learning-theoretic understanding of how to meta-learn an initialization is largely restricted to the convex Lipschitz setting . We relax both assumptions to study the meta-learning of online algorithms over piecewise-Lipschitz functions , which can be nonconvex and highly discontinuous . As no-regret online learning over such functions is impossible in-general , we study the case of piecewise-Lipschitz functions with dispersed discontinuities that do not concentrate in any small compact subset of the domain [ 11 ] . Such functions arise frequently in data-driven algorithm design , where the goal is to learn the optimal parameter settings of algorithms for difficult ( often NP-Hard ) problems over a distribution or sequence of instances [ 4 ] ; for example , a small change to the metric in cluster linkage can lead to a discontinuous change in the classification error [ 7 ] . In this paper , we also demonstrate that such losses are relevant in the setting of adversarial robustness , where we introduce a novel online formulation . For both cases , the associated problems are often solved across many time periods or for many different problem domains , resulting in natural multi-task structure that might improve performance . To the best of our knowledge , ours is the first theoretical study of meta-learning in both of these application settings . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In the single-task setting the problem of learning dispersed functions can be solved using simple methods such as the exponentially-weighted forecaster . To design an algorithm for learning to initialize online learners in this setting , we propose a method that optimizes a sequence of datadependent upper-bounds on the within-task regret [ 29 ] . The result is an averaged bound that improves upon the regret of the single-task exponential forecaster so long as there exists an initial distribution that can compactly contain many of the within-task optima of the different tasks . Designing the meta-procedure is especially challenging in our setting because it involves online learning over a set of distributions on the domain . To handle this we study a “ prescient ” form of the classic follow-the-regularized leader ( FTRL ) scheme that is run over an unknown discretization ; we then show the existence of another algorithm that plays the same actions but uses only known information , thus attaining the same regret while being practical to implement . To demonstrate the usefulness of our method , we study this algorithm in two settings . Multi-task data-driven algorithm design . We consider data-driven tuning of the parameters of combinatorial optimization algorithms for hard problems such as knapsack and clustering . The likely intractability of these problems on worst case instances have led to several approaches to study them in more realistic settings , such as smoothed analysis [ 44 ] and data-driven algorithm configuration [ 4 ] . Our setting is more realistic than those considered in prior work . It is more challenging than learning from iid instances [ 27 , 12 ] , but at the same time less pessimistic than online learning over adversarial problem instances [ 11 ] as it allows us to leverage similarity of problem instances coming from different but related distributions . We instantiate our bounds theoretically on several problems where the cost functions are piecewise-constant in the tuned parameters , allowing our meta-procedure to learn the right initial distribution for exponential forecasters . This includes well-known combinatorial optimization problems like finding the maximum weighted independent set ( MWIS ) of vertices on a graph , solving quadratic programs with integer constraints using algorithms based on the celebrated Goemans-Williamson algorithm , and mechanism design for combinatorial auctions . Then we consider experimentally the problem of tuning the right α for the α-Lloyd ’ s family of clustering algorithms [ 15 ] . In experimental evaluations on two datasets—a synthetic Gaussian mixture model and the well-known Omniglot dataset from meta-learning [ 33 ] —our meta-procedure leads to improved clustering accuracy compared to single-task learning to cluster . We also study our results for a family of greedy algorithms for the knapsack problem introduced by [ 27 ] and obtain similar results for a synthetic dataset . The results holds for both one-shot and five-shot tasks . Online robust meta-learning . The second instantiation of our meta-learning procedure is to a new notion of adversarial robustness for the setting of online learning , where our results imply robust meta-learning in the presence of outliers . In this setting , the adversary can make ( typically small ) modifications to some example x ∈ X , which can result in potentially large changes to the corresponding loss value lh ( x ) , where h ∈ H is our hypothesis . For instance , consider the well-studied setting of adversarial examples for classification of images using deep neural networks [ 36 , 17 ] . Given a neural network f , the adversary can perturb a datapoint x to a point x′ , say within a small Lp-ball around x , such that f ( x ) = f ( x′ ) but the true label of x′ does not match x , and therefore lf ( x ) 6= lf ( x′ ) . In general , under the adversarial influence , we observe a perturbed loss function l̃h ( x ) = lh ( x ) + ah ( x ) . Typically we are interested in optimizing both the perturbed loss l̃h ( x ) , i.e . measuring performance relative to optimum for adversarially perturbed losses , and the true loss lh ( x ) ( performance on the unobserved , unperturbed loss ) . For example , in the online learning setting , [ 1 ] consider perturbed loss minimization for linear dynamical systems , while [ 41 ] look at true { 0 , 1 } loss minimization in the presence of adversarial noise . Our approach ensures that regret for both the perturbed and true loss are small , for piecewise-Lipschitz but dispersed adversaries . 2 Preliminaries and initialization-dependent learning of dispersed functions . In this section we introduce our setup and notation for online learning of piecewise-Lipschitz functions in a multi-task environment . We then generalize existing results for the single-task setting in order to obtain within-task regret bounds that depend on both the initialization and the task data . This is critical for both defining a notion of task similarity and devising a meta-learning procedure . Meta-learning setup Following past setups [ 2 , 21 , 29 ] , for some T , m > 0 and all t ∈ [ T ] and i ∈ [ m ] we consider a meta-learner faced with a sequence of Tm loss functions ` t , i : C 7→ [ 0 , 1 ] over a compact subset C ⊂ Rd that lies within a ball B ( ρ , R ) of radius R around some point ρ ∈ Rd . Algorithm 1 Exponential Forecaster 1 : Input : step size parameter λ ∈ ( 0 , 1 ] , initialization w : C → R≥0 . 2 : Initialize w1 = w 3 : for i = 1 , 2 , . . . , m do 4 : Wi : = ∫ C wi ( ρ ) dρ 5 : Sample ρi with probability proportional to wi ( ρi ) , i.e . with probability pi ( ρi ) = wi ( ρi ) Wi 6 : Suffer ` i ( ρi ) and observe ` i ( · ) 7 : For each ρ ∈ C , set wi+1 ( ρ ) = e−λ ` i ( ρ ) wi ( ρ ) Here we used the notation [ n ] = { 1 , . . . , n } . Before each loss function ` t , i the meta-learner must pick an element ρt , i ∈ C before then suffering a loss or cost ` t , i ( ρt , i ) . For a fixed t , the subsequence ` t,1 , . . . , ` t , m defines a task for which we expect a single element ρ∗t ∈ C to do well , and thus we will use the within-task regret on task t to describe the quantity Rt , m = m∑ i=1 ` t , i ( ρt , i ) − ` t , i ( ρ∗t ) where ρ∗t ∈ arg min ρ∈C m∑ i=1 ` t , i ( ρ ) ( 1 ) In the single-task setting the goal is usually to show that Rt , m is sublinear in m , i.e . that the average loss decreases with more rounds . A key point here is that the functions we consider can have numerous global optima . In this work we will assume , after going through the m rounds of task t , that we have oracle access to a single fixed optimum for t , which we will refer to using ρ∗t and use in both our algorithm and to define the task-similarity . Note that in the types of applications we are interested in—piecewise-Lipschitz functions—the complexity of computing optima scales with the number of discontinuities . In the important special case of piecewise-constant functions , this dependency becomes logarithmic [ 19 ] . Thus this assumption does not affect the usefulness of the result . Our goal will be to improve the guarantees for regret in the single-task case by using information obtained from solving multiple tasks . In particular , we expect average performance across tasks to improve as we see more tasks ; to phrase this mathematically we define the task-averaged regret R̄T , m = 1 T T∑ t=1 Rt , m = 1 T T∑ t=1 m∑ i=1 ` t , i ( ρt , i ) − ` t , i ( ρ∗t ) ( 2 ) and claim improvement over single-task learning if in the limit of T →∞ it is smaller than Rt , m . Note that for simplicity in this work we assume all tasks have the same number of rounds within-task , but as with past work our results are straightforward to extend to the more general setting . Learning piecewise-Lipschitz functions We now turn to our target functions and within-task algorithms for learning them : piecewise-Lipschitz losses , i.e . functions that are L-Lipschitz w.r.t . the Euclidean norm everywhere except on measure zero subsets of the space ; here they may have arbitrary jump discontinuities so long they still bounded between [ 0 , 1 ] . Apart from being a natural setting of interest due to its generality compared to past work on meta-learning , this class of functions has also been shown to have important applications in data-driven algorithm configuration [ 11 ] ; there these functions represent the cost , e.g . an objective value or time-complexity , of algorithms for difficult problems such as integer programming , auction design , and clustering . This literature has also shown lower bounds demonstrating that no-regret learning piecewise-Lipschitz function is impossible in general , necessitating assumptions about the sequence . One such condition is dispersion , which requires that the discontinuities are not too concentrated . Definition 2.1 ( [ 11 ] ) . The sequence of random loss functions ` 1 , . . . , ` m is said to be β-dispersed with Lipschitz constant L if , for all m and for all ≥ m−β , we have that , in expectation over the randomness of the functions , at most Õ ( m ) functions ( the soft-O notation suppresses dependence on quantities beside , m and β , as well as logarithmic terms ) are not L-Lipschitz for any pair of points at distance in the domain C. That is , for all m and for all ε ≥ m−β , E max ρ , ρ′∈C ‖ρ−ρ′‖2≤ ‖ ∣∣ { i ∈ [ m ] | ` i ( ρ ) − ` i ( ρ′ ) > L‖ρ− ρ′‖2 } ∣∣ ≤ Õ ( m ) ( 3 ) Assuming a sequence of m β-dispersed loss functions and initial distribution w1 set to the uniform distribution over C and optimize the step size parameter , the exponential forecaster presented in Algorithm 1 achieves sublinear regret Õ ( √ dm log ( Rm ) + ( L+1 ) m1−β ) . While this result achieves a no-regret procedure , its lack of dependence on both the task-data and on the chosen initialization makes it difficult to meta-learn . In the following theorem , we generalize the regret bound for the exponential forecaster to make it data-dependent and hyperparameter-dependent : Theorem 2.1 . Let ` 1 , . . . , ` m : C 7→ [ 0 , 1 ] be any sequence of piecewise L-Lipschitz functions that are β-dispersed . Suppose C ⊂ Rd is contained in a ball of radius R. The exponentially weighted forecaster ( Algorithm 1 ) has expected regret Rm ≤ mλ + log ( 1/Z ) λ + Õ ( ( L + 1 ) m 1−β ) , where Z = ∫ B ( ρ∗ , m−β ) w ( ρ ) dρ∫ C w ( ρ ) dρ for ρ∗ the optimal action in hindsight . The proof of this result adapts past analyses of Algorithm 1 ; setting step-size λ appropriately recovers the previously mentioned bound . The new bound is useful due to its explicit dependence on both the initialization w and the optimum in hindsight via the log ( 1/Z ) term . Assuming w is a ( normalized ) distribution , this effectively measures the overlap between the chosen initialization and a small ball around the optimum ; we thus call − logZ = − log ∫ B ( ρ∗ , m−β ) w ( ρ ) dρ∫ C w ( ρ ) dρ the negative log-overlap of initialization w ( . ) with the optimum ρ∗ . We also obtain an asymptotic lower bound on the expected regret of any algorithm by extending the argument of [ 10 ] to the multi-task setting . We show that for finite D∗ we must suffer Ω̃ ( m1−β ) regret , which limits the improvement we can hope to achieve from task-similarity . Theorem 2.2 . There is a sequence of piecewise L-Lipschitz β-dispersed functions ` i , j : [ 0 , 1 ] 7→ [ 0 , 1 ] , whose optimal actions in hindsight arg minρ ∑m i=1 lt , i ( ρ ) are contained in some fixed ball of diameter D∗ , for which any algorithm has expected regret Rm ≥ Ω̃ ( m1−β ) .

Summary: This paper proposed a meta-learning approach to learn piecewise-Lipschitz functions in the multi-task setting. Simply applying a traditional single-task exponential forecasting method multiple times cannot make the regret bound diminishes as the number of tasks $T$ increases. To tackle this issue, they first attempt to obtain a bound that involves the task similarity term, and then the authors propose two meta-learning algorithms, based on the observation that the regret bound depends on two things: learning rate and initialization probability choosing. Hence, they proposed two meta-learning algorithms, to efficiently learn to choose better initialization and learning rate. The theoretical results show that this algorithm can provably improve the baseline of applying a single-task algorithm $T$ times.