Learning Surrogate Losses

1 INTRODUCTION . In reality , a large set of loss functions can not be directly minimized by gradient-based methods because they are either piece-wise continuous , non-differentiable , or non-decomposable ( Zhang et al. , 2018 ) . For example , binary classification models are often evaluated using the miss-classification rate ( MCR ) , Area under the ROC curve ( AUC ) , F1 measure ( F1 ) , Jaccard Similarity Index ( JAC ) , Average Precision ( AP ) , Equal Error Rate ( EER ) , or the Mathew Correlation Coefficient ( MCC ) . On the other hand , learning-to-rank models are measured through the Normalized Discounted Cumulative Gain ( NDCG ) , or the Mean Average Precision ( MAP ) . To illustrate the point , Figure 1 shows the challenging surfaces of common binary classification losses . Unfortunately , there exists no tractable omni-solver so far , i.e . no off-the-shelf optimization strategy to train prediction models for the aforementioned category of loss functions . It is worth pointing out that non-gradient-based approaches , such as evolutionary computing , are intractable from a runtime perspective . While there exists a plethora of methods that tackle various aspects of particular losses , there is no single gradient-based method that can optimize any loss in a seamless manner . Researchers have so far focused on deriving smooth surrogate relaxations ( approximations ) to the true loss functions ( Berman et al. , 2018 ; Eban et al. , 2017 ) , in a way that first- and second-order optimization techniques can benefit from the derivative of the surrogates with respect to the parameters of prediction models . For instance , the widely-applied cross-entropy loss is a surrogate relaxation of the miss-classification rate . However , these explicit relaxations are hand-crafted individually for each loss and do not generalize to other losses . In this paper , we present the first off-the-shelf optimizer for arbitrary loss functions . In contrast to the related work , this work proposes a new perspective on minimizing loss functions , by defining surrogate losses as meta-level neural networks that approximate the desired true non-differentiable losses . Our method does not need the gradient information of the true loss with respect to the parameters of the prediction model and treats the loss as a black-box function . In addition , we introduce a set-based surrogate network that computes the loss over the training set , being invariant to the order of instances , in order to accurately handle non-decomposable losses . The surrogate learning problem is formalized as a bilevel programming task ( Colson et al. , 2007 ; Franceschi et al. , 2018 ) that is trained through a concurrent optimization algorithm . This paper shows that universal surrogates , which are trained without paying attention to a specific dataset , are sub-optimal compared to surrogates learned in a per-dataset manner . Results on nine datasets demonstrate that learning surrogates produces more accurate prediction models than state-of-the-art baselines with regard to diverse loss functions . 2 RELATED WORK . Due to the non-differential and non-continuous nature of most real-life losses , early works deployed the proxies of the miss-classification rate ( e.g . cross-entropy ) as universal proxy losses , despite their sub-optimal performance ( Cortes & Mohri , 2004 ) . Subsequent approaches relied on designing smooth relaxations of the losses . As an example , the pairwise ranking loss is a common surrogate of the AUC measure ( Gao & Zhou , 2015 ; Chen et al. , 2009 ) . On the other , hand , the F-measure is another typical loss that can not be optimized directly due to its non-decomposable nature over instances . The initial papers tackling F1 focused instead on the empirical utility maximization paradigm ( Ye et al. , 2012 ) . Later on , researchers addressed F1 by optimizing the hyper-parameters of standard binary classifiers , either the cost-sensitive weights of the classification loss ( Puthiya Parambath et al. , 2014 ) , or the threshold of the estimated target values ( Lipton et al. , 2014 ; Koyejo et al. , 2014 ) . Nevertheless , the non-decomposable trait of F1 remains unresolved ( Zhang et al. , 2018 ) and recent works have explored directions to improve hyper-parameter tuning with tighter bounds ( Bascol et al. , 2019 ) . Instead of relying on explicit surrogates , another research direction handles non-convex losses by means of the direct loss method ( Hazan et al. , 2010 ) , which minimizes a surrogate loss by embedding the true loss as a correction term . This method was recently extended to optimize neural networks ( Song et al. , 2016 ) . It assumes the loss can be decomposed into per-instance sub-losses and the authors derived an explicit decomposition of the average precision ( Song et al. , 2016 ) . Unfortunately , per-instance dis-aggregations are not trivially feasible in other cases ( e.g . F1 ) , making the direct loss optimization technique an impractical off-the-shelf option . Two more recent papers have offered relaxation surrogates for non-decomposable losses , concretely the AUC and the Jaccard Index ( known as Intersection over Union in the computer vision community ) . The first method defines relaxation forms for the building blocks of a confusion matrix ( e.g . true positives , true negatives etc . ) and combines the building block relaxations to create a final surrogate for losses like the AUC ( Eban et al. , 2017 ) . However , this model does not handle cases where the loss is not expressible into confusion matrix blocks , for instance the Jaccard index . On the other hand , the second paper proposes the Lovasz soft-max as a smooth approximation to the Jaccard index ( Berman et al. , 2018 ) , which is based on a generic decomposition of sub-modular ( decomposable ) losses for sets ( Yu & Blaschko , 2015 ) . Besides that , a stream of recent papers has focused on meta-learning for loss functions . The ” Learning to Teach ” paradigm ( Fan et al. , 2018 ) proposes a meta-level teacher/controller that continuously updates the loss function for a prediction model based on its progress . The work has been recently extended to enable a gradient-based learning of the teacher/controller ( Wu et al. , 2018 ) . However , this approach does not extend to non-decomposable loss functions which are defined over the full set of instances . A parallel stream of research has elaborated the concept of ” Learning to Learn ” ( Li & Malik , 2017 ) , or ” Learning to Optimize ” ( Chen et al. , 2017 ) , which proposes to directly learn the amount of update values that are applied to the parameters of the prediction model . In the proposed framework a controller uses per-parameter learning curves comprised of the loss values and derivatives of the loss with respect to each parameters ( Chen et al. , 2017 ) . This method suffers from two drawbacks that prohibit its direct applicability to arbitrary losses : i ) for large prediction models it is computationally infeasible to store the learning curve of every parameter , and ii ) there is no gradient information for non-differentiable losses . An alternative approach towards learning loss functions analyzes the usage of discriminative adversarial networks ( dos Santos et al. , 2017 ) . The idea focuses on discriminating the probability of a given target variable value into either being an estimated target or the true one , therefore acting as a form of surrogate loss ( dos Santos et al. , 2017 ) . Last , but not least , our method shares similarities with the concept of error-critic model for function approximation ( Pang & Werbos , 1998 ) . In contrast to the prior work , we propose the first off-the-shelf optimization method that seamlessly minimizes any loss function . In Section 4.3 we empirically compare the proposed method against multiple state-of-the-art relaxations , with regards to minimizing popular binary classification losses , such as AUC , F1 , Jaccard Index and the miss-classification rate . 3 SURROGATE LEARNING . Data mini-batches ( x , y ) : = { ( x1 , y1 ) , . . . , ( xN , yN ) } ofN instances each , with features x ∈ RN×M and ground-truth targets y ∈ YN , are drawn from a dataset D with a sampling distribution PD ( x , y ) , where typically P is the uniform distribution . The target domain can be binary y ∈ { 0 , 1 } N , or nominal among C categories y ∈ { 1 , . . . , C } N . Ultimately , the purpose of a prediction model is to estimate a target variable ŷ ( x ; α ) : RN×M → RN , where the prediction model has parameters α . The estimations ŷ ∈ RN need to accurately match the given ground-truth target variable y ∈ YN with regards to a desired loss function ` ( y , ŷ ) : RN × YN → R. Therefore , supervised learning focuses on computing the optimal parameters α∗ that minimize the following objective . α∗ = argmin α E ( x , y ) ∼PD ( x , y ) ` ( y , ŷ ( x ; α ) ) ( 1 ) To minimize the aforementioned objective through first- or second-order optimization , it is necessary to define the gradients ∂ ` ∂α . Unfortunately , in most real-world cases the loss represents step functions that are only piece-wise continuous ( MCR , F1 , AUC , etc. ) . Therefore , the derivatives are either zero , or undefined at the function steps , which prohibits a direct optimization of these losses . For this reason , a smooth surrogate relaxations of the loss functions is used instead of the true loss . Arguably the most popular surrogate is cross-entropy , which is a relaxation of the miss-classification rate . Yet , such non-parametric relaxation functions are not trivially derivable when the loss is nondecomposable into per-instance components ( e.g . F1 , AUC ) , because such losses are defined as performance measures over an entire set of instances . Instead of deriving one explicit hand-crafted function ˆ̀ for the surrogate of every loss function ` , we propose a method that parameterizes and learns the surrogate for any demanded loss function through an off-the-shelve procedure . Neural networks are a good choice for parametrizing the surrogate loss ˆ̀given their universal approximation capability ( Hornik , 1991 ) . However , the surrogate loss network must be a permutation-invariant set model , whose output must remain fixed given different orders of instances within a dataset mini-batch , i.e . ˆ̀ ( y1 , . . . , yN , ŷ1 , . . . , ŷN ) = ˆ̀ ( yπ ( 1 ) , . . . , yπ ( N ) , ŷπ ( 1 ) , . . . , ŷπ ( N ) ) for any index permutation π . Our method uses the KolmogorovArnold representation theorem ( Tikhomirov , 1991 ) and defines the surrogate loss of Equation 2 as a composition function h of per-instance functions g. This type of aggregation was recently applied to learning neural networks over sets of instances ( Zaheer et al. , 2017 ) . ˆ̀ ( y , ŷ ) = h ( 1 N N∑ i=1 g ( yi , ŷi ) ) ( 2 ) where g : R2 → RQ , and h : RQ → R are deep forward neural networks . The first function g extracts Q latent error components for the predictions on each instance ( e.g . latent representations of false positive , false negative , etc . ) , while the aggregation produces set-wise performance indicators ( e.g . potentially latent representation of the count of true positives , false positives , error rate , etc. ) . The function h creates nonlinear combinations of set-wise performance indicators to produce complex latent error metrics such as precision , recall , F1 , etc . It has been proven that with sufficient capacities for g and h , any permutation-invariant set function ( hence any loss ) can be approximated via the Kolmogorov-Arnold ( KA ) representation theorem ( Za- heer et al. , 2017 ) . In particular , it is worth mentioning that due to being an instance of the KA superposition , the proposed surrogate model of Equation 2 benefits from existing error bound guarantees of such a representation when the functions g and h are deep neural networks ( Montanelli & Yang , 2019 ) . The remaining sections of this paper detail a novel method for optimizing surrogate losses .

Summary: This paper proposes a method of learning loss functions in addition to the learning of predictors. Since it's not easy to optimize loss functions that evaluate the accuracy, surrogate loss functions have been widely employed. The design of the surrogate loss is problem-dependent, and handcraft is required. This paper tries to tackle this problem from the viewpoint of meta-learning, i.e., the surrogate loss learning. Typically, deep neural networks (DNN) are used to design a surrogate loss that approximates the original loss while maintaining the tractability of the optimization. Some convergence properties of the proposed method are analyzed. Some empirical studies showed the efficiency of the proposed method to the state-of-the-art baselines.