Relational Surrogate Loss Learning

1 INTRODUCTION . Evaluation metrics matter in machine learning since it depicts how well we want the models to perform . Nevertheless , most of them are non-differentiable and non-decomposable , thus we can not directly optimize them during training but resort to loss functions ( or surrogate losses ) , which serve exactly as a proxy of task metrics . For example , pose estimation task uses percentage of correct keypoints ( PCK ) ( Yang & Ramanan , 2012 ) to validate point-wise prediction accuracy , but it often adopts mean square error ( MSE ) as loss function . Neural machine translation task takes the sentence-level metric BLEU ( Papineni et al. , 2002 ) to evaluate the quality of predicted sentences , while using word-level cross-entropy loss ( CE Loss ) in training . Besides this manual proxy , some works ( Grabocka et al. , 2019 ; Patel et al. , 2020 ) propose to learn surrogate losses which approximate the metrics using deep neural networks ( DNN ) , so the optimization of metrics can be relaxed to a differentiable space . For example , taking predictions and labels as input , ( Grabocka et al. , 2019 ) approximates the outputs of surrogate losses and evaluation metrics by minimizing their L2 distances . Moreover , recent work even involves the prediction networks into the surrogate loss learning by alternatively updating the loss and predictions , i.e. , they train the surrogate losses after every epoch during training , then use the latest optimized losses to train prediction networks in the next epoch . For instance , ( Grabocka et al. , 2019 ) mainly focuses on the simple binary classification while LS-ED ( Patel et al. , 2020 ) chooses to adopt the surrogate losses in the post-tuning stage ( fine-tuning the models learned by original losses ) and achieves promising improvements . However , these methods often suffer from heavy computational consumption and do not perform well on large-scale challenging datasets . ∗Correspondence to : Shan You < youshan @ sensetime.com > . Both manual and learned surrogate losses follow an exact recovery manner ; namely , the surrogate losses should approximate the target metrics rigorously , and optimizing the surrogate loss is supposed to improve the evaluation metrics accordingly . However , this assumption does not always hold due to the approximation gap , bringing bias to the optimization and leading to sub-optimal results . Instead of pursuing an exact recovery of the evaluation metric , we are reminded of the purpose of metrics , which is to distinguish the performance of models . If a model has a smaller loss than the other model , its metric ought to be better . Nevertheless , current surrogate losses usually have weak relation with the evaluation metrics ( e.g. , CE Loss & BLEU in Figure 1 ( b ) ) . Ideally , the surrogate loss should maintain strong relation of evaluation metric to all models . In this paper , we leverage the ranking correlation as the relation between surrogate losses and evaluation metrics . Then a natural question raises , if the loss functions only require accurate relative rankings to discriminate the models , why do we need to approximate the metrics exactly ? In this way , we propose a method named Relational Surrogate Loss ( ReLoss ) to maximize this rank correlation directly . Concretely , our ReLoss directly leverages the simple Spearman ’ s rank correlation ( Dodge , 2008 ) as the learning objective . By adopting differentiable ranking method , the ranking correlation coefficient can be maximized through gradient descent . Compared to exactly recovering the metrics , our correlation-based optimization is much easier to learn , and our ReLoss , which is simply constructed by multi-layer perceptions , aligns well with the metrics and obtains significantly better correlations compared to the original losses . For example , the commonly used loss MSE in pose estimation only has 46.71 % Spearman ’ s rank correlation coefficient with the evaluation metric PCK , while our ReLoss enjoys 84.72 % relative improvement ( see Table 1 and Figure 1 ) . Our ReLoss generalizes well to various tasks and datasets . We learn ReLoss using randomly generated data and pre-collected network outputs , then the learned losses are integrated into the training of prediction networks as normal loss functions ( e.g. , cross-entropy loss ) , without any further finetuning . Note that we use the same surrogate losses with the same weights in each task , and we find that it is sufficient to obtain higher performance . Compared to previous works , our method is much easier to optimize and enjoys significant efficiency and performance improvements . Extensive experiments on the synthetic dataset and large-scale challenging datasets demonstrate our effectiveness . Moreover , our method outperforms the state-of-the-art methods in human pose estimation and machine reading comprehension tasks . For example , on human pose estimation task , our ReLoss outperforms the state-of-the-art method DARK ( Zhang et al. , 2020 ) by 0.2 % on COCO test-dev set ; on machine reading comprehension task , we achieve new state-of-the-art performance on DuReader 2.0 test set , outperforming all the competitive methods , and even obtain 7.5 % better ROUGE-L compared to human performance . 2 RELATED WORK . Surrogate loss learning . Since most of the metrics in deep learning tasks are non-differentiable and non-decomposable ( e.g. , accuracy , F1 , AUC , AP , etc . ) , surrogate losses aim to approximate the metrics to make them differentiable using neural networks . ( Grabocka et al. , 2019 ) first proposes to learn surrogate losses by approximating the metrics of tasks through a neural network , and the losses are optimized jointly with the prediction model via bilevel optimization . ( Patel et al. , 2020 ) learns the surrogate losses via a deep embedding where the Euclidean distance between the prediction and ground truth corresponds to the value of the metric . However , it is hard to obtain a precise prediction by directly optimizing the surrogate loss with such a strong constraint . We remind that the role of loss functions is to determine which model is better , but with the unavoidable existence of approximation gap , this determinability does not always hold . In addition , these methods both train the surrogate losses alternately with prediction networks , resulting in noticeable efficiency and generability deduction compared to regular losses . In our paper , instead of only focusing on point-to-point recovery , which ignores the rankings between relative values of metrics , we ease the optimization constraint by explicitly learning our ReLoss with rank correlation , and enjoy significant performance and efficiency improvements . Differentiable sorting & ranking . Differentiable sorting and ranking algorithms ( Adams & Zemel , 2011 ; Grover et al. , 2018 ; Blondel et al. , 2020 ; Petersen et al. , 2021 ) can be used in training neural networks with sorting and ranking supervision . Recent approach ( Blondel et al. , 2020 ) proposes to construct differentiable sorting and ranking operators as projections onto the permutahedron , i.e. , the convex hull of permutations , and using a reduction to isotonic optimization . ( Petersen et al. , 2021 ) proposes differentiable sorting networks by relaxing their pairwise conditional swap operations . In this paper , we can use any of these differentiable ranking algorithms to generate differentiable ranking vectors , then directly optimize the rank correlation coefficient for the supervision of our surrogate losses . The algorithm in ( Petersen et al. , 2021 ) is adopted for better performance . 3 PRELIMINARIES . For a given task with a metric function M ( y , ŷ ) , where y and ŷ denote the predicted labels and ground-truth labels , respectively , its loss function L ( y , ŷ ) can be formulated as : L ( y , ŷ ) = f ( y , ŷ ) , ( 1 ) where f can be any function with output ∈ R1 . In this paper , we tend to use a learned DNN ( fDNN ) with weights θl as a surrogate loss , i.e. , L ( y , ŷ ; θl ) = fDNN ( y , ŷ ; θl ) . ( 2 ) The surrogate losses are learned with the networks ’ outputs y and the corresponding metric values M ( y , ŷ ) , i.e. , θ∗l = argmin θl Os ( L ( y , ŷ ; θl ) , M ( y , ŷ ) ) , ( 3 ) where Os is the learning objective of surrogate loss . The prediction networks with weights θm are then optimized by descending the learned surrogate losses L ( y , ŷ ; θ∗l ) , i.e. , θ∗m = argmin θm L ( y , ŷ ; θ∗l ) . ( 4 ) Approximation-based optimization . To learn a surrogate loss w.r.t . a metric , an intuitive idea is to approximate the metric ’ s outputs , i.e. , learn the surrogate losses by minimizing the distances between the outputs of surrogate losses and their corresponding metric values , which is adopted in previous works ( Grabocka et al. , 2019 ; Patel et al. , 2020 ) , their learning objective Os is Os ( L ( y , ŷ ; θl ) , M ( y , ŷ ) ) =‖ L ( y , ŷ ; θl ) −M ( y , ŷ ) ‖22 , ( 5 ) we call this optimization as approximation-based optimization . However , it is hard for a DNN to fully recover the evaluation metric . We conduct toy experiments using a random weighted DNN with output ∈ R1 as an evaluation metric , and then the surrogate loss is learned using limited observations of the metric . As illustrated in Figure 2 ( a ) , since approximating a random network with random inputs is challenging , the errors between surrogate loss learned by approximation-based optimization and metric values are noticeably large . In order to validate the effectiveness of losses in training , we then train the input data with metric or learned losses , as shown in Figure 2 ( c ) , we illustrate the curves of metric values w.r.t . learned input data during training , and directly using metric as loss function obtains best metric value ( lower is better ) , but the performance of input data using approximation-based loss is getting worse . 4 LEARNING RELATIONAL SURROGATE LOSS . 4.1 RELATION AS RANK CORRELATION . Based on the previous discussion , the prior works adopt an unnecessary constraint by enforcing the surrogate losses to fully recover the evaluation metrics . However , the loss function only needs to have the same ranking relation to the metrics , i.e. , we just need to make the surrogate losses have the same ranking as metrics . In this paper , we obtain the relation between surrogate losses and evaluation metrics by using rank correlation as the learning objective , which we call correlationbased optimization . The relation between surrogate losses and evaluation metrics is measured by ranking correlation , which is a statistic that measures the relationship between rankings of the same variable . A ranking correlation coefficient measures the degree of similarity between two rankings and can be used to assess the relation ’ s significance . If the surrogate loss fully correlates to the evaluation metric , the descent of loss value will always obtain better metric values . Spearman ’ s rank correlation . For optimization of surrogate losses , we use the most commonly used Spearman ’ s rank correlation ( Dodge , 2008 ) . For two vectors a and b with size n , the Spearman ’ s rank correlation is defined as : ρS ( a , b ) = Cov ( ra , rb ) Std ( ra ) Std ( rb ) = 1 n−1 ∑n i=1 ( rai − E ( ra ) ) ( rbi − E ( rb ) ) Std ( ra ) Std ( rb ) , ( 6 ) where ra is the rank vector of a , Cov ( ra , rb ) is the covariance of the rank vectors , Std ( ra ) denotes the standard derivation of ra .

Summary: - This paper proposes a relational surrogate loss learning method (ReLoss) inspired by the fact that the evaluation metric and loss are used to distinguish whether one model is better or worse than another.   - This paper provides extensive experiments that demonstrate the effectiveness of the proposed method. The performance and efficiency compared to existing surrogate loss methods are significant, and the performance compared to original losses is seem to be significant on various tasks.