Relational Surrogate Loss Learning

1 INTRODUCTION . Evaluation metrics matter in machine learning since it depicts how well we want the models to perform . Nevertheless , most of them are non-differentiable and non-decomposable , thus we can not directly optimize them during training but resort to loss functions ( or surrogate losses ) , which serve exactly as a proxy of task metrics . For example , pose estimation task uses percentage of correct keypoints ( PCK ) ( Yang & Ramanan , 2012 ) to validate point-wise prediction accuracy , but it often adopts mean square error ( MSE ) as loss function . Neural machine translation task takes the sentence-level metric BLEU ( Papineni et al. , 2002 ) to evaluate the quality of predicted sentences , while using word-level cross-entropy loss ( CE Loss ) in training . Besides this manual proxy , some works ( Grabocka et al. , 2019 ; Patel et al. , 2020 ) propose to learn surrogate losses which approximate the metrics using deep neural networks ( DNN ) , so the optimization of metrics can be relaxed to a differentiable space . For example , taking predictions and labels as input , ( Grabocka et al. , 2019 ) approximates the outputs of surrogate losses and evaluation metrics by minimizing their L2 distances . Moreover , recent work even involves the prediction networks into the surrogate loss learning by alternatively updating the loss and predictions , i.e. , they train the surrogate losses after every epoch during training , then use the latest optimized losses to train prediction networks in the next epoch . For instance , ( Grabocka et al. , 2019 ) mainly focuses on the simple binary classification while LS-ED ( Patel et al. , 2020 ) chooses to adopt the surrogate losses in the post-tuning stage ( fine-tuning the models learned by original losses ) and achieves promising improvements . However , these methods often suffer from heavy computational consumption and do not perform well on large-scale challenging datasets . ∗Correspondence to : Shan You < youshan @ sensetime.com > . Both manual and learned surrogate losses follow an exact recovery manner ; namely , the surrogate losses should approximate the target metrics rigorously , and optimizing the surrogate loss is supposed to improve the evaluation metrics accordingly . However , this assumption does not always hold due to the approximation gap , bringing bias to the optimization and leading to sub-optimal results . Instead of pursuing an exact recovery of the evaluation metric , we are reminded of the purpose of metrics , which is to distinguish the performance of models . If a model has a smaller loss than the other model , its metric ought to be better . Nevertheless , current surrogate losses usually have weak relation with the evaluation metrics ( e.g. , CE Loss & BLEU in Figure 1 ( b ) ) . Ideally , the surrogate loss should maintain strong relation of evaluation metric to all models . In this paper , we leverage the ranking correlation as the relation between surrogate losses and evaluation metrics . Then a natural question raises , if the loss functions only require accurate relative rankings to discriminate the models , why do we need to approximate the metrics exactly ? In this way , we propose a method named Relational Surrogate Loss ( ReLoss ) to maximize this rank correlation directly . Concretely , our ReLoss directly leverages the simple Spearman ’ s rank correlation ( Dodge , 2008 ) as the learning objective . By adopting differentiable ranking method , the ranking correlation coefficient can be maximized through gradient descent . Compared to exactly recovering the metrics , our correlation-based optimization is much easier to learn , and our ReLoss , which is simply constructed by multi-layer perceptions , aligns well with the metrics and obtains significantly better correlations compared to the original losses . For example , the commonly used loss MSE in pose estimation only has 46.71 % Spearman ’ s rank correlation coefficient with the evaluation metric PCK , while our ReLoss enjoys 84.72 % relative improvement ( see Table 1 and Figure 1 ) . Our ReLoss generalizes well to various tasks and datasets . We learn ReLoss using randomly generated data and pre-collected network outputs , then the learned losses are integrated into the training of prediction networks as normal loss functions ( e.g. , cross-entropy loss ) , without any further finetuning . Note that we use the same surrogate losses with the same weights in each task , and we find that it is sufficient to obtain higher performance . Compared to previous works , our method is much easier to optimize and enjoys significant efficiency and performance improvements . Extensive experiments on the synthetic dataset and large-scale challenging datasets demonstrate our effectiveness . Moreover , our method outperforms the state-of-the-art methods in human pose estimation and machine reading comprehension tasks . For example , on human pose estimation task , our ReLoss outperforms the state-of-the-art method DARK ( Zhang et al. , 2020 ) by 0.2 % on COCO test-dev set ; on machine reading comprehension task , we achieve new state-of-the-art performance on DuReader 2.0 test set , outperforming all the competitive methods , and even obtain 7.5 % better ROUGE-L compared to human performance . 2 RELATED WORK . Surrogate loss learning . Since most of the metrics in deep learning tasks are non-differentiable and non-decomposable ( e.g. , accuracy , F1 , AUC , AP , etc . ) , surrogate losses aim to approximate the metrics to make them differentiable using neural networks . ( Grabocka et al. , 2019 ) first proposes to learn surrogate losses by approximating the metrics of tasks through a neural network , and the losses are optimized jointly with the prediction model via bilevel optimization . ( Patel et al. , 2020 ) learns the surrogate losses via a deep embedding where the Euclidean distance between the prediction and ground truth corresponds to the value of the metric . However , it is hard to obtain a precise prediction by directly optimizing the surrogate loss with such a strong constraint . We remind that the role of loss functions is to determine which model is better , but with the unavoidable existence of approximation gap , this determinability does not always hold . In addition , these methods both train the surrogate losses alternately with prediction networks , resulting in noticeable efficiency and generability deduction compared to regular losses . In our paper , instead of only focusing on point-to-point recovery , which ignores the rankings between relative values of metrics , we ease the optimization constraint by explicitly learning our ReLoss with rank correlation , and enjoy significant performance and efficiency improvements . Differentiable sorting & ranking . Differentiable sorting and ranking algorithms ( Adams & Zemel , 2011 ; Grover et al. , 2018 ; Blondel et al. , 2020 ; Petersen et al. , 2021 ) can be used in training neural networks with sorting and ranking supervision . Recent approach ( Blondel et al. , 2020 ) proposes to construct differentiable sorting and ranking operators as projections onto the permutahedron , i.e. , the convex hull of permutations , and using a reduction to isotonic optimization . ( Petersen et al. , 2021 ) proposes differentiable sorting networks by relaxing their pairwise conditional swap operations . In this paper , we can use any of these differentiable ranking algorithms to generate differentiable ranking vectors , then directly optimize the rank correlation coefficient for the supervision of our surrogate losses . The algorithm in ( Petersen et al. , 2021 ) is adopted for better performance . 3 PRELIMINARIES . For a given task with a metric function M ( y , ŷ ) , where y and ŷ denote the predicted labels and ground-truth labels , respectively , its loss function L ( y , ŷ ) can be formulated as : L ( y , ŷ ) = f ( y , ŷ ) , ( 1 ) where f can be any function with output ∈ R1 . In this paper , we tend to use a learned DNN ( fDNN ) with weights θl as a surrogate loss , i.e. , L ( y , ŷ ; θl ) = fDNN ( y , ŷ ; θl ) . ( 2 ) The surrogate losses are learned with the networks ’ outputs y and the corresponding metric values M ( y , ŷ ) , i.e. , θ∗l = argmin θl Os ( L ( y , ŷ ; θl ) , M ( y , ŷ ) ) , ( 3 ) where Os is the learning objective of surrogate loss . The prediction networks with weights θm are then optimized by descending the learned surrogate losses L ( y , ŷ ; θ∗l ) , i.e. , θ∗m = argmin θm L ( y , ŷ ; θ∗l ) . ( 4 ) Approximation-based optimization . To learn a surrogate loss w.r.t . a metric , an intuitive idea is to approximate the metric ’ s outputs , i.e. , learn the surrogate losses by minimizing the distances between the outputs of surrogate losses and their corresponding metric values , which is adopted in previous works ( Grabocka et al. , 2019 ; Patel et al. , 2020 ) , their learning objective Os is Os ( L ( y , ŷ ; θl ) , M ( y , ŷ ) ) =‖ L ( y , ŷ ; θl ) −M ( y , ŷ ) ‖22 , ( 5 ) we call this optimization as approximation-based optimization . However , it is hard for a DNN to fully recover the evaluation metric . We conduct toy experiments using a random weighted DNN with output ∈ R1 as an evaluation metric , and then the surrogate loss is learned using limited observations of the metric . As illustrated in Figure 2 ( a ) , since approximating a random network with random inputs is challenging , the errors between surrogate loss learned by approximation-based optimization and metric values are noticeably large . In order to validate the effectiveness of losses in training , we then train the input data with metric or learned losses , as shown in Figure 2 ( c ) , we illustrate the curves of metric values w.r.t . learned input data during training , and directly using metric as loss function obtains best metric value ( lower is better ) , but the performance of input data using approximation-based loss is getting worse . 4 LEARNING RELATIONAL SURROGATE LOSS . 4.1 RELATION AS RANK CORRELATION . Based on the previous discussion , the prior works adopt an unnecessary constraint by enforcing the surrogate losses to fully recover the evaluation metrics . However , the loss function only needs to have the same ranking relation to the metrics , i.e. , we just need to make the surrogate losses have the same ranking as metrics . In this paper , we obtain the relation between surrogate losses and evaluation metrics by using rank correlation as the learning objective , which we call correlationbased optimization . The relation between surrogate losses and evaluation metrics is measured by ranking correlation , which is a statistic that measures the relationship between rankings of the same variable . A ranking correlation coefficient measures the degree of similarity between two rankings and can be used to assess the relation ’ s significance . If the surrogate loss fully correlates to the evaluation metric , the descent of loss value will always obtain better metric values . Spearman ’ s rank correlation . For optimization of surrogate losses , we use the most commonly used Spearman ’ s rank correlation ( Dodge , 2008 ) . For two vectors a and b with size n , the Spearman ’ s rank correlation is defined as : ρS ( a , b ) = Cov ( ra , rb ) Std ( ra ) Std ( rb ) = 1 n−1 ∑n i=1 ( rai − E ( ra ) ) ( rbi − E ( rb ) ) Std ( ra ) Std ( rb ) , ( 6 ) where ra is the rank vector of a , Cov ( ra , rb ) is the covariance of the rank vectors , Std ( ra ) denotes the standard derivation of ra .

Summary: The authors introduce a relational surrogate loss learning method (ReLoss) for replacing the original losses. The rationale and intuition behind are well-grounded. Experiments on various tasks and ablation studies prove the validity.