Learning to Predict Trustworthiness with Steep Slope Loss

1 Introduction . Classification is a ubiquitous learning problem that categorizes objects according to input features . It is widely used in a range of applications , such as robotics [ 1 ] , environment exploration [ 2 ] , medical diagnosis [ 3 ] , etc . In spite of the successful development of deep learning methods in recent decades , high-performance classifiers would still have a chance to make mistakes due to the improvability of models and the complexity of real-world data [ 4 , 5 , 6 , 7 ] . To assess whether the prediction yielded by a classifier can be trusted or not , there are growing efforts towards learning to predict trustworthiness [ 8 , 9 ] . These methods are evaluated on small-scale datasets , e.g. , MNIST [ 10 ] , where the data is relatively simple and existing classifiers have achieved high accuracy ( > 99 % ) . As a result , there are a dominant proportion of correct predictions and the trustworthiness predictors are prone to classify incorrect predictions as trustworthy predictions . The characteristics that the simple data is easy-to-classify aggravate the situation . To further understand the prowess of predicting trustworthiness , we study this problem on the real-world large-scale datasets , i.e. , ImageNet [ 11 ] . This is a challenging theme for classification in terms of boundary complexity , class ambiguity , and feature dimensionality [ 12 ] . As a result , failed predictions are inevitable . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . A general illustration of predicting trustworthiness [ 9 , 13 ] is shown in Fig . 1a . The trustworthiness predictor [ 13 ] that is based on the maximum confidence have been proven to be unreliable [ 8 , 14 , 15 , 16 ] . Instead , Corbiere et al . [ 9 ] propose the true class probability ( TCP ) that uses the confidence w.r.t . the ground-truth class to determine whether to trust the classifier ’ s prediction or not . Nevertheless , the classification confidence is sensitive to the data . As shown in Fig . 1b , TCP predicts that all the incorrect predictions ( 0.9 % in predictions ) are trustworthy on MNIST [ 10 ] and predicts that all the incorrect predictions ( ∼16 % in predictions ) are trustworthy on ImageNet [ 11 ] . To comprehensively understand this problem , we follow the learning scheme used in [ 9 ] and use two state-of-the-art backbones , i.e. , ViT [ 7 ] and ResNet [ 5 ] , as the trustworthiness predictors . For simplicity , we call the “ trustworthiness predictor '' an oracle . We find that the oracles trained with cross entropy loss [ 17 ] , focal loss [ 18 ] , and TCP confidence loss [ 9 ] on ImageNet are prone to overfit the training samples , i.e. , the true positive rate ( TPR ) is close to 100 % while the true negative rate ( TNR ) is close to 0 % . To improve the generalizability of oracles , we propose a novel loss function named the steep slope loss . The proposed steep slope loss consists of two slide-like curves that cross with each other and face in the opposite direction to separate the features w.r.t . trustworthy and untrustworthy predictions . It is tractable to control the slopes by indicating the heights of slides . In this way , the proposed loss is able to be flexible and effective to push the features w.r.t . correct and incorrect predictions to the well-classified regions . Predicting trustworthiness is similar to as well as different from conventional classification tasks . On one hand , predicting trustworthiness can be formulated as a binary classification problem . On the other hand , task-specific semantics are different between the classification task and predicting trustworthiness . The classes are referred to visual concepts , such as dog , cat , etc. , in the classification task , while the ones in predicting trustworthiness are abstract concepts . The trustworthiness could work on top of the classes in the classification task . In other words , the classes in the classification task are specific and closed-form , while trustworthiness is open-form and is related to the classes in the classification task . The contribution of this work can be summarized as follows . • We study the problem of predicting trustworthiness with widely-used classifiers on ImageNet . Specifically , we observe that a major challenge of this learning task is that the cross entropy loss , focal loss , and TCP loss are prone to overfit the training samples , where correct predictions are dominant over incorrect predictions . • We propose the steep slope loss function that improves the generalizability of trustworthiness predictors . We conduct comprehensive experiments and analyses , such as performance on both small-scale and large-scale datasets , analysis of distributions separability , comparison to the class-balanced loss , etc. , which verify the efficacy of the proposed loss . • To further explore the practicality of the proposed loss , we train the oracle on the ImageNet training set and evaluate it on two variants of ImageNet validation set , i.e. , the stylized validation set and the adversarial validation set . The two variants ’ domains are quite different from the domain of the training set . We find that the learned oracle is able to consistently differentiate the trustworthy predictions from the untrustworthy predictions . 2 Preliminaries . In this section , we first recap how a deep learning model learns in the image classification task . Then , we show how the task of predicting trustworthiness connects to the classification task . Supervised Learning for Classification . In classification tasks , given a training sample , i.e. , image x ∈ Rm and corresponding ground-truth label y ∈ Y = { 1 , . . . , K } , we assume that samples are drawn i.i.d . from an underlying distribution . The goal of the learning task is to learn to find a classifier f ( cls ) ( · ; θ′ ) with training samples for classification . θ′ is the set of parameters of the classifier . Let f ( cls ) θ′ ( · ) = f ( cls ) ( · ; θ′ ) . The optimization problem is defined as f ∗ ( cls ) θ′ = argmin f ( cls ) θ′ R̂ ( f ( cls ) θ′ , ` ( cls ) , Dtr ) , ( 1 ) where f∗ ( cls ) θ′ is the learned classifier , R̂ is the empirical risk , ` ( cls ) is a loss function for classification , and Dtr is the set of training samples . Supervised Learning for Predicting Trustworthiness . In contrast to the learning task for classification , which is usually a multi-class single-label classification task [ 4 , 5 , 6 , 7 ] , learning to predict trustworthiness is a binary classification problem , where the two classes are positive ( i.e. , trustworthy ) or negative ( i.e. , untrustworthy ) . Similar to [ 9 ] , given a pair ( x , y ) and a classifier f ( cls ) θ′ , we define the ground-truth label o for predicting trustworthiness as o = { 1 , if argmax f ( cls ) θ′ ( x ) = y 0 , otherwise ( 2 ) In other words , the classifier correctly predicts the image ’ s label so the prediction is trustworthy in hindsight , otherwise the prediction is untrustworthy . The learning task for predicting trustworthiness follows a similar learning framework in the classification task . Let fθ ( · ) be an oracle ( i.e. , a trustworthiness predictor ) . A generic loss function ` : Rm×R → R≥0 , where R≥0 is a non-negative space and m is the number of classes . Given training samples ( x , y ) ∈ Dtr , the optimization problem for predicting trustworthiness is defined as f∗θ = argmin fθ 1 |Dtr| |Dtr|∑ i=1 ` ( fθ ( xi ) , oi ) , ( 3 ) where |Dtr| is the cardinality of Dtr . Particularly , we consider two widely-used loss functions for classification and the loss function used for training trustworthiness predictors as baselines . They are the cross entropy loss [ 17 ] , focal loss [ 18 ] , and TCP confidence loss [ 9 ] . Let p ( o = 1|θ , x ) = 1/ ( 1 + exp ( −z ) ) be the trustworthiness confidence , where z ∈ R is the descriminative feature produced by the oracle , i.e. , z = fθ ( x ) . The three loss functions can be written as ` CE ( fθ ( x ) , o ) = −o · log p ( o = 1|θ , x ) − ( 1− o ) · log ( 1− p ( o = 1|θ , x ) ) , ( 4 ) ` Focal ( fθ ( x ) , o ) = −o · ( 1− p ( o = 1|θ , x ) ) γ log p ( o = 1|θ , x ) − ( 1− o ) · ( p ( o = 1|θ , x ) ) γ log ( 1− p ( o = 1|θ , x ) ) , ( 5 ) ` TCP ( fθ ( x ) , y ) = ( fθ ( x ) − p ( ŷ = y|θ′ , x ) ) 2 . ( 6 ) In the focal loss , γ is a hyperparameter . In the TCP confidence loss , ŷ is the predicted label and p ( ŷ = y|θ′ , x ) is the classification probability w.r.t . the ground-truth class . Consequently , the learned oracle would yield z to generate the trustworthiness confidence . In the cases of ` CE and ` Focal , the oracle considers a prediction is trustworthy if the corresponding trustworthiness confidence is greater than the positive threshold 0.5 , i.e. , p ( o = 1|θ , x ) > 0.5 . The predictions whose trustworthiness confidences are equal to or lower than the negative threshold 0.5 are viewed to be untrustworthy . In the case of ` TCP , the positive threshold is also 0.5 , but the negative threshold correlates to the number of classes in the classification task . It is defined as 1/K in [ 9 ] . 3 Methodology . In this section , we first introduce the overall learning framework for predicting trustworthiness . Then , we narrow down to the proposed steep slope loss function . At last , we provide the generalization bound that is related to the proposed steep slope loss function . 3.1 Overall Design . Corbière et al . [ 9 ] provide a good learning scheme for predicting trustworthiness . Briefly , it first trains a classifier with the training samples . Then , the classifier is frozen and the confidence network ( i.e. , trustworthiness predictor ) is trained ( or fine-tuned ) with the training samples . In general , we follows this learning scheme . This work focuses on the trustworthiness on the predictions yielded by the publicly available pretrained classifiers , i.e. , ViT [ 7 ] and ResNet [ 5 ] . We use the pre-trained backbones as the backbones of the oracles for general purposes . In this sense , the set of the oracle ’ s parameters can be split into two parts , one is related to the backbone and the other one is related to the head , i.e. , θ = { θbackbone , θhead } . θbackbone is used to generated the intermediate feature xout and θhead = { w , b } are usually the weights of a linear function to generate the discriminative feature z = w > xout + b . With the classifier , the oracle , a pre-defined loss , and the training samples , we can optimize problem ( 3 ) to find the optimal parameters for the oracle .

Summary: This paper tests methods for training a classifier for predicting trustworthiness-- formulated here as predicting whether a given image will be classified correctly or not by an target classifier -- on large scale datasets and finds that using existing loss functions like uncertainty estimates/focal loss on this binary classification task cannot detect untrustworthy samples, all samples are classified as positive resulting in 100% FPR (0% TNR). They attempt to fix this shortcoming by introducing a steep slope loss which essentially separates features w.r.t. correct and incorrect predictions from each other, and the loss is bounded in its magnitude. The paper extensively tests whether its possible to predict trustworthiness with two oracle/target classifier combinations on Imagenet and its stylized/adversarial test sets.