Learning to Predict Trustworthiness with Steep Slope Loss
Authors: Yan Luo
Yongkang Wong
Mohan Kankanhalli
Qi Zhao
Venue: NeurIPS 2021 Poster (2021)
Keywords: Deep leearning
Classification
Trustworthiness

Abstract: Understanding the trustworthiness of a prediction yielded by a classifier is critical for the safe and effective use of AI models. Prior efforts have been proven to be reliable on small-scale datasets. In this work, we study the problem of predicting trustworthiness on real-world large-scale datasets, where the task is more challenging due to high-dimensional features, diverse visual concepts, and a large number of samples. In such a setting, we observe that the trustworthiness predictors trained with prior-art loss functions, i.e., the cross entropy loss, focal loss, and true class probability confidence loss, are prone to view both correct predictions and incorrect predictions to be trustworthy. The reasons are two-fold. Firstly, correct predictions are generally dominant over incorrect predictions. Secondly, due to the data complexity, it is challenging to differentiate the incorrect predictions from the correct ones on real-world large-scale datasets. To improve the generalizability of trustworthiness predictors, we propose a novel steep slope loss to separate the features w.r.t. correct predictions from the ones w.r.t. incorrect predictions by two slide-like curves that oppose each other. The proposed loss is evaluated with two representative deep learning models, i.e., Vision Transformer and ResNet, as trustworthiness predictors. We conduct comprehensive experiments and analyses on ImageNet, which show that the proposed loss effectively improves the generalizability of trustworthiness predictors. The code and pre-trained trustworthiness predictors for reproducibility are available at \url{https://github.com/luoyan407/predict_trustworthiness}.

Learning to Predict Trustworthiness with Steep Slope Loss
Yan Luo', Yongkang Wong', Mohan S Kankanhalli', Qi Zhao
i Department of Computer Science & Engineering, University of Minnesota
& School of Computing, National University of Singapore
1uoxx6488umm. edu, yongkang- wongdnus. lecaanussed.. edu. sg. mohan@comp nus edu. sg, qzhao@cs umn.edi edu
Abstract
Understanding the trustworthiness of a prediction yielded by classifier is crit- ical for the safe and effective use of AI models. Prior efforts have been proven be reliable on small-scale datasets. In this work, we study the problem of predicting trustworthiness on real-world large-scale datasets, where the task is more challenging due high-dimensional features, diverse visual concepts, and a large number of samples. In such a setting, we observe that the trustworthi- ness predictors trained with prior-art loss functions, i.e., the cross entropy loss, focal loss, and true class probability confidence loss, are prone to view both correct predictions and incorrect predictions to trustworthy. The reasons are two-fold. Firstly, correct predictions dictions. Secondly, due to the data complexity, it is challenging to differentiate the incorrect predictions from the correct ones real-world large-scale datasets. To improve the generalizability of trustworthiness predictors, we propose a novel steep slope loss to separate the features w.r.t. correct predictions from the ones w.r.t. incorrect predictions by two slide-like curves that oppose each other. The proposed loss is evaluated with two representative deep learning models, i.e., Vision Transformer and ResNet, as trustworthiness predictors. We conduct com- prehensive experiments and analyses on ImageNet, which show that the proposed loss effectively improves the generalizability of trustworthiness predictors. The code and pre-trained trustworthiness predictors for reproducibility available httpps///ggthbb...//////o/////// wtwp:rrwwwwtttttttooroeeeeoooooe
generally dominant over incorrect pre-
Introduction
Classification is a ubiquitous learning problem that categorizes objects according to input features. It
widely used in a range applications, such as robotics 11. environment exploration 2, medical diagnosis [3]. etc. In spite of the successful development of deep learning methods in recent decades, classifiers would still have a chance to make mistakes due to the improvability of
high-performan
models and the complexity of real-world data [ 5 6 77.
To assess whether the prediction yielded by classifier can be trusted not, there growing efforts towards learning to predict trustworthiness g 9.. These methods are evaluated on small-scale datasets, e.g., MNIST [10. where the data relatively simple and existing classifiers have achieved high accuracy (> 99%). As a result, there are a dominant proportion of correct predictions and the trustworthiness predictors are prone classify incorrect predictions as trustworthy predictions. The characteristics that the simple data is to-cassiyy aggravate the situation. To further understand the prowess of predicting trustworthiness, we study this problem on the real-worid large- scale datasets, i.e., ImageNet [I]]. This is a challenging theme for classification in terms of boundary complexity. class ambiguity, and feature dimensionality [12]
As a result, failed predictions are inevitable.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Ground truih: woll'
MNIST
Input
Predict
Classifier
Dog
Traowwthiies profese
Amai
% Actal
The prodiction incorrest
Input
Prodict
Oracle
.N (Trustworthiness)
TSR-000
TTR 198-1 TVD- 19000
(a)
(b)
Figure 1: Conceptual illustrations of trustworthiness prediction. (a) shows the process of predicting trustworthiness where the oracle is the trustworthiness predictor. The illustration in (b) shows that the task is challenging on ImageNet, where TCP's confidence loss [9] is used in this example. The confidence that is greater (lower) than the positive (negative) threshold would be classified as a trustworthy (untrustworthy) prediction. Usually, both the positive threshold and the negative threshold are 0.5, but the negative threshold is #f classes in the case of TCP.
A general illustration of predicting trustworthiness 9 13 shown in Fig. la The trustworthiness predictor [3] that is based on the maximum confidence have been proven to he unreliable [ 14, [5] [6]. Instead, Corbiere et al. g propose the true class probability (TCP) that uses the confidence w.r.t. the ground-truth class to determine whether to trust the classifier's ' prediction or not. Nevertheless, the classification confidence is sensitive the data. As shown in Fig. Tb TCP predicts that all the incorrect predictions (0.9% in predictions) are trustworthy on MNIST 10 and predicts that all the incorrect predictions (~16% in predictions) are trustworthy on ImageNet [1]
To comprehensively understand this problem, we follow the learning scheme used in and use two state-of-the-ar backbones, i.e., VIT 7 and ResNet [5), as the trustworthiness predictors. For simplicity, we call the "trustworthiness predictor" an oracle. We find that the oracles trained with cross entropy loss [17). focal loss [18], and TCP confidence loss [9] on ImageNet
prone overfit the training samples, i.e., the true positive rate (TPR) is close to 100% while the true negative rate (TNR) is close 0%. To improve the generalizability of oracles, we propose a novel loss function named the steep slope loss. The proposed steep slope loss consists of two slide-like curves that cross with each other and face in the opposite direction to separate the features w.r.t. trustworthy and untrustworthy predictions. It is tractable to control the slopes by indicating the heights of slides. In this way, the proposed loss is able be flexible and effective push the features w.r.t. correct and incorrect predictions to the well-classifedd regions.
Predicting trustworthiness is similar On one hand, predicting trustworthiness can be formulated as a binary classification problem. On the other hand, task-specific semantics are different between the classification task and predicting trustworthiness. The classes are referred to visual concepts, such as dog, cat, etc., in the classification task, while the ones in predicting trustworthiness are abstract concepts. The trustworthiness could work on top of the classes in the classification task. In other words, the classes in the classification task are specific and closed-form, while trustworthiness is open-form form and related the classes in the classification task.
well as different from conventional classification tasks.
The contribution of this work can be summarized as follows.
We study the problem of predicting trustworthiness with widely-used classifiers ImageNet. Specifically, we observe that a major challenge of this learning task is that the cross entropy loss, focal loss, and TCP loss are prone overfit the training samples, where correct predictions are dominant over incorrect predictions.
We propose the steep slope loss function that improves the generalizability of trustworthiness predictors. We conduct comprchensive experiments and analyses, such as performance on both small-scale and large-scale datasets, analysis of distributions separability, comparison to the bllannad loss, etc., which verify the efficacy of the proposed loss.
To further explore the practicality of the proposed loss, we train the oracle on the ImageNet training set and evaluate on two variants of ImageNet validation set, i.e., the stylized validation set and the adversarial validation set. The two variants' domains are quite different

---

from the domain of the training set. We find that the learned oracle is able to consistently differentiate the trustworthy predictions from the untrustworthy predictions.
2 Preliminaries
In this section, we first recap how deep learning model learns in the image classification task. Then, we show how the task of predicting trustworthiness connects the classification task.
Supervised Learning for Classification. In classification tasks, given a training sample, i.c., image
Th' and corresponding ground-truth label y E y/ {i..... K. we assume that samples are drawn i..d. from an underlying distribution. The goal of the learning task is to learn to find a classifier f(cle) (c; 0') with training samples for classification. is the set of parameters of the classifier. Let plde) for ()) f(cle) ('). The optimization problem is defined as
for (ela)) arg min R(
plots) ,,(cle), ,rr),
(1)
plcta
pe(cls) is the learned classifier, R is the empirical risk, g(cls) is a loss function for classification, where fgr and Drr is the set of training samples.
Supervised Learning for Predicting Trustworthiness. In contrast to the learning task for classifi- cation, which is usually multi-class single-label classification task 4 6 71. learning to predict trustworthiness is a binary classification problem, where the two classes are positive (i.c., trustworthy)
or negative (i.e., untrustworthy). Similar to (9, given a pair (æ, y) and classifier p(ole) we define
the ground-truth label o for predicting trustworthiness as
1, if arg max ((cls) (x)
(x)) =
(2)
0, otherwise
In other words, the classifier correctly predicts the image's label so the prediction is trustworthy in hindsight, otherwise the prediction is untrustworthy.
The learning task for predicting trustworthiness follows a similar learning framework in the clas- sification task. Let fo(-) be an oracle (i.e., a trustworthiness predictor). A generic loss function L: TM x + Rzo. where R2 is a non-negative space and m is the number of classes. Given training samples (æ, y) E Dir, the optimization problem for predicting trustworthiness is defined as
Dell
fō arg min fo
((fo(z;),
(3)
D
where Derl is the cardinality of Der-
Particularly, we consider two widely-used loss functions for classification and the loss function used for training trustworthiness predictors as baselines. They are the cross entropy loss [17, focal loss 18]. and TCP confidence loss 19. Let p(o 1|6, x>) = 1/(1 + cxp(-2)) be the trustworthiness confidence, where z R is the descriminative feature produced by the oracle, i.e., 2 = fo(x:). The three loss functions can be written as
(())) -0 loggloo 110, x) (1- o) log(1 p(o 1|0, x)),
(4)
(Soc - (1 p(o 110,x))) x)) p(o = 10,
(5)
(1 (p(o= = 1|0, x))² log(1 - p(o = 1 x)),
(rcr(fo(z),)) v) = (fo(z) p(ū = -²)²
(6)
In the focal loss, 7 is a hyperparameter. In the TCP confidence loss, 7 is the predicted label and p(ū = y/0', æz) is the classification probability w.r.t. the ground-truth class.
Consequently, the learned oracle would yield z to generate the trustworthiness confidence. In the cases of tce and & the oracle considers a prediction is trustworthy if the corresponding trustworthiness confidence is greater than the positive threshold 0.5, i.e., p(o = 100, > 0.5. The predictions whose trustworthiness confidences are equal to or lower than the negative threshold 0.5 are viewed to untrustworthy. In the case of ÊTCP, the positive threshold also 0.5, but the negative threshold correlates to the number of classes in the classification task. It is defined as KK in 99.
3

---

3.
Dew 60
14
Joess
wir5
wids
Spece
(a)
(b)
Figure 2: Conceptual workflow of the proposed steep slope loss (a) and graph comparison between the proposed loss and the conventional losses (b). In (b), the cross entropy loss and focal loss plotted in blue and black, respectively. The TCP confidence loss is square error and varies with the classification confidence. Therefore, it is not plotted here.
Methodology
In this section, first introduce the overall learning framework for predicting trustworthiness. Then, we narrow down to the proposed steep slope loss function. At last, we provide the generalization bound that is related the proposed steep slope loss function.
3.1 Overall Design
Corbière et al. [9] provide a good learning scheme for predicting trustworthiness. Briefly, first trains a classifier with the training samples. Then, the classifier is frozen and the confidence network (i.e., trustworthiness predictor) is trained (or fine-tuned) with the training samples. In general, we follows this learning scheme.
This work focuses on the trustworthiness on the predictions yielded by the publicly available pre- trained classifiers, i.e., VIT (7) and ResNet (5]. We use the pre-trained backbones as the backbones of the oracles for general purposes. into two parts, one is related to the backbone and the other one is related to the head, i.e., 0 [Obackbone: 0h Obackbone is used to generated the intermediate feature out and 0head = {w, b} are usually the weights of linear function to generate the discriminative feature w zut + With the classifier, the oracle, pre-defined loss, and the training samples, we can optimize problem (3 to find the optimal parameters for the oracle.
this sense, the set of the oracle's parameters can be split
3.2 Steep Slope Loss
The conceptual workflow the proposed steep slope loss is shown in Fig. 2a The core idea is that we exploit the graph characteristies of the exponential function and the softsign function establish two slides such that the features z w.r.t. the positive class ride down the positive slide to the right bottom and the features z w.r.t. the negative class ride down the negative slide the left bottom. is defined as the signed distance to the hyperplane (i.e., oracle head). Given an image be broken down into
fo(x) can
x out
x put
feracthone (x).
(7)
The signed distance to the hyperplane has geometric interpretation: its sign indicates in which half-space ont and its absolute value indicates how far out away from the hyperplane.
It is desired that the signed distance of out with ground-truth label 1 (o 0) tends towards
too (-oo) much possible. To this end, we define the steep slope (SS) loss function follows
atz
(ss(fo(z),o) =. exp
+ exp(-a*)) +(1 o)
exp
exp(-a")
Pounive slide
Negative slide
where a a= E + control the slope of the positive slide and the negative slide, respectively. If w.r.t. the positive class is on the left-hand side of = = O, minimizing the loss would push the point on the hill down to the bottom, i.c., the long tail region indicating the well-classified region. Similarly,

---

w.L.t the negative class would undergo a similar process. exp(-a*) and exp(-a are vertical shifts for the positive and negative slides such that E.s has a minimum value O. Note that the proposed steep slope loss is in the range [0, maximum (exp(a*) - exp(-at)) exp(a-)) exp(-a-)]], whereas the cross entropy loss and the focal loss are in the range [0, +00). The proposed steep slope loss can work with the output of the linear function as well. This is because the signed distance and the output
of the linear function have a proportional relationship with each other, i.e.,
T"=="< x w T x= + b. [w]]
Essentially, as shown in Fig. 2b the cross entropy loss, focal loss, and steep slope loss work in similar manner to encourage 2 w.r.t. the positíve class to move the right-hand side of the positive threshold O and encourage w.r.t. the negative class move the left-hand side of the negative threshold O, which is analogous to the sliding motion. The proposed steep slope loss is more tractable to control the steepness of slopes than the cross entropy loss and focal loss. This leads an effective learning to yield discriminative feature for predicting trustworthiness.
3.3 Generalization Bound
With the proposed steep slope loss, are interested in the generalization bound of trustworthiness predictors. For simplicity, we simplify a trustworthiness predictor as f E F, where F is a finite hy- pothesis set. The risk of predicting trustworthiness is defined as R(f) y) (Ess( (f (xx) o]], where P is the underlying joint distribution of (æt, o). As P is inaccessible, a common practice is to use em- pirical risk minimization (ERM) to approximate the risk [19], i.e. = b Σ. (f:)) o.).
The following theorem provides an insight into the correlation between the generalization bound and the loss function in the learning task for predicting trustworthiness.
Denote maximum ( ) - exp(-at), exp(a - exp(-a as fmaz Lss E [0, SS Assume F is finite hypothesis set, for any ò > 0, with probability at least 1 - 6, the following inequality holds for all F:
Theorem 3.1.
log [F] + log
[R(J) - RD(
maaz
2
The proof sketch is similar to the generalization bound provided in [20] and the detailed proof is provided in the appendix A
A desired characteristic of the proposed steep slope loss is that it is in a certain range determined by a and a=,, as discussed in Section 3.2 This leads to the generalization bound shown Theorem 3.1 The theorem implies that given generic classifier, as the number of training samples increases, the empirical risk would be close to the true risk with a certain probability. On the other hand, the cross entropy loss, focal loss, and TCP loss are not capped in a certain range. They do not fit under Hoeffding's 's inequality to derive the generalization bounds.
Related Work
Loss Function. Loss function is the key to search for optimal parameters and has been extensively Specifically, the cross entropy loss may be the most
studied a range of learning tasks 9 18 21 22
widely-used loss for classification 211, however, it is not optimal for all cases. Lin et al. [18] propose the focal loss to down-weight the loss w.r.t. well-classified examples and focus on minimizing the loss w.r.t. misclassified examples by reshaping the cross entropy loss function. Liu et al. 22 introduce the large margin softmax loss that is based on cosine similarity between the weight vectors and the feature vectors. Nevertheless, it requires more hyperparameters and entangles with the linear function in the last layer of the network. Similar to the idea of focal loss, the proposed steep slope loss is flexible to reshape the function graphs to improve class imbalance problem. TCP's confidence loss is used to train the confidence network for predicting trustworthiness
9.. We adopt TCP as a baseline.
Trustworthiness Prediction. Understanding trustworthiness of classifier has been studied in the past decade g 9
Hendryeks and Gimpel [13] aim to detect the misclassified examples according to maximum class probability. This method depends on the ranking of confidence scores and is proved to be unreliable (8) Monte Carlo dropout (MCDropout) intends to understand trustworthiness through the lens of uncertainty estimation 223. However, it is difficult to distinguish
13
23
24
5

---

the trustworthy prediction from the incorrect but overconfident predictions 19]. Jiang et al. g propose a confidence measure named trust score, which is based on the distance between the classifier and modified hearest-neighhor classifier on the test examples. The shortcomings of the method are the poor scalability and computationally expensive. Recently, instead of implementing a standalone trustworthiness predictor, Moon et al. [24] use the classifier's weights to compute the correctness ranking. The correctness ranking is taken into account in the loss function such that the classification accuracy is improved. This method relies on pairs of training samples for computing the ranking. Moreover, it is unclear if the classifier improved by the correctness ranking outperforms the pre- trained classifier when applying the method on large-scale datasets, e.g., ImageNet. In contrast, the learning scheme to train the trustworthiness predictor (i.e., confidence network) in [9] is standard and does not affect the classifiers. Its efficacy has been verified on small-scale datasets. In this work, follows the learning scheme used in 9 and focus on predicting trustworthiness on complex dataset.
Imbalanced Classification. In classification task, the ratio of correct predictions to incorrect pre- dictions is expected to be large due to the advance deep learning methods. This is aligned with the nature of imbalanced classification [25] 26 27 28 29], which generally employ re-sampling and re-weighting strategies to solve the problem. However, predicting trustworthiness is different from imbalanced classification as it is not aware of the visual concepts but the correctness of the predictions, whereas the classification task relies on the visual concepts. Specifically, the re-samlling based methods [25] 26, 29] may not be suitable for the problem of predicting trustworthiness. It is difficult to determine if a sample is under-represented (over-represented) and should be ever-sampled (under-same sander-sampled). The re- re-weighting based methods 26 28
can be applied to any generic loss func- tions, but they hinge on some hypothesis related to imbalanced data characteristics. For instance, the laanebdlanced loss (28 is based the effective number w.r.t. class, which presumes number of samples is known. However, the number of samples w.r.t. a class is not always assessible, e.g.. in the online learning scheme (30 31. Instead of assuming some hypothesis, we follow the standard learning scheme for predicting trustworthiness g g.
Experiment & Analysis
In this section, we first introduce the experimental set-up. Then, we report the performances of baselines and the proposed steep slope loss on ImageNet, followed by comprehensive analyses.
Experimental Set-Up. We use ViT B/16 M and ResNet-50 5 as the classifiers, and the respective backbones are used as the oracles' backbones. We denote the combination of oracles and classifiers as (O, C). There are four combinations in total, i.e., (ViT, VIT), (ViT, RSN), (RSN, ViT), and (RSN, RSN), where RSN stands for ResNet. In this work, we adopt three baselines, i.e., the cross entropy loss 21. focal loss [8), and TCP confidence loss (9, for comparison purposes.
The experiment is conducted on ImageNet [TI. which consists of 1.2 million labeled training images and 50000 labeled validation images. It has 1000 visual concepts. Similar to the learning scheme in [9] the oracle is trained with training samples and evaluated on the validation set. During the training process of the oracle, the classifier works in the evaluation mode so training the oracle would not affect the parameters of the classifier. Moreover, we conduct the analyses about how well the learned oracle generalizes to the images in the unseen domains. To this end, we apply the widely-
style transfer method (32] and the functional adversarial attack method [33 to generate two variants of the validation set, i.e., stylized validation set and adversarial validation set. Also, we adopt ImageNet-C 34 for evaluation, which is used for evaluating robustness to common corruptions.
The oracle's backbone is initialized by the pre-trained classifier's backbone and trained by fine-tuning using training samples and the trained classifier. Training the oracles with all the loss functions uses the same hyperparameters. such as learning rate, weight decay, momentum, batch size, etc. The details for the training process and the implementation are provided in Appendix B
For the focal loss, we follow [18] to use 7 = 2, which leads to the best performance for object detection. For the proposed loss, we use a 1 and a = 3 for the oracle that is based on ViT's backbone, while we use a = and a = 5 for the oracle that is based on ResNet's backbone.
Following 9. we use FPR-95%-TPR. AUPR-Error, AUPR-Succent and AUC as the metrics. FPR- 9P% is the false positive rate (FPR) when true positive rate (TPR) is equal to 95%. AUPR the area under the precision-recall curve. Specifically, AUPR-Success considers the correct prediction

---

Performance on the ImageNet validation set. The mean and the standard deviation of scores are computed over three runs. The oracles are trained with the ImageNet training samples. The classifier is used the evaluation mode. Ace is the classification accuracy and helpful to understand the proportion of correct predictions. SS stands for the proposed steep slope loss.
Table I:
(0,C)
Loss
Acct
FPR-95%-
AUPR-
AUPR-
AUC1
TPR
TNR
TPR..
Errort
Success
CE
83,90
93.01=0.17
15.80=00.56
84.25+00.57
51.62=0.86
99.99=0.01
0.02+00002
Focal
1X
83,90
93.37+0.52
15.31+0.44
84.76+0.50
52.38+0.77
99.15+0.14
1.35+00.2
(VIT. VIT)
TCP SS
83,90
88.38:10.23
12.96-0.10
87.63-10.15
60.14:10.7
99.73:1..02
0.00
83,90
80.48100.6
10.26:10:03
93.01:0.100
73.68:0.27
87.52.10.95
38.27:1..48
CE
68.72
93,43:10.28 28
30,90=10335
69.13:0.36
51.24=0.63
99,90=0.04
0.20+00.00
(VIT, RSN)
Focal
18
68.72
93 0.5
30.97+0.36
69.07=0033
51.26=0.62
93.66++0.29
0.71+00.33
TCP
68.72
0.
03.56
79.04=1.91
66.23=1.02
94,25.10.96
0,00+00000
68.72
07.89+0.33
20.91=0.05
85.39=0.16
74.31=0021
68.32::0.41
6.653+#0.62
CE Focal
83,90
03.29+0053
14.74+0.17
85.40+0.20
53.43+00.8
100000=0.00
0.00++0.00
18
83,90
94.60+0.53
14.98+0021
03.10-+
52.37+00551
100.00
0.00++0.0
(RSN, VIT)
TCP
83,90
91. 03910.499
12.12.1..1
86, 16.10:10.15
56.55:10:46
100.00
0.00100.00
SS
83,90
88.70-10.08
11.6910.04
90.01=0.10
64.34:10.16
96.20.10.73
1.00
CE
68.72
29.41=0.188 94.84=0.27
70.79+0.19
52.36=0.41
100.00=0.00
0.00+0.00
(RSN, RSN)
Focal
18
68.72
95. 06.169
29.92+00338
70.23++0.44
51.43=0.50
99.86++0.05
0.08
TCP
68.72
88.81
24.46:0.12
77.79=0.29
62.73=0.14
99.23110.14
0.0010.00
68.72
86.21+00.44
22.53+0.03
81.88=0110
67.92-t0.11
79.20++2.50
42.09+3.77
as the positive class, whereas AUPR- considers the incorrect prediction as the positive class. AUC is the area under the receiver operating characteristic curve, which is the plot of TPR versus FPR. Moreover, we use TPR and true negative rate (TNR) as additional metries because they assess overfitting issue, e.g., TPR=100% and TNR=0% imply that the trustworthiness predictor is prone to view all the incorrect predictions to be trustworthy.
Performance on Large-Scale Dataset. The result on ImageNet are reported in Table I We have two key observations. Firstly, training with the cross entropy loss, focal loss, and TCP confidence loss lead to overfitting the imbalanced training samples, i.c., the dominance of trustworthy predic- tions. Specifically, TPR is higher than 99% while TNR is less than 1 1 in all cases. Secondly, the performance of predieting trustworthiness is contingent on both the oracle and the classifier. When a high-performance model (i.e.. ViT) is used as the oracle and a relatively low- oowppeffrraacc model (i.e., ResNet) is used as the classifier, cross entropy loss and focal loss achieve higher TNRs than the loss functions with the other combinations. In contrast, the two losses with (ResNet, ViT) lead to the lowest TNRs (i.e., 0%).
Despite the combinations of oracles and classifiers, the proposed steep slope loss can achieve significantly higher TNRs than using the other loss functions, while it achieves desirable performance on FPR-95%-TPP AUPR-Success, and AUC. This verifies that the proposed loss is effective to improve the generalizability for predicting trustworthiness. Note that the scores of AUPR-Erre and TPR yielded by the proposed loss are lower than that of the other loss functions. Recall that AUPR-Error aims inspect how easy detect failures and depends on the negated trustworthiness confidences w.r.t. incorrect predictions 19. The AUPR-Error correlates to TPR and TNR. When TPR close to 100% and TNR close to 0%, indicates the oracle prone to view all the predictions to be trustworthy. In other words, almost all the trustworthiness confidences are on the right-hand side of p(o== 1 x) = 0.5. Consequently, when taking the incorrect prediction as the positive class, the negated confidences are smaller than -0.5. On the other hand, the oracle trained with the proposed loss intends to yield the ones w.r.t. incorrect predictions that are smaller than 0.5. In general, the negated confidences w.r.t. incorrect predictions are greater than the ones yielded by the other loss functions. In summary, a high TPR score and a low TNR score leads to a high AUPR-Error.
To intuitively understand the effects of all the loss functions, we plot the histograms of trustworthiness confidences w.r.t. true positive (TP), false positive (FP), true negative (TN), and false negative (FN) in Fig- The result confirms that the oracles trained with the baseline loss functions are prone to predict overconfident trustworthiness for incorrect predictions, while the oracles trained with the proposed loss can properly predict trustworthiness for incorrect predictions.
7

---

100.00
11,
18.
(a) (VIT. +IT)+ + CE
(b) (VIT. VIT Focal
(c) (VIT. VIT)+ TCP
(d) (VIT. VIT)+ + SS
110 18.99
pi 1)9.4)
18..0
plo
10.
(c) (VIT, RSN)+ CE
(f) (VIT, RSN)+ Focal
(VIT, RSN)+ TCP
(h) (ViT, RSN)+ SS
Figure 3: Histograms of trustworthiness confidences w.r.t all the loss functions on the ImageNet validation set. The oracles that are used to generate the confidences are the ones used in Table The histograms generated with (RSN, ViT) (RSN, RSN)are provided in Appendix
Table 2: Performance on MNIST and CIFAR-10.
Dataset
Loss
Acct
FPR-95% TPR.
AUPR- Error
AUPR- Success
AUC1
TPR:
TNR1
MCP [K]
99.10
5.56
35.05
99.99
98.63
99.89
8.89
MCDropout
23
99.10
5.26
38.50
99.99
98.65
TrustScore
99.10
10.00
35.88
99.98
98.20
MNIST
TCP
99.10
3.33
45.89
98.82
0.00
TCPf
99.10
3.33
45.88
99,99
98.82
99.72
0.00
SS
99.10
2.22
40.86
99.99
98.83
100.00
0.00
MCP 13
92.19
47.50
45.36
91.53
99.64
6.66
MCDropout
23
92.19
49,02
46.40
99.27
92.08
TrustScore g
92.19
55.70
38.10
98.76
88.47
CIFAR-10
TCP
9
92.19
44.94
49.94
99.24
92.12
99.77
0.00
TCP
92.19
45.07
49.89
99 99.24
92.12
97.88
0.00
92.19
44.69
50.28
99.26
92.22
98.46
28.04
Separability between Distributions of Correct Predictions and Incorrect Predictions. As ob-
served in Fig. 3 the confidences w.r.t. correct and incorrect predictions follow Gaussian-like distri- butions. Hence, we can compute the separability between the distributions of correct and incorrect predictions from probabilistic perspective. Given the distribution of correct predictions (1 of) and the distribution of correct predictions N2(u2, o2), we use the average Kullback-Leible (KL) divergence (NV NG) (35) and Bhattacharyya distance (N N22 (36) to measure the separability, More details and the quantitative results are reported in Appendix G In short, the proposed loss leads larger separability than the baseline loss functions. This implies that the proposed loss more effective to differentiate incorrect predictions from correct predictions.
Performance Small-Scale Datasets. We also provide comparative experimental results small- scale datasets, i.e., MNIST 10] and CIFAR- 10 (37]. The results are reported in Table The proposed both MNIST and CIFAR-10. and additionally
loss outperforms TCPi metric 9PR-95%-TPP achieved good performance on metrics AUPR-Emot and TNR on CIFAR-10. This shows the proposed loss is able adapt relatively simple data. More details can be found in Appendix B.]]
Generalization to Unseen Domains. In practice, the oracle may run into the data in the domains that are different from the ones of training samples. Thus, it is interesting find out how well the learned oracles generalize to the unseen domain data. Using the oracles trained with the ImageNet training set (i.e., the ones used in T evaluate it the stylized ImageNet validation set [32]. adversarial ImageNet validation set 33 and corrupted ImageNet validation set 34]. (ViT, ViT) is used in the experiment.
The results on the stylized ImageNet, adversarial ImageNet, and ImageNet-C are reported Table 3 More results on ImageNet-C are reported in Table Al As all unseen domains are different from the

---

Table 3: Performance the stylized ImageNet validation set, the adversarial ImageNet validation set, and one (Defocus blur) of validation sets in ImageNet-C. Defocus blus severity. (ViT, VIT) is used in the experiment and the domains of the two validation sets are different
the highest level of
from the one of the training set that is used for training the oracle. The corresponding histograms are
available in Appendix
More results ImageNet-C can be found in Table
Dataset
Loss
Acct
FPR-95%-
AUPR- Errort
AUPR-
AUC
TPR
TNR
TPR.
Success
15.94
95.52
84.18
15.86
49.07
99.99
0.02
Stylized
Focal 18
15.94
95.96
85.90
14,30
46.01
99.71
0.25
ICP
9
15.94
93.42
80.17
21.25
57.29
99.27
0.00
SS
15.94
89.38
75.08
34.39
67.68
44.42
81.22
CE
6.14
94.35
93.70
6.32
51.28
99.97
0.06
Focal
18
6.15
93.67
93.48
6.56
52.39
99.06
1.43
Adversarial 33
TCP
6.11
93.94
92.77
7.55
55.81
99.71
0.00
SS
6.16
90.07
90.09
13.07
65.36
87.10
24.33
CE
31.83
94.46
68.56
31.47
50.13
99.15
1.07
Focal 18
31.83
94.98
66.87
33.24
51.28
96.70
3.26
Defocus blur
TCP 9
31.83
93.50
64.67
36.05
54.27
96.71
4.35
SS
31.83
90.18
57.95
48.80
64.34
77.79
37.29
one the training set, the classification accuracies are much lower than the ones Table The adversarial validation set also more challenging than the stylized validation set and the corrupted validation set. As a result, the difficulty affects the scores across all metrics. The oracles trained with the baseline loss functions are still prone recognize the incorrect prediction be trustworthy. The proposed loss consistently improves the performance on FPR-95%- TPR, AUPR-Sucer AUC, and TNR. Note that the adversarial perturbations are computed on the fly B3 Instead truncating the sensitive pixel values and saving into the images files, we follow the experimental settings in [33] to evaluate the oracles on the fly. Hence, the classification accuracies w.r.t. various loss function are slightly different but are stably around 6. 14%.
Selective Risk Analysis. Risk-coverage curve is important technique for analyzing trustworthiness through the lens of the rejection mechanism in the classification task [9] [88]. In the context of predicting trustworthiness, selective risk is the empirical loss that takes into account the decisions. i... to trust or not to trust the prediction. Correspondingly, coverage is the probability mass of the non-rejected region. As can see in Fig. 4a| the proposed loss leads significantly lower risks, compared to the other loss functions. We present the risk-coverage curves w.r.t. all the combinations of oracles and classifiers in Appendix They consistently exhibit similar pattern.
Ablation Study. In contrast to the compared loss functions, the proposed loss has more hyperparam- eters to be determined, i.e., a and a- As the proportion of correct predictions usually larger than that of incorrect predictions, we would prioritize a over a such that the oracle is able to recognize certain amount of incorrect predictions. In other words, at, and then freeze a and search for a+ Fig. 4b and 4c show how the loss, TPR, and TNR vary with various a this analysis, the combination (ViT, ViT) is used and a 1. We see that
first search for o by freezing
soleetive risk (25)
stoup slope Loss
14-
4.5
22
1.0
TPR.
TNR
20
α
(a)
Figure 4: Analyses based on (VIT, VIT). (a) are the curves of risk vs. coverage. Selective risk
represents the percentage of errors the remaining validation set for a given coverage. (b) are the curves of loss vs. a (c) are TPR and TNR against various o.

---

a = 3 achieves the optimal trade-off between TPR and TNR. We follow similar search strategy to determine at and 5 for training the oracle with ResNet backbone.
Effeets of Using = - w out + b. Using the signed distance as 2, i.c., 2 a geometric interpretation as shown in Fig. 2a However, the main-stream models 15
10
ae
+-b
has
7
use
w x + Therefore, we provide the corresponding results appendix F which are
generated by the proposed loss taking the output of the linear function as input. In comparison with the results of using w= ====""'4 [e]
w x b yields comparable scores of FPR- 95R, AUPR-Error, AUPR-Success, and AUC. Also, TPR and TNR are moderately different between 2 = m +h and 2 = un not + h, when a+ and a are fixed. This implies that TPR and TNR are sensitive to | This is because the normalization by ||/|| would make more dispersed value than the variant without normalization. In other words, the normalization leads to tailed distributions while no normalization leads to short-tailed distributions. Given the same threshold, TNR (TPR) is determined by the location of the distribution of negative (positive) examples and the extent of short/long tails. Our analysis the histograms generated with and without ||w]|| normalization verifies this point.
using
Steep Slope Loss vs. Class-Balanne Balanced Loss. We compare the proposed loss to the class-balanced loss 28]. which is based on re-weighting strategy. The results are reported in Appendix [H] Overall, the proposed loss outperforms the of predicting trustworthiness is different from that of imbalanced data classification.
loss, which implies that the imbalance characteristics
Conclusion
In this work, we study the problem of predicting trustworthiness on a large-scale dataset. We observe that the oracle, i.e., trustworthiness predictor, trained with the cross entropy loss, focal loss, and TCP confidence loss lean towards viewing incorrect predictions be trustworthy due overfitting. To improve the generalizability of the oracles, we propose the steep slope loss that encourages the features w.r.t. correct predictions and incorrect predictions to be separated from each other. We evaluate the proposed loss on ImageNet through the lens of the trustworthiness metrics, selective classification metric, and separability of distributions, respectively. Experimental results show that the proposed loss is effective in improving the generalizability of trustworthiness predictors.
7 Societal Impact
Learning high-accume models is a long-standing goal in machine learning. Nevertheless, due to the complexity of real-world data, there is still gap between state-of-the-ar classification models and perfect one. Hence, there a critical need to understand the trustworthiness of classification models, i.e, differentiate correct predictions and incorrect predictions, in order to safely and effectively apply the models in real-world tasks. Models with the proposed loss achieve considerably better performance with various trustworthiness metrics. They also show generalizability with both in- distribution and -iittiddittiittiionn images at a large scale. In addition, while existing trustworthiness models focus high TPR and tend view all incorrect predictions be trustworthy (i.e., TNR close to 0), false positives may lead to high consequent cost in some real-world scenarios such as critical applications in medicine (e.g., false positive leading to unnecessary and invasive tests treatment such as biopsies or surgery, or harmful side effects in medicine) and security (e.g., counter terrorism). It is thus of great importance for trustworthiness model flexibly trade off between TPR and TNR. To this end, the proposed loss allows the underlying distributions of positive and negative examples to be more separable, enabling a more effective trade-off between them.
Acknowledgments and Disclosure of Funding
This research was funded in part by the NSF under Grants 1908711, 1849107, and in part supported by the National Research Foundation, Singapore under its Strategic Capability Research Centres Funding Initiative. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore.
10

---

References
[1] Daniel Leidner, Christoph Borst, Alexander Dietrich, Michael Beetz, and Alin Albu-Schäffer. Classifying compliant manipulation tasks for automated planning in robotics. In IEEE/RS/ International Conference on Intelligent Robots and Systems, pages 1769-1776, 2015.
[2] Antoine Caté, Lorenzo Perozzi, Erwan Gloaguen, and Martin Blouin. Machine learning as a tool for geologists. The Leading Edge, 36(3):215-219 2017.
[3] José Antonio Sanz, Mikel Galar, Aranzazu Jurio, Antomio Brugos, Miguel Pagola, and Humberto Bustince. Medical diagnosis of cardiovascular diseases using an interval-valued fuzzy rule-based classification system. Applied Soft Computing, 20: 103-111, 2014.
[4] Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097-1105, 2012.
[5] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Puttern Recognition, pages 770-778, 2016.
[6] Mingxing Tan and Quoe V. Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 4105-6114, 2019.
[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbomn Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[8] Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust a classifier. In Advances in Neural Information Processing Systems, pages 5546-5557, 2018.
[9] Charles Corbière, Nicolas Thome, Avner Bar- Mer, Matthieu Cord, and Patrick Pérez. Ad- dressing failure prediction by learning model confidence. In Advances in Neural Information Processing Systems, pages 2902-2913, 2019.
[10] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2778-2324, 1998.
[11] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Puttern Recognition, pages 248-255, 2009.
[12] Mitra Basu and Tin Kam Ho. Data complexity in pattern recognition. Springer Science & Business Media, 2006.
[13] Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and -istr--distribution examples in neural networks. International Conference on Learning Representations, 2017.
[14] Foster Provost, Tom Fawcett, and Ron Kohavi. The case against accuracy estimation for comparing induction algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, page 445-453, 1998.
[15] Ian Goodfellow. Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
[16] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Pattern Recognition, pages 427-436, 2015.
[17] Kevin P Murphy. Machine learning: probabilistic perspective. MIT press, 2012.
[18] Tsung- Yi Lin, Priya Goyal, Ross Girshick. Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In IEEE International Conference on Computer Vision, pages 2999-3007, 2017.
[19] Vladimir N Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5)988-999, 1999.
[20] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018

---

[21] David R Cox. Regression models and life-tables Journal of the Royal Statistical Society: Series B (Methodological), 34(2):187-22) 1972.
[22] Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. Large-margin softmax loss for convolutional neural networks. In International Conference on Machine Learning, volume 48 of JMLR Workshop and Conference Proceedings, pages 507-516, 2016.
[23] Yarin Gal and Zoubin Ghahramani. Dropout as Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1050-1059, 2016.
[24] Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. Confidence-aware learning for deep neural networks. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 7044-7044, 2020.
[25] Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang. Learning deep representation for imbalanced classification. IEEE Conference on Computer Vision and Pattern Recognition. pages 5375-5384, 2016.
[26] Yiru Wang, Weihao Gan, Jie Yang, Wei Wu, and Junjie Yan. Dynamic curriculum learning for imbalanced data classification. In IEEE/CVF International Conference Computer Vision, pages 5016-5025, 2019.
[27] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbal- anced datasets with bibellddsttibution-awaar margin loss. In Advances in Neural Information Processing Systems, pages 1565-1576, 2019.
[28] Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
9269, 2019.
[29] Jaehyung Kim, Jongheon Jeong, and Jinwoo Shin. M2m: Imbalanced classification via major- to-minor translation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13893-13902, 2020.
[30] Koby Crammer, Jaz Kandola, and Yoram Singer. Online classification on a budget. In Advances in Neural Information Processing Systems, pages 225-232--22222222222222222222--1 232, 2003.
[31] Yahav Bechavod, Christopher Jung, and Steven Z. Wu. Metric-free individual fairness in online learning. In Advances in Neural Information Processing Systems, pages 11214-1122 1225, 2020.
[32] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. ImageNet-trained CNNs biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
[33] Cassidy Laidlaw and Soheil Feizi. Functional adversarial attacks. In Advances in Neural Information Processing Systems, pages 10408-10418, 2019.
[34] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[35] S. Kullback and R. A. Leibler. On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1):79- 86, 1951.
[36] Anil Bhattacharyya. On measure of divergence between two multinomial populations. Sankhyā: The Indian Journal of Statistics, pages 401-406, 1946.
[37] Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto, 2009.
Master's thesis,
[38] Yonatan Geifman and Ran El- Yaniv. Selective classification for deep neural networks. In Advances in Neural Information Processing Systems, pages 4885 4894, 2017.
12


Reviews:
[{"review_id": "w68HuzQMoqy", "reviewer": "Reviewer_8gNS", "summary": "The paper proposes Steep Slope loss, a loss function that is designed to improve performance in the binary classification task of prediction trustworthiness. The paper reports that common loss functions used in this domain such as cross-entropy loss, focal loss, and true classification probability loss (TCP), all suffer from an issue where correct predictions tend to dominate over incorrect predictions, due to issues of class imbalance and evaluations on simple datasets where classifiers have achieved very high accuracy. \n\nSteep Slope loss solves this issue by controlling the slope parameters on both the negative and positive classification thresholds with two hyperparameters that control the derivative and gradient updates for datapoints that are correctly classified and incorrectly classified separately. These two controlled slopes are a function of the signed distance of the output of the classifier from the hyperplane of classification.\n\nThe paper evaluates this loss on the MNIST, CIFAR, and Imagenet datasets and shows empirical improvements on multiple binary classification metrics such as AUPRC, AUROC, and FPR95, and importantly, SS loss shows marked improvements in the TNR, therefore being more robust to detecting incorrectly classified points.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths of the Work - \n\nThe paper is well written, motivates the problem of trustworthiness classification well, and derives a well-argued formulation for Steep Slope loss. The empirical results are quite strong, and the datasets studied are comprehensive as well.\n\nThe formulation for SS loss seems like it will be quite beneficial for many binary classification tasks as well, especially in order to improve model calibration on in-domain data. This is a promising loss function to add to the family for more than trustworthiness applications as well. \n\nThe comparisons to class-imbalance loss and clear reasons for why SS loss is a better choice also adds to the strength of the argument in a domain where the \"positive class\" and \"negative class\" statistics are unknown and depend on the accuracy of the classifier being studied. \n\nShowing the robustness of this loss to unseen domains such as the stylized and adversarial Imagenet test sets also strengthens the method, and the TNR numbers in Table 2 are quite impressive compared to the baselines.\n\n### Comments and Suggestions - \n\nThe paper is a good contender for acceptance in its current form. The appendices are comprehensive, and the following points, if addressed, would strengthen my opinion of acceptance further.\n\n1. It would be interesting to see a numerical analysis of gradients of the Steep Slope loss function, whether there are vanishing gradient issues, or numerical stability issues due to the exponent.\n2. Another potential test dataset to evaluate the robustness of SS loss could be the Imagenet-Corrupted datasets, defined in [1]. Trustworthiness as a function of dataset shift can be an interesting property to map, to strengthen the adoption of SS loss to domains where there can be covariate shift from the training regime to an unknown test regime.\n3. Why are the TNR results so sensitive to the normalization by $||w||$? I would be curious to know if the authors have a hypothesis for why it is better to normalize the affine outputs with a norm of the weights?\n4. The paper mentions that all the oracles were trained with the same hyperparameters (LR, weight decay, momentum etc). This doesn't make much sense to me, as each of these loss functions have very different training behaviours and gradient updates, and the number of steps required for convergence would be different. Further, since the loss  landscape for these loss functions are different, the optimal hyperparameter space would be different for each of the loss functions. It might be that the SS loss is in a good hparam space for the current hparams, but the other losses are not.\n5. Could the authors clarify how the picked the values of $\\alpha^+$ and $\\alpha^-$ for different datasets, or point me to the relevant section in the paper in case I missed it?\n6. From the histogram plots, it seems like that a lot of True Positive Examples get misclassified, with the tradeoff that more True Negative Examples are accurately classified. It would be good for the authors to discuss the pros and cons of this tradeoff, especially in real-world applications of trustworthiness (medical applications etc), which can strengthen arguments for their method.\n7. A further comparison to non-loss functions method in the trustworthiness sphere could be good, such as the trust score [2].\n\nNitpicks - \n\nLine 45: \"state-of-the-art\"\n\nLine 84: should be $f_\\theta'$ instead of $f_\\theta$\n\nLine 136: \"hyperplane\"\n\nLine 137: $x_{\\text{out}}$ instead of $x_{\\text{in}}$\n\n\n### Originality of the work\n\nTo my knowledge, the formulation of the SS loss is novel, and the parametrization of positive and negative slopes to aid with the class imbalance present in trustworthiness classification has not been done before. There is rich literature in the field of proper scoring rules and binary classification for parametrized binary loss functions [1], but I have not seen any research in the field of trustworthiness that makes use of these parametrized loss functions. This work has definite novelty in scaling the methods to Imagenet and ViT, whereas previous work in this field focused on MNIST and CIFAR, which are much easier tasks with higher classification accuracy.\n\n### Quality and clarity\n\nThe paper is well-written, hyperparameter tuning and ablation experiments are comprehensive, and the proof for the generalization bound seems correct to me. The authors have picked valid metrics to display the pros and cons of their method, though a further discussion on balancing the tradeoff between TPR and TNR would be good.\n\n### Significance\n\nIn my opinion, the empirical results are impressive and push the envelope for binary classification in the trustworthiness sphere. The implementation of the loss function is also straightforward in a standard deep learning package, and can be plugged into existing pipelines with a one-line change. The authors also perform experiments on a large scale dataset such as Imagenet and report state-of-the-art numbers for their method and some baselines. The risk analysis plots are also an additional way of testing the robustness of these methods, and further results on synthetic and adversarial versions of the test dataset should be considered the norm for future analysis in this space.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have responded N/A to this prompt in the checklist. However, I would advise the authors to add a small discussion about the importance of trading off between TPR and TNR for critical applications such as in medicine, where the robustness of their method to TNR would make a strong statement.", "review_text": "", "extra_scores": {"ethical_concerns": "Not to my knowledge.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "jgEYdKKKLw", "reviewer": "Reviewer_R52P", "summary": "The authors find that past trustworthiness predicting methods are prone to overfitting to the training set where correct predictions are dominant. The label imbalance makes the trustworthiness predictors over-confident to even incorrect predictions. To solve the problem, this work proposes a new training loss named steep slope loss, which aims at improving the generalizability of trustworthiness predictors. The intuition behind this is to emphasize the negative class during training. The experiments are conducted in ImageNet. The results show the shortcomings of past methods with cross-entropy loss, focal loss, and TCP loss and then demonstrate the effectiveness of the proposed method. ", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*Strengths\n- The paper reveals the shortcomings of past works in large-scale datasets like ImageNet, which is an interesting problem.\n- The paper proposes an effective method to improve the generalizability for trustworthiness predictors.\n\n*Weaknesses\n- The paper lacks a theoretical study about why the proposed loss is better than the previous methods.\n- It is not clear to the reviewer the empirical and theoretical difference between the proposed method and simply up-weighting the negative samples in trustworthiness learning.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have adequately addressed the limitations. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "20"}, "extra_text": {}}, {"review_id": "RYve4SDZKk", "reviewer": "Reviewer_m14X", "summary": "The paper proposes the steep slope loss for learning trustworthiness under a supervised training setup. A pre-trained neural network is further trained on the steep slope loss to learn trustworthiness as a binary classification problem. The approach is evaluated on ImageNet for a ResNet and a ViT model. In addition, evaluations on MNIST and CIFAR-10 are provided for comparisons with the related work. In all experiments, the proposed approach shows good performance.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Paper strengths: \n\n+ The paper addresses an open problem where there is not a standard solution. It's well-written and mostly easy to follow.\n\n+ The steep slope loss is well-motivated. The visual illustration of the loss in Fig. 2 helps to immediately follow how it works. \n\n+ The evaluation relies on the standard metrics for the presented problem. Also, recent approaches are included in the evaluations.  It's clear that the proposed loss works well for most of the metrics. \n\n+ The paper is the first one to perform a large-scale evaluation for trustworthiness estimation.\n\n+ Unseen domain evaluation: the proposed approach makes use of trustworthiness estimation for identifying wrong predictions in a second domain. This is a novel application for trustworthiness that has not been presenter before.\n\n\nPaper weaknesses: \n\n- The small-scale evaluation is important for the paper because one can compare it with the prior work. They should have been part of the main paper.\n\n- It's not clear whether the trustworthiness estimator (oracle) and the classifier are two separate models. If so, it's important to discuss the inference time of the presented algorithms. The trustworthiness estimator would probably double the inference. \n\n- Comparison with Trust Scores (To trust or not to trust a classifier, NIPS, 2018): The trust scores is a related approach for trustworthiness estimation. It would be supportive for the paper to compare with this approach.\n\n\nImprovements:\n\n- line 96: m is not defined.\n\n- Eq. 4 / Eq. 5: what is the first minus?\n\n\nOverall: \n\nThe paper presents an interesting approach for trustworthiness estimation. The proposed loss function learns when a deep neural network makes untrustworthy predictions. The main innovation of the paper is the evaluation of ImageNet using two complex models. The proposed study brings trustworthiness estimation closer to real-world problems. It's the first one to consider ImageNet with recent network architectures such as the transformer.\n\n\nAfter rebuttal comments:\n\nThe rebuttal addressed my points. In addition, the major concerns on the theoretical study, the difference between the proposed method and negative sample up-weighting, and Imagenet-Corrupted evaluation are discussed in the rebuttal. The proposed approach is the standard evaluation protocol for trustworthiness estimation. The approach does not have to be evaluated for out-of-distribution detection. The current protocol is completely fine and similar to the prior work. For these reasons, I retain my positive evaluation since the paper has enough merit to be accepted. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "-", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2-3"}, "extra_text": {}}, {"review_id": "2nj9oMWO0o", "reviewer": "Reviewer_DFoG", "summary": "This paper tests methods for training a classifier for predicting trustworthiness-- formulated here as predicting whether a given image will be classified correctly or not by an target classifier -- on large scale datasets and finds that using existing loss functions like uncertainty estimates/focal loss on this binary classification task cannot detect untrustworthy samples, all samples are classified as positive resulting in 100% FPR (0% TNR). They attempt to fix this shortcoming by introducing a steep slope loss which essentially separates features w.r.t. correct and incorrect predictions from each other, and the loss is bounded in its magnitude. The paper extensively tests whether its possible to predict trustworthiness with two oracle/target classifier combinations on Imagenet and its stylized/adversarial test sets.", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n2.1) Clear and well-presented: The contributions are clearly stated and well-motivated, acknowledgement is given to the existing body of work to the best of my knowledge. Experiments are performed on large-scale and existing loss functions fail to differentiate between samples. The paper then proceeds to describe a sufficient solution, results are clear and well-presented-- it seems a good step forward.\n\n2.2) Very significant results and interesting transferability: The increase in TNR seems very significant compared to other loss functions, essentially going from near 0 to very significant distinguishability--which says steep slope loss is very useful. The results hold true for stylized and adversarial sets as well with equally significant degrees. The phenomena of ViT being able to detect RSN trustworthy samples far better than the other way around is a very interesting observation, the difference seems stark enough to go beyond simply it having more classification power. \n\n2.3) Reproducible & nice ablations: The code is provided alongside with detailed readme which I wanted to highlight  (I didn't verify the code but it should work well). The ablation with the balanced loss was well-though of and addressed the question I was thinking of.\n\nWeaknesses:\n\n3.1) Formulation unconvincing [Critical]: In my opinion, trustworthiness of a classifier’s output depends on the classifier. Consequently, the input to the trustworthiness predictor should have that output of the target classifier passed (softmax probability/logit etc) alongside the image. This formulation has only the implicit encoding by using supervised samples of a classifier without providing explicit knowledge about the classifier itself seems badly designed (in principle) and I strongly oppose this.\n\n3.2) Unfair comparisons [Critical]: Past works to the best of my knowledge the trustworthiness (typically used for out-of-distribution detection rather than rejecting test samples) is determined by the probability outputs of the target classifiers (a function of the target classifier). The calibrated probability estimates for the same are obtained by using loss functions like cross-entropy (with some calibration) {13} and focal loss [1] -- not training a separate binary classifier with these loss functions as done here, along with assumption of having access to just the image. Recent (parallel) work [2] has benchmarked these methods on large-scale datasets which indicates these scale well to larger datasets. A fair comparison in my opinion should be with (older-- {13},[1]) classifiers using the output of target networks to determine trustworthiness. \n\nOverall:\n\nThe paper is well-written and the improvements are strong. The proposed approach seems simple, accurate and interesting. I mainly have the above concerns regarding the framework introduced and the comparisons made. Overall, I am on the fence but will be willing to increase my rating if the weaknesses are addressed. Note that I might have missed important contribution novelty (summary/strengths) or misunderstood claims (weaknesses) since I am not familiar with the uncertainty/OOD/classification-with-rejection domain.\n\n[1] Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H. S. Torr, Puneet K. Dokania, Calibrating Deep Neural Networks using Focal Loss, NeurIPS 2020\n\n[2] Rui Huang, Yixuan Li, MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Space, CVPR 2021\n\nPost-rebuttal:\n\nI am not convinced by the rebuttal and choose to be skeptical of the proposed approach. \n\nI think the crux of disagreement lies here:\n\n -  The trustworthiness scores are indeed dependent on the classifier.\n\nThe rebuttal and the paper explicitly say it only depends on the image input which makes it a very different problem formulation departing from previous literature [r1]. I want to stress that the difference is not between access to intermediate layers or the last of the unknown classifier, but that there should be access to the unknown classifier at all. [r1] in contrast does assume access, infact stating that they fix the unknown classifier and make it accessible to the trustworthiness predictor. \n\nIn this work as I understand, ideally they should have trained another classifier to predict trustworthiness (of the unknown classifier) given only the soft information of predictions by that unknown classifier on training data. Otherwise it's confusing how a ViT unknown model, ResNet trustworthiness predictor could be used. I hope this clarifies the point I was trying to make.\n\n(Note that [r1] chooses to take the intermediate layers but could just additionally have incorporated the last layer as well, however in this work they cannot assume access to the underlying classifier.)\n\nThe defense of miscalibrated classifiers and adversarial attacks seem not convincing to me for two reasons:\n\n- Calibration has been surprisingly accurate and existing methods seem to alleviate the problem to a large degree on both in-distribution and OOD data. It seems like a more straightforward comparison to make if allowed access to the classifier-- which the authors could just say is unfair because that formulation specifically requires access which they cannot assume (which is my original objection).\n- Adversarial attacks can be made on the trustworthiness predictor as well. I am pondering why dependence on the classifier makes it different, the trustworthiness predictor could ignore the classifiers outputs entirely and choose to rely more on the image.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No substantial negative societal impact that must've been discussed in my view but the section was missing from the draft.\n", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper develops a novel loss function for predicting the "trustworthiness" of a classifier.  This involves training a separate classifier, on existing predictions and corresponding outcomes, that makes a binary prediction about whether a predicted class can be trusted.  The authors show that existing loss functions do not generalize well and instead propose a novel steep slope loss.  The authors present theoretical generalization bounds and empirical evidence on a variety of non-IID validation sets that their method outperforms existing losses.  Overall the reviewers thought that the paper was well written, the contributions significant and the empirical evidence convincing. 
 One criticism was regarding the amount of access required by the method to the classifier of interest (there was significant debate about this and seems to have been resolved to an acceptable level).   Two reviewers voted high confidence accepts (7, 8) and two low confidence borderline scores (5, 6).  Thus the recommendation is to accept the paper.   Please try to incorporate feedback from the reviewers (e.g. clarification about access to the classifier) in the camera ready version.
Decision: Accept (Poster)