Training Uncertainty-Aware Classiﬁers with Conformalized Deep Learning
Abstract
Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty and estimate reliable probabilities.
In particular, they tend to be overconﬁdent.
We begin to address this problem in the context of multi-class classiﬁcation by developing a novel training algorithm producing models with more dependable uncertainty estimates, without sacriﬁcing predictive power.
The idea is to mitigate overconﬁdence by minimizing a loss function, inspired by advances in conformal inference, that quantiﬁes model uncertainty by carefully leveraging hold-out data.
Experiments with synthetic and real data demonstrate this method can lead to smaller conformal prediction sets with higher conditional coverage, after exact calibration with hold-out data, compared to state-of-the-art alternatives.
Introduction
The predictions of deep neural networks and other complex machine learning (ML) models affect important decisions in many applications [1–6], including autonomous driving, medical diagnostics or security monitoring.
Prediction errors in those contexts can be costly or even dangerous, which makes dependable and explainable uncertainty estimation essential.
Unfortunately, deep neural networks are not designed to understand uncertainty and they are easily prone to overﬁtting; consequently, they may lead to overconﬁdence [7–9].
Overconﬁdence is especially problematic in the face of aleatory uncertainty [10], which refers to situations in which the outcome to be predicted is intrinsically noisy.
Unlike the complementary epistemic uncertainty, aleatory uncertainty cannot be eliminated by training a more ﬂexible model or by increasing the sample size.
For example, think of prognosticating
COVID-19 survival [11, 12], assessing genetic disease predisposition [13–15], or anticipating credit card defaults [16].
In those applications, the outcome of interest is potentially complicated and likely depends on many unmeasured variables.
Therefore, no practical model may achieve perfect accuracy, but models that can offer practitioners principled measures of conﬁdence for each individual-level prediction naturally tend to be more useful and trustworthy.
Among many existing techniques for estimating uncertainty in ML predictions [7–9], conformal inference [17], stands out for its ability to provide ﬁnite-sample guarantees without unrealistic algorithmic simpliﬁcations and without strong assumptions about the data generating process or the underlying sources of uncertainty.
Conformal inference is designed to convert the output of any ML model into a prediction set of likely outcomes whose size is automatically tuned using hold-out data , in such a way that the same procedure applied to future test data will yield prediction sets that are well calibrated in a frequentist sense.
In particular, these sets have provable marginal coverage ; i.e., at the 90% level this means the outcome for a new random test point is contained in the output set 90% of the time.
The hold-out data are utilized to evaluate conformity scores , or goodness-of-ﬁt scores, whose ordering determines how the sets are to be expanded so that the desired fraction of test points is guaranteed to be covered.
A limitation of conformal inference is that it involves of two distinct phases, training and calibration, that are generally not designed to work together efﬁciently.
The calibration algorithm takes as input pre-trained models that may already be overconﬁdent, and this is sub-optimal because bad habits are harder to correct after they become entrenched.
As a result of this uncoordinated two-step approach, conformal predictions may be either unnecessarily conservative or overconﬁdent for certain types of test cases, which can make them unreliable [18, 19] and unfair [20].
We address this challenge by developing a new ML training method that synergetically combines the learning and calibration phases of conformal inference, reducing the overconﬁdence of the predictions calculated by the output model.
The idea is to minimize a new loss function designed to measure the discrepancy in distribution between the conformity scores computed by the current model estimate and those of an imaginary oracle that can leverage perfect knowledge of the data generating process to construct the most informative and reliable possible predictions.
As we will demonstrate empirically, applying standard conformal inference (based on independent calibration data) to models trained with our novel algorithm leads to smaller prediction sets with more accurate coverage for all test points.
In other words, the “uncertainty-aware” ML models trained with the proposed algorithm tend to work more efﬁciently within a standard conformal inference framework compared to black-box models, which perhaps should not be surprising.
Further, we will also show that the models obtained with our method produce relatively reliable prediction sets even if the post-training conformal calibration step is omitted, although the latter remains necessary in theory to guarantee valid coverage.
Related work
Many methods have been developed to mitigate overconﬁdence in ML [21–34], for example by allowing an agnostic output, by suitable post-processing [7, 35–40], or through early stopping [41].
Additional relevant research includes that of [42–46, 46–52], and these will provide us with informative benchmarks.
However, unlike conformal inference, these methods have no frequentist guarantees in ﬁnite samples, and largely rely on loss functions targeting the accuracy of best-guess predictions, without explicitly addressing uncertainty during training.
We build upon conformal inference [53– 58], pioneered by [17], which typically deals with off-the-shelf models [17, 19, 59, 60].
Although other very recent works have proposed leveraging ideas from this ﬁeld to improve training [61–66], this paper is novel as it combines the adaptive conformity scores of [19] with a completely new uncertainty-aware loss function.
This departs from [62, 63], which sought to minimize the cardinality of the prediction sets, and from [61, 64–66], which utilized conformal inference ideas for tuning low-dimensional hyper-parameters as opposed to fully guiding the training of all model parameters.
Although we focus on classiﬁcation [19], our method could be repurposed for regression [58, 67] and other supervised tasks [60, 68, 69].
Conformal inference can also be utilized to test hypotheses and calibrate probabilities [70], and our work could be extended to those problems.
Relevant background on conformal inference 2.1 Uncertainty quantiﬁcation via conformal prediction sets
Consider a data set of i.i.d.
(or sometimes simply exchangeable ) observations ( X i , Y i ) n +1 i =1 sampled from an arbitrary unknown distribution P XY .
Here, X i ∈ R p contains p features for the i th sample, and Y i ∈ { 1 , . . .
, K } =
[ K ] denotes its label, which we assume to be one of K possible categories.
The goal is to train a model on n data points, ( X i , Y i ) ni
=1 , and construct a reasonably small prediction set ˆ C n,α ⊆
[ K ] for Y n +1 given X n +1 such that, for some ﬁxed level α ∈ (0 , 1) ,
[Equation] [Equation]
This property is called marginal coverage because it treats ( X i , Y i ) n +1 i =1 as all random.
If α = 0 .
1 , it ensures Y n +1 is contained in the prediction sets 90% of the time.
Marginal coverage is practically feasible but not fully satisfactory, as it is not as reliable and informative as conditional coverage :
[Equation]
Conditional coverage would give one conﬁdence that ˆ C n,α ( x ) contains the true Y for any individual data point, which is stronger than (1).
For example, imagine a population in which color is a feature and 90% of samples are blue while the others are red.
Then, 90% marginal coverage is attained by any prediction sets that contain the true Y for all blue samples but never do for the red ones.
Valid coverage can be obtained conditional on a given protected category [20], but it is impossible to guarantee (2) more generally without unrealistically strong assumptions [71, 72].
Thus, a typical compromise is to construct prediction sets with marginal coverage and hope they are also reasonably valid conditional on X .
For multi-class classiﬁcation, a solution is offered by the conformity scores developed in [19], which are reviewed below as the starting point of our contribution.
2.2 Review of adaptive conformity scores for classiﬁcation
Imagine an oracle knowing the conditional distribution of Y given X , namely P Y | X , and think of how it would construct the smallest possible prediction sets C oracle α ( x ) with exact 1 − α conditional coverage.
For any x ∈ X and y ∈ [ K ] , deﬁne π y ( x )
=
P
[ Y = y
| X = x ] .
Then, the oracle would output the smallest subset S ⊆
[ K ] such that (cid:80) y ∈ S π y ( x ) ≥ 1 − α .
In truth, this set may have coverage strictly larger than 1 − α due to the discreteness of Y ; however, exact coverage can be achieved by introducing a little extra randomness [20].
For any nominal coverage level τ ∈ (0 , 1) and any random noise variable u ∈ (0 , 1) , let S be the function with input x , u , π , and τ that computes the set of most likely labels up to (but possibly excluding) the one identiﬁed by the above deterministic oracle: S ( x, u ; π, τ ) ⊆
[ K ] .
Note that the explicit expression of S is written in Appendix A1.1.
If u = U is a uniform random variable independent of everything else, it is easy to verify that the following oracle prediction sets have exact conditional coverage at level τ = 1 − α :
[Equation]
For example, if π 1 ( x ) = 0 . 3 , π 2 ( x ) = 0 .
6 , and π 3 ( x ) = 0 .
1 , then C oracle 0 .
2 ( x ) = { 1 , 2 } with probability 2 / 3 , and C oracle 0 .
2 ( x ) = { 2 } with probability 1 / 3 .
Of course, this oracle is only a thought experiment because P Y | X is generally unknown.
Therefore, to construct practical prediction intervals, π must be replaced by an ML model ˆ π .
Then, conformal inference is needed to ensure the output sets based on the possibly inaccurate ML model at least satisfy marginal coverage (1).
Conformal inference [19] begins by training any classiﬁer on part of the data, indexed by D 1 ⊂ [ n ] , to ﬁt an approximation ˆ π of the unknown π .
After substituting ˆ π into the oracle rule S , the hold-out data indexed by D 2 = [ n ] \ D 1 are leveraged to adjust the prediction sets as to empirically achieve the desired coverage.
More precisely, deﬁne the following conformity scores W i for all observations in D 2 , and for the test point n + 1 .
Intuitively, W i is the smallest τ ∈ [0 , 1] such that S ( X i , U i ; ˆ π, τ ) contains the true Y i , while U i is a uniform random variable independent of everything else:
[Equation]
These statistics are observed for all i ∈ D 2 , but not for n + 1 .
Deﬁne also ˆ τ n,α as the (cid:100) (1 − α )(1 + |D 2 | ) (cid:101) largest element of { W i } i ∈D 2 .
Intuitively, ˆ τ n,α is the smallest τ ∈ [0 , 1] such that
S ( X i ; ˆ π, τ ) contains a fraction 1 − α of the hold-out data in D 2 .
Then, the output prediction set for a new X n +1 is C n,α ( X n +1 )
=
S ( X n +1 , U n +1 ; ˆ π, ˆ τ n,α ) .
This has 1 − α marginal coverage due to the exchangeability of the calibration and test data [19].
In fact, Y n +1 / ∈ C n,α ( X n 1 ) implies
W n +1 > ˆ τ n,α , and by exchangeability the probability of this event is smaller than α ; see [58].
Unlike alternative approaches based on different scores [17, 18, 53], this solution would yield prediction sets equivalent to those of the oracle [19] if ˆ π = π .
Although generally ˆ π (cid:54) =
π , the above prediction sets often achieve relatively high conditional coverage [19] in practice.
Our goal is to further improve their empirical performance by training the ML model to be more deliberately aware of uncertainty.
Methods 3.1 The distribution of the adaptive conformity scores
The conformity scores deﬁned in (4) are uniformly distributed conditional on X = x , for any x , if ˆ π = π .
This property was hinted without proof in [19] and it serves as the starting point of our contribution.
Note that all mathematical proofs can be found in Appendix A2.
Proposition 1.
The distribution of the conformity scores W i in (4) is uniform conditional on X i if ˆ π = π .
That is, P [ W ( X, Y, U ; π ) ≤ β | X = x ] = β for all β ∈ (0 , 1) , where ( X, Y ) is a random sample from P X,Y , and U ∼ Uniform[0 , 1] independent of everything else.
Further, W i | X i is uniform if and only if S ( X i ; ˆ π, 1 − α ) has conditional coverage at level 1 − α for all α ∈ [0 , 1] .
As we seek accurate conditional coverage, this result suggests training ˆ π as to produce scores that are approximately uniform on hold-out data, at least marginally.
Therefore, we will evaluate (4) on hold-out data while training ˆ π , encouraging the conformity scores to follow a uniform distribution.
3.2 An uncertainty-aware conformal loss function
We develop a loss function that approximately measures the deviation from uniformity of the conformity scores deﬁned in (4) by combining classical non-parametric tests for equality in distribution with fast algorithms for smooth sorting and ranking [73, 74].
This loss is combined with the traditional cross entropy as to also promote accurate predictions, and it can be approximately optimized by stochastic gradient descent (it will generally be non-convex).
The novel uncertainty-aware component of this loss only sees the hold-out samples through the lens of a non-parametric test applied to the empirical score distribution within a subset of the data.
Therefore, it provides little incentive to overﬁt compared to a traditional loss targeting point-wise predictive accuracy, such as the cross entropy.
By contrast, it discourages overconﬁdent predictions which would yield non-uniform scores, as we shall see below.
This solution is outlined in Figure A1, Appendix A1.2, and detailed below.
First, the n training samples are partitioned into two subsets, I
and I 2 such that I 1 ∪I 2 =
[ n ] = D 1 .
Here we assume the training data are indexed by D 1 = [ n ] ; this notation is slightly different from
Section 2, but it is simple and does not introduce ambiguity because the additional calibration data in D 2 remain untouched during training.
The training algorithm approximately minimizes a loss function (cid:96) consisting of two additive components, each evaluated on one subset of the data:
[Equation]
Above, the hyper-parameter λ ∈ [0 , 1] controls the relative weights of the two components.
The (cid:96) a component is evaluated on the data in I 1 , and its purpose is to seek high predictive accuracy, as customary.
For example, this could be the cross entropy:
[Equation]
where ˆ π is the output of the ﬁnal softmax layer.
The novel uncertainty-aware component (cid:96) u is evaluated on I 2 , and its role is to mitigate overconﬁdence.
Concretely, conformity scores W i are evaluated according to (4) for all i ∈
I 2 , and their empirical distribution is compared to the ideal uniformity expected if ˆ π = π .
Ideally, we would like to quantify this discrepancy by directly applying a powerful non-parametric test, for example by computing the Cramér-von Mises [75, 76] or Kolmogorov-Smirnov [77, 78] test statistics.
Concretely, in the latter case,
[Equation]
where ˆ F |I 2 | ( · ) is the empirical cumulative distribution function (CDF) of W i for i ∈
I 2 : ˆ F |I 2 | ( w ) =
(1 / |I 2 | ) (cid:80) i ∈I 2 1 [ W i ≤ w ] .
Unfortunately, ˆ F |I 2 | ( · ) is not differentiable, which makes the overall loss intractable to minimize.
This requires introducing some approximations in (cid:96) u , as explained next.
3.3 Differentiable approximations
The empirical CDF of the conformity scores is not differentiable because it involves sorting, which is a non-smooth operation.
Further, these scores themselves are not differentiable in the model parameters θ because they involve ranking and sorting the estimated class probabilities ˆ π .
In fact, the score (4) can be computed in a closed form,
[Equation]
where U i is a uniform random variable independent of everything else; see Appendix A1.1 for details.
Fortunately, there exist fast approximate algorithms for differentiable sorting and ranking that work well in combination with standard back-propagation [73, 74].
Note that evaluating W i in (8) requires accessing elements of (ˆ π (1) ( X i ) , . . .
, ˆ π ( K ) ( X i )) through a θ -dependent index, r ( Y i , ˆ π ( X i )) , which is also non-differentiable.
Therefore, indexing by r ( Y i , ˆ π ( X i )) must be approximated with a smooth linear interpolation; see Appendix A1.2 for further details.
In conclusion, the (cid:96) u loss in (7) is approximated by evaluating a differentiable version of the scores in (4) as described above, and then by replacing their empirical CDF with a differentiable approximation obtained with the same techniques from [74].
This procedure, combined with stochastic gradient descent for ﬁtting the model parameters θ , is summarized in Algorithm 1.
Although here we assume I
= I 2 for simplicity, this algorithm can easily accommodate I 1 (cid:54)
=
I 2 .
A more technically detailed version of Algorithm 1 is provided in Appendix A1.2, and an open-source software implementation of this method is available online at https://github.com/bat-sheva/conformal-learning .
Input: Data { X i , Y i } ni =1 ; hyper-parameter λ ∈ [0 , 1] , learning rate γ > 0 , batch size M ;
Randomly initialize the model parameters θ (0) ;
Randomly split data subsets
∪
Set
); for do
Randomly divide
; for do
Evaluate
Generate
Evaluate
˜ W i for all i in batch b of I 2 , using U i and a differentiable approximation of (4)
Evaluate the gradient
(cid:96) a ( θ ( t ) ) of (cid:96) a in (6) using the data in batch b of I
Evaluate the gradient ∇ ˜ (cid:96) u
( θ ( t ) ) of a differentiable approximation ˜ (cid:96) u of (cid:96) u in (7) using the differentiable scores ˜ W i in batch b of I
− λ
· ∇ λ · ∇
3.4 Theoretical analysis
The uncertainty-aware loss function in Algorithm 1 can be justiﬁed theoretically by noting that it is approximately minimized (although possibly non-uniquely) by the imaginary oracle model π , which yields the smallest possible prediction sets with exact conditional coverage.
This analysis focuses on the original version of the loss function deﬁned in (5)–(7), ignoring for simplicity the additional subtleties introduced by the differentiable approximations described in Section 3.3.
Proposition 2.
The loss function (cid:96) ≥ 0 in (5) is bound from above by (cid:96) 0 + δ(cid:96) , where (cid:96) 0 ≥
attains value zero if ˆ π = π , and δ(cid:96) =
O P (1 / √ M ) as I 2 → ∞ .
Of course, Algorithm 1 does not minimize (5) exactly because it involves solving a high-dimensional non-convex optimization problem that is difﬁcult to study theoretically.
Yet, it is possible to prove at least a weak form of convergence for its stochastic gradient descent, whose solution may not however necessarily approach a global minimum.
This analysis is in Appendix A2 for lack of space.
Numerical experiments 4.1 Experiments with synthetic data
The performance of Algorithm 1 is investigated here on synthetic data that mimic a multi-class classiﬁcation problem in which most samples are relatively easy to classify but a few are unpredictable.
Speciﬁcally, data are simulated with 100 independent and uniformly distributed features X =
( X 1 , . . .
, X 100 ) ∈ [0 , 1] 100 and a label Y ∈
[ K ] , for K = 6 .
The ﬁrst feature controls whether the sample is intrinsically difﬁcult to classify, while the next two features determine the most likely labels; all other features are useless.
On average, 20% of the samples are impossible to classify with absolute conﬁdence.
This conditional distribution is written explicitly in Appendix A3.1.
The conditional class probabilities ˆ π are estimated as the output of a ﬁnal softmax layer in a fully connected neural network implemented with PyTorch [79]; see Appendix A3.1 for more information about network architecture and training details.
This model is ﬁtted separately with Algorithmand three benchmark techniques including traditional cross entropy minimization and focal loss minimization [80].
Unfortunately, we cannot directly compare to the recent methods of [62, 63] as originally implemented by those authors for lack of openly available computer code.
Instead, we consider a hybrid benchmark that combines elements of Algorithm 1 with the main idea of [63], essentially seeking a model that yields small conformal prediction sets, irrespective of conditional coverage; see Appendix A1.3 for details about this benchmark.
As early stopping can help mitigate overﬁtting [41], it is informative to also investigate its effect on each of the aforementioned learning algorithms.
For this purpose, we generate an additional validation set of 2000 independent data points and use it to preview the out-of-sample accuracy and loss value at each epoch.
Then, the best versions of each model according to these two early stopping criteria are saved during training.
After training each model, 10,000 additional independent samples are utilized to calibrate split-conformal prediction sets with 90% marginal coverage, as explained in Section 2.2.
All models, including those trained with our method, undergo conformal calibration with these independent data points prior to constructing the prediction sets, as this step is necessary to theoretically guarantee marginal coverage.
Further, we ensure the comparisons between different models are always fair by giving each training algorithm access to exactly the same data set, regardless of whether its inner workings involve sample splitting to evaluate separately distinct components of the loss function.
Figure 1 compares the performance of conformal prediction sets obtained with each model
fortest points as a function of the number of training samples, averaging over 50 independent experiments.
The prediction sets are evaluated in terms of their average size and coverage conditional on X 1 ≤ δ ; i.e., separately for the “hard” samples.
For each learning algorithm, the results corresponding to the model achieving the highest conditional coverage among the fully trained and two early stopped alternatives are reported.
This allows us to focus on the overall behaviour of different losses while accounting for possible differences in the optimal choices of early-stopping strategies.
[Figure]
Figure 1 does not visualize marginal coverage because that is guaranteed to be above 90%.
The results show our algorithm yields the prediction sets with highest conditional coverage and smallest size, especially when the number of training samples is large.
By contrast, the conditional coverage obtained with the cross entropy does not visibly improve as the training data set grows, suggesting systematic over-conﬁdence with hard-to-classify samples.
The focal loss improves upon the cross entropy by reducing the average size of the prediction sets, but it does not lead to higher conditional coverage.
Finally, the hybrid algorithm increases conditional coverage slightly compared to the cross entropy, but it does not lead to smaller prediction sets.
This is likely because the hybrid loss does not introduce very different incentives compared to the cross entropy, as the latter already attempts to maximize the estimated probability of the observed labels, thus effectively seeking small conformal prediction sets without necessarily high conditional coverage.
Note that the focal loss is applied here with hyper-parameter γ = 1 , as we have found larger values to yield lower accuracy; see Figure A2.
In this section, our method is applied with the hyper-parameter λ = 0 .
2 in (5); additional results obtained with different values of λ are in Figures A3–A4, Appendix A3.2.
The improved performance of our method can be understood by looking at the distribution of the corresponding conformity scores for test data, as shown in Figure A6 for the model trained onsamples.
Our method leads to scores that are more uniformly distributed than those obtained with the cross entropy, which is indicative of more reliable uncertainty quantiﬁcation.
In fact, our estimated conditional probabilities ˆ π are more similar to the true oracle π compared to those obtained by minimizing the cross entropy; see Figure A7.
All ﬁgures here refer to fully trained models, without early stopping; analogous results with early stopping are presented later.
It is worth emphasizing that post-training conformal calibration is always necessary to guarantee marginal coverage, regardless of how the model is trained.
At the same time, it is interesting to observe that models trained with our method tend to estimate more reliable probabilities compared to the benchmarks.
Therefore, it should not be surprising that our models lead to larger prediction sets with relatively high marginal coverage if we skip the post-training conformal calibration step; see Figure A5.
Figure A8 compares the performance of the prediction sets obtained with each method when covariate shift occurs in the test data.
In particular, here we imagine that at test time the uncertainty-controlling feature X 1 is sampled uniformly from [0 , a ] with a ≤ 1 , so that lower values of a correspond to higher proportions of intrinsically hard-to-classify samples.
Of course, in this case marginal coverage is no longer guaranteed for the same reason why conditional coverage in Figure 1 is not always controlled.
As expected, all models produce smaller set sizes with higher marginal coverage for a closer to 1, consistently with Figure 1, but our method outperforms the benchmarks.
Several additional results are in Appendix A3.2.
Figure A9 reports on experiments in which the number K of labels is varied, ranging from 4 to 12 .
These results show our methods leads to prediction sets with consistently smaller size and typically higher conditional coverage compared to all benchmarks.
Figure A10 reports on experiments in which the proportion of hard-to-classify samples is varied, ranging from 0 .
1 to 0 .
All models lead to higher conditional coverage for larger δ , as implied by the ﬁxed marginal coverage, but our method consistently achieves it with smaller prediction sets.
Figures A11–A14 report on experiments with models trained using early stopping based on maximum prediction accuracy on the validation data.
Again, our method achieves higher conditional coverage and smaller prediction sets relative to the benchmarks.
Figures A15–A18 report analogous results obtained with early stopping based on minimum validation loss.
Figures A19–A (resp.
A21–A22) show the distribution of conformity scores and the estimated class probabilities, as in Figures A6–A7, from models trained with early stopping based on validation predictive accuracy (resp. loss).
Finally, Figures A23–A24 (resp.
A25–A26) show the distribution of conformity scores and the estimated class probabilities from the focal loss (res.
hybrid) models.
4.2 Experiments with CIFAR-10 data
Convolutional neural networks guided by the conformal loss are trained on the publicly available
CIFAR-10 image classiﬁcation data set [81] (10 classes), and the models thus obtained are compared to those targeting the three benchmark losses considered before.
As these data are not too hard to classify, we make the problem more interesting by randomly applying RandomErasing [82] to some images—a form of corruption that makes images very hard to recognize.
The number of training samples is varied from 3000 to 45000.
See Appendix A3.3 for details.
To measure the performance of conformal prediction sets based on each model, we set aside 5000 calibration and test observations prior to training.
All models are calibrated after training, as in the previous section, in order to guarantee valid marginal coverage.
The proportions of corrupt images in the calibration and test sets are 0.2, while that in the training set is varied.
All models are calibrated as in [19], seeking 90% marginal coverage.
Their performance is measured on the test data in terms of the size of the output prediction sets and their coverage conditional on the indicator of corruption.
Further, the test accuracy of each model is evaluated based on the misclassiﬁcation rate of its best-guess label.
Figureshowcases two example test images, respectively intact and corrupted by RandomErasing, along with their corresponding conditional class probabilities calculated by different models fully trained on 45000 data points.
This shows the model trained with our conformal loss is not as overconﬁdent when dealing with the corrupted images as that minimizing the cross entropy.
[Figure]
Figure A27 summarizes the performance of the prediction sets obtained with each fully trained model, as a function of the number of training samples.
Median performance measures are reported overexperiments with random data subsets.
The conformal loss leads to higher coverage for intrinsically hard images, and smaller prediction sets for the easy ones.
As shown in Figure A28, this improvement is associated with higher test accuracy for the models targeting our loss, consistently with their increased robustness to overﬁtting.
As overﬁtting can also be mitigated by early stopping, we report in Figure A29 the corresponding results obtained with early stopping based on maximum accuracy on a validation data set of size 2000 (or 5000, training with 45000 samples).
In this case, all models achieve similar test accuracy, but those trained with our method lead to conformal prediction sets with (slightly) higher conditional coverage and smaller size, especially if trained on many samples and compared to the focal loss.
These conclusions are summarized in Table 1 in the case of 45000 training samples, which includes also the results corresponding to models trained with early stopping based on validation accuracy.
See Table A1 for results with early stopping based on validation loss.
Overall, the models targeting the conformal loss perform best when trained fully or with early stopping based on accuracy, and they tend to achieve higher coverage for the hard images while enabling smaller prediction sets with valid coverage for the easy cases.
A similar summary for the models trained with 3000 samples is in Table A2; there, the advantage of the conformal loss compared to the hybrid method is not as marked as in the large-sample experiments.
Note that the models in Table 1 have accuracy below 83% due to the corrupted images in the training data; by comparison, 89% accuracy could be achieved by applying the cross-entropy method on the clean data set [83].
In Appendix A3.4, Figures A30 and A31 report on results similar to those in Figures A27 and A29, respectively, but using prediction sets directly constructed based on the probability estimates computed by each model, without conformal calibration.
Consistently with the experiments described in the previous section, these results show our models lead to prediction sets with relatively high marginal coverage, especially if the sample size is large.
Of course, post-training conformal calibration remains necessary to guarantee the nominal coverage level in ﬁnite samples.
Figures A32 and A visualize some concrete examples of prediction sets obtained with and without post-training conformal calibration, respectively for intact and corrupt images.
These results conﬁrm the prediction sets obtained with our method tend to be less overconﬁdent compared to the benchmarks, whether or not post-training conformal calibration is applied to guarantee marginal coverage.
[Table]
Figure A34 reports performance measures as in Figure A27, after ﬁxing the number of training samples to 45000 and varying the corruption proportion.
The model trained with the conformal loss leads to smaller prediction sets with higher conditional coverage compared to the benchmarks, and its test accuracy does not decrease as the proportion of corrupted training images increases.
Figure A35 demonstrates early stopping mitigates overﬁtting and allows the benchmarks to maintain relatively high accuracy as the training corruption proportion increases.
However, the conformal loss outperforms even if all models are trained with early stopping based on validation accuracy.
Figurereports on experiments similar to those in Table 1 but performed under covariate shift.
Here, the proportions of corrupt images in the training and calibration data sets are still ﬁxed to 0.2, while the proportion of corrupt images in the test set is varied from 0.2 (no covariate shift) to 1.0 (extreme covariate shift).
In these experiments, no model is theoretically guaranteed to achieve valid marginal coverage due to the covariate shift, but the model trained with our method remains approximately valid because it has practically high conditional coverage.
In conclusion, these results conﬁrm the model trained with our method can provide reliable uncertainty estimates for both intact and the corrupt images, while the benchmark models tend to be overconﬁdent in the hard-to-classify cases.
[Figure]
The distributions of conformity scores obtained with different methods on test data are compared in
Figures A36–A39.
These results conﬁrm the scores obtained with the conformal loss tend to be more uniform compared to the benchmarks, especially if the training sample size is large.
If few training samples are available, the focal loss yields scores that are slightly closer to being uniform, but at the cost of lower accuracy and conditional coverage (Table A2).
In Figures A40–A43, the performances of all models are compared separately for each of the 10 possible test labels.
These results suggest corrupt images with true labels 4,5, or 6 are most difﬁcult to classify, and their prediction sets have the lowest coverage.
However, this issue is mitigated by models fully trained to minimize the proposed conformal loss.
Further, the results conﬁrm that models trained with our loss yield more informative prediction sets for the easier unperturbed images, while achieving the desired coverage rate.
4.3 Experiments with credit card data
In this section, we analyze a publicly available credit card default data set [84] containingobservations of 23 features and a binary label.
Approximately 22% of the labels are equal to 1.
The data are randomly divided into 16800 training samples, 4500 calibration samples, andtest samples, while the remaining 4200 samples are utilized to determine early stopping rules.
All experiments are repeated 20 times with independent data subsets.
The conformal and hybrid losses are implemented using 70% of the training samples for evaluating the cross entropy component.
The model architecture is as in Section 4.1; see Appendix A3.5 for further implementation details.
The performances of conformal prediction sets for models trained with different losses are measured in terms of their respective sizes and coverage conditional on the true label being equal to 1.
As the samples with label 1 are a minority, constructing prediction sets with valid coverage for their group is an interesting problem that may be relevant in the context of algorithmic fairness [20].
Note that it is possible to calibrate conformal prediction sets in such a way as to achieve perfect label-conditional coverage [17, 19], at the cost of higher data usage, but here we focus on training uncertainty-aware models as to approximately achieve this goal without explicit label-conditional calibration.
Therefore, we apply a slightly modiﬁed version of Algorithm 1 in which the empirical distribution of the hold-out conformity scores is evaluated separately on the samples from each class, and then the sum of these two instances of (7) is utilized as (cid:96) u .
The same strategy is also incorporated into the hybrid method.
Table 2 compares the average performances of each model and shows our method achieves the highest conditional coverage.
Here, early stopping is applied based on minimum validation loss, as the alternative accuracy criterion achieved lower conditional coverage for all models.
Our conformal models have relatively high classiﬁcation errors on these data, but this can be explained by the high class imbalance.
Thus, the error rate in Table 2 may be less informative as a performance metric than the corresponding F-score reported in Figure A44, which is higher (better) for the models trained with our method.
Finally, Figure A45 and Table A3 in Appendix A3.6 demonstrate our model leads to conformity scores that are closer to being uniformly distributed on test data, as expected.
[Table]
Discussion
The conformal loss function presented in this paper mitigates overconﬁdence in deep neural networks and can lead to smaller prediction sets with higher conditional coverage compared to standard benchmarks.
This contribution is practically relevant for many applications in which overﬁtting may occur, but it is especially useful when dealing with noisy data that do not allow very accurate out-of-sample predictions.
One limitation of the proposed method is that it is more computationally expensive than its benchmarks.
For example, training a conformal loss model on 45000 images in the
CIFAR-10 data set took us approximately 20 hours on an Nvidia P100 GPU, while training models with the same architecture to minimize the cross entropy or focal loss only took about 11 hours.
Another limitation of the conformal loss is that a relatively large number of training samples appears to be required for signiﬁcant performance improvements.
Nonetheless, the ability of an ML model to more openly admit ignorance when asked to provide an unknown answer is a valuable achievement for which it may sometimes be worth investing additional training resources.
Future research may explore extensions of our method for uncertainty-aware learning to problems beyond multi-class classiﬁcation or to ML models other than deep neural networks, as well as further applications to real-world data sets.
Focusing more closely on image classiﬁcation, it would be interesting to combine the method proposed in this paper with data augmentation techniques that can further increase predictive accuracy [83], especially if the available training set is limited [85].
Further, it could be useful to combine our method with different random image corruption techniques as well as data augmentation in order to improve robustness to covariate shift and adversarial cases.
Such extensions are not straightforward because data augmentation may violate the sample exchangeability assumptions, but recent theoretical advances in conformal inference may open the door to a principled solution [86, 87].
We plan to explore these directions in future work.
Software implementing the proposed method is available online at https://github.com/ bat-sheva/conformal-learning , with all code needed to reproduce the numerical experiments.
Acknowledgements
M. S. and Y. Z. are supported by NSF grant DMS 2210637.
M. S. is also supported by an Amazon
Research Award. B. E. and Y. R. were supported by the Israel Science Foundation (grant No. 729/21).
Y. R. thanks the Career Advancement Fellowship, Technion, for providing research support.
The authors also thank four anonymous reviewers for helpful comments and an insightful discussion.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]