N OVELTY DETECTION USING ENSEMBLES WITH REGULARIZED DISAGREEMENT
Anonymous authors
Paper under double-blind review
A BSTRACT
Despite their excellent performance on in-distribution (ID) data, deep neural networks often conﬁdently predict on out-of-distribution (OOD) samples that come from novel classes instead of ﬂagging them for expert evaluation.
Even though conventional OOD detection algorithms can distinguish far OOD samples, current methods that can identify near OOD samples require training with labeled data that is very similar to these near OOD samples.
In turn, we develop a new ensemble-based procedure for semi-supervised novelty detection (SSND) that only utilizes a mixture of unlabeled ID and OOD samples to achieve good detection performance on near OOD data.
It crucially relies on regularization to promote diversity on the OOD data while preserving agreement on ID data.
Extensive comparisons of our approach to state-of-the-art SSND methods on standard image data sets (SVHN/CIFAR-10/CIFAR-100) and medical image data sets reveal signiﬁcant gains with negligible increase in computational cost 1 .
I NTRODUCTION
Despite achieving great in-distribution (ID) performance, deep neural networks (DNN) often have trouble dealing with samples in the test set that are out-of-distribution (OOD), i.e. test inputs that are unlike the data seen during training.
For example, DNNs often make incorrect predictions with high conﬁdence when new unseen classes emerge over time (e.g. undiscovered bacteria Ren et al.
new diseases Katsamenis et al.
or when data suffers from distribution shift (e.g. corruptions
Lu et al.
(2019), environmental changes Kumar et al.
If the labels of these OOD samples follow the same conditional probability distribution as the training set Shimodaira (2000), it is most desirable to output calibrated uncertainty estimates in addition to a single predicted value, as in
Neal (1996); Gal & Ghahramani (2016); Malinin & Gales (2018); Lakshminarayanan et al.
In contrast, when this assumption does not hold and some inputs might come from previously unseen classes, we would like to detect such novel samples and bring them to the attention of experts.
More speciﬁcally, in novelty detection , we want to detect inputs x that satisfy P X ( x ) < α for a small threshold α , where P X is the marginal training (ID) distribution.
Previous works that have tackled this problem are known as open-set recognition and OOD detection methods.
In what follows, we use the terms novelty detection (ND) and OOD detection interchangeably.
Related problems such as aforementioned uncertainty estimation or one-class (OC) classiﬁcation have slightly different objectives but also include methods that are applicable to novelty detection.
[Table]
Numerous novelty detection methods are successful for simple benchmarks where the OOD samples are far from the training samples (such as SVHN vs CIFAR10).
As Winkens et al.
(2020) and a number of concurrent works (Tack et al., 2020; Fort et al., 2021) recently noted, these methods however have subpar performance on near OOD data, for instance when OOD samples are drawn from unseen classes from the same data set (e
[Table]
g. CIFAR100 vs CIFAR10).
A recent method that is reported to have high near OOD detection performance involves tuning large models pretrained on ImageNet21k (Fort et al., 2021) and, hence, arguably relies on using during training OOD data that contains classes that are close to the unseen CIFAR classes during test time.
However, in many scientiﬁc applications that use, for instance, medical or satellite images, such large data sets for pretraining are not available.
In particular, truly novel classes will be inherently dissimilar from any previously available data while still sharing some of the same characteristics.
Instead, a more realistic scenario assumes access to a small batch of unlabeled test data that includes
ID and OOD samples without knowing which are the outliers.
We can use this set during training to ultimately perform well on future samples (i.e. semi-supervised novelty detection (SSND) (Blanchard et al., 2010)) or to detect outliers from that particular test batch itself (reminiscent of transductive OOD detection (Scott & Blanchard, 2008)).
Even though using unlabeled data has the potential to improve detection performance as argued in Scott & Blanchard (2009), existing SSND methods for deep neural networks (Kiryo et al., 2017; Yu & Aizawa, 2019; Guo et al., 2020; Zhang et al., 2020b) do not leverage the unlabeled set well enough to improve near OOD detection performance, as shown in Figure 1.
This includes previous methods that try to obtain a diverse ensemble and use it for novelty detection, such as MCD (Yu & Aizawa, 2019).
In this paper, we introduce Ensembles with Regularized Disagreement (ERD), an ensemble method for SSND that successfully takes advantage of the unlabeled data to achieve the right amount of disagreement between the model predictions.
The models in our ensemble are trained to not only ﬁt the training data but also the artiﬁcially labeled samples from the unlabeled set, while aiming to achieve good validation accuracy.
In particular,
• we argue why regularizing disagreement is crucial for OOD detection with ensemble methods.
• we give a justiﬁcation, backed by theoretical arguments, as to why training with early stopping and with artiﬁcial labels assigned to the unlabeled set achieves the right amount of disagreement.
• we test our method on many near OOD tasks, including medical data, demonstrating signiﬁcant gains with a negligible increase in computation cost compared to vanilla ensembles.
R ELATED WORK AND A TAXONOMY OF OOD DETECTION METHODS
In this section we give an overview of approaches that are related to ours and categorize them with respect to 1) data availability and 2) the ﬁnal objective.
Further, we list a few representative approaches in Table 1 and refer the reader to surveys such as Bulusu et al.
(2020) for a thorough literature overview.
2. T AXONOMY ACCORDING TO DATA AVAILABILITY
In this section we present methods that are ordered by a decreasing necessary amount of labeled
OOD data or data similar to OOD data at test time.
Supervised and augmented unsupervised ND.
We call methods that use labeled OOD samples from the test distribution for calibration or hyperparameter tuning supervised ND (SND) (Yu &
Aizawa, 2019; Lee et al., 2018; Liang et al., 2018; Ruff et al., 2020).
Alternatively, some works use during training either real, known outliers Hendrycks et al.
Malinin & Gales (2018) or
OOD-like synthetically generated data to simulate the test OOD distribution, in what we call the augmented unsupervised ND (A-UND).
However, the performance of these methods relies on proxy
OOD data that is similar to test OOD.
Concurrent works also propose to use models large-scale models that are pretrained on datasets that contain classes similar to the novel classes (Fort et al., 2021; Reiss & Hoshen, 2021), referred to as pretrained unsupervised ND .
Semi-supervised novelty detection (SSND, ours).
This setting is the one we study in this work.
We assume that, apart from class-labeled training data, we also have access to a small batch of unlabeled data drawn from the test distribution.
It consists of ID and OOD data without explicitly knowing which samples are ID and OOD.
SSND methods attempt to detect OOD samples from a new batch drawn from the same test distribution (Blanchard et al., 2010; M˜unoz-Mar´ı et al., 2010; du Plessis et al., 2014; Liu et al., 2018).
This is similar to the goal of approaches that use unlabeled data for learning with augmented classes (U-LAC)
(Da et al., 2014; Guo et al., 2020;
Zhang et al., 2020b).
The SSND setting is related to transductive novelty detection (Scott & Blanchard, 2008;
Guo et al., 2020), where the test set coincides with the unlabeled set used for training.
Unsupervised novelty detection (UND).
In this less restrictive setting, only ID data is available during training (Lakshminarayanan et al., 2017; Sastry & Oore, 2019; Nalisnick et al., 2019;
Choi et al., 2018), which generally leads to a poorer performance.
Even though Scott & Blanchard (2009) prove for shallow models that unlabeled data available in SSND should signiﬁcantly improve detection, it remains unclear how well deep neural networks can leverage the unlabeled data set effectively.
2. T AXONOMY ACCORDING TO OVERALL OBJECTIVE
We now discuss the different objectives of methods in the literature that can be used to detect OOD samples and explicitly list those that do not explicitly mention this keyword.
Learning the ID marginal P X .
Since we deﬁne OOD samples as all x for which P X ( x ) < α , if we had access to the marginal training distribution P X , we would have perfect OOD detection for any x (cid:54)∈ X ID .
Generative models (Choi et al., 2018; Akc¸ay et al., 2018; Nalisnick et al., 2019), however, struggle to learn the density from ﬁnite samples when the data is high-dimensional.
Alternatively, one-class classiﬁcation (M˜unoz-Mar´ı et al., 2010; Ruff et al., 2020; Sohn et al., 2021) and
PU learning approaches (du Plessis et al., 2014; Kiryo et al., 2017) try to directly learn a discriminator between ID and OOD data, but tend to produce indistinguishable representations for inliers and outliers when the ID distribution consists of many diverse classes.
Learning P X using label information.
(ours) When the ID training set has class labels, one can take advantage of that additional information to distinguish points in the support of P X from OOD data, for instance, by using the intermediate representations of trained neural networks (Lee et al., 2018; Sastry & Oore, 2019).
Often the task is to also simultaneously predict well on ID data, known in the literature as the problem open-set recognition
(Zhang et al., 2020a;
Geng et al., 2021).
Learning uncertainty estimates for P Y | X .
Uncertainty estimates optimized for minimal calibration error can naturally be used for OOD samples.
Many uncertainty quantiﬁcation methods are based on a Bayesian framework (Gal & Ghahramani, 2016; Malinin & Gales, 2018; Graves, 2011;
Blundell et al., 2015) or calibration improvement (Liang et al., 2018; Hafner et al., 2019) – neither of them however perform as well as other OOD methods mentioned above (Ovadia et al., 2019).
P ROPOSED METHOD
In this section we introduce our proposed algorithm, ERD, and provide a principled justiﬁcation for the key ingredients that lead to the improved performance of our method.
3. T HE COMPLETE ERD PROCEDURE
Recall that we have access to both a labeled training set S
= { ( x i , y i ) }
ni =1 ∼ P , where x i ∈
X are covariates and
y i ∈ Y are discrete labels, and an unlabeled set U , which contains both ID and unknown OOD samples.
Moreover, we initialize the models of the ensemble using weights pretrained on S .
Input: Train set S , ID Validation set V , Unlabeled set U ,
Model ˜ f pretrained on S , Ensemble size K
Result:
ERD ensemble { f y i } Ki =
Sample K different labels { y 1 , ..., y K } from Y for c ← { y 1 , ..., y K } do // fine-tune K models f
c ← Initialize ( ˜ f ) ( U, c ) ← { ( x, c ) : x ∈ U } f c ← RegularizedFineTuning ( f c , S ∪ ( U, c ); V ) return { f y i } Ki =
Input: Ensemble { f y i } Ki =1 , Test set T , O = ∅ , Threshold t 0 ,
Disagreement metric ρ Result: O , i.e. the OOD elements of T for x ∈ T do // run hypothesis test if ( Avg ◦ ρ )( f y 1 , ..., f y K )( x ) > t 0 then O ← O ∪ { x } else predict x using ˜ f
return O
The entire training procedure is described in Algorithm 1.
For training a single model in the ensemble, we assign a label c ∈ Y to all the unlabeled samples in U , resulting in the c -labeled set that we denote as ( U, c ) :
= { ( x, c ) : x ∈ U } .
We then ﬁne-tune a classiﬁer f c on the union S ∪ ( U, c ) of the correctly-labeled training set S , and the unlabeled set ( U, c ) .
The model f c that we output is regularized to have high validation accuracy on the validation set while still achieving a low training error on S ∪ ( U, c ) .
We create an ensemble of K classiﬁers f c by choosing a different artiﬁcial label c ∈ Y for every model 4 .
Finally, during test time in Algorithm 2, any samples with an aggregate disagreement metric (described in Section 3.2) larger than a threshold value t 0 are ﬂagged as OOD while the rest can be predicted using the initial model trained on the labeled training set.
Intuitively, the training procedure encourages the models to produce different predictions on the
OOD samples in U , while regularization prevents them from ﬁtting the incorrect label c and hence disagreeing on the ID points.
We argue in Section 3.2 why regularized disagreement is essential and in Section 3.3 why we can ﬁnd such models using early stopping.
[Figure]
3. E NSEMBLES WITH REGULARIZED DISAGREEMENT
We now discuss how we can use ensembles with disagreement detect OOD samples and why the right amount of diversity is crucial.
Note that we can cast the OOD detection problem as a hypothesis test with null hypothesis H 0 : x ∈ X ID .
Our Algorithm 2 tests the null hypothesis by using an ensemble-based disagreement score: The null hypothesis is rejected and we report x as OOD ( positive ) if the score is larger than a threshold t 0 (Section 4.3 elaborates on the choice of t 0 ).
Ideally, the test should have high power (ﬂag true OOD as OOD) and low false positive rate (avoid ﬂagging true ID as OOD).
For simplicity of illustration, let us assume a training set with binary labels and a semi-supervised novelty detection setting as depicted in Figure 2 a). If we obtain two different models as in Figure also present a version of ERD trained from random
[Figure]
b), the ensemble agrees on the blue and red area and disagrees on the gray area depicted in Figurec).
If we ﬂag as OOD all samples in the region where the models disagree, in this scenario we have high power and a low false positive rate.
To extend the concept of disagreement for a multi-class model and multi-model ensemble, we propose to use the average disagreement between the softmax outputs of the K models in the ensemble
[Equation]
where ρ is a measure of disagreement between the softmax outputs of two predictors, for example the total variation distance ρ TV ( f i ( x ) , f j ( x ))
= 12 (cid:107) f
i ( x ) − f j ( x ) (cid:107) 1 used in our experiments 5 .
We provide a thorough discussion on the soundness of this statistic for disagreeing models and compare it with previous metrics in Appendix B.
We would like to emphasize that the two models in Figure 2 are just diverse enough to obtain both high power and low false positive rate at the same time.
In particular, previous ensemble methods usually do not reach the “sweet spot”: either they have too little disagreement as in Figurea), resulting in low power, or they disagree too much, resulting in high false positive rate as in
Figure 3 c).
In these ﬁgures, one can easily see that at the the sweet spot of what we call regularized disagreement in Figure 3 b), the models are quite diverse while maintaining high ID validation accuracy.
Supported by previous empirical and theoretical observations, we now argue that we can ﬁnd the sweet spot via early-stopped ﬁne-tuning on the artiﬁcally labeled unlabeled set.
3. D ISAGREEMENT VIA ARTIFICIAL LABELS AND REGULARIZATION VIA EARLY STOPPING
In this section we show how we can obtain a neural network that achieves the right amount of disagreement using unlabeled data and regularization via early stopping (refer to the following theoretical works on early stopping implicitly restricting model complexity Yao et al.
; Raskutti et al.
; Wei et al.
Previous methods have also tried to leverage a set of unlabeled data to obtain more diverse ensembles for OOD detection (Yu & Aizawa, 2019; Jain et al., 2020) or for better predictive performance(Bennett et al., 2002; Zhang & Zhou, 2010).
The Maximum discrepancy method (MCD), the only candidate that can be used for our large-scale datasets, tends to result in ensembles that do not disagree enough on OOD data, resulting in subpar performance (see Figure 9a in Appendix B).
We argue that Algorithm 1 when regularized via early stopping succeeds based on two observations in the literature: empirically, neural networks can ﬁt arbitrary labels perfectly (Zhang et al., 2016) while on the other hand, noisy samples with incorrect labels are often ﬁt after the correctly labeled samples Yilmaz & Heckel (2019);
Li et al.
Song et al.
; Liu et al.
; Xia et al.
This motivates our procedure that 1) ﬁne-tunes models to ﬁt artiﬁcial labels c during training, while we 2) use the model at an earlier stopping time that achieves high validation accuracy while enough of the samples in the U have been ﬁt.
We now give a more detailed argument why such a stopping time can lead to good regularized disagreement.
Recall that in our approach, in addition to the correct labels of the ID training set S , each member of the ensemble tries to ﬁt one label c to the entire unlabeled set U that can be further partitioned into
[Figure]
[Equation]
where U ID :
= U ∩ X ID and U OOD :
=
U \ U ID .
Moreover, assuming that the labels of an ID input x is deterministically y ∗ ( x ) , we can partition the set ( U ID , c ) (see Figure 4a) into a subset of effectively “correctly labeled” samples ( U c ID , c ) and “incorrectly labeled” samples ( U ¬ c ID , c ) : ( U
[Equation]
c ID , c ) :=
[Equation]
x, c
[Equation]
: x ∈ U ID with y ∗ ( x
[Equation]
Note that ( U ¬ c ID , c ) can be viewed as the noisy samples of our entire training set.
Assuming the
OOD samples are representative, if the models ﬁt ( U OOD , c ) perfectly to different artiﬁcial labels c , then they will disagree on the OOD data in the labeled set, as well as on unseen similar OOD samples, leading to a diverse ensemble.
However, if the models also ﬁt the samples in ( U ¬ c ID , c ) , the ensemble becomes too diverse, as shown in Figure 4 a).
As noted before, early stopping prevents models from ﬁtting training samples with label noise, which, in our case, amounts to the incorrectly labeled subset ( U ¬ c ID , c ) .
In Proposition A.1 in Appendix A we provide a rigorous proof based on previous theoretical results that establishes the existence of such an optimal stopping time under mild assumptions on the data distributions.
To ﬁnd the best stopping time in practice, we use a validation set of labeled ID points to select an intermediate checkpoint before convergence.
As a model starts to ﬁt ( U ¬ c ID , c ) , i.e. the wrongly labeled ID samples in U ID , it also predicts the label c on some validation ID points, leading to a decrease in validation accuracy, as shown in Figure 5.
In our experiments, we wait for one epoch to allow for the ﬁne-tuning to have any effect at all, and then pick the iteration with the largest validation accuracy (indicated by the vertical line in the ﬁgure).
In Appendix I we show that the trend depicted in Figure 5 persists for various different data sets.
Finally, we note that instead of early stopping we could also explicitly regularize using dropout or weight decay.
However, running a grid search to select the right hyperparameters can be more computationally expensive than simply using one run of the training process to select the optimal stopping time.
E XPERIMENTAL RESULTS
In this section we evaluate the OOD detection performance of ERD for deep neural networks on several image data sets.
We ﬁnd that our approach outperforms all baselines on difﬁcult OOD detection scenarios.
In addition, we discuss some of the trade-offs that impact ERD’s performance.
4. D ATA SETS
We report results on near and far OOD detection scenarios using standard data sets and a recent
OOD detection benchmark for medical images Cao et al.
(2020) (see Appendix D for details):
Easy/Far OOD data.
ID and OOD samples come from strikingly different data sets (e.g. CIFAR10 or CIFAR100 as ID and SVHN as OOD).
These are the settings considered in the majority of the literature and on which most baselines perform well.
Hard/Near OOD data.
The OOD data consists of “novel” classes that resemble the ID samples.
For the standard image data sets we consider half of the classes as ID, and the other half as OOD.
For the medical benchmarks near OOD data consists of unseen diseases.
The similarities between the ID and the OOD classes make these settings signiﬁcantly more challenging.
For all scenarios, we used a labeled training set (e.g. 40K samples for CIFAR10), a validation set with ID samples (e.g. 10K samples for CIFAR10) and an unlabeled test set where half of the samples are ID and the other half are OOD (e.g. 5K ID samples and 5K OOD samples for CIFAR vs SVHN).
For evaluation, we use a holdout set containing ID and OOD samples in the same proportions as the unlabeled set.
In Appendix E.1 we show that ERD can also successfully identify the outliers from the unlabeled set used for ﬁne-tuning.
Furthermore, in Appendix E.4 we present results obtained with a smaller unlabeled set of only 1K samples.
4. B ASELINES
We compare our method against a wide range of baselines that require different access to OOD data for training, as indicated in Table 1.
Semi-supervised novelty detection.
We primarily compare our method to approaches that can be used in the semi-supervised setting for deep neural networks, in which a small set of unlabeled ID and OOD samples is available.
The MCD method Yu & Aizawa (2019) trains an ensemble of two classiﬁers with different types of predictive distributions on the unlabeled samples: one model gives high-entropy predictions, while the other has low entropy.
Notably, this method uses oracle OOD data for hyperparameter tuning.
Furthermore, positive-unlabeled (PU) learning du Plessis et al.
(2014) considers a binary classiﬁcation setting, in which the labeled data comes from one class (i.e.
ID samples, in our case), while the unlabeled set contains a mixture of samples from both classes.
Crucially, PU learning methods, like nnPU Kiryo et al.
(2017), require oracle knowledge of the ratio of OOD samples in the unlabeled set.
In addition to these methods, we propose two more baselines that use an unlabeled set.
Firstly, we present a version of the Mahalanobis approach ( Mahal-U ) that is calibrated using the unlabeled set.
Secondly, since PU learning requires access to the OOD ratio of the unlabeled set, we also consider a less burdensome alternative: a binary classiﬁer trained to separate the training data from the unlabeled set and regularized with early stopping like our method.
Unsupervised novelty detection.
When it comes to methods that use no OOD data for training, the current SOTA on the usual benchmarks is the Gram method Sastry & Oore (2019).
Other approaches that use no OOD data include vanilla ensembles Lakshminarayanan et al.
(2017), deep generative models that tend to give undesirable results for OOD detection Kirichenko et al.
(2020), or various Bayesian approaches that are often poorly calibrated on OOD data Ovadia et al.
Other methods.
We also compare with Outlier Exposure Hendrycks et al.
and Deep Prior
Networks (DPN) Malinin & Gales (2018) which use TinyImages as known outliers during training, irrespective of the OOD set used for evaluation (A-UND).
On the other hand, the Mahalanobis baseline Lee et al.
(2018) is tuned on samples from the same OOD distribution used for evaluation (SND).
Preliminary analyses revealed that generative models and one-class classiﬁcation methods perform poorly on near OOD data sets (see also Appendix E.2).
Choice of hyperparameters.
For all the baselines, we use the default hyperparameters suggested by their authors on the same ID data set.
The Gram method requires laborious hyperparameter tuning for multi-layer perceptron (MLP) models, so we do not consider it for the MNIST and FMNIST data
[Table]
sets.
For the binary classiﬁer and nnPU, we pick hyperparameters only to optimize the loss on an
ID validation set.
We defer the details regarding training the models to Appendix C.
4. O UR METHOD AND EVALUATION
ERD.
We present results for two ﬂavors of our method.
For one, we ﬁne-tune each model in the ensemble with early stopping, starting from weights that are pretrained on the labeled ID set S (ERD).
We also train the models from random initializations (ERD++) that can obtain slightly better
OOD detection at the cost of more training iterations.
We train ensembles of K = 5 MLP models for MNIST and FMNIST and ResNet
He et al.
(2016) networks for the other settings (results for other architectures and ensemble sizes are presented in Appendix E.7 and E.8, respectively).
We choose the arbitrary label assigned to the unlabeled set at random, without replacement (see
Appendix E.8 for a discussion on the impact of the choice of arbitrary label).
For each model in the ensemble we perform post-hoc early stopping: we train for 10 epochs (100 epochs for ERD++) and select the iteration with the lowest validation loss.
The other hyperparameters for training are chosen to maximize validation accuracy on the ID data.
Evaluation.
As in standard hypothesis testing problems, choosing different thresholds for rejecting the null hypothesis leads to different false positive and true positive rates (FPR and TPR, respectively).
The ROC curve follows the FPR and the TPR for all possible threshold values and the area under the curve (AUROC; larger values are better) captures the performance of a statistical test without having to select a speciﬁc threshold.
In addition to the AUROC, we also use the TNR at a
TPR of 95% (TNR@95; larger values are better) for evaluation.
4. M AIN RESULTS
Table 2 summarizes the main empirical results.
On the easy scenarios (top part of the table) most methods achieve near-perfect OOD detection with AUROC close to 1 .
However, on the novelty detection scenarios (bottom part), ERD has a clear edge over other baselines, even when they are calibrated on oracle OOD data, or when they use the true OOD ratio of the unlabeled set, e.g. nnPU.
The substantial gap between ERD and other approaches, both in average AUROC and average
TNR@95, indicates that our method lends itself well to practical situations when accurate OOD detection is critical.
We note that repeated runs of ERD show a small variance σ 2 < 0 .
01 in the detection metrics.
In Appendix E we show that our method successfully identiﬁes OOD samples with mild distribution shift (e.g. corrupted CIFAR10 Hendrycks & Dietterich (2019),
CIFAR10v
Recht et al.
(2018), ObjectNet Barbu et al.
(2019)), which provides further evidence that ERD is well-suited for the most difﬁcult of OOD detection tasks.
For the medical OOD detection benchmark we show in Figure 6a the average AUROC achieved by some representative baselines taken from Cao et al.
Our method improves the average
[Figure]
AUROC from
91 , compared to best
We refer for precise details on the methods.
Appendix F contains more results, as well as additional baselines.
C ONCLUSION
In this section, we discuss the possible limitations of using ERD in practice before summarizing the advantages compared to other methods and outlining potential avenues for future work.
Discussion on potential limitations of ERD and SSND.
First, as mentioned in Section 3.3, ERD requires the OOD samples in the unlabeled set to be representative of the OOD distribution.
Even though this may not be realistic for anomaly detection where outliers are rare, it may very well be feasible when a novel class (such as a disease like COVID-19) emerges at inference time.
We empirically investigate the impact on the performance of ERD of the size and the ratio of OOD samples in the unlabeled set ( | U [Equation] | [Equation]
ID | + | U OOD | ).
We ﬁnd that there is a broad spectrum of values for which ERD maintains a good performance, as indicated in Figure 6b (see also Appendix H). Moreover, in
Appendix I and J we provide insight as to why near OOD data affects the performance of ERD.
Another limitation concerns the general applicability of SSND: the OOD data in the unlabeled set used for ﬁne-tuning needs to match the OOD data at test time.
In the following we argue using an example that i) it may be inherently valuable to predict the OOD samples in the unlabeled set itself; and ii) ERD allows for a different test OOD distribution, at the cost of a slight delay.
Consider, for instance, a medical center that uses an automated system for real-time diagnosis and an ofﬂine system which runs at the end of each week, for novelty detection.
All the X-rays collected during the week constitute the unlabeled set U that the SSND method may then use for training.
If a quickly spreading novel disease circulates the patient population, the detection model can identify the OOD samples that are then shown to the scarcely available experts.
While the experts are examining the peculiar X-rays in the course of the next week, the model helps to collect more instances of the same new condition and can already encourage clinicians to practice extra caution when diagnosing these patients.
Since the novelty detection algorithm is run every week, new diseases are identiﬁed with a delay of at most a week – the time it takes to collect an unlabeled set.
Since new diseases emerge seldomly and the beneﬁts of even delayed identiﬁcation greatly outweigh the waiting time, SSND approaches are particularly suitable to this practical scenario.
Summary and future work.
We would like to stress once again that a signiﬁcant advantage of the
SSND setting is that it does not require any labeled or oracle OOD data during ﬁne-tuning, unlike many other related works summarized in Table 1.
The only other approach that achieves comparable performance to our method (see Appendix E.2) uses a large transformer model pretrained on a much larger data set (Fort et al., 2021).
At the same time, computationally, ERD reaches the optimal stopping time within the ﬁrst 10 epochs on all the data sets we consider, which amounts to around 6 minutes of training time if the models in the ensemble are ﬁne-tuned in parallel on NVIDIA 1080 Ti GPUs.
Other ensemble diversiﬁcation methods require training different models for each hyperparameter choice and have training losses that cannot be easily parallelized.
In conclusion, we propose a procedure that succeeds in exploiting unlabeled data to generate an ensemble with regularized disagreement, with remarkable novelty detection performance.
We leave as future work a thorough investigation into the inﬂuence of the labeling scheme of the unlabeled set on the sample complexity of the method, as well as an analysis of the trade-off governed by the complexity of the model class of the classiﬁers.
A T HEORETICAL STATEMENTS
Deﬁnition A.1 ( ( (cid:15), ρ ) -clusterable data set) .
We say that a data set D = { ( x i , y i ) } ni
=1 is ( (cid:15), ρ )- clusterable for ﬁxed (cid:15) > 0 and ρ ∈ [0 , 1] if there exists a partitioning of it into subsets { C 1 , ..., C K } , which we call clusters , each with their associated unit-norm cluster center c
i , that satisfy the following conditions:
• (cid:83) Ki
=1 C i = D and C i ∩ C j = ∅ , ∀ i, j ∈ [ K ] ; • all the points in a cluster lie in the (cid:15) -neighborhood of their corresponding cluster center,
i
e. || x − c
i || 2 ≤ (cid:15) for all x ∈ C i and all i ∈
[ K ] ; • a fraction of at least 1 − ρ of the points in each cluster C i have the same label, which we call the cluster label and denote y ∗ ( c i ) .
The remaining points suffer from label noise; • if two cluster C i and C j have different labels, then their centers are 2 (cid:15) far from each other,
i
e. || c i − c
j || 2 ≥ 2 (cid:15) ; • the clusters are balanced i
e. for all i ∈ [ K ] , α 1 nK ≤ | C i | ≤ α 2 nK , where α 1 and α 2 are two positive constants.
In our case, for a ﬁxed label c ∈ Y , we assume that the set S ∪ ( U, c ) is ( (cid:15), ρ ) -clusterable into K clusters.
We further assume that each cluster C i only includes a few noisy samples from ( U ¬ c ID , c ) , i.e. | C i ∩ ( U ¬ c ID ,c ) | | C i | ≤ ρ and that for clusters C i whose cluster label is not c , i.e. y ∗ ( c i ) (cid:54)
=
c , it holds that C i ∩ ( U OOD , c )
= ∅ .
We deﬁne the matrices C :=
[ c 1 , ..., c K ] T ∈ R K × d and Σ :
= ( CC T ) (cid:74) E g
[ φ (cid:48)
( Cg ) φ (cid:48)
( Cg ) T ] , with g ∼ N (0 , I d ) and where (cid:74) denotes the elementwise product.
We use (cid:107) · (cid:107) and λ min ( · ) to denote the spectral norm and the smallest eigenvalue of a matrix, respectively.
For prediction, we consider a 2-layer neural network model with p hidden units, where p (cid:38)
K 2 (cid:107) C (cid:107) 4 λ min (Σ) 4 .
We can write this model as follows:
[Equation]
The ﬁrst layer weights W are initialized with random values drawn from N (0 , 1) , while the last layer weights v have ﬁxed values: half of them are set to 1 /p and the other half is − 1 /p .
We consider activation functions φ with bounded ﬁrst and second order derivatives, i.e. | φ (cid:48)
( x ) | ≤ Γ and φ (cid:48)(cid:48)
( x ) ≤ Γ .
We use the squared loss for training, i.e. L ( W ) = 12 (cid:80) ni =
( y i − f ( x i ; W )) 2 and take gradient descent steps to ﬁnd the optimum of the loss function, i.e. W τ +
=
W τ − η ∇L ( W τ ) , where the step size is set to η (cid:39)
[Equation]
We can now state the following proposition:
Proposition A.1.
Assume that ρ ≤ δ/ 8 and (cid:15) ≤ αδλ min (Σ) 2 /K 2 , where δ is a constant such that δ ≤ 2 |Y− 1 | and α is a constant that depends on Γ .
Then it holds with high probability 1 − 3 /K 100 −
Ke − 100 d over the initialization of the weights that the neural network trained on S ∪ ( U, c ) perfectly ﬁts S , ( U c ID , c ) and ( U OOD , c ) , but not ( U ¬ c ID , c ) , after T = c 4 (cid:107) C (cid:107) 2 λ min (Σ) iterations.
This result shows that there exists an optimal stopping time at which the neural network predicts the correct label on all ID points and the label c on all the OOD points.
As we will see later in the proof, the proposition is derived from a more general result which shows that the early stopped model predicts these labels not only on the points in U but also in an (cid:15) -neighborhood around cluster centers.
Hence, an ERD ensemble can be used to detect holdout OOD samples similar to the ones in
U , after being tuned on U .
This follows the intuition that classiﬁers regularized with early stopping are smooth and generalize well.
The clusterable data model is generic enough to include data sets with non-linear decision boundaries.
Moreover, notice that the condition in Proposition A.1 is satisﬁed when S ∪ ( U ID , c ) is ( (cid:15), ρ ) - clusterable and ( U OOD , c ) is (cid:15) -clusterable
and if the cluster centers of ( U OOD , c ) are at distance at least 2 (cid:15) from the cluster centers of S ∪ ( U ID , c ) .
A situation in which these requirements are met is, for instance, when the OOD data comes from novel classes, when all classes (including the unseen ones that are not in the training set) are well separated, with cluster centers at least 2 (cid:15) away in Euclidean distance.
In addition, in order to limit the amount of label noise in each cluster, it is necessary that the number of incorrectly labeled samples in ( U ¬ c ID , c ) is small, relative to the size of
S .
In practice, we only need that the decision boundary separating ( U OOD , c ) from S is easier to learn than the classiﬁer required to interpolate the incorrectly labeled ( U ¬ c ID , c ) , which is often the case, provided that ( U OOD , c ) is large enough and the OOD samples come from novel classes.
We now provide the proof for Proposition A.1:
Proof.
We begin by restating a result from Li et al.
Theorem A.1 (Li et al.
Let D := { ( x i , y i )
} ∈ R d × Y be an ( (cid:15), ρ ) -clusterable training set, with (cid:15) ≤ c 1 δλ min (Σ) 2 /K 2 and ρ ≤ δ/ 8 , where δ is a constant that satisﬁes δ ≤ 2 |Y|− 1 .
Consider a two-layer neural network as described above, and train it with gradient descent starting from initial weights sampled i.i.d.
from N (0 , 1) .
Assume further that the step size is η = c 2 K n (cid:107) C (cid:107) 2 and that the number of hidden units p is at least c 3 K 2 (cid:107) C (cid:107) 4 λ min (Σ) 4 .
Under these conditions, it holds with probability at least 1 − 3 /K 100 − Ke − 100 d over the random draws of the initial weights, that after T = c 4 (cid:107) C (cid:107) 2 λ min (Σ) gradient descent steps, the neural network x (cid:55)→ f ( x ; W T ) predicts the correct cluster label for all points in the (cid:15) -neighborhood of the cluster center,
namely: arg max y ∈Y | f ( x ; W T ) − ω
( y ) |
=
y ∗ ( c i ) , for all x with (cid:107) x − c
i (cid:107) 2 ≤ (cid:15) and all clusters i ∈
[ K ] , (3) where ω : Y → { 0 , 1 } |Y| yields one-hot embeddings of the labels.
The constants c 1 , c 2 , c 3 , cdepend only on Γ .
Notice that, under the assumptions introduced above, the set S ∪ ( U, c ) is ( (cid:15), ρ ) -clusterable, since the incorrectly labeled ID points in ( U ¬ c ID , c ) constitute at most a fraction ρ of the clusters they belong to.
As a consequence, Proposition A.1 follows directly from Theorem A.1.
B D ISAGREEMENT SCORE FOR OOD DETECTION
As we argue in Section 3.3, Algorithm 1 produces an ensemble that disagrees on OOD data, and hence, we want to devise a scalar score that reﬂects this model diversity.
Previous works Lakshminarayanan et al.
; Ovadia et al.
ﬁrst average the softmax predictions of the models in the ensemble and then use the entropy as a metric, i.e. ( H ◦ Avg )( f 1 ( x ) , ..., f K ( x )) :
= − (cid:80) |Y| i =
( f ( x ))
i log( f ( x ))
i where f ( x ) :
= 1 K (cid:80) K i =
1 f i ( x ) and ( f ( x ))
i is the i th element of f ( x ) ∈ [0 , 1] |Y| 8 .
We argue later that averaging discards information about the diversity of the models.
Recall that our average pairwise disagreement between the outputs of K models in an ensemble reads:(
[Equation]
) , ..., f
[Equation]
( x )) :=
K (
[Equation]
− 1) (cid:88) i (cid:54) =
j ρ ( f
[Equation]
where ρ is a measure of disagreement between the softmax outputs of two predictors, for example the total variation distance ρ TV ( f i ( x ) , f j ( x ))
= 12 (cid:107) f
i ( x ) − f j ( x ) (cid:107) 1 used in our experiments.
We brieﬂy highlight the reason why averaging softmax outputs ﬁrst like in previous works relinquishes all the beneﬁts of having a more diverse ensemble, as opposed to the proposed pairwise score in Equation 4.
Recall that varying thresholds yield different true negative and true positive rates (TNR and TPR, respectively) for a given statistic.
In the sketch in Figure 7 we show that the score we propose, ( Avg ◦ ρ ) , achieves a higher TNR compared to ( H ◦ Avg ) , for a ﬁxed TPR, which is a common way of evaluating statistical tests.
Notice that the detection region for ( H ◦ Avg ) is always limited to a band around the average model for any threshold value t 0 .
In order for the ( H ◦ Avg ) to have large TPR, this band needs to be wide, leading to many false positives.
Instead, our disagreement score exploits the diversity of the models to more accurately detect OOD data
We now provide further quantitative evidence to support the intuition presented in Figure 7.
The aggregation metric is tailored to exploit ensemble diversity, which makes it particularly beneﬁcial for
ERD.
On the other hand, Vanilla Ensembles only rely on the stochasticity of the training process and the random initializations of the weights to produce diverse models, which often leads to classiﬁers that are strikingly similar as we show in Figure 8 for a few 2D data sets.
As a consequence, using our disagreement score ( Avg ◦ ρ ) for Vanilla Ensembles can sometimes hurt OOD detection performance.
To see this, consider the extreme situation in which the models in the ensemble are identical, i.e. f 1 = f 2 .
Then it follows that ( Avg ◦ ρ )( f 1 ( x ) , f 2 ( x ))
= 0 , for all test points x and for any function ρ that satisﬁes the distance axioms.
We note that the disagreement score that we propose takes a form that is similar to previous diversity scores, e.g. Zhang & Zhou (2010); Yu & Aizawa (2019).
In the context of regression, one can measure uncertainty using the variance of the outputs metric previously employed in works such as Gal & Ghahramani (2016).
However, we point out that using the output variance requires that the ensemble is the result of sampling from a random process (e.g. sampling different training data for the models, or sampling different parameters from a posterior).
In our framework, we obtain the ensemble by solving a different optimization problem for each of the models by assigning a different label to the unlabeled data.
Therefore, despite their similarities, our disagreement score and the output variance are, on a conceptual level, fundamentally different metrics.
Table 3 shows that ( Avg ◦ ρ ) leads to worse OOD detection performance for Vanilla Ensembles, compared to using the entropy of the average softmax score, ( H ◦ Avg ) , which was proposed in prior work.
However, if the ensembles are indeed diverse, as we argue is the case for our method
ERD (see Section 3.3), then there is a clear advantage to using a score that, unlike ( H ◦ Avg ) , takes diversity into account, as shown in Table
[Table] [Table]
.91 / 0.66 CIFAR100[0:49]
CIFAR100[50:99] 0.78 / 0.
Average 0.87 / 0.
We highlight once again that other methods that attempt to obtain diverse ensembles, such as MCD, fail to train models with sufﬁcient disagreement, even when they use oracle OOD for hyperparameter tuning (Figure 9a).
[Figure]
C E XPERIMENT DETAILS C. B ASELINES
In this section we describe in detail the baselines with which we compare our method and describe how we choose their hyperparameters.
For all baselines we use the hyperparameters suggested by the authors for the respective data sets (e.g. different hyperparameters for CIFAR10 or ImageNet).
For all methods, we use pretrained models provided by the authors.
However, we note that for the novel-class settings, pretraining on the entire training set means that the model is exposed to the
OOD classes as well, which is undesirable.
Therefore, for these settings we pretrain only on the split of the training set that contains the ID classes.
Since the classiﬁcation problem is similar to the original one of training on the entire training set, we use the same hyperparameters that the authors report in the original papers.
Moreover, we point out that even though different methods use different model architectures, that is not inherently unreasonable when the goal is OOD detection, since it is not clear if a complex model is more desirable than a smaller model.
For this reason, we use the model architecture recommended by the authors of the baselines and which was used to produce the good results reported in their
• Vanilla Ensembles Lakshminarayanan et al.
(2017): We train an ensemble on the training set according to the true labels.
For a test sample, we average the outputs of the softmax probabilities predicted by the models, and use the entropy of the resulting distribution as the score for the hypothesis test described in Section 3.2.
We use ensembles of 5 models, with the same architecture and hyperparameters as the ones used for ERD.
Hyperparameters are tuned to achieve good validation accuracy.
• Gram method Sastry & Oore (2019):
The Gram baseline is similar to the Mahalanobis method in that both use the intermediate feature representations obtained with a deep neural network to determine whether a test point is an outlier.
However, what sets the Gram method apart is the fact that it does not need any OOD data for training or calibration.
We use the pretrained models provided by the authors, or train our own, using the same methodology as described for the Mahalanobis baseline.
For OOD detection, we use the code published by the authors.
We note that for MLP models, the Gram method is difﬁcult to tune and we could not ﬁnd a conﬁguration that works well, despite our best efforts and following the suggestions proposed during our communication with the authors.
• Deep Prior Networks (DPN) Malinin & Gales (2018): DPN is a Bayesian Method that trains a neural network (Prior Network) to parametrize a Dirichlet distribution over the class probabilities.
We train a WideResNet WRN-28-10 for 100 epochs using SGD with momentum 0 .
9 , with an initial learning rate of 0 . 01 , which is decayed by 0 .
at epochs 50 , 70 , and 90 .
For MNIST, we use EMINST/Letters as OOD for tuning.
For all other settings, we use TinyImages as OOD for tuning.
• Outlier Exposure Hendrycks et al.
(2019): This approach makes a model’s softmax predictions close to the uniform distribution on the known outliers, while maintaining a good classiﬁcation performance on the training distribution.
We use the WideResNet architecture (WRN) Zagoruyko & Komodakis (2016).
For ﬁne-tuning, we use the settings recommended by the authors, namely we train for 10 epochs with learning rate 0 .
For training from scratch, we train for 100 epochs with an initial learning rate of 0 . 1 .
When the training data set is either CIFAR10/CIFAR100 or ImageNet, we use the default WRN parameters of the author’s code, namely 40 layers, 2 widen-factor, droprate 0 .
When the training dataset is SVHN, we use the author’s recommended parameters of 16 layers,widen-factor and droprate 0 . 4 .
All settings use the cosine annealing learning rate scheduler provided with the author’s code, without any modiﬁcations.
For all settings, we use TinyImages as known OOD data during training.
In Section E.5 we show results for known OOD data that is similar to the OOD data used for testing.
• Mahalanobis Lee et al.
The method pretrains models on the labeled training data.
For a test data point, it uses the intermediate representations of each layer as “extracted features”.
It then performs binary classiﬁcation using logistic regression using these extracted features.
In the original setting, the classiﬁcation is done on “training” ID vs “training” OOD samples (which are from the same distribution as the test OOD samples).
Furthermore, hyperparameter tuning for the optimal amount of noise is performed on validation ID and OOD data.
We use the WRN-28-10 architecture, pretrained for 200 epochs.
The initial learning rate is 0 . 1 , which is decayed at epochs 60 , 120 , and 160 by 0 .
We use SGD with momentum 0 .
9 , and the standard weight decay of 5 · 10 − 4 .
The code published for the Mahalanobis method performs a hyperparameter search automatically for each of the data sets.
The following baselines assume the same Unknown OOD setting as ERD, in which one has access to both a labeled ID training set S and an unlabeled set with an unknown mixture of ID and OOD samples U .
• Non-negative PU learning (nnPU) Kiryo et al.
The method trains a binary predictor to distinguish between a set of known positives (in our case the ID data) and a set that contains a mixture of positives and negatives (in our case the unlabeled set).
To prevent the interpolation of all the unlabeled samples, Kiryo et al.
(2017) proposes a regularized objective.
It is important to note that most training objectives in the PU learning literature require that the ratio between the positives and negatives in the unlabeled set is known or easy to estimate.
For our experiments we always use the exact OOD ratio to train the nnPU baseline.
Therefore, we obtain an upper bound on the AUROC/TNR@95.
If the ratio is estimated from ﬁnite samples, then estimation errors may lead to slightly worse OOD detection performance.
We perform a grid search over the learning rate and the threshold that appears in the nnPU regularizer and pick the option with the best validation accuracy measured on a holdout set with only positive samples (in our case, ID data).
• Maximum Classiﬁer Discrepancy (MCD) Yu & Aizawa (2019):
The MCD method trains two classiﬁers at the same time, and makes them disagree on the unlabeled data, while maintaining good classiﬁcation performance.
We use the WRN-28-10 architecture as suggested in the paper.
We did not change the default parameters which came with the author’s code, so weight decay is 10 − 4 , and the optimizer is SGD with momentum 0 .
When available (for CIFAR10 and CIFAR100), we use the pretrained models provided by the authors.
For the other training datasets, we use their methodology to generate pretrained models: We train a WRN-28-10 for 200 epochs.
The learning rate starts at 0.1 and drops by a factor of 10 at 50% and 75% of the training progress.
• Mahalanobis-U
This is a slightly different version of the Mahalanobis baseline, for which we use early-stopped logistic regression to distinguish between the training set and an unlabeled set with ID and OOD samples (instead of discriminating a known OOD set from the inliers).
The early stopping iteration is chosen to minimize the classiﬁcation errors on a validation set that contains only ID data (recall that we do not assume to know which are the OOD samples).
In addition to these approaches that have been introduced in prior work we also propose a strong novel baseline that that bares some similarity to PU learning and to ERD.
• Binary classiﬁer
The approach consists in discriminating between the labeled ID training set and the mixed unlabeled set, that contains both ID and OOD data.
We use regularization to prevent the trivial solution for which the entire unlabeled set is predicted as OOD.
Unlike PU learning, the binary classiﬁer does not require that the OOD ratio in the test distribution is known.
The approach is similar to a method described in Scott & Blanchard (2008) which also requires that the OOD ratio of the unlabeled set is known.
We tune the learning rate and the weight of the unlabeled samples in the training loss by performing a grid search and selecting the conﬁguration with the best validation accuracy, computed on a holdout set containing only ID samples.
We note that the binary classiﬁer that appears in Section F in the medical benchmark, is not the same as this baseline.
For more details on the binary classiﬁer that appears in the medical data experiments we refer the reader to Cao et al.
C. T RAINING CONFIGURATION FOR ERD
For ERD we always use hyperparameters that give the best validation accuracy when training a model on the ID training set.
In other words, we pick hyperparameter values that lead to good ID generalization and do not perform further hyperparameter tuning for the different OOD data sets on which we evaluate our approach.
We point out that, if the ID labeled set is known to suffer from class imbalance, subpopulation imbalance or label noise, any training method that addresses these issues can be used instead of standard empirical risk minimization to train our ensemble (e.g. see the works of Sagawa et al.
; Li et al.
(2020) etc).
For MNIST and FashionMNIST, we train ensembles of 3-layer MLP models with ReLU activations.
Each intermediate layer has 100 neurons.
The models are optimized using Adam, with a learning rate of 0 .
001 , for 10 epochs.
For SVHN, CIFAR10/CIFAR100 and ImageNet, we train ensembles of ResNet20 He et al.
The models are initialized with weights pretrained for 100 epochs on the labeled training set.
We ﬁne-tune each model for 10 epochs using SGD with momentum 0 .
9 , and a learning rate of 0 .
The weights are trained with an (cid:96) 2 regularization coefﬁcient of 5 e − 4 .
We use a batch size of 128 for all scenarios, unless explicitly stated otherwise.
We used the same hyperparameters for all settings.
For pretraining, we perform SGD for 100 epochs and use the same architecture and hyperparameters as described above, with the exception of the learning rate that starts at 0 . 1 , and is multiplied by 0 .at
epochs 50 , 70 and 90 .
Apart from ERD, which ﬁne-tunes the ensemble models starting from pretrained weights, we also present in the Appendix results for ERD++.
This variant of our method trains the models from random initializations, and hence needs more iterations to converge, making it more computationally expensive than ERD.
We train all models in the ERD++ ensembles for 100 epochs with a learning rate that starts at 0 .
1 , and is multiplied by 0 .
2 at epochs 50 , 70 and 90 .
All other hyperparameters are the same as for ERD ensembles.
For the medical data sets, we train a Densenet-121 as the authors do in the original paper Cao et al.
For ERD++, we do not use random weight initializations, but instead we start with the
ImageNet weights provided with Tensorﬂow.
The training conﬁguration is exactly the same as for
ResNet20, except that we use a batch size of 32 due to GPU memory restrictions, and for ﬁne tuning we use a constant learning rate of 10 − 5 .
D ID AND OOD DATA SETS D. D ATA SETS
For evaluation, we use the following image data sets: MNIST Lecun et al.
Fashion MNIST
Xiao et al.
(2017), SVHN Netzer et al.
(2011), CIFAR10 and CIFAR100 Krizhevsky (2009).
For the experiments using MNIST and FashionMNIST the training set size is 50K, the validation size is 10K, and the test ID and test OOD sizes are both 10K. For SVHN, CIFAR10 and CIFAR100, the training set size is 40K, the validation size is 10K, and the unlabeled set contains 10K samples: 5K are ID and 5K are OOD.
For evaluation, we use a holdout set of 10K examples (half ID, half
OOD).
For the settings that use half of the classes as ID and the other half as OOD, all the sizes are divided by 2.
D. S AMPLES FOR THE SETTINGS WITH NOVEL CLASSES
[Figure]
E M ORE EXPERIMENTS E. E VALUATION ON THE UNLABELED SET
In the main text we describe how one can leverage the unlabeled set U to obtain an OOD detection algorithm that accurately identiﬁes outliers at test time that similar to the ones in U .
It is, however, possible to also use our method ERD to ﬂag the OOD samples contained in the same set U used for ﬁne-tuning the ensemble.
In Table 4 we show that the OOD detection performance of ERD is similar regardless of whether we use U for evaluation
[Table]
or a holdout test set T drawn from the same distribution as U .
[Table]
E. C OMPARISON WITH OTHER RELATED WORKS
We compare our method to more OOD detection approaches.
For various reasons we did not run these methods ourselves on the data sets for which we evaluate our method in Section 4 (e.g. resource constraints, code not available, unable to replicate published results, poor performance reported by the authors etc).
We collected the AUROC numbers presented in Table 5 from the papers that introduce each method.
We note that our approach shows an excellent overall performance, almost matching the AUROC of the best performing method of Fort et al.
(2021) which uses large scale visual transformer models pretrained on a superset of the OOD data, i.e. ImageNet21k.
Furthermore, we note that generative models (Nalisnick et al., 2019; Choi et al., 2018; Akc¸ay et al., 2018) and one-class classiﬁcation approaches (Ruff et al., 2020; Tack et al., 2020; Sohn et al., 2021) showed generally bad performance, in particular on near OOD data.
When the ID training set is made up of several diverse classes, it is difﬁcult to represent accurately all the ID data, and only the
ID data.
[Table]
E. OOD DETECTION FOR DATA WITH COVARIATE SHIFT
In this section we evaluate the baselines and the method that we propose on settings in which the
OOD data suffers from covariate shift Shimodaira (2000).
The goal is to identify all samples that come from the shifted distribution, regardless of how strong the shift is.
Notice that mild shifts may be easier to tackle by domain adaptation algorithms, but when the goal is OOD detection they pose a much more difﬁcult challenge.
We want to stress that in practice one may not be interested in identifying all samples with distribution shift as OOD, since a classiﬁer may still produce correct predictions on some of them.
In contrast, when data suffers from covariate shift we can try to learn predictors that perform well on both the training and the test distribution, and we may use a measure of predictive uncertainty to identify only those test samples on which the classiﬁer cannot make conﬁdent predictions.
Nevertheless, we use these covariate shift settings as a challenging OOD detection benchmark and show in Table 7 that our method ERD does indeed outperform prior baselines on these difﬁcult settings.
We use as outliers corrupted variants of CIFAR10 and CIFAR100 Hendrycks & Dietterich (2019), as well as a scenario where ImageNet Deng et al.
(2009) is used as ID data and ObjectNet Barbu et al.
(2019) as OOD, both resized to 32x32.
Figure 11 shows samples from these data sets.
The Gram and nnPU baselines do not give satisfactory results on the difﬁcult CIFAR10/CIFAR100 settings in
Table 2 and thus we do not consider them for the distribution shift cases.
For the Unknown OOD methods (i.e. MCD, Mahal-U and ERD/ERD++) we evaluate on the same unlabeled set that is used for training (see the discussion in Section E.1).
Furthermore, we present results on distinguishing between CIFAR10 Krizhevsky (2009) and CIFAR10v2 Recht et al.
(2018), a data set meant to be drawn from the same distribution as CIFAR (generated from the Tiny Images collection Torralba et al.
In Recht et al.
(2019), the authors argue that CIFAR10 and CIFAR10v2 come from very similar distributions.
They provide supporting evidence by training a binary classiﬁer to distinguish between them, and observing that the accuracy that is obtained of 52.9% is very close to random.
Our experiments show that the two data sets are actually distinguishable, contrary to what previous work has argued.
First, our own binary classiﬁer trained on CIFAR10 vs CIFAR10v2 obtains a test accuracy of 67%, without any hyperparameter tuning.
The model we use is a ResNet20 trained for 200 epochs using SGD with momentum 0.9.
The learning rate is decayed by 0.2 at epochs 90, 140, 160 and 180.
We use 1600 examples from each data set for training, and we validate usingexamples from each data set.
[Table]
Our OOD detection experiments (presented in Table 6) show that most baselines are able to distinguish between the two data sets, with ERD achieving the highest performance.
The methods which require OOD data for tuning (Outlier Exposure and DPN) use CIFAR
[Table] [Table]
E. R ESULTS WITH A SMALLER UNLABELED SET
We now show that our method performs well even when the unlabeled set is signiﬁcantly smaller.
In particular, we show in the table below that ERD maintains a high AUROC and TNR@95 even when only 1,000 unlabeled samples are used for ﬁne-tuning (500 ID and 500 OOD)
[Table]
ERD
[Table]
E. M ORE RESULTS FOR O UTLIER E XPOSURE
[Table]
The Outlier Exposure method needs access to a set of OOD samples during training.
The numbers we report in the rest of paper for Outlier Exposure are obtained by using the TinyImages data set as the OOD samples that are seen during training.
In this section we explore the use of an OOD train data set that is more similar to the OOD data observed at test time.
This is a much easier setting for the Outlier Exposure method: the closer OOD train is to OOD test , the easier it will be for the model tuned on OOD train to detect the test OOD samples.
In Table 9 we focus only on the settings with corruptions.
For each corruption type, we use the lower severity corruption as OOD train and evaluate on the higher severity data and vice versa.
We report for each metric the average taken over all corruptions (A), and the value for the worst-case setting (W)
[Table]
E. R ESULTS ON MNIST AND F ASHION MNIST
[Table]
For FashionMNIST we chose this particular split (i.e. classes 0,2,3,7,8 vs classes 1,4,5,6,9) because the two partitions are more similar to each other.
This makes OOD detection more difﬁcult than the 0-4 vs 5-9 split.
E. V ANILLA AND ERD E NSEMBLES WITH DIFFERENT ARCHITECTURES
In this section we present OOD detection results for Vanilla and ERD ensembles with different architecture choices, and note that the better performance of our method is maintained across model classes.
Moreover, we observe that ERD beneﬁts from employing more complex models, like the
WideResNet.
[Table]
In this section we show OOD detection results with our method using a smaller number of models for the ensembles. We notice that the performance is not affected substantially, indicating that the computation cost of our approach could be further reduced by ﬁne-tuning smaller ensembles
[Table]
[Table]
Impact of the choice of arbitrary labels. Furthermore, we note that in the table we report averages over 3 runs of our method, where for each run we use a different subset of Y to assign arbitrary labels to the unlabeled data. We do this in order to assess the stability of ERD ensembles to the choice of the arbitrary labels and notice that the OOD detection performance metrics do not vary signiﬁcantly. Concretely, the standard deviations are consistently below 0 .
01 for all data sets for the
AUROC metric, and below 0 .
07 for the TNR@95 metric.
F M EDICAL OOD DETECTION BENCHMARK
The medical OOD detection benchmark is organized as follows.
There are four training (ID) data sets, from three different domains: two data sets with chest X-rays, one with fundus imaging and one with histology images.
For each ID data set, the authors consider three different OOD scenarios:
Use case 1: The OOD data set contains images from a completely different domain, similar to our category of easy OOD detection settings.
Use case 2: The OOD data set contains images with various corruptions, similar to the hard covariate shift settings that we consider in Section E.3.
Use case 3: The OOD data set contains images that come from novel classes, not seen during training.
The authors evaluate a number of methods on all these scenarios.
The methods can be roughly categorized as follows:
Data-only methods: Fully non-parametric approaches like kNN.
Classiﬁer-only methods: Methods that use a classiﬁer trained on the training set, e.g. ODIN Liang et al.
(2018), Mahalanobis Lee et al.
ERD falls into this category as well.
Methods with Auxiliary Models: Methods that use an autoencoder or a generative model, like a Variational Autoencoder or a Generative Adversarial Network.
Some of these approaches can be expensive to train and difﬁcult to optimize and tune.
We stress the fact that for most of these methods the authors use (known) OOD data during training.
Oftentimes the OOD samples observed during training come from a data set that is very similar to the OOD data used for evaluation.
For exact details regarding the data sets and the methods used for the benchmark, we refer the reader to Cao et al.
In addition, in Figure 14 we present the average taken over only the novel-class settings in the medical benchmark.
We observe that the performance of all methods is drastically affected, all of them performing much worse than the average presented in Figure 13.
This stark decrease in AUROC and TNR@95 indicates that novelty detection is indeed a challenging task for OOD detection methods even in realistic settings.
Nevertheless, our method maintains a better performance than the baselines.
In Figures 15, 16, 17 we present AUROC and AUPR (Area under the Precision Recall curve) for
ERD for each of the training data sets, and each of the use cases.
Figure 13 presents averages over all settings that we considered, for all the baseline methods in the benchmark.
Notably, ERD performs well consistently across data sets.
The baselines are ordered by their average performance on all the settings (see Figure 13).
For all medical benchmarks, the unlabeled set is balanced, with an equal number of ID and OOD samples (subsampling the bigger data set, if necessary).
We use the unlabeled set for evaluation.
G E FFECT OF LEARNING RATE AND BATCH SIZE
We show now that our method ERD is not too sensitive to the choice of hyperparameters.
We illustrate this by varying the learning rate and the batch size, the hyperparameters that we identify as most impactful.
As Figure 18 shows, many different conﬁgurations lead to similar OOD detection performance.
H A DDITIONAL FIGURE SHOWING THE DEPENDENCE ON THE UNLABELED SET CONFIGURATION
The conﬁguration of the unlabeled set (i.e. the size of the unlabeled set, the ratio of OOD samples in the unlabeled set) inﬂuences the performance of our method, as illustrated in Figure 6b.
Below, we show that the same trend persists for different data sets too, e.g. when we consider CIFAR10 as
ID data and SVHN as OOD data.
I L EARNING CURVES FOR OTHER DATA SETS
In addition to Figure 5, we present in this section learning curves for other data sets as well.
The trend that persists throughout all ﬁgures is that the arbitrary label is learned ﬁrst on the unlabeled
OOD data.
Choosing a stopping time before the validation accuracy starts to deteriorate prevents the model from ﬁtting the arbitrary label on unlabeled ID data.
Impact of near OOD data on training ERD ensembles.
The learning curves illustrated in Figure 20 provide insight into what happens when the OOD data is similar to the ID training samples and the impact that has on training the proposed method. In particular, notice that for CIFAR10[0-4] vs CIFAR10[5-9] in Figure 20d, the models require more training epochs before reaching an accuracy on unlabeled OOD samples of 100%. The learning of the arbitrary label on the OOD samples is delayed by the fact that the ID and OOD data are similar, and hence, the bias of the correctly labeled training set has a strong effect on the predictions of the models on the OOD inputs. Since we early stop when the validation accuracy starts deteriorating (e.g. at around epoch 8 in Figure 20d), we end up using models that do not interpolate the arbitrary label on the OOD samples.
Therefore, the ensemble does not disagree on the entirety of the OOD data in the unlabeled set, which leads to lower OOD detection performance.
Importantly, however, our empirical evaluation reveals that the drop in performance for ERD ensembles is substantially smaller than what we observe for other
OOD detection methods, even on near OOD data sets.
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
J E VOLUTION OF DISAGREEMENT SCORE DURING FINE - TUNING
In this section we illustrate how the distribution of the disagreement score changes during ﬁne-tuning for ID and OOD data.
Thus, we can further understand why the performance of the ERD ensembles is impacted by near OOD data.
Figure 21 reveals that for far OOD data (the left column) the disagreement scores computed on OOD samples are well separated from the disagreement scores on ID data (note that disagreement on OOD data is so concentrated around the maximum value of 2 that the boxes are essentially reduced to a line segment).
On the other hand, for near OOD data (the right column) there is sometimes signiﬁcant overlap between the disagreement scores on ID and OOD data, which leads to the slightly lower
AUROC values that we report in Table 2.
The ﬁgures also illustrate how the disagreement on the ID data tends to increase as we ﬁne-tune the ensemble for longer, as a consequence of the models ﬁtting the arbitrary labels on the unlabeled ID samples.
Conversely, in most instances one epoch sufﬁces for ﬁtting the arbitrary label on the OOD data.
We need to make one important remark: While in the ﬁgure we present disagreement scores for the ensemble obtained after each epoch of ﬁne-tuning, we stress that the ﬁnal ERD ensemble need not be selected among these.
In particular, since each model for ERD is early stopped separately,
[Figure]
potentially at a different iteration, it is likely that the ERD ensemble contains models ﬁne-tuned for a different number of iterations.
Since we select the ERD ensembles from a strictly larger set, the ﬁnal ensemble selected by the our proposed approach will be at least as good at distinguishing ID and OOD data as the best ensemble depicted in Figure 21.