Novelty detection using ensembles with regularized disagreement
Authors: Alexandru Tifrea
Eric Petru Stavarache
Fanny Yang
Venue: ICLR 2022 Submitted (2022)
Keywords: out-of-distribution detection
novelty detection
ensembles
ensemble diversity
outlier detection
regularization

Abstract: Despite their excellent performance on in-distribution (ID) data, deep neural networks often confidently predict on out-of-distribution (OOD) samples that come from novel classes instead of flagging them for expert evaluation. Even though conventional OOD detection algorithms can distinguish far OOD samples, current methods that can identify near OOD samples require training with labeled data that is very similar to these near OOD samples. In turn, we develop a new ensemble-based procedure for \emph{semi-supervised novelty detection} (SSND) that only utilizes a mixture of unlabeled ID and OOD samples to achieve good detection performance on near OOD data. It crucially relies on regularization to promote diversity on the OOD data while preserving agreement on ID data. Extensive comparisons of our approach to state-of-the-art SSND methods on standard image data sets (SVHN/CIFAR-10/CIFAR-100) and medical image data sets reveal significant gains with negligible increase in computational cost.


Under review as a conference paper at ICLR 2022
NOVELTY DETECTION USING ENSEMBLES WITH REGULARIZED DISAGREEMENT
Anonymous authors Paper under double-bling review
ABSTRACT
Despite their excellent performance on in-distribution (ID) data, deep neural net- works often confidently predict on out-of-distributie (OOD) samples that come from novel classes instead of flagging them for expert evaluation. Even though conventional OOD detection algorithms can distinguish far OOD samples, cur- rent methods that can identify near OOD samples require training with labeled data that is very similar to these near OOD samples. In turn, develop new ensemble- only utilizes mixture of unlabeled ID and OOD samples to achieve good detec- tion performance on near OOD data. It crucially relies on regularization to pro- mote diversity the OOD data while preserving agreement II) data. Extensive comparisons of our approach to tiite-of-th-arr SSND methods on standard im- age data sets C significant gains with negligible increase in computational cost
procedure for supervise novelty detection (SSND) that
and medical image data sets reveal
INTRODUCTION
Despite achieving great in-distribution (ID) performance, deep neural networks (DNN) often have trouble dealing with samples in the test set that are out-of-distribution (OOD), i.c. test inputs that are unlike the data seen during training. For example, DNNs often make incorrect predictions with high confidence when new unseen classes emerge over time (e-g. undiscovered bacteria Ren al. (2019).
new discases Katsamenis et al. 22200). ) or when data suffers from distribution shift (e.g. corruptions Lu et al. (2019). environmental changes Kumar et al. (2020)).
If the labels of these OOD samples follow the same conditional probability distribution as the training set Shimodaira (2000), it is most desir- able to output calibrated uncertainty estimates in addition
Novel class 000
Gram
Vanilla
Ensemble
MCD
single predicted value, in Neal (1996); Gal & Ghahramani] (2016); Ma-
MetialUU
TNR@TPR=92
0.4
ERD (Ours)
linin & Gales (2018); Lakshminarayannnn et (2117). In contrast, when this assumption does hold and some inputs might come from pre- viously unseen classes, we would like to detect such novel samples and bring them to the at- tention of experts. More specifically, in novelty detection, we want to detect inputs x that sat- isfy (x) < for small threshold a, where Px is the marginal training (ID) distribution.
0.2
Figure 1: in the SSND setting. Our method shows better perfor- mance on challenging near OOD data sets. Averages over the data sets presented in Table
Comparison of methods that are applicable
Previous works that have tackled this problem are known as open-set recognition and OOD detection methods. In what follows, we use the terms novelty detection (ND) and OOD detection interchangeably. Related problems such aforemen- tioned uncertainty estimation or one-class (OC) classification have slightly different objectives but also include methods that are applicable to novelty detection.
Our code is publicly available at http::/bit.ll//3aaa000

---

Under review as a conference paper at ICLR 2022
Table
Taxonomy of novelty detection methods, categorized according to data availability (horizontal axis) and overall objective (vertical axis). We highlight the insemble-base methods.
Different OOD A-UND
Syathetic OOD A-UND
P.UND
SND
SNND
UND
Lear Px
[RH21]
G3KRB13,
OC clasa.. ISLYIP2, TMDS20]
PU [KNPS17]
Generutive c.g. [AABIS), OC classif [SPSSWL]]
DKT19,
RVGBM20)
Px
VIT
Mahullanahis
Contrnatvve [TMIS20
SSND
for shallow
Gram [SD19], Contrastive (WRRSN20). OpenHyhrid (LLGG22)
using y
[PRL20]
[1111118]
1A20]
models [MBCICCIO BLSID]. (DW714, 27M7200
MCD [YA19)
U-LAC
Uncertainty Prix
ODIN (LLS]8]
DEN [MG18].
Calibrute images [LLES18].
using
GAN moise
Bayesian
methods e.g.
Outlier
Expo-
Vanilla Ensemble [LPB17]
(GG16)
sure [HMD199
(HILID19) or samples (JJL))20)
Numerous novelty detection methods are successful for simple benchmarks where the OOD samples are far from the training samples (such as SVHN vs CIFAR 10). As Wiinkens et al. (2020) and a number of concurrent works (Tack et al. 2020 Fort et al., (02T)) recently noted, these methods however have subpar performance on near OOD data, for instance when OOD samples are drawn from unseen classes from the same data set (e.g. CIFARIO0 vs CIFARI0). A recent method that is reported to have high near OOD detection performance involves tuning large models pretrained on ImageNet21k (Fort al. 202T and, hence, arguably relies using during training OOD data that contains classes that are close to the unseen CIFAR classes during test time. However, in many scientific applications that use, for instance, medical satellite images, such large data sets for pretraining are not available. In particular, truly novel classes will be inherently dissimilar from any previously available data while still sharing some of the same characteristics.
Instead, a more realistic scenario assumes access to small batch of unlabeled test data that includes ID and OOD samples without knowing which are the outliers. We can use this set during train- ing to ultimately perform well on future samples (i.e. semi-supervised novelty detection (SSND) (Blanchard et al. (0100) or to detect outliers from that particular test batch itself (reminiscent of transductive OOD detection (Scott & Blanchard, (008)). Even though using unlabeled data has the potential to improve detection performance as argued in Scott & Blanchard (2009), existing SSND methods for deep neural networks (Kiryo et al.) 2017 Yu & Aizawal [219;; Guo et al. 2020 Zhang
ct al. 2020b) do not leverage the unlabeled set well enough to improve near OOD detection perfor- mance, as shown in Figure I This includes previous methods that try to obtain diverse ensemble and use it for novelty detection, such as MCD Yu & Aizawa 2019).
In this paper, we introduce Ensembles with Regularized Disagreement (ERD), ensemble method for SSND that successfully takes advantage of the unlabeled data to achieve the right amount of disagreement between the model predictions. The models in our ensemble are trained not only fit the training data but also the artificially labeled samples from the unlabeled set, while aiming to achieve good validation accuracy. In particular.
we argue why regularizing disagreement is crucial for OOD detection with ensemble methods.
we give justification, backed by theoretical arguments, to why training with early stopping and with artificial labels assigned to the unlabeled set achieves the right amount of disagreement.
we test our method on many near OOD tasks, including medical data, demonstrating significant gains with a negligible increase in computation cost compared to vanilla ensembles.
RELATED WORK AND A TAXONOMY OF OOD DETECTION METHODS
In this section give an overview of approaches that are related to and categorize them with respect to 1) data availability and 2) the final objective. Further, we list a few representative approaches in Table and refer the reader surveys such as Bulusu al. (202 for a thorough literature overview.
2.1 TAXONOMY ACCORDING TO DATA AVAILABILITY
In this section we present methods that are ordered by a decreasing necessary amount of labeled OOD data or data similar to OOD data at test time.
2

---

Under review as a conference paper at ICLR 2022
Supervised and augmented unsupervised ND. We call methods that use labeled OOD samples from the test distribution for calibration or hyperparameter tuning supervised ND (SND) Yu &
Aizawa 2019: Lee et al., 2018 Liang et al. 2018; Ruff et al. 2020h. Alternatively, some works use during training either real, known outliers Hendrycks et al. 2019); Malinin & Gales (2018) or
OOD-like synthetically generated data to simulate the test OOD distribution, in what we call the augmented unsupervised ND (A-UND). However, the performance of these methods relies on proxy OOD data that is similar to test OOD, Concurrent works also propose to use models large-scale models that are pretrained on datasets that contain classes similar to the novel classes (Fort et al., 2021 Reiss& & Hoshen 2021), referred as pretrained unsupervised ND.
Semi-supervised novelty detection (SSND, ours). This setting is the one we study in this work 2 We assume that, apart from class-labeled training data, we also have access to small batch of unlabeled data drawn from the test distribution. It consists of ID and OOD data without explicitly knowing which samples are ID and OOD. SSND methods attempt detect OOD samples from new batch drawn from the same test distribution (Blanchard et al. 2010: Mūnoz-Mari et al., 2010 du Plessis et al., 2014 Liu al. 2018). This is similar to the goal of approaches that use unlabeled data for learning with augmented classes (U-LAC) (Da et al.) 2014 Guo et al., 22020) Zhang et al. 2020b). The SSND setting is related to transductive novelty detection (Scott & Blanchard 2008 Guo et al. 2020). where the test set coincides with the unlabeled set used for training.
Unsupervised novelty detection (UND). In this less restrictive setting, only ID data is avail- able during training (Lakshminarayanan et al. 2017 (Sastry & Oore) 2019; Nalisnick et al.) 22019] Choi et al. 2018). which generally leads to a poorer performance. Even though /Scott& & Blanchard (209) prove for shallow models that unlabeled data available in SSND should significantly im- prove detection, it remains unclear how well deep neural networks can leverage the unlabeled data set effectively.
2.2 TAXONOMY ACCORDING TO OVERALL OBJECTIVE
We now discuss the different objectives of methods in the literature that can be used to detect OOD samples and explicitly list those that do not explicitly mention this keyword.
Learning the ID marginal Px- Since define OOD samples as all for which Px (x)) a, if
we had access to the marginal training distribution Px. we would have perfect OOD detection for any x XID. Generative models (Choi et al., 2018 Akçay al.. 2018 Nalisnick et 2019).
however, struggle to learn the density from finite samples when the data is dimeesiinn.. Alter-
natively, one-class classification (Mūnoz- Marí et al., 2010 Ruff et al., 2020 Sohn et al. (021] and PU learning approaches (du Plessis al. | [0014] 4 Kiryo et al. (211)) try to directly learn discrimina- tor between ID and OOD data, but tend to produce indistinguishable representations for inliers and outliers when the ID distribution consists of many diverse classes.
Learning Px using label information. (ours) When the ID training set has class labels, one can take advantage of that additional information to distinguish points in the support of Px from OOD data, for instance, by using the intermediate representations of trained neural networks (Lee al., 2018 Sastry& & Oore 2019). Often the task is to also simultancously predict well on ID data, known in the literature as the problem open-set recognition (Zhang et al.) 2020a Geng et al. 2021).
Learning uncertainty estimates for Prix- Uncertainty estimates optimized for minimal calibra- tion error can naturally be used for OOD samples. Many uncertainty quantification methods based on a Bayesian framework (Gal & Ghahramani (016)] Malinin & Gales) 2018)] Graves] 2011 Blundell al. 2015)) or calibration improvement (Liang al. [2018] Hafner et 2019 neither of them however perform as well as other OOD methods mentioned above (Ovadia et al. 2019).
PROPOSED METHOD
In this section we introduce our proposed algorithm, ERD, and provide principled justification for the key ingredients that lead to the improved performance of our method.
We use the same definition of SSND as the survey by Bulusu et al. (2020), whereas some works use the
term to refer to SND (Gomitz et al. 2013 Daniel et al. 2019 Ruff et al. 2020 UND (Song al. 2017
Akçay et al. 2018) according our taxonomy.
3

---

Under review as a conference paper at ICLR 2022
3.1 THE COMPLETE ERD PROCEDURE
Recall that we have access to both labeled training set S = { Bi B:) ~ P. where I; E X are covariates and y/ E y are discrete labels, and an unlabeled set U, which contains both ID and unknown OOD samples. Moreover, we initialize the models of the ensemble using weights pretrained on S
Algorithm 1: Fine-tuning the ERD ensemble
Algorithm 2: OOD detection with ERD
Input: Train set S, ID Validation set V, Unlabeled set U, Model pretrained on S. Ensemble size K Result: ERD ensemble f.f. Y.K
Input: Ensemble {f R-m1 Test set T, Threshold to. Disagrecment metric p
Result: O, i... the OOD elements T for E T do // run hypothesia test if (Avg p (x) > to then (2{
Sample K different labels [UI,..., yk from y
for (y1...., yk do // fine-tune K models
fo c Initializer() (f)
(U, t [(x,, c) EU
else
fe Regularized (fa, (U.,))
predict using f
return 1m
return O
The entire training procedure is described in Algorithm I For training single model in the ensem- ble, we assign a label с E y to all the unlabeled samples in U, resulting in the c-labeled set that we denote as cU, == { c) x E U}. We then fine-tune a classifier fe on the union S U (U, c) of the carrectly-labeled training set S. and the unlabeled set (U,c) The model fe that we output is regularized to have high validation accuracy on the validation set while still achieving low training error on S U (U, c). We create an ensemble of K classifiers fc by choosing different artificial label e E y/ for every mode Finally, during test time in Algorithm 2 any samples with an aggregate disagreement metric (described in Section 3.2 larger than a threshold value are flagged as OOD while the rest can be predicted using the initial model trained on the labeled training set.
Intuitively, the training procedure encourages the models to produce different predictions the OOD samples in U, while regularization prevents them from fitting the incorrect label c and hence disagreeing the ID points. We argue in Section (.2 why regularized disagreement is essential and in Section 3.3 why we can find such models using early stopping.
O set of ID cata
+ set of ODD data
mocel
ensemble disagrees
modeltersmblee prediction 8008 predictoons in U
o labeled set S
unlabeled set J
model II
identified OOD in U
2
2
8
(b) Two models in the ensemble Figure 2: Left: Sketch of the SSND setting. Middle and Right: OOD detection with diverse ensemble.
(a) SSND setting
(c) Ensemble predictions
3.2 ENSEMBLES WITH REGULARIZED DISAGREEMENT
We now discuss how we can use ensembles with disagreement detect OOD samples and why the right amount of diversity is crucial. Note that we can cast the OOD detection problem hypothesis test with null hypothesis Ho : I E XID. Our Algorithm 2 tests the null hypothesis by using an ensemble-based disagreement score: The null hypothesis is rejected and we report x as OOD (positive) if the score is larger than a threshold in (Section 4.3 elaborates on the choice of 1))- Ideally, the test should have high power (flag true OOD OOD) and low false positive rate (avoid flagging true ID as OOD).
For simplicity of illustration, let assume a training set with binary labels and a semi-supervised novelty detection setting as depicted in Figure al.). If we obtain two different models as in Figure
IIn Section 4 we also present a version of ERD trained from random initializations, i.c. ERD++ *Choosing the ensemble size K 5 sufficient for all of experiments even for y ~ 50 or larger.
4

---

Under review as a conference paper at ICLR 2022
ID data e aboled set S
of OOD data ensemble models
ensembibe disggrees ident fied OOD
modelleroenioo prediciion
prodict ons U
#
25
18
83
85
Too little disagreement b) Just enough disagreement
c) Too much disagreement
Figure 3: Varying degrees of ensemble disagreement and how that influences what regions are flagged OOD,
b), the ensemble agrees on the blue and red area and disagrees on the gray area depicted in Figure 2 c). If we flag as OOD all samples in the region where the models disagree, in this scenario we have high power and low false positive rate. To extend the concept of disagreement for multi-class model and multi-model ensemble, we propose to use the average disagreement between the softmax outputs of the models in the ensemble
(Avg
E E((A)))),()))) = K(K 1) i#3
(1)
where p is measure of disagreement between the softmax outputs of two predictors, for example the total variation distance (rv (z), fz() = 2 (z) fz) used in our experiments We provide thorough discussion the soundness of this statistic for disagreeing models and compare it with previous metrics in Appendix B
We would like emphasize that the models in Figure 2 are just diverse enough to obtain both high power and low false positive rate at the same time. In particular, previous ensemble methods usually do not reach the "sweet spot": either they have too little disagreement in Figure a). resulting in low power, or they disagree too much, resulting in high false positive rate as in Figure c). In these figures, one can easily see that at the the sweet spot of what we call regularized disagreement in Figure 3 b). the models are quite diverse while maintaining high ID validation accuracy. Supported by previous empirical and theoretical observations, we now argue that we can find the sweet spot via early-stopped fine-tuning on the artifically labeled unlabeled set.
3.3 DISAGREEMENT VIA ARTIFICIAL LABELS AND REGULARIZATION VIA EARLY STOPPING
In this section we show how we can obtain a neural network that achieves the right amount of disagreement using unlabeled data and regularization via early stopping (refer the following the- oretical works on early stopping implicitly restricting model complexity Yao et al. (007)) Raskutti al. (013); Wei et al. (2017). Previous methods have also tried to leverage set unlabeled data to obtain more diverse ensembles for OOD detection Yuu & Aizawa 2019 Jain et al., 2020
or for better predictive performance (Bennett et al., 2002) Zhang
Zhou, 2010 The Maximum
discrepancy method (MCD), the only candidate that can be used for our parge-scale datasets, tends to result in ensembles that do not disagree enough on OOD data, resulting in subpar performance (see Figure 9a Appendix B
We argue that Algorithm | when regularized via early stopping succeeds based on two observations in the literature: empirically, neural networks can fit arbitrary labels perfectly (Zhang et al. 2016) while on the other hand, noisy samples with incorrect labels are often fit after the correctly labeled samples Yilmaz& & Heckel (2019); Li et al. (2020); [Song al. (2020); Liu et al. (2020); Xia et al. (021I) This motivates our procedure that I) fine-tunes models to fit artificial labels c during training, while 2) use the model at earlier stopping time that achieves high validation accuracy while enough of the samples in the U have been fit. We now give a more detailed argument why such a stopping time can lead good regularized disagreement.
Recall that in our approach, in addition to the correct labels of the ID training set 5, each member of the ensemble tries to fit one label the entire unlabeled set U that can be further partitioned into
'We also expect other distance metrics to be similarly effective.
5

---

Under review as a conference paper at ICLR 2022
set of ID data
set of OOD data
model trained with * labeled U model trained with * -labeledd U
mode predictions
labeled set 5
artiiiially labeled U
CoOD.
( c
(U cj
a) Two models trained to fit artifically labeled data
b) Effect of regularization on trained models
Figure 4: Regularization prevents individual models trained on S U (U) from fitting (UTTS, c).
(U,c) (UID, c) U (UoOD. c) ((2,() c) E UND U ((I,c): c) x E UOOD), where UID == U XID and UOOD == UID- Moreover, assuming that the labels of an ID input x is deterministically y" (x), we can partition the set (UID, c) (see Figure 4p) into a subset of effectively "correctly labeled" samples (Ui..c) c) and "incorrectly labeled" samples (UTD.c). c):
(UD) c) = {(x, c) x E UID with (x) + cj (Uib.c) = [(x,c) : e UID with y²(x) c}.
Note that (UID' e) can be viewed as the noisy samples of our entire training set. Assuming the OOD samples are representative, if the models fit (UooD, e) perfectly to different artificial labels then they will disagree on the OOD data in the labeled set, as well as unseen similar OOD samples, leading to a diverse ensemble. However, the models also the samples in (UTo e). the ensemble becomes too diverse, shown in Figure a). As noted before, early stopping prevents models from fitting training samples with label noise, which, our case, amounts to the incorrectly labeled subset (UTTs c). In Proposition A.1 in Appendix A we provide rigorous proof based on previous theoretical results that establishes the existence of such an optimal stopping time under mild assumptions the data distributions.
To find the best stopping time in practice, we use a validation set of labeled ID points to select an intermediate checkpoint before convergence. As model starts to fit (Uips c). i.e. the wrongly labeled ID samples in UID it also predicts the label c on some validation ID points, leading to decrease in validation accuracy, as shown in Figure to allow for the fine-tuning to have any effect at all, and then pick the iteration with the largest validation accuracy (indicated by the vertical line in the figure). Appendix we show that the trend depicted in Figure persists for various different data sets.
In our experiments, we wait for one epoch
Finally, we note that instead of carly stopping we could also explicitly regularize using dropout weight decay. However, rumning grid search to select the right hyperparameters be more computationally expensive than simply using one run of the training process to select the optimal stopping time.
S (correct labels)
(Ubr c) (arbitrary label)
(Ucoo, c) (arbitrary label)
Accuracy
Validation (ID; correct labels)
0.5
2
6
10
Epochs
Figure 5: Accuracy during fine-tuning model pretrained on S (epoch 0 indicates values obtained with the initial pretrained weights). The samples in (Uoon, c) are fit first, while the model reaches high accuracy on (UmD, e) much later. We fine-tune for at least one epoch then early stop when validation accuracy starts decreasing after 7 epochs (vertical line). The model is trained on SVHN[0:4] as ID and SVHN[S:9] as OOD.

---

Under review as a conference paper at ICLR 2022
4 EXPERIMENTAL RESULTS
In this section we evaluate the OOD detection performance of ERD for deep neural networks on several image data sets. We find that our approach outperforms all baselines on difficult OOD detection scenarios. In addition, we discuss some of the trade-offs that impact ERD's performance.
4.1
DATA SETS
We report results on near and far OOD detection scenarios using standard data sets and a recent OOD detection benchmark for medical images Cao al. 2020) (see Appendix[D]oo details):
Easy/Far OOD data. ID and OOD samples come from strikingly different data sets (e.g. CI- FAR10 or CIFARIO0 as ID and SVHN as OOD). These are the settings considered the majority of the literature and on which most baselines perform well.
Hard/Near OOD data. The OOD data consists of "novel" classes that resemble the ID samples. For the standard image data sets consider half of the classes ID, and the other half OOD. For the medical benchmarks near OOD data consists of unseen diseases. The similarities between the ID and the OOD classes make these settings significantly more challenging.
For all scenarios, we used a labeled training set (e.g. 40K samples for CIFAR10); a validation set with ID samples (e.g. 10K samples for CIFAR10) and an unlabeled test set where half of the sam- ples are ID and the other half are OOD (e.g. 5K ID samples and 5K OOD samples for CIFARIO vs SVHN). For evaluation, we use a holdout set containing ID and OOD samples the same pro- portions as the unlabeled set. In Appendix E. we show that ERD can also successfully identify the outliers from the unlabeled set used for fine-tuning. Furthermore, in Appendix E.4 results obtained with a smaller unlabeled set of only IK samples.
we present
4.2 BASELINES
We compare our method against wide range of baselines that require different access to OOD data for training, as indicated in Table
Semi-supervised novelty detection. We primarily compare our method to approaches that can be used in the semi-supervised setting for deep neural networks, in which a small set of unlabeled II and OOD samples is available. The MCD method Yu & Aizawa (2019) trains an ensemble of two classifiers with different types of predictive distributions on the unlabeled samples: one model gives high-entropy predictions, while the other has low entropy. Notably, this method uses oracle OOD data for hyperparameter tuning. Furthermore, positive-unlabele 2014) considers binary classification setting, in which the labeled data comes from one class (i.e. ID samples, in our case), while the unlabeled set contains a mixture of samples from both classes. Crucially, PU learning methods, like nnPU Kiryo et al. (2017), require oracle knowledge of the ratio of OOD samples in the unlabeled set.
(PU) learning du Plessis et al.
In addition to these methods, we present a version of the Mahalanobis approach (Mahal-U) that is calibrated using the unlabeled set. Secondly, since PU learning requires access the OOD ratio of the unlabeled set, also consider a less burdensome alternative: a binary classifier trained to separate the training data from the unlabeled set and regularized with early stopping like our method.
propose two more baselines that use unlabeled set. Firstly,
Unsupervised novelty detection. When it comes to methods that use no OOD) data for training, the current SOTA on the usual benchmarks is the Gram method [Sastry & Oore (2019). Other ap- proaches that use no OOD data include vanilla ensembles Lakshminarayanal et al. (2017). deep generative models that tend to give undesirable results for OOD detection Kirichenko et al. 2020)) or various Bayesian approaches that are often poorly calibrated on OOD data Ovadia et al. (019)).
Other methods. We also compare with Outlier Exposure Hendrycks et al. 2019) and Deep Prior Networks (DPN) Malinin & Gales (2018) which use TinyImages as known outliers during training, irrespective of the OOD set used for evaluation (A-UND). On the other hand, the Mahalanobis baseline Lee et al. (2018) is tuned on samples from the same OOD distribution used for evaluation (SND). Preliminary analyses revealed that generative models and one-class classification methods perform poorly on near OOD data sets (see also Appendix F.2)
Choice of hy prrepaaraeserrs. For all the baselines, by their authors on the same ID data set. The Gram method requires laborious hyperparameter tuning for llyri perceptron (MLP) models, so we do not consider it for the MNIST and FMNIST data
use the default hyperparameters suggested
7

---

Under review as a conference paper at ICLR 2022
Table 2: AUROC and TNR@95 for different OOD detection scenarios (the numbers in squared brackets indicate the ID or OOD classes). We highlight the best ERD variant and best baseline. The asterisk marks baselines proposed in this paper. nnPU and MCD (*) use oracle information about the OOD data.
Other settings
Unknown OOD
ID datta
OOD data
Easembles
Vanilla
Gram
DPN
OR
Mahal,
maPur
MCD
Mahal-U:
Classit.
Bli
ERD
AUROC T/ /NTR 1
SVIIN
CIFARIO
0.88
0.97
1.00
1.00
0.99/0.98
0.97 1.00
10.85
0.99 0.95
1.00 1.00
1.00/ 0.99
CIARRD SVHN
0.92/0.78
10.98
0.95/0.85
0.99 /0.89
0.99/0..5
1.00
1.00/0.998
0.99 0..96
1,00/ 1.00
1.00/ 1.00
CIFAR100 SVIN
0.44/0,88
0.99/0.97
0.77/0.44
0.82 /0,50
0,98/0.90
1,00/1000
0.97 10.73
0.98/ 0.92
1,00/1000
1,00 1.00
FMNIST
FMNIST 0.6/0.007
[0.233.7.8)
[1,45669]
0,77/0.15
0.66/0.12
0.77/0.20
0.95/
0.78/0.30
0.82/030
0.5/0066
0.94/
SVHN
[0:4]
[5:9]
SVHN
0.92/0.69
0.11/0311
0,87/0.19
0.85 0.52
0.92/0.71
0.9610.73
0.91 /0.51
0.9/0.63
0.81 00.00
0,95/
CIFARIO
[0:4]
CIFARIO
0.80/0399
0.67/0.15
0.82/0.32
0.82 /0.41
0.79/
0.61/
0.09/0.25
0.64/
0.85/043
0.53/0.70
CO
[0:49]
CIFAR100
[50:99]
10.85
0.71
0.70/0.26
0.74/0.11
0.72/0..0
0.53/0066
0.70//.26
0.72/0.19
0.66/0.13
0.82 0.44
Average
0.450.52
0.86
/0.57
0.84
0.54/0.54
6.88
0.86/
0.36/0.55
0.86/0.66 0.60
0.89 0066
0.95 0.77
sets For the binary classifier and nnPU, we pick hyperparameters only to optimize the loss on an ID validation set. We defer the details regarding training the models to Appendix C
4.3
OUR METHOD AND EVALUATION
ERD. We present results for two flavors of our method. For one, we fine-tune each model in the ensemble with early stopping, starting from weights that are pretrained on the labeled ID set S (ERD). We also train the models from random initializations (ERD++) that can obtain slightly better OOD detection at the cost of more training iterations. We train ensembles of K 5 MLP models for MNIST and FMNIST and ResNet20 He et al.] (2016) networks for the other settings (results for other architectures and ensemble sizes are presented in Appendix E.7 and E.8 respectively). We choose the arbitrary label assigned to the unlabeled set at random, without replacement (see Appendix E.8 for a discussion on the impact of the choice of arbitrary label). For each model in the ensemble we perform post-hoc early stopping: we train for 10 epochs (100 epochs for ERD++) and select the iteration with the lowest validation loss. The other hyperparameters for training are chosen maximize validation accuracy on the ID data.
Evaluation. As in standard hypothesis testing problems, choosing different thresholds for reject- ing the null hypothesis leads to different false positive and true positive rates (FPR and TPR, re- spectively). The ROC curve follows the FPR and the TPR for all possible threshold values and the area under the curve (AUROC; larger values are better) captures the performance of statistical test without having to select specific threshold. In addition to the AUROC, we also use the TNR at TPR of 95% (TNR @95; larger values are better) for evaluation
4.4
MAIN RESULTS
Table 2 summarizes the main empirical results. On the easy scenarios (top part of the table) most methods achieve near-perfee near-perfect OOD detection with AUROC close to 1. However, on the novelty detection scenarios (bottom part), ERD has a clear edge over other baselines, even when they are calibrated on oracle OOD data, or when they use the true OOD ratio of the unlabeled set, e.g. unPU. The substantial gap between ERD and other approaches, both in average AUROC and average TNR@95 indicates that our method lends itself well to practical situations when accurate OOD detection is critical. We note that repeated runs of ERD show a small variance o² 0.01 in the detection metrics. In Appendix E we show that our method successfully identifies OOD samples with mild distribution shift (e.g. corrupted CIFARIO |IFARRO/ & Dietterich (2019), CIFAR1OV2 Recht et al. (2018), ObjectNet Barbbu et al. (201)), which provides further evidence that ERD is well-suited for the most difficult of OOD detection tasks.
For the medical OOD detection benchmark we show in Figure 6a the average AUROC achieved by some representative basclines taken from Cao et al. 2020). Our method improves the average
We note that the code provided by the authors does not include the configurations required for MLP models. IIn practice. choosing good rejection threshold is important. A recent work Liu et al. (0188) proposes a criterion for setting the threshold that is tailored specifically to the SSND setting. Altematively, one can choose the threshold to achieve a desired FPR, which we can estimate using validation set of ID samples.
8

---

Under review as a conference paper at ICLR 2022
10C Secorat 3CE PR0
Unlabeted sett sizee
2000
0.8
200
67
100
0.35
0.75
Proportion of OOD In the unlabeled set
OOD detection performance on medical data
(b) Effect of OOD proportion on detection
Figure 6: Left: AUROC averaged over all scenarios the medical OOD detection benehmark. The values for the baselines are computed using the code from Cao et al. (2020). Right: The AUROC of ERD as the number
and proportion of ID (C]FARI0(0:4) and OOD (CIFARI0[5:9] samples in the unlabeled set are varied.
AUROC from 0.85 to 0.91, compared to the best baseline. We refer the reader to Cao et al. (2020) for precise details on the methods. Appendix Appendix[Fcconainn more results, as well as additional baselines.
CONCLUSION
In this section, we discuss the possible limitations of using ERD in practice before summarizing the advantages compared to other methods and outlining potential avenues for future work.
Discussion potential limitations of ERD and SSND. First, mentioned in Section 3.3 ERD requires the OOD samples in the unlabeled set to be representative of the OOD distribution. Even though this may not be realistic for anomaly detection where outliers rare, it may very well be feasible when a novel class (such as a disease like COVID-19) emerges at inference time. We em- pirically investigate the impact on the performance ERD of the size and the ratio of OOD samples in the unlabeled set TnD UibI-UUooD! We find that there is a broad spectrum of values for which ERD maintains a good performance, as indicated in Figure 6b (see also Appendix H). Moreover, in Appendix [[ and ! we provide insight as to why near OOD data affects the performance of ERD.
Woop
Another limitation concerns the general applicability of SSND: the OOD data in the unlabeled set used for fine-tuning needs match the OOD data test time. In the following we argue using an example that i) it may be inherently valuable to predict the OOD samples in the unlabeled set itself; and ii) ERD allows for different test OOD distribution, at the cost of a slight delay.
Consider, for instance, medical center that uses an automated system for real-time diagnosis and an offline system which runs at the end of each week, for novelty detection. All the X-rays collected during the week constitute the unlabeled set U that the SSND method may then use for training. If a quickly spreading novel disease circulates the patient population, the detection model can identify exam-
the OOD samples that are then shown to the scarcely available experts. While the experts ining the peculiar X-rays in the course of the next week, the model helps to collect more instances of the same new condition and agnosing these patients. Since the novelty detection algorithm is run every week, new diseases are identified with delay of at most week the time takes collect an unlabeled set. Since new dis- eases emerge seldomly and the benefits of even delayed identification greatly outweigh the waiting time, SSND approaches are particularly suitable this practical scenario.
already encourage clinicians to practice extra caution when di-
Summary and future work. We would like stress once again that significant advantage of the SSND setting is that it does not require any labeled or oracle OOD data during fine-tuning, unlike many other related works summarized in Table The only other approach that achieves comparable performance to our method (see Appendix (.2) uses large transformer model pretrained on a much larger data (Fort et al. 2021). At the same time, computationally. ERD reaches the optimal stopping time within the first 10 epochs on all the data sets we consider, which amounts to around minutes of training time if the models in the ensemble are fine-tuned in parallel on NVIDIA 1080 Ti GPUs Other ensemble diversification methods require training different models for each hyperparameter choice and have training losses that cannot be easily parallelized.
In conclusion, we propose procedure that succeeds in exploiting unlabeled data to generate an ensemble with regularized disagreement, with remarkable novelty detection performance. We leave as future work thorough investigation into the influence of the labeling scheme of the unlabeled set on the sample complexity of the method, as well as an analysis of the trade-off governed by the complexity of the model class of the classifiers.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Samet Akçay, Amir Atapour- Abarghouu, and T. Breckon. GANomaly: Semi-supervised anomaly detection via adversarial training. In ACCV, 2018.
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. ObjectNet: large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems 32, pp. 9453 9463. 2019.
Kristin P. Bennett, Ayhan Demiriz, and R. Maclin. Exploiting unlabeled data in ensemble methods. Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 2002.
Gilles Blanchard, Gyemin Lee, and Clayton Scott. Semi-supervised novelty detection. Journal of Machine Learning Research, 11:2973-3009, 2010.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. In Proceedings of the 32th International Conference on Machine Learning, 2015.
Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K. Varshney, and Dawn Song. Anomalous instance detection in deep learning: A survey. arXiv preprint (eXXxx:2003306779 2020.
Tianshi Cao, Chin-Wei Huang, David Yu-Tung Hui, and Joseph Paul Cohen. benchmark of medical out of distribution detection. arXiv preprint arXiv:2007.04250 2020.
Hyunsun Choi, Eric Jang, and Alexander Alemi. WAIC, but why? Generative ensembles for robust anomaly detection. arXiv preprint 18100v11 2018.
Qing Da, Yang Yu, and Zhi-Hua Zhou. Learning with augmented class by exploiting unlabeled data. In AAAI, 2014.
Tal Daniel, Thanard Kurutach, and Aviv Tamar. Deep variational semi-supervised novelty detection. arXiv preprint arXiv: 0,971, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
Marthinus du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. In Advances in Neural Information Processing Systems 27, 2014.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distributtio detection. arXiv preprint arXiv:2106.03000 2021.
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. volume 48 of Proceedings of Machine Learning Research, pp- 1050-1059, 2016.
Chuanxing Geng, Sheng-Jun Huang, and Songcan Chen. Recent advances in open set recognition: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43:3614-3631, 2021.
Nico Gornitz, Marius Kloft, Konrad Rieck, and Ulf Brefeld. Toward supervised anomaly detection. Artif. Int. Res., 46:235-262, 2013.
Alex Graves. Practical variational inference for neural networks. In Advances Neural Information Processing Systems 24, pp. 3348-2356. 2011.
Lan-Zhe Guo, Zhen-Yo Yu Zhang, Yuan Jiang, Yu-Feng Li, and Zhi-Hua Zhou. Safe deep semi- supervised learning for unseen-class unlabeled data. In Proceedings of the 37th International Conference on Machine Learning, Proceedings Machine Learning Research, pp. 3897-3906, 2020.
10

---

Under review as a conference paper at ICLR 2022
Danijar Hafner, Dustin Tran, Timothy P. Lillicrap, Alex Irpan, and James Davidson. Noise trastive priors for functional uncertainty. In UAI, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In Proceedings of the International Conference on Learning Repre- sentations, 2019.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In Proceedings of the International Conference on Learning Representations, 2019.
Siddhartha Jain, Ge Liu, Jonas Mueller, and David Gifford. Maximizing overall diversity for im- proved uncertainty estimates in deep ensembles. Proceedings of the AAAI Conference on Artificial Intelligence, 34:4264 4271, 2020.
lason Katsamenis, Eftychios Protopapadakis, Athanasios Voulodimos, Anastasios Doulamis, and Nikolaos Doulamis. Transfer learning for COVID-19 pneumonia detection and classification in chest X-ray images. medRxiv, 2020.
Polina Kirichenko, Pavel Izmailov, and Andrew G Wilson. Why normalizing flows fail to detect out- of-distribution data. In Advances in Neural Information Processing Systems 33, pp. 20578-20589, 2020.
Ryuichi Kiryo, Gang Niu, Marthinus du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In Advances in Neural Information Processing Systems 30, 2017.
Alex Krizhevsky, Learning multiple layers of features from tiny images. Technical report, 2009.
Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. arXiv preprint arXiv: 1361, 2020.
Balaji Lakshminarayanan. Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems 30, pp. 6402-6411 2017.
Yann Lecun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, pp. 2278-2324, 1998.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Process- ing Systems 3/, pp. 7167-7177. 2018.
Mingchen Li, Mahdi Soltanolkotabi, and Samet Oymak. Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. Proceedings of Machine Learning Research, pp. 4313-4324, 2020.
Shiyu Liang Yixuan Li, and R. Srikant. Enhancing the reliability of nut-of-disttibution image detection in neural networks. In Proceedings of the International Conference on Learning Repre- sentations, 2018.
Hao Liu and Pieter Abbeel. Hybrid discriminnative-geeraiii training via contrastive learning. arXiv preprint arXiv:2007.09077, 2020.
Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda. Early-learning regularization prevents memorization of noisy labels. 2020.
Si Liu, Risheek Garrepalli, Thomas Dietterich, Alan Fern and Dan Hendrycks. Open category detection with PAC guarantees. Proceedings of the 35th International Conference on Machine Learning, pp. 3169-3178, 2018
11

---

Under review as a conference paper at ICLR 2022
Alex X. Lu, Amy X. Lu, Wiebke Schormann, David W. Andrews, and Alan M. Moses. The cells out of sample (COOS) dataset and benchmarks for measuring sut-oe-sample generalization of image classifiers. arXiv preprint arXiv:1906.07288. 2019.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks, In Advances in Neural Information Processing Systems 32, pp. 7047-7058, 2018.
Jordi Münoz-Marí, Francesca Bovolo, Luis Gómez-Chova, Lorenzo Bruzzone, and Gustavo Camp- Valls. Semi-supervised one-class support vector machines for classification of remote sensing data. IEEE Transactions Geoscience and Remote Sensing, 08:3188-3197 2010.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't t know? In Proceedings of the International Conference on Learning Representations, 2019.
Radford M. Neal. Bayesian Learning for Neural Networks. Springer-Verlant 1996.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning, In NIPS Workshop Deep Learning and Unsupervised Feature Learning 201 2011.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan and Jasper Snock. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems 32. pp. 13991-14008 2019.
Garvesh Raskutti, Martin Wainwright, and Bin Yu. Early stopping and non-parametric regression: An optimal data-dependent stopping rule. arXiv preprint
2013.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do CIFAR-10 classi- fiers generalize to CIFAR-10? arXiv preprint arXiv:1806.000 00451, 2018.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet? arXiv preprint 19X2. 2019.
Tal Reiss and Yedid Hoshen. Mean-shifted contrastive loss for anomaly detection. arXiv preprint
arXiv:2/06.00 033444 2021.
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. In Advances in Neural Information Processing Systems 32, pp. 14707-14718, 2019.
Lukas Ruff, Robert A. Vandermeulen, Nico Görnitz, Alexander Binder, Emmanuel Müller, Klaus- Robert Müller, and Marius Kloft. Deep semi-supervised anomaly detection. In Proceedings of the International Conference on Learning Representations, 2020.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In Proceedings of the International Conference Learning Representations, 2020.
Chandramouli Shama Sastry and Sageev Oore. Detecting out-of-distribution examples with in- distribution examples and gram matrices. arXiv preprint 11X1v:1912.12510, 2019.
Clayton Scott and Gilles Blanchard. Transductive anomaly detection. Technical report, 2008.
Clayton Scott and Gilles Blanchard. Novelty detection: Unlabeled data definitely help. In Proceed- ings of the Twelth International Conference on Artificial Intelligence and Statistics, volume pp. 464-471, 2009.
Vikash Sehwag, Mung Chiang, and Prateek Mittal. (SSD): A unified framework for self- outlier detection. In International Conference on Learning Representations, 2021.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning and Inference, 90:227-244, 2000.
12

---

Under review as a conference paper at ICLR 2022
Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, and Tomas Pfister. Learning and evaluating representations for deep one-class classification. In Proceedings of the International Conference on Learning Representations, 2021.
Hongchao Song. Zhuqing Jiang, Aidong Men, and Bo Yang. A hybrid semi-supervised anomaly detection model for high-dimensional data. Computational Intelligence and Neuroscience, 2017.
Hwanjun Song, Minseok Kim, Dongmin Park, and Jae-Gil Lee. How does early stopping help generalization against label noise? arXiv preprint arXiv: 1.08099, 2020.
Jihoon Tack, Sangwoo Mo. Jongheon Jeong, and Jinwoo Shin. CSI: Novelty detection via con- trastive learning distributionally shifted instances. In Advances in Neural Information Pro- cessing Systems 33, pp. 1189--11852, 2020.
Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. JEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1958-1970, 2008.
Yuting Wei, Fanny Yang, and Martin J Wainwright. Early stopping for kernel boosting algorithms: A general analysis with localized complexities. In Advances Neural Information Processing Systems 30, pp. 2065-6075. 2017.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R. Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, Taylan Cemgil, S. M. Ali Eslami, and Olaf Ronneherger. Contrastive training for improved out-of-distribution detection, arXiv preprint arXiv:2007.05560 2020.
Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nannan Wang, Zongyuan Ge. and Yi Chang. Robust early-learning: Hindering the memorization of noisy labels. In Proceedings of the Inter- national Conference on Learning Representations, 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. marking machine learning algorithms, 2017.
A novel image dataset for bench-
Yuan Yao, Lorenzo Rosasco, and Andrea Caponnetto. On early stopping in gradient descent learn- ing. Constructive Approximation, pp. 289-315, 2007.
Fatih Furkan Yilmaz and Reinhard Heckel. Image recognition from raw labels collected without annotators. arXiv preprint arXiv:
2019.
Qing Yu and Kiyoharu Aizawa. Unsupervised duito--distribution detection by maximum classifier discrepancy. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference (BMVC), 2016.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv: 161 1.0530,, 2016.
Hongjee Zhang, Ang Li, Jie Guo, and Yanwen Guo. Hybrid models for open set recognition. In ECCV, 2020a.
Min-Ling Zhang and Zhi-Hua Zhou. Exploiting unlabeled data to enhance ensemble diversity. Data Mining and Knowledge Discovery, 26:98-129, 2010.
Yu-Jie Zhang, Peng Zhao, Lanjihong Ma. and Zhi- Zui Zhou. An unbiased risk estimator for learning with augmented classes. Advances in Neural Information Processing Systems, volume 33, pp- 10247-10255, 2020b.
13

---

Under review as a conference paper at ICLR 2022
A THEORETICAL STATEMENTS
Definition A.1 p)-clustteee data set), We say that a data set D {ar, Bi) is (e, p)- clusterable for fixed e > 0 and p [0, 1] if there exists a partitioning of it into subsets (C1,..., Ck ) which we call clusters, each with their associated unit-norm cluster center Co that satisfy the fol- lowing conditions:
- C. = D and C,n - Vi,j [K];
all the points in cluster lie in the c-neighhorhood of their corresponding cluster center, i.e. - S for all x E C; and all i E [K];
fraction of at least 1 of the points in each cluster Cs have the same label, which call the cluster label and denote y" (ci). The remaining points suffer from label noise;
if two cluster C; and Cj have different labels, then their centers are 2e far from each other, i.e. ||e - -cyll 2 2e;
the clusters are balanced i.e. for all i E [K] S |C S 0r R where on and 02 are two positive constants.
In our case, for a fixed label с E y. we assume that the set S U (U, c) is -clustccle into K clusters. We further assume that each cluster C only includes a few noisy samples from (UTD, c), iee. [C:nCUTs.el p and that for clusters C₁ whose cluster label is not i.e. v*(() + it holds [all that ( c) 0.
We define the matrices C = [cI,..,,ck] E Rxd and E == (CCT) E 7 with g (0, Ia) and where O denotes the elementwise product. We use || and Amin (-) denote the spectral norm and the smallest eigenvalue of a matrix, respectively.
For prediction, we consider a 2-layer neural network model with p hidden units, where p 2 KCI Aman. (2)T.
We can write this model follows:
x fz; W) of (Wwx),
The first layer weights W are initialized with random values drawn from No0 1), while the last layer weights v have fixed values: half of them are set to 1/p and the other half -1/p. We consider activation functions e with bounded first and second order derivatives, i.e. [0'(x)] < r
and d²(x) S I. We use the squared loss for training, i.c. L(W) = Ei - f(x;; W))² and
take gradient descent steps to find the optimum of the loss function, i.c. W r+1 Wr -WWnnVWWW,
where the step size is set to 1j n
K
We Proposition A.1. Assume that p S 6/8 and c S. ox6 (X) where 8 is constant such that
state the following proposition:
ò < 2 -- and a is a constant that depends on l.. Then it holds with high probability 1 - 3 K200
Ke 100d over the initialization of the weights that the neural network trained on SU(U,)) perfectly iterations.
fis S. (Ui and (UooD, c), but not (UID; c), after T C4 Xman [E)
||C!|
This result shows that there exists an optimal stopping time at which the neural network predicts the correct label on all ID points and the label e on all the OOD points. As we will see later in the proof, the proposition is derived from more general result which shows that the early stopped model predicts these labels not only on the points in U but also in an e-neighborhood around cluster centers. Hence, ERD ensemble can be used to detect holdout OOD samples similar to the ones in U, after being tuned on U. This follows the intuition that classifiers regularized with early stopping smooth and generalize well.
The clusterable data model is generic enough to include data sets with non-linear decision bound- aries. Moreover, notice that the condition in Proposition i. satisfied when SU (UID. c) is (e, p)- clusterable and (UoOD: c) is e-clusterable and if the cluster centers of (UooD= c) are at distance at least 2c from the cluster centers of S U (UnD, c). A situation in which these requirements met
14

---

Under review as a conference paper at ICLR 2022
is, for instance, when the OOD data comes from novel classes, when all classes (including the un- seen ones that are not in the training set) are well separated, with cluster centers at least 2c away in Euclidean distance. In addition, in order to limit the amount of label noise in each cluster, it is necessary that the number of incorrectly labeled samples in (UDD, e) is small, relative to the size of S.
In practice, we only need that the decision boundary separating (UooD: from S is easier to learn than the classifier required interpolate the incorrectly labeled (VID c). which is often the case, provided that (UooD, e) is large enough and the OOD samples come from novel classes.
We now provide the proof for Proposition A.I
Proof. We begin by restating a result from Li et 2020
Theorem A.1 eLi al. (0200). Let D == ((Ti, Us)) E Rd x y be (llpscccute training set, with S Ci 6 (2) K and p I 5/8, where 6 a constant that satisfies 5 S -y- Consider a two-layer neural network as described above, and train it with gradient descent starting from initial weights sampled i..dd from NO, 1). Assume further that the step size is 1) = C2 भा|टp]र and that the
number of hidden units p is at least C3 (2) Under these conditions, it holds with probability at
least 1 1- - -ee-ee-oo 100d over the random draws of the initial weights, that after = C4 Xmiin (E)
||C]1
gradient descent steps, the neural network + f(x; WT) predicts the correct cluster label for all points in the e-neighborhood of the cluster center, namely:
arg max (f;; Wr) - w(v)l = yy (ci), for all with ||x ain-c,le cilla S and all clusters i E [K], (3) yEy
where w y + {0, 1 yields one-hot embeddings of the labels. The constants C12 C2 Ca C4 depend only on T.
Notice that, under the assumptions introduced above, the set SU is (c, clcluttrrabe since the incorrectly labeled ID points in (Ups, c) constitute at most a fraction p of the clusters they belong to. As a consequence, Proposition A. follows directly from Theorem A.I
follows
B DISAGREEMENT SCORE FOR OOD DETECTION
As we argue in Section 3.3 Algorithm | produces ensemble that disagrees on OOD data, and hence, we want to devise scalar score that reflects this model diversity. Previous works Laksh- minarayanan et al. (2017); Ovadia et al. (2019) first average the softmax predictions of the mod- els in the ensemble and then use the entropy as a metric, i.e. (H o (vg)(f(
= Σ> (f())); log(f(zz) where f(z) := L K fi()) and (f(x)); is the 2t element of f(x) [0, y We argue later that averaging discards information about the diversity of the models.
Recall that our average pairwise disagreement between the outputs of K models in an ensemble reads
(Avg o (s). = K(K - 1) E [p(A(t),(((o))
(4)
1$3
where p is measure of disagreement between the softmax outputs of two predictors, for example the total variation distance (Iv(fi(z), (x)) f1l)(()))))))))))))()) - [1(x)] used in our experiments.
We briefly highlight the reason why averaging softmax outputs first like in previous works relin- quishes all the benefits of having a more diverse ensemble, as opposed to the proposed pairwise
We abuse notation slightly and denote our disagreement metric (Avgop) to contrast it with the ensemble entropy metric (H o Avg). which first takes the average of the softmax outputs and only afterwards computes the score.
We abuse notation slightly and denote our disagreement metric as (Avgop) to contrast it with the ensemble entropy metric (H o Avg), which first takes the average of the softmax outputs and only afterwards computes the score.
15

---

Under review as a conference paper at ICLR 2022
red
blue
red
blue
ID support
(2 classes)
Ensemble models
Averaged model
blue
blue
True positives
True negatives
False negatives
red
False positives
(H o Avg)
(Avg p)
Figure 7:
Cartoon illustration showing diverse ensemble of linear binary classitiers. We compare OOD detection performance for two aggregation scores: (H Avg) (Left)) and (Avg o p) with (x), f2())
1 Lagu(f(()) (=)) (Right). The two metrics achieve similar TPRs, but using AHg) instead of our score, (Avgop), leads to more false positives, since the former simply flags as OOD a band around the averaged model (solid black line) and does not take advantage of the ensemble's 's diversity.
score in Equation Recall that varying thresholds yield different true negative and true positive rates (TNR and TPR, respectively) for given statistic. the sketch in Figure ? we show that the score we propose, (Avg p). achieves a higher TNR compared to (H Avg), for fixed TPR, which is a common way of evaluating statistical tests. Notice that the detection region for (H Avg) is always limited to band around the average model for any threshold value to. In order for the (H Avg) to have large TPR, this band needs to be wide, leading to many false positives. Instead, disagreement score exploits the diversity of the models more accurately detect OOD data.
Figure 8: Relying only on the randomness of SGD and of the weight initialization to diversify models is not enough, as it often yields similar classifiers. Each column shows different predictor trained from random initializations with Adam. All models have the same 1-hidden layer MLP architecture.
We now provide further quantitative evidence to support the intuition presented in Figure 7 The aggregation metric is tailored exploit ensemble diversity, which makes particularly beneficial for ERD. On the other hand, Vanilla Ensembles only rely on the stochasticity of the training process and the random initializations of the weights produce diverse models, which often leads classifiers that are strikingly similar as we show in Figure for a few 2D data sets. As a consequence, using our disagreement score (Avgop) for Vanilla Ensembles can sometimes hurt OOD detection performance. To see this, consider the extreme situation in which the models in the ensemble are identical, i.e. fi = f2. Then it follows that (Avg o (J (I), f2(x)) = 0, for all test points æ and for any function P that satisfies the distance axioms.
We note that the disagreement score that we propose takes a form that is similar to previous diversity scores, e.g. Zhang & Zhou (2010); Yu & Aizawal 2019). In the context of regression, can measure uncertainty using the variance of the outputs metric previously employed in works such point out that using the output variance requires that
Gal & Ghahramani (2016). However, the ensemble is the result of sampling from a random process (e.g. sampling different training data for the models, or sampling different parameters from posterior). In our framework, obtain the ensemble by solving different optimization problem for each of the models by assigning a different label to the unlabeled data. Therefore, despite their similarities, our disagreement score and the output variance are, conceptual level, fundamentally different metrics.
16

---

Under review as a conference paper at ICLR 2022
Table shows that (Avg p) leads to worse OOD detection performance for Vanilla Ensembles, compared to using the entropy of the average softmax score, (H Avg), which was proposed in prior work. However, if the ensembles are indeed diverse, as argue is the case for our method ERD (see Section 3.3) then there is a clear advantage to using a score that, unlike (H o Avg), takes diversity into account, as shown in Table 3
Table 3: The disagreement score that we propose (Avg p) exploits ensemble diversity and benefits in partic- ular ERD ensembles. OOD detection performance is significantly improved when using (Avg o p) compared to the previously proposed (H o Avg) metric. Since Vanilla Ensemble are not diverse enough, a score that relies on model diversity can hurt OOD detection performance. We highlight the AUROC and the TNR@ @95 obtained with the score function that is best for Vanilla Ensemble and the best for ERD.
Vanilla Ensembles (H a Aug)
Vanilla Ensembles (Avg AUROC
ID data
OOD data
BRD (H o Avg)
BRD (Avg
SVHN
CIFARIO
0196
0.85
/
CIARRIO
SVHN
0.72
1,000
CIFARION 100
SVHN
0.84
0.79//0.46
0.36/
1.80 1.00
SVHN[0.]
SVHN[559]
F0.69
0.91 0.069
[1944 0.66
0.94 / 0.66
CHARI0104]
CIARR0ES:9
0.30
0.80 0.897.0.9
/0.65
0.91 00,66
CIFAR1000-22
(FAR100(50-99)
0.78
0.7670.34
0..3
0.81
Average
0.87.0..0
0.36/0.59
0.77/0.64
0.94/0.75
We highlight once again that other methods that attempt to obtain diverse ensembles, such as MCD, fail train models with sufficient disagreement, even when they use oracle OOD for hyperparameter tuning (Figure Da>.
0.2
ID scores
0.2
ID scores
OOD scores
OOD scores
0.1
0.1
2
2
Disagreement score
Disagreement score
(a) Not enough diversity (MCD)
Regularized diversity (ERD)
Figure 9: Distribution of disagreement scores on ID and OOD data for an ensemble that is not diverse enough (Left), and ensemble with regularized disagreement (Right). Note that MCD is carly-stopped using oracle OOD data. ID=CIFARI0JO:4), OOD=CIFARI0(5:9)
C EXPERIMENT DETAILS
C.I BASELINES
In this section we describe in detail the baselines with which we compare our method and describe how we choose their hyperparameters. For all baselines we use the hyperparameters suggested by the authors for the respective data sets (e.g. different hyperparameters for CIFARIO or ImageNet). For all methods, we use pretrained models provided by the authors. However, we note that for the novel-class settings, pretraining OOD classes as well, which is undesirable. Therefore, for these settings we pretrain only on the split of the training set that contains the ID classes. Since the classification problem is similar to the original one of training on the entire training set, we use the same hy hyperparameters that the authors report in the original papers.
the entire training set means that the model is exposed to the
Moreover, we point out that even though different methods use different model architectures, that is not inherently unreasonable when the goal is OOD detection, since is not clear if a complex model is more desirable than a smaller model. For this reason, we use the model architecture recommended by the authors of the baselines and which was used to produce the good results reported in their
17

---

Under review as a conference paper at ICLR 2022
published works. For Vanilla Ensembles and for ERD we show results for different architectures in Appendix E.7
Vanilla Ensembles Laaaaissssasssmmaaaaaaaaaaa et al. 20177: We train an ensemble on the training set according the true labels. For test sample, average the outputs of the softmax probabilities predicted by the models, and use the entropy of the resulting distribution as the We use ensembles of 5 models, with the same architecture and hyperparameters as the ones used for ERD. Hyperparameters are tuned to achieve good validation accuracy.
score for the hypothesis test described in Section 3.2
* Gram method [Sastry& & Oore (2019): The Gram baseline is similar to the Mahalanobis method in that both use the intermediate feature representations obtained with a deep neural network to determine whether a test point is outlier. However, what sets the Gram method apart is the fact that it does not need any OOD data for training or calibration. We use the pretrained models provided by the authors, train our own, using the same methodology as descrihed for the Mahalanobis baseline. For OOD detection, we use the code published by the authors. We note that for MIP models, the Gram method is difficult to tune and we could not find a configuration that works well, despite our best efforts and following the suggestions proposed during our communication with the authors.
Deep Prior Networks (DPN) Malinin & Gales (2018): DPN is a Bayesian Method that trains a neural network (Prior Network) to parametrize a Dirichlet distribution over the class probabilities. We train WideResNet WRN-28-10 10 for 100 epochs using SGD with momentum 0.9, with an initial learning rate of 0.01, which is decayed by 0.2 at epochs 50, 70, and 90. For MNIST, we use EMINST/Letters as OOD for tuning. For all other settings, we use Tmnylmages as OOD for tuning.
Outlier Exposure Hendrycks et al. (2019): This approach makes a model's softmax pre- dictions close to the uniform distribution on the known outliers, while maintaining good classification performance on the training distribution. We use the WideResNet architec- ture (WRN) Zagoruyko & Komodakis (2016). For fine-tuning, we use the settings rec- ommended by the authors, namely we train for 10 epochs with learning rate 0.001. For training from scratch, we train for 100 epochs with an initial learning rate of 0.1. When the training data set is either CIFAR RO/CIFO//ARIIIIIII 100 or ImageNet, we use the default WRN parameters of the author's code, namely 40 layers, 2 widen-factor, droprate 0.3. When the training dataset is SVHN, we use the author's recommended parameters of 16 layers, 4 widen-factor and droprate 0.4. All settings use the cosine annealing learning rate sched- uler provided with the author's 's code, without any modifications. For all settings, we use Tinylmages as known OOD data during training. In Section E.5 we show results for known OOD data that is similar to the OOD data used for testing.
Mahalanobis Lee et al. 2018): The method pretrains models on the labeled training data. For a test data point, uses the intermediate representations of each layer as "extracted fea- tures". It then performs binary classification using logistic regression using these extracted features. In the original setting, the classification is done on "training" ID vs "training" OOD samples (which are from the same distribution as the test OOD samples). Further- more hyperparameter tuning for the optimal amount of noise is performed on validation ID and OOD data. We use the WRN-28-10 architecture, pretrained for 200 epochs. The initial learning rate is 0.1, which is decayed at epochs 60, 120, and 160 by 0.2. We use SGD with momentum 0.9, and the standard weight decay of 5 10 4, The code published for the Mahalanobis method performs a hyperparameter search automatically for each of the data sets.
The following baselines assume the same Unknown OOD setting as ERD, in which one has access to both a labeled ID training set S and an unlabeled set with an unknown mixture of ID and OOD samples U.
Non-negative PU learning (nnPU) Kiryo et al. (2017): The method trains a binary predic- tor to distinguish between a set of known positives (in our case the ID data) and a set that contains mixture of positives and negatives (in our case the unlabeled set). To prevent the interpolation of all the unlabeled samples, Kiryo et al. (2017) proposes a regularized objective. It is important to note that most training objectives in the PU learning literature
18

---

Under review as a conference paper at ICLR 2022
require that the ratio between the positives and negatives in the unlabeled set is known or easy to estimate. For our experiments we always use the exact OOD ratio to train the nnPU baseline. Therefore, estimated from finite samples, then estimation errors may lead to slightly worse OOD de- tection performance. We perform grid search over the learning rate and the threshold that appears in the nnPU regularizer and pick the option with the best validation accuracy measured on a holdout set with only positive samples (in our case, ID data).
obtain an upper bound on the AUROC/TNR @95. If the ratio is
Maximum Classifier Diserepancy (MCD) Yu & Aizawa 2019): The MCD method trains two classifiers at the same time, and makes them disagree on the unlabeled data, while maintaining good classification performance. We use the WRN-28-10 architecture sug- gested in the paper. We did not change the default parameters which came with the author's code, so weight decay is 10-A, and the optimizer is SGD with momentum 0.9. When avail- able (for CIFAR1O 10 and CIFAR 100), we use the pretrained models provided by the authors. For the other training datasets, we use their methodology to generate pretrained models: We train a WRN-28-16 10 for 200 epochs. The learning rate starts at 0. and drops by factor of 10 at 50% and 75% of the training progress.
Mahalanobis-U: U: This is a slightly different version of the Mahalanobis baseline, for which we use carly-stopped logistic regression to distinguish between the training set and an un- labeled set with ID and OOD samples (instead of discriminating known OOD set from the inliers). The early stopping iteration is chosen to minimize the classification errors on validation set that contains only ID data (recall that we do not assume to know which the OOD samples).
In addition to these approaches that have been introduced in prior work, we also propose a strong novel baseline that that bares some similarity to PU learning and to ERD.
Binary classifier The approach consists in discriminating between the labeled ID training set and the mixed unlabeled set, that contains both ID and OOD data. We use regularization to prevent the trivial solution for which the entire unlabeled set is predicted as OOD, Unlike PU learning, the binary classifier does not require that the OOD ratio in the test distribution is known. The approach is similar to a method described in /Soott & Blanchard (2008) which also requires that the OOD ratio of the unlabeled set known. We tune the learning rate and the weight of the unlabeled samples in the training loss by performing a grid search and selecting the configuration with the best validation accuracy, computed on a holdout set containing only ID samples. We note that the binary classifier that appears in Section F in the medical benchmark, is not the same as this baseline. For more details on the binary classifier that appears in the medical data experiments we refer the reader to Cao et al. (2020).
C.2 TRAINING CONFIGURATION FOR ERD
For ERD we always use hyperparameters that give the best validation accuracy when training a model on the ID training set. In other words, we pick hyperparameter values that lead to good ID generalization and do not perform further hyperparameter tuning for the different OOD data sets on which we evaluate our approach. We point out that, if the ID labeled set is known to suffer from class imbalance, subpopulation imbalance or label noise, any training method that addresses these issues can be used instead of standard empirical risk minimization to train our ensemble (e.g. see the works of Sagawa et al. (2020): Li et al. (2020) etc).
For MNIST and FaShionMNIIST, we train ensembles of 3-layer MIP models with ReLU activations. Each intermediate layer has 100 neurons. The models are optimized using Adam, with a learning rate of 0.001, for 10 epochs.
For SVHN, CIFAR 10/CIFARI00 and ImageNet, we train ensembles of ResNet20 He et al. (016)) The models are initialized with weights pretrained for 100 epochs on the labeled training set. We
fine-tune each model for 10 epochs using SGD with momentum 0.9, and learning rate of 0.001.
The weights are trained with an l2 regularization coefficient of 5e - 4. We use a batch size of 128 for all scenarios, unless explicitly stated otherwise. We used the same hyperparameters for all settings.
19

---

Under review as a conference paper at ICLR 2022
For pretraining, we perform SGD for 100 epochs and use the same architecture and hyperparameters as described above, with the exception of the learning rate that starts at 0.1, and multiplied by 0.2 at epochs 50, 70 and 90.
Apart from ERD, which fine-tunes the ensemble models starting from pretrained weights, we also present in the Appendix results for ERD++. This variant of our method trains the models from random initializations, and hence needs more iterations to converge, making more computationally expensive than ERD. We train all models in the ERD++ ensembles for 100 epochs with learning rate that starts at 0.1, and is multiplied by 0.2 at epochs 50, 70 and 90. All other hyperparameters the same for ERD ensembles.
For the medical data sets, we train a Densenet- 121 as the authors do in the original paper Cao et al. (2020). For ERD++ we do not use random weight initializations, but instead we start with the ImageNet weights provided with Tensorflow. The training configuration is exactly the same as for ResNet20, except that we use batch size of due GPU memory restrictions, and for fine tuning use constant learning rate of 10-b.
D ID AND OOD DATA SETS
D.1 DATA SETS
the following image data sets: MNIST Lecun et al. (998). Fashion MNIST Xiao et al. (2017), SVHN Netzer et al. (201T), CIFARIO and CIFAR 100 Krizhevsky (2009).
For evaluation,
For the experiments using MNIST and FashionMNIST the training set size is 50K, the validation size is 10K, and the test ID and test OOD sizes are both 10K. For SVHN, CIFARIO 10 and CIFAR 100, the training set size is 40K, the validation size is 10K, and the unlabeled set contains 10K samples: 5K are ID and 5K are OOD. For evaluation, we use holdout set of 10K examples (half ID, half OOD). For the settings that half of the classes ID and the other half as OOD, all the sizes divided by 2.
D.2 SAMPLES FOR THE SETTINGS WITH NOVEL CLASSES
MNIST(O:4]
CIFAR10[O:4]
label
I
2
3
4
MNIST(0:9]
CIAR1005:991
label 5
label
label
label B
label 9
label
label 6
label 7
label
label
5
6
7
8
9
FashionMNIST 0,2.3,7,8
SVHN[0:4]
label
label
label
label 7
label
label
label
label 2
label
label
4D
0.thionMNIST.1.45..00
SVHNIS
label
label
label
5
$6
17
(a)
(b)
Figure 10: (a) Data samples for the MNIST/FashionMNISTT splits. (b) Data samples for the CI- FAR10/SVHN splits.
20

---

Under review as a conference paper at ICLR 2022
E MORE EXPERIMENTS
E. EVALUATION ON THE UNLABELED SET
In the main text we describe how one can leverage the unlabeled set to obtain an OOD detection algorithm that accurately identifies outliers at test time that similar to the ones in U. It is, however, possible to also use our method ERD to flag the OOD samples contained in the same set U used for tun-nt the ensemble. In Table 4 we show that the OOD detection performance of ERD is similar regardless whether we use U for evaluation, or a holdout test set T drawn from the same distribution as U.
Table 4: Comparison between the OOD detection performance of ERD when using holdout test set T for evaluation, or the same unlabeled set U that was used for finettuning the models.
TD data
OOD data
ERD ca AUROC
ERD
SVIN
CIFARIO
CIFARIO
SVIIN
1,00/ 1.00
1.00/ 1.00
CIFARI00
SVHN
1.00/ 1.00
1.00/ 1.00
FMNIST
FM5ISTI14569
0.94/0067
0.94/0.67
SVHN[044]
SVHN[5.9]
0.957
0.96/0.79
CIARRIQ0411
CHARR1O1591
0,95//0.70
0.92 00.69
CIFAR1000-49
[FAR100[50:999
0.82/ 0:44
0.50/
Average
0.95/0.79
0.95 0..79
E.2 COMPARISON WITH OTHER RELATED WORKS
We compare our method to more OOD detection approaches. For various reasons we did not run these methods ourselves on the data sets for which we evaluate our method in Section[[[c.. resource constraints, code not available, unable to replicate published results, poor performance reported by the authors etc). We collected the AUROC numbers presented in Table 5 from the papers that introduce each method. We note that approach shows excellent overall performance, almost matching the AUROC of the best performing method of Fort et al. 2021) which uses large scale visual transformer models pretrained
superset of the OOD) data, f.e. ImageNer21k.
Furthermore, we note that generative models (Nalisnick et al. 2019 Choi et al. [018 Akçay et al. 2018) and one-class classification approaches (Ruff' al.) 2020 Tack et al. [2020] Sohn et al.) 2021 showed generally bad performance, in particular on near OOD data. When the ID training set is made up of several diverse classes, it is difficult to represent accurately all the ID data, and only the ID data.
Table 5: AUROC numbers collected from the literature for number of relevant OOD detection methods. We note that the method of Fort et al. 2021 (') uses a large scale visual transformer models pretrained on superset of the OOD data, I.e. ImageNet2Ik, while the method of Sehwag et al. (021) (*) uses oracle OOD samples for training from the same data set as test OOD. For the settings with random classes, the numbers are averages over draws and the standard deviation is always strictly smaller than 0.01 for our method.
ID data
DOD data
Fur et
Zhang
Winkens
Tack
Sebwag
Lig
Zhung
ERD (oun)
ERD++
2021
20220
Abbrel
2020
2021
2020
CIFARIO CIFARIDO
CIFARIOI
0.95
0.92
0.92
0.93
0.1
0.92
095
CIFARID
0.35
0.78
0.78
0.1
0.9
SVHN:
SVHN:
randum
random
0.94
0.91
0.91
0.1
classss
classes
CIFARIE:
CIFARID: 4
random
0.94
0.35
0.94
0.7
classes
classes
E.3 OOD DETECTION FOR DATA WITH COVARIATE SHIFT
In this section we evaluate the baselines and the method that we propose on settings in which the OOD data suffers from covariate shift Shimodaira (000)). The goal is to identify all samples that
21

---

Under review as a conference paper at ICLR 2022
come from the shifted distribution, regardless of how strong the shift is. Notice that mild shifts may be easier to tackle by domain adaptation algorithms, but when the goal is OOD detection they pose much more difficult challenge.
We want to stress that in practice one may not be interested in identifying all samples with distri- bution shift OOD, since classifier may still produce correct predictions some of them. In contrast, when data suffers from covariate shift we can try to learn predictors that perform well on both the training and the test distribution, and we may identify only those test samples on which the classifier cannot make confident predictions. Never- theless, we use these covariate shift settings as challenging OOD detection benchmark and show in Table that that our method ERD does indeed outperform prior basclines on these difficult settings.
measure of predictive uncertainty to
We outliers corrupted variants of CIFARIO and CIFARI00 Hendrycks & Dietterich (2019). as well as a scenario where ImageNet Deng et al. (2009) is used ID data and ObjectNet Barbu et al. (2019) as OOD, both resized 32x32. Figure 11 shows samples from these data sets. The Gram and nnPU baselines do not give satisfactory results the difficult CIFAR 10/CIFARION settings in Table 2 and thus we do not consider them for the distribution shift cases. For the Unknown OOD methods (i.e. MCD, Mahal-U and ERD/ERD++) we evaluate on the same unlabeled set that is used for training (see the discussion in Section
CHARID severity
Brightnass
Conrras:
Defoous blur
Backic
15g
IrageNts
OlécaNel
Chais
Chain by mexton
Cloin by bockmmnt
Charn by sinaporr
Tipors
T-bitt
Roched goot Ner
Gruesan noico
Impueer reits
pCg comppptcoo
suumiity
Brightnwss
Chstiaas
Bledi:
ToU
Jong compreccion
Motion
bur
Figure 11: Left: Samples from ImageNet and ObjectNet taken from the original paper by Barbu et al. (2019). Right: Data samples for the corrupted CIFAR 10-C data set.
Furthermore, present results on distinguishing between CIFAR1O Krizhevsky (2009) and CI- FAR10v2 Recht et al.] (2018), a data set meant to be drawn from the same distribution as CIFAR10 (generated from the Tiny Images collection Torralba et al. (2008)) In Re/hh al. (211)) the authors argue that CIFARI0 10 and CIFARIOV come from very similar distributions. They provide supporting evidence by training a binary classifier distinguish between them, and observing that the accuracy that is obtained of 52.9% is very close to random.
Our experiments show that the two data sets are actually distinguishable, contrary to what previous work has argued. First, our own binary classifier trained CIFARIO vs CIFARIOv2 obtains test accuracy of 67%, without any hyperparameter tuning. The model we use is ResNer20 trained for 200 epochs using SGD with momentum 0.9. The learning rate is decayed by 0.2 at epochs 90, 140, 160 and 180. We use 1600 examples from each data set for training, and we validate using 400 examples from each data set.
22

---

Under review as a conference paper at ICLR 2022
Table 6: OOD detection performance on CIFARIO CIFARIOV2
Vanilla
ID data
OOD data
Ensembles
DPN
QE
Mahal
MCD
Mahal-U
BRD
BRD+
AUROC TNR
CIFARIO CBARION
0.641073
0.09
/..4/
0.55/0.08
0.58/0.10
0,5670007
0.76/0.26
Our OOD detection experiments (presented in Table 6 show that most baselines are able to distin- guish between the two data sets, with ERD achieving the highest performance. 'he methods which require OOD data for tuning (Outlier Exposure and DPN) use CIFARIO0.
Table 7: OOD detection performance on data with covariate shift. For ERD and vanilla ensembles, train ResNet20 models for each setting, The evaluation metrics are computed on the unlabeled set.
OOD data
Vanilla Ensembles
DPN
OE
Mahal
MCD
Mahal-U
ERD
AURRCC 1 TTNR@99
0.68
0.33/0311
0.8410.53
0.82/0.50
0.75/0.38
CIFARIO
CIFARIO-C sev (WW
0.517
047/0.03
0.52/0.06
0.008
0.52/0.06
0.55/0.07
0.68/0.11
0.86
CIFARIO
CIFARIO sev (
0,470.49
0.9/0.60
0.86/0.54
0,4/00800
0,951 0.84
0.8800.3
1,00/.999
1.00/ 1.00
CIFARIO
CIFARIO-C 5(W)
0.60/0110
0.72/0.10
063/0.11
0.7810.27
0,60/0066
0.68/0.12
0.880085
1.00 1,00
CIFARIOO
CIEARIOD-C 2
0.68 0.20
0.62 /0.18
0.15/0.19
0.82 0.48
0.22
/0,22
0.94/0.76
0.97 0.88
CIFAR100
100-C
0.52/6.06
0.32/0.03
0.52/0.06
0.52/0..6
0.55/0.0
0.71
CD
CIFAR
0.78/0.37
0.74
0.76/0.57
0.11 7066
034/0055
0.99
1.00/0.9
CIFAR100
CIFAR CB-CCoer sev 5 (WT
0.64/0,14
/0.12
0.62/0.13
0,71 0.79
0.60/0110
0.63 /0.13
0.96/
0.98/ 11,99
Tiny ImageNet
Tiny ObjectNet
0.82/0.49
0.70 0,32
0.39/ 0.37
0.75/ 0.25
0.99 0.98
0.72/ 0.25
0.98/ 0.88
0.99 0.99
Average
0.67/0.23
0.67
0.76
0.74/
0.70 0.27
0.91 /0.71
E.4 RESULTS WITH A SMALLER UNLABELED SET
We now show that our method performs well even when the unlabeled set is significantly smaller. In particular, we show in the table below that ERD maintains high AUROC and TNR@95 even when only 1,000 unlabeled samples are used for fine-tuning (500 ID and 500 OOD).
Table 8: Experiments with test set of size 1, 1,000, with an equal number of ID and OOD test samples. For ERD and vanilla ensembles, we train 5 ResNet20 models for each setting. The evaluation metrics are computed on the unlabeled set.
Vanilla Easembles
data
00D data
DPN
OE
Mahal.
MCD
Malnl-U
ERD
AUROC: /
SVHN
CIFARIO
0.97 0.88
0.97 /0.5
0:99/0.95
CIFARIO
SVIN
0.770078
0.95/0.85
0.97/0.89
0.99 0.96
1,00/098
0.99/0.96
1.00/1.00
CIFAR100
SVHN
0.8470.48
0.77/0.44
0.82/0.50
0.98 0.90
0.97/0..3
0.98/0.92
0.99/1.10 1.00
SVLEN[O:1]
SVEIN[5:9]
0.92/0.6
0.37 /.119
0.85/0.55
0.92
0.91 70.51
0.91/00 1.9/
0.97 70886
0.30/0.39
70.32
0.64/0.13
0.50
10FAR10000-91
[00[50.99]
0.7870.35
070/0.336
0.74/0.31
0.72/0.30
0.20/0.36
0.73/0.19
00.38
CIFARIO
CIFARIO-C sev ((A)
0.70
00.21
CIFARIO
CIARIOOC sev (WW)
0.51/0.00 0.05
0.47 /0.05
0.52/ 0.06
0.58 10,8
0.52 0006
0.55 0.07
0.57 / 00.09
CIFARIO
CIFARIO-C sev (A)
0.44/0.9
0.99/0.00
0.86 / 0.54
0,84
0.95 0.95/0344
0.88 0.63
0.99 / 0..55
CIFARIO
CIFAR (W)
0.60/0.10
0.72/0.10
01,7
CIFAR100
CIFARIOG 100-C CC (A)
0.68/0.20
0.62/0.15
0.65/0..0
0.82 0.44
0.72/0..9
/.27/
0.84 / 0.48
CIFAR
0.52/00.6
0.32/0.05
CIFARLOO
CIEAR100 sev (A)
0.78/0.57 0.37
0,36
0.75
0.9210.72
0.91 70.65
0.84/0.55
0.96 0,80
CIFARI00 100
CIFAR 100-C sev (W)
0.6470.14
0.11/
0.62/ 0.13
0.7110.19
0.10/0.10
0.63 0..13
0,51 / 0.25
Average
0.57
0.72/0.34
0.75 0..3
0.82 0.51
0.78/0..4
0.73 0.42
0.87 0.62
23

---

Under review as a conference paper at ICLR 2022
E.5
MORE RESULTS FOR OUTLIER EXPOSURE
Table 9: Results for Outlier Exposure, when using the same corruption type, but with higher/lower
severity, as OOD data seen during training.
ID data
OOD data
OE (trained on sev5)
OE (trained sev2)
AUROC t
CIFARIO
CIFARIO-C sev 2 (A)
0.89
N/A
CIFARIO
CIFARIO-C sev 2 (W)
0.65
N/A
CIFARIO
CIFARIO-C sev 5 (A)
N/A
0.98
CIFARTO
CIFARIO-C sev 5 (W)
N/A
0.78
CIFAR LOO
CIFARIO0 1O0-C sev (A)
0.85
N/A
CIFAR 100
CIFAR100-C sev (W)
0.59
N/A
CIFARIOS 100
CIFAR100-C sev (A)
N/A
0.97
CIFARI00
CIFAR100-C sev (W)
N/A
0.67
Average
0.87
0.98
The Outlier Exposure method needs access to a set of OOD samples during training. The numbers report in the rest of paper for Outlier Exposure are obtained by using the Tinylmages data set as the OOD samples that are seen during training. In this section we explore the use of an OOD data that is more similar to the OOD data observed test time. This is much easier setting for the Outlier Exposure method: the closer OOD is to OOD the easier it will be for the model tuned on OOD detect the test OOD samples.
In FableeWa we focus only on the settings with corruptions. For each corruption type, we use the lower severity corruption as OODinin and evaluate the higher severity data and vice versa. We report for each metric the average taken over all corruptions (A), and the value for the worst-case setting (W).
E.6 RESULTS ON MNIST AND FASHIONMNIST
Table 10: Results on MNIST/FashionMNIST settings. For ERD and vanilla ensembles, we train 3-hidden layer MLP models for each setting. The evaluation metrics are computed on the unlabeled set.
OUD
Vanilla Ensemblee
Mahal.
Mahal-U
Classif.
AUROC 1 /TNRR995
MNIST FMIISTT
/0.1
1.00
1,00
1.00/
MNIST
0.87 F0.42
1 1,00
DU68
/0.16
0.99 1.00/ 1.00 1.00 1.00
0.99
/..9
1.00/100 1.00
1,00
1.00
1,00/ 1.00
MINISTTO:41
MNISTI5991
0.77
0.99
0.95/01
/0.78
0.99 0.98
0.99 / 11.57
/.96/
0.99
0.99
0.99/0994
0.94
0.99
/1.96
0.99/0..5 0.97
FMNISTIU227788
FMNISTILA-569)
0.77/0.15
/0.7
0.78/0.30
8:95/
Average
0.82/0.30
0.78 0.82/0.51 0.99 /..9 0.96 0.95/0.83 0.96/0.90 0.98/0.91
0.98/0.91
For FashionMNIST we chose this particular split (i.e. classes 0,2,3.7,8 vs classes 1,4.5,6,9) because the partitions more similar to each other. This makes OOD detection more difficult than the 0-4 vs 5-9 split.
E.7 VANILLA AND ERD ENSEMBLES WITH DIFFERENT ARCHITECTURES
In this section we present OOD detection results for Vanilla and ERD ensembles with different architecture choices, and note that the better performance of our method is maintained across model classes. Moreover, we observe that ERD benefits from employing more complex models, like the NedeResNet.
24

---

Under review as a conference paper at ICLR 2022
Table 11: Results with three different architectures for Vanilla and ERD ensembles. All ensembles comprise 5 models. For the comuptiion data sets, we report for each metric the average taken over all corruptions (A), and the value for the worst-case setting (W). The evaluation metrics are computed on the unlabeled set.
ResNer20
WideResN:s-28-10
Vanilta Fissembles
Vanilia
Vanilla
Fasemilles
AUROC T /TTNR
SVHN
CI
0..8
0.99/0.94
0.97 /0.88
0.99 00.9/
0.96 00.85
1.00/0999
CIFARIO
SVEI
0.88/ 0..6
1,90/ 1.00
0.92/
100/1.00 1.00
0.94 70.81
1.00/ 1.00
SVIIN[O:]]
SVIIN 5:9]
0.89/0.66
0,93/0063
0.92/ 0.92/0.69
0.94/0.66
0.91 /0.62
0.96/0.76
CIFAR
10[5-9]
0,91/063
0.91/0.66
CIFARIO
10-C
0.66/0.11
0.94/0.79
/0.20
0.60/0.18
0.58
CIFARIO
CIFARIO-C sev (W)
0.51
0.19
0.51
CIFARIO
CIRARIO-C sev (A)
0.80/0.44 00.41
/.99/
0.84/0049
1.00/ 0:99
0.54 /0.47
1.00/ 1.00
CIFARID
CIFARIO-C sev (W)
0.58 /0.10
0.95/0.72
0.60/0.10
0.98 0..9
0.59/
0.99/0.00
Average
0.75/0.00
0,92/0.73
0.78/0.45
0.93/
0.78 00,8/
0.96/0.84
E.8 IMPACT OF THE ENSEMBLE SIZE AND OF THE CHOICE OF ARBITRARY LABEL
In this section show OOD detection results with method using a smaller number of models for the ensembles. We notice that the performance is not affected substantially, indicating that the computation cost of our approach could be further reduced by fine-tuning smaller ensembles.
Table 12: Results obtained with smaller ensembles for HRD. All numbers are averages over runs, method's 's stability with
where we use different set of arbitrary labels for each run illustrate respect the choice of labels to be assigned to the unlabeled set. We note that the standard deviations small S 0.01 for the AUROC values and S 0.08 for the TNR @95 values).
K=2
K=3
K=4
data
OOD data
ERD
BRD++
ERD
ERD++
ERD
ERD++
AUROC' 1 /TNRG95 1
SVHN
CIFAR10
0.99/0.95
0.99/0.9
0.99/ 0.997
1.00/0.99
0.99/0.99
1.00/0.99
CI
SVHN
1.00/1000
1,00/ 1.00
1.00 /.000
1,00/ 1.00
1.00/1.00
1.00 1.00/1.00
1.00/
SVHN[O-4]
SVHN[S:9]
0.95/0.69
0.94/0.6
00.73
0.95/0.55
/0.76
0.96/0.77
CIPAR [IFAR10(044]
CIFAR 10[5:9]
/..9/0.55
0.92/0.58
0.89/0. 10.57
0.94/0.70
0.90/0.5
0.95 /0.73
CIFARL00[0-19]
CIFAR [00[30999
0.8100.00
0.62/0.33
08.40.41
0.84/0.44
0.81/044
0.84/0.44
Average
0.94/0.77
0.95/0.78
0.94/0778
0.95/081
0.94/0.79
0.96/0.82
Impact of the choice of arbitrary labels. Furthermore, we note that the table we report aver- ages over 3 runs of our method, where for each run we use different subset of y to assign arbitrary labels to the unlabeled data. We do this in order to assess the stability ERD ensembles to the choice of the arbitrary labels and notice that the OOD detection performance metrics do not vary significantly. Concretely, the standard deviations are consistently below 0.01 for all data sets for the AUROC metric, and below 0.07 for the TNR @95 metric.
F MEDICAL OOD DETECTION BENCHMARK
The medical OOD detection benchmark is organized as follows. There are four training (ID) data sets, from three different domains: two data sets with chest X-rays, one with fundus imaging and one with histology images. For each ID data set, the authors consider three different OOD scenarios:
1. Use 1: The OOD data set contains images from completely different domain, similar to our category of easy OOD detection settings.
2. Use case 2: The OOD data set contains images with various corruptions, similar to the hard covariate shift settings that we consider in Section
3. Use case 3: The OOD data set contains images that come from novel classes, not seen during training.
25

---

Under review as a conference paper at ICLR 2022
Hard OOD data (unseen diseases)
Training data
Easy OOD data
ooo
Valuily gl
Figure 12:
Samples from the medical image benchmark. There are 3 ID data sets containing frontal and lateral chest X-rays and retinal images. Hard OOD samples contain images of diseases that not present in the training set.
The authors evaluate number of methods all these scenarios. The methods can be roughly categorized as follows:
1. Data-only methods: Fully non-parametric approaches like kNN.
2. Classificr-only methods: Methods that use classifier trained on the training set, e.g. ODIN Liang al. 2018). Mahalanobis Lee et al. 2018). ERD falls into this category as well.
3. Methods with Auxiliary Models: Methods that use an autoencoder or generative model, like Variational Autoencoder Generative Adversarial Network. Some of these ap- proaches can be expensive to train and difficult to optimize and tune.
We stress the fact that for most of these methods the authors use (known) OOD data during training. Oftentimes the OOD samples observed during training come from data set that very similar to the OOD data used for evaluation. For exact details regarding the data sets and the methods used for the benchmark, we refer the reader Cao al. 20200.
0.5
0.89
AUROC
0.4
0.71
0.73
0.74
0.75
0.79
0.79
0.80
3.81
0.83
0.60
0.61
0.63
Prob. Diredtals
Scoree Sive
Fuetura ketn
Adede- Anttle
Recovat AERICE
Sargla Layar Maria.
Adtedt scmn
Mahusarcaas
Recurnt ALl
VAERCE RNN--
1001-
Recmnot VAgHeSe
Reczort VARRCE
Soery chassteer
Resshaal ADMOS
Brb
Figure 13: AUROC averaged over all scenarios in the medical OOD detection benchmark Cao et al. 2020 The values for all the baselines computed using code made available by the authors of Cao et al. (2020). Notably, most of the baselines assume oracle knowledge of OOD data at training time.
26

---

Under review as a conference paper at ICLR 2022
In addition, in Figure 14 present the average taken over only the novel-class settings in the med- ical benchmark. We observe that the performance of all methood is drastically affected, all of them performing much worse than the average presented in Figure [3 This stark decrease in AUROC and TNR @95 indicates that novelty detection is indeed a challenging task for OOD detection meth- ods even in realistic settings. Nevertheless, method maintains better performance than the baselines.
0.5
AUROC
0.8
0.71
0.74
0.61
0.58
0.53
0.59
0.54
0.55
0.59
0.55
0.33
0.57
0.50
0.57
0.35
0.46
1.49
Score Sule
OGIN
Afaccs mot-1
Featar keo
Aovee--tile Atdl-1
Recorat Azace
VAracE-Eni 2wo
acace nact-romm romn-
dirgie layer stads.
VASHiSE pASHiE--Rtee- stre-
Recunat All
VARACE- 1a0--
Recanat VAGMNE
Roaort VASBCE
Binnry donaderr
Raconal ARMGE
50b
510++
threshate
Figure 14: AUROC averaged over the novel-class scenarios in the medical OOD detection bench- mark Cao et al. 2020), i.e. only case 3.
In Figures L5
16 17 we present AUROC and AUPR (Area under the Precision Recall curve) for
ERD for each of the training data sets, and each of the cases. Figure [ averages over all settings that we considered, for all the baseline methods the benchmark. Notably, ERD performs well consistently across data sets. The baselines ordered by their average performance on all the settings (see Figure 13
100
90
75
ausilery eacelarr
100
Prab tireanoia
Scare suaa
acCiCA A
Feeture ken
Resconat Asscc
Lastace Dttl
REBCC
Single toyer balie.
Vagees Sait
Manaaandaa
Rerurat All
tataace Khite
Recronet VAgeeg
Reccautt VACACC
grnsky clossiter
Reconct AEMEE
BRD
5
Figure 15: Comparison between ERD and the various baselines on the NIH chest X-ray data set, for use case (top), use case (middle) and use case 3 (bottom). Baselines ordered as in Figure [3]
27

---

Under review as a conference paper at ICLR 2022
100
50
AURDC, ditt-onll
AUROC, clessiter only AUPRC, classitir only
GURDC, with aupilery aUPAA, waxilery
AUPRC, datewnrly
Prob threahot
Scare SVW4
HEaice baWe 1
Fentur teen
AEMEE stot
terconst sERCE
LREEEE Et0ly 1
Khik
BEBICE AUBiCAA A
Strgle layer Hatte.
Nitest exw-
IONDEEE 3300-5
Shhalanubbl
MAEDEE-XR MACRCE-AWW-- X3068
Reconat VAENNSE
Becunet VAraCE:
linory uacather
Becoust ALMGE
EB0 ERQ
Figure 16: use case (top), use case (middle) and use case (bottom). Baselines ordered as in Figure I3
Comparison between ERD and the various baselines on the PC chest X-ray data set, for
100
AUROC, data-anly defsenny
AUROC, ClaSSt any thassffe only
with aubilary unciilrry
Prob theatraid
Scare SYN
NEBICE shtp 1
Feature from
HEMOE Stot-
Ietconst sERIEE
VRCEMSE coty
BCBICS BRCCSAOO Khite-i
Sinttte tayer Matte.
BCMISE 3
VIENECE A302-
Stahaloncioo
MACECE- X----3
Racconat VElENiN
Sinery clacather Rlccenst VALEACE
Bertonst AENSE
DPD
GDD++
Figure 17: Comparison between ERD and the various baselines on the DRD fundus imaging data set, for use case (top), use case (middle) and use case 3 (bottom). Baselines ordered as in Figure 13
For all medical benchmarks, the unlabeled set is balanced, with equal number of ID and OOD samples (subsampling the bigger data set, if necessary). We use the unlabeled set for evaluation.
G EFFECT OF LEARNING RATE AND BATCH SIZE
We show now that our method ERD is not too sensitive to the choice of hyperparameters. We illustrate this by varying the learning rate and the batch size, the hyperparameters that identify as most impactful. As Figure 18 shows, many different configurations lead to similar OOD detection performance.
28

---

Under review as a conference paper at ICLR 2022
0.77
0.75
bs 128
0.7
0.73
0. 7.75
0.76
D.75
0.74
Batch size
05
0.71
0.74
0.76
0.74
0.72
0.78
0.72
0.71
0.75
0.72
0.71
0.09
1.0.001
0.005
0.0.0
00.0
Ir0.1
Initial learning rate
Figure 18: AUROCs obtained with an ensemble of WRN-28- 10 models, as the initial learning rate and the batch size are varied. We used the hardest setting, CIFAR100:0-3 as ID, and CIFAR 100:50- 100 OOD.
H ADDITIONAL FIGURE SHOWING THE DEPENDENCE ON THE UNLABELED SET CONFIGURATION
The configuration of the unlabeled set (i.e. the size of the unlabeled set, the ratio of OOD samples in the unlabeled set) influences the performance of our method, as illustrated Figure 6b Below, we show that the same trend persists for different data sets too, e.g. when we consider CIFAR1O as ID data and SVHN as OOD data.
:0000
Unlabeled set size
0.95
1000
0.9
200
0.8
40
0.75
0.0005
0.001
0.005
0.01
0.05
0.1
0.25
0.5
0.75
0.9
Proportion of OOD in the unlabeled set
Figure 19: The AUROC of ERD the number and proportion of ID (CIFARI0) and OOD (SVHN) samples in the unlabeled set are varied.
I LEARNING CURVES FOR OTHER DATA SETS
In addition to Figure we present in this section learning curves for other data sets as well. The trend that persists throughout all figures is that the arbitrary label is learned first on the unlabeled OOD data. Choosing stopping time before the validation accuracy starts deteriorate prevents the model from fitting the arbitrary label on unlabeled ID data.
Impact of near OOD data on training ERD ensembles. The learning curves illustrated in Fig- ure provide insight into what happens when the OOD data similar to the training samples and the impact that has on training the proposed method. In particular, notice that for CIFAR10[0-4] vs CIFAR10[5-9 in Figure 20d the models require more training epochs before reaching an aceu- racy unlabeled OOD samples of 100%. The learning of the arbitrary label on the OOD samples delayed by the fact that the ID and OOD data are similar, and hence, the bias of the correctly la- beled training set has strong effect the predictions of the models the OOD inputs. Since
29

---

Under review as a conference paper at ICLR 2022
early stop when the validation accuracy starts deteriorating (e.g. at around epoch 8 in Figure 20d we end up using models that do not interpolate the arbitrary label on the OOD samples. Therefore, the ensemble does not disagree on the entirety of the OOD data in the unlabeled set, which leads to lower OOD detection performance. Importantly, however, our empirical evaluation reveals that the drop in performance for ERD ensembles is substantially smaller than what we observe for other OOD detection methods, even on near OOD data sets.
(connet latei)
Coomect lase)) (arbiraryy (obe) Validatioo (1)) label)
C foriitary
Valider (10; labels)
10
10
Epocts
Epochs
(a) SVHN; OOD = CIFARIO.
(b) ID SVHN0004: OOD = SVHN[5-9].
[cornect labels) (Uaa Q (whhiirary labef)
Conrect laoek)
Olp +) (arbittary Iobed)
(Moor 43 Carbbrary label)
(Dyee, c) (ertitrary lsbe)
Valiide (10; correct labels)
Velidatioo (20) correct label)
Epocts
Spochs
(c) ID CIFARIO; OOD SVHN.
(d) ID CIFAR10(O-4); OOD CIFAR10|5-91.
(coneet labed] (Max C torbitaar
(comect lase))
(arbitaryy lobe) Validatior (10) correct label)
Accusey
Valider (10; labels)
Accueacy
10
10
Epochs
Epochs
(c) ID CIFARIO0:: OOD = SVHN.
(f) ID = CIFAR100(0-49) OOD = CIFAR100|(0-99))
Figure 20: Accuracy measured while fine-tuning a model pretrained on S (epoch 0 indicates values obtained with the initial pretrained weights). The samples in (Uoon, c) are fit first, while the model reaches high accuracy on (UID,c) c) much later. We fine-tune for at least epoch and then early stop when the validation accuracy starts decreasing.
J EVOLUTION OF DISAGREEMENT SCORE DURING FINE-TUNING
In this section we illustrate how the distribution of the disagreement score changes during fine-tuning for ID and OOD data. Thus, we can further understand why the performance of the ERD ensembles impacted by near OOD data.
Figure 21 reveals that for far OOD data (the left column) the disagreement scores computed on OOD samples are well separated from the disagreement scores on ID data (note that disagreement on OOD data is so concentrated around the maximum value of 2 that the boxes are essentially reduced to a line segment). On the other hand, for near OOD data (the right column) there is sometimes significant overlap between the disagreement scores on ID and OOD data, which leads to the slightly lower AUROC values that we report in Table 2
The figures also illustrate how the disagreement the ID data tends to increase ensemble for longer, as a consequence of the models fitting the arbitrary labels on the unlabeled ID samples. Conversely, in most instances epoch suffices for fitting the arbitrary label the OOD data.
fine-tune the
We need to make one important remark: While in the figure present disagreement scores for the ensemble obtained after each epoch of fine-tur fine-tuning we stress that the final ERD ensemble need not be selected among these. In particular, since each model for ERD is early stopped separately,
30

---

Under review as a conference paper at ICLR 2022
-000
000
Dreggnmmentt
Epocts
(a) ID SVHN: OOD CIFARIO.
(b) ID SVHN[O-4]: OOD SVHN[5-9].
ID data -000 62t2
1D data
000 6203
Dinggamment
Epochs
EPOCTS
(c) ID = CIFARIO; OOD = SVHN.
(d) ID = CIFARI0(0-4) 4]; OOD CIFARI0|5-99 99..
1D 000
000
Otaggrement
Epochs
Spacts
(c) ID = CIFAR100; OOD SVHN.
(f) ID CIFAR CAIAARIOOO0-40 100[0-49]; OOD (00AR190/50-9))
Figure 21: The distribution of the disagrcement score measured during fine-tuning on ID and OOD data (bluc and orange boxes, respectively). The box indicates the lower and upper quartiles the distribution, while the middle line represents the median and the whiskers show the extreme values. Notice that the distributions of the scores are easier distinguish for far OOD data (left column), and tend to overlap more for near OOD settings (right column).
potentially at different iteration, it is likely that the ERD ensemble contains models fine-tuned for a different number of iterations. Since we select the ERD ensembles from a strictly larger set, the final ensemble selected by the our proposed approach will be least as good at distinguishing II) and OOD data as the best ensemble depicted in Figure 21
31


Reviews:
[{"review_id": "lCr2-0IOjq2", "reviewer": "Reviewer_7CPC", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "-The setting of the paper assumes that a mixture of in- and out-distribution samples is available during training and the authors argue for the validity of this assumption in the conclusion section of the paper. While I agree that this specific setting can come up in real-world applications, I think it is still important to evaluate unseen out-distributions. For example, if I train a model on medical images using their procedure, can the model detect \"garbage samples\", for example uniform noise, that might result from a technical failure in the image capturing pipeline. Another interesting setting might be to only train a CIFAR10 model on the first 50 CIFAR100 classes in the unlabeled pool available during training and evaluate OOD-detection performance on the last 50 classes from CIFAR100 and other standard benchmarks such as AUROC against SVHN or Uniform and Gaussian Noise. The paper clearly demonstrates that the model is able to beat baseline methods such as nnPU and MCD if the distribution of out-of-distribution samples stays the same between train and test time, however, I think it important to also evaluate robustness with respect to unseen out-of-distributions, especially as methods such as Outlier Exposure have shown that they can generalize to various unseen out-distributions. \nIt might also be interesting to replace the unlabeled set U with a general image dataset with enough variance, such as a subset of 80 million tiny images, OpenImages or YFCC-100M which all contain some samples of the standard CIFAR10/100 classes but also a vast number of unseen classes. Using such a general dataset might also lead to a better generalization to unseen out-distributions, similar to Outlier Exposure and it would allow to computer AUROC of a CIFAR10 model against all CIFAR100 classes, which is a challenging benchmark that is often looked at in related papers. \n\n-The paper shows strong improvements over a wide variety of baseline methods in terms of AUROC and TNR for several tasks based on SVHN, FMNIST, CIFAR10, CIFAR100 and a medical benchmark. \n\n-The success of the method largely relies on the models in the ensemble having sufficiently different outputs on OOD samples. The authors argue that during training, the model will first first the correctly labeled samples in U_ID before fitting the wrongly labeled ones. \nIn the appendix, the authors give further theoretical insight and state that under some assumptions, there exists a checkpoint after T iterations where the model fits the correctly labeled samples in U_ID but not the incorrectly labeled ones. In practice, this T value can not be calculated so the final checkpoint is selected based on accuracy on a validation set that only contains in-distribution samples. However, I am not sure how this relates to disagreement on actual out-of-distribution samples, which is arguably the goal of the training scheme to achieve best OOD-performance. In detail, what happens if all models in the ensemble predict the same label c on an out-distribution sample, wouldn't all models still learn this output before we end training? This systematic bias might be a problem if the OOD classes are similar to the in-distribution classes, for example train or busses might be similar to trucks in CIFAR10 and as the method does not seem to contain an explicit way to generate diverse initialization for their fine-tuning scheme. \nFigure 9 in the appendix gives hope that one has sufficient disagreement,, however, I am not totally convinced that the first 5 CIFAR10 classes are sufficiently close to the last 5 classes to completely eliminate this concern. \n\n-I liked that the authors show that the method works with varying proportions of OOD in the unlabeled dataset unless U consists of 95% in-distribution samples. \n\n-The authors also propose to use a different disagreement score and from Table 3 it seems like at least on some tasks, especially CIFAR100 vs SVHN it greatly increases results. While I value this as a contribution from the authors, it would be interesting to see (if applicable) how ensemble-based baseline methods (such as MCD) perform with this disagreement score. This could further verify that both the new disagreement score and the training scheme are important parts of ERD and the improvements are not mostly caused by the disagreement score. \n\n-The analysis in section 3.3 assumes that the label is a deterministic function of the datapoint. Typically, one assumes that p(y|x) is not deterministic and even in practice this might not always be true, for example if an image contains both a cat and a dog. Could your theory be applied to this more general setting or would your training algorithm pick up those samples as having high disagreement? ", "summary_of_the_paper": "The paper presents an ensemble-based semi-supervised learning method for novelty detection. \nThe goal of their training scheme is to create an ensemble of models that has a high disagreement on the out-of-distribution (OOD) samples in the unlabeled set. The training resembles a self-training algorithm that labels the unlabeled pool and uses implicit regularization via early-stopping to find a \"sweet spot\" in terms of disagreement on OOD samples between the models in the ensemble.\nThe final decision of whether an input is considered as out- or in-distribution is based on a hypothesis test using the average disagreement between the softmax outputs in the ensemble. In-distribution samples should be accurately classified with high confidence and show little disagreement in the ensemble while out-distribution samples should be detectable as the different models will produce different outputs. ", "summary_of_the_review": "Overall, while there are relatively few technical novelties in the paper, the given results demonstrate that this simple scheme is able to improve performance over baseline methods in their evaluation.\nFor me, generalization to unseen OOD-sets is still a very important part of any novelty detection method and should be evaluated. \nAs the method heavily relies on implicit regularization of disagreement between models in the ensemble, I think there could be more experiments in the paper that show that there is sufficient disagreement even on close OOD tasks and how this changes during fine-tuning. \nAlso, it would be nice to clearly demonstrate that the improvements over the baselines methods are caused by both the new disagreement score and the early-stopping-based training. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Wne84q6UBFV", "reviewer": "Reviewer_QNSM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n1. Strong empirical Results on Hard/Near OOD settings.\n2.Theoretical insights on the benefits of training with early stopping and the usage of artificial labels \nWeakness:\n1. The setting of this framework is too restrictive and thus makes the proposed method less practical for real life applications.  For instance, the proposed method requires a carefully selected validation set with ID samples and an unlabeled test set where half of the samples are ID and the other half are OODs. In practice, it is difficult, and somes impossible, to predefine the OOD samples and thus it is not likely to prepare such a high-quality validation set.  Also, it is necessary to investigate how the distributions of the validation set will affect the final performance. Moreover,  as the OOD samples are relatively rare to obtain, the resulting validation set could be highly imbalanced. Hence, using the validation accuracy to guide the model learning is definitely not an optimal solution. \n2. The experimental comparisons are not fair. The proposed methods are based on esnemble of 5 networks and this will linearly increase the computational complexity. Thus, it is necessary to have a more fair ``Vanilla Ensembles\" by training all the baseline method 5 times with some randmization techniques, such as random sampling on the training set (bagging).\n3. The training pipeline of the proposed method is complicated and it is not clear how will each part affect the final performance. For instance, the proposed method finetune K models, where each model will choose one different artificial label on the unlabled set. How will the choice of this artificial label affect the final performance?", "summary_of_the_paper": "The authors develop an ensemble-based procedure for semi-supervised novelty detection (SSND). It utilizes a mixture of unlabeled ID and OOD samples to perform on near OOD data. A regularization technique is further used  to promote diversity on the OOD data while preserving agreement on ID data.", "summary_of_the_review": "The proposed method have some empirical results on the Hard/Near OOD. However, this relies heavily on the carefully selected  and high-quality validation sets.  Also, the experimental comparison is not fair.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FuShoxB8u-A", "reviewer": "Reviewer_Etvi", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is generally well-written, however, there is some confusion listed as follows:\n\n1.\tHow is the ensemble number selected? What is the validation dataset for choosing it?\n2.\tSome claims are vague in the submission, for example, in section 3.3, the authors claimed that the Maximum discrepancy method (MCD) tends to result in ensembles that do not disagree enough on OOD data and thus subpar performance. Is there any proof or empirical evidence behind such claims?\n3.\tIn figure 5, is it possible that the decrease of validation accuracy is due to the fitting on the labels of the OOD data rather than the noisy in-distribution labels? One quick experiment to verify this is using clean labels for ID data and then fitting on a new OOD dataset with artificial labels to see the accuracy change on the validation dataset.\n4.\tIn figure 5, how is the early stopping tolerance scalar 7 selected? Is there any sensitivity analysis on it?\n5.\tIt is improper to claim that “However, running a grid search to select the right hyperparameters can be more computationally expensive than simply using one run of the training process to select the optimal stopping time”. Since early stopping requires you to do inference per training epoch and it may be more expensive than tuning the hyperparameters using grid search.\n6.\tHow is the proposed method compared to self-supervised-based approaches, such as CSI[1] and SSD[2]? Since these methods do not require ID labels at all, it has fewer requirements than the proposed approach. How is the method compared to different uncertainty scores, such as energy scores [3]?\n\n\n[1]Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. In Advances in Neural Information Processing Systems, pages 11839–11852, 2020\n\n[2]Vikash Sehwag, Mung Chiang, and Prateek Mittal. Ssd: A unified framework for self-supervised outlier detection. In Proceedings of the 9th International Conference on Learning Representations, pages 1–17,2021\n\n[3] Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li, Energy-based Out-of-distribution Detection, in NeurIPS 2020.\n\n", "summary_of_the_paper": "Even though conventional OOD detection algorithms can distinguish far OOD samples, current methods that can identify near OOD samples require training with labeled data that is very similar to these near OOD samples. In turn, the authors develop a new ensemble-based procedure for semi-supervised novelty detection (SSND) that only utilizes a mixture of unlabeled ID and OOD samples to achieve good detection performance on near OOD data.", "summary_of_the_review": "Some confusion exists both empirically and theoretically. I recommend weak-reject at this stage.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FUl3741Z8Qi", "reviewer": "Reviewer_VFKa", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Problem setting\n- In the introduction, the authors mentioned that we have access to the unlabeled test data that includes ID and OOD samples (and using this for training). \n- It means that we already have access to the OOD and this may not be applicable to discovering new classes in the inference time that we have never seen before.\n- OOD means that the samples from the out of the training set distribution. However, if we use some OOD samples during training, those samples are no longer OOD.\n- If we use this for a testing batch, we should retrain the model with a new testing batch. This may be not practical in some online settings. \n- It would be good to mention the limitations clearly. Like changing the title as \"transductive novelty detection instead of just novelty detection)\n\n2. Number of unlabeled samples\n- It seems like the number of unlabeled samples are also important for fine-tuning.\n- This is because if we have too small a number of unlabeled samples, it will not provide enough randomness to the ensemble.\n- It would be great if we have this additional analysis with different numbers of unlabeled data. (including very small number of unlabeled samples)\n\n3. Number of label categories\n- How about if the problem is binary classification?\n- In that case, K should be 2. (We can increase K but it does not have an impact).\n- In that case, I think the randomness that the ensemble can get is not enough.\n\n4. Validation set\n- Based on Section 3, it seems like the validation set is the subset of ID.\n- In that case, the best validation performance would be \"before\" fine-tuning. Because, before fine-tuning, we only use the ID samples to train the model.\n- With newly labeled unlabeled data, if the number of labeled categories is large, most of them are incorrectly labeled. Therefore, I think the validation error should be decreased from the beginning.\n- The authors said that they wait for 1 epoch. However, it is not sure whether this rule of thumbs can be applicable to the general datasets.\n\n5. Ablation studies\n- It would be also good if the methods can work with different K. Especially, if the label categories are small (e.g., binary classification).\n- It would be good if the methods can work with small unlabeled sets like 10, 20. \n- It would be good if the methods can work with different OOD data ratios in the unlabeled data. (Like 1%, 2% OOD and 99%, 98% ID samples in the test set)", "summary_of_the_paper": "The authors proposed a novel transductive novelty detection method using the disagreements of the ensemble models.\nMore specifically, the authors directly utilize the unlabeled test set samples, providing different labels for those unlabeled samples, and training multiple models with fine-tuning. Then, this framework tries to identify the OOD samples using the disagreement of those samples.\nThe authors provide some promising experimental results to discover the OOD samples in transductive settings.", "summary_of_the_review": "Strengths\n- The proposed method is intuitively making sense. \n- The main idea \"Introducing the randomness to multiple models using different label assignments to the unlabeled datasets\" would be promising to discover the samples which are not presented in training set.\n- The experimental results are clear and concrete. The baselines are somewhat complete and the comparisons seem fair.\n\nWeakness\n- We need another training procedure for different testing set which is less practical. \n- Experimental settings should be more diverse. More ablation studies are needed. \n- Robustness of the model training is one of the biggest concerns about this paper. Relying on the validation error for early stopping to avoid overfitting to the ID unlabeled samples seem somewhat heuristic.\n\n--------------------------------------------------\n\nI am appreciate to get the detailed responses from the authors in this rebuttal.\nI carefully read all the reviews from other reviewers and authors response.\nHowever, there are still remaining concerns.\n\n1. Practicality of the settings\n- Not only me, other reviewers also concerned on the setting of this paper.\n- Also, the authors did not answer about re-training problem with new test set. Retraining again and again for new test set would be very painful.\n\n2. Fundamental limitations of the methods\n- The robustness of the proposed method is highly related to the diversity of the submodels for the ensemble.\n- However, this diversity is significantly limited if the number of unlabeled data is small or the problem is the binary classification.\n- Even though the authors provide some interesting results with this setting, the fundamental limitations of the model itself remain concerns.\n\n3. Early stopping\n- As I mentioned above, the best validation performance would be the model \"before the fine-tuning\" because there is no distribution mismatch between training and validation. \n- It is unclear how fundamentally address this problem even after reading the rebuttals.\n\nTherefore, even though the authors provide some additional experiments (thank you), I am going to stay on my score (6) and cannot provide more than the score (6).\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6GHGtDtAo--", "reviewer": "Reviewer_iZuG", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- simple idea; easy to understand\n- a theoretical justification of the approach is provided (in the appendix) \n- good results\n\nWeaknesses:\n\n- The semi-supervised requirement of the approach limits its applicability. The availability of clean labeled data is rare in practical scenarios.\n\n- The main set of experiments assume that the unlabeled data used during training follows the same distribution of the data used for testing.\n\n- The paper does not clearly identify the limitations of the proposed approach and scenarios in which the approach would not work. \n\nThe scenarios depicted in Figures 2 and 3, while useful to convey the overall idea behind the methodology, show an ideal case only. What happens if the ID classes present a non negligible overlap, and this overlap is also reflected in the unlabeled ID data? Would the early stopping criterion still be effective to achieve the desired disagreement in the ensemble? When does the proposed technique break? Under which conditions? What happens when the OOD do not belong to specific classes with a well-defined support? What happens when there are fewer OOD data. This latter point is somewhat discussed in the appendix, but it should be covered in the main paper as well. \n\nThe data used for Figure 1 should be specified.\n\nThe authors need to make it clear in Algorithm 1 that the validation set V is made up of in-distribution data.\n\nHow was the ensemble size K=5 chosen?\n\nIt's unclear whether the values in Table 2 are averages of multiple runs, as they should be. Statistical significance should also be reported.\n\nIn Table 2, for each dataset half of the classes were used to construct the ID data, and half for the OOD data. How was the desired ratio between ID and OOD achieved? Were the classes for OOD data sub-sampled? This is not discussed in the paper.\n\n", "summary_of_the_paper": "The authors introduce a semi-supervised ensemble approach to novelty detection. The main idea consists in generating base classifiers that disagree on the out-of-distribution data (ODD). An early-stop criterion is used to achieve the wanted level of disagreement among the component of the ensemble.\n\nThe main assumption is the availability, for training, of clean labeled in-distribution (ID) data, and of unlabeled data which include both ID data and out-of-distribution (OOD) data. The unlabeled data follows the same distribution of the data the ensemble is tested on.\n\nEach base classifier is first trained on the labeled in-distribution (ID) data. For a given base classifier, the unlabeled data is randomly assigned to one of the labels (a different label is chosen for each base classifier), and the classifier is tuned on the resulting labeled data. The tuning is stopped when the optimal tradeoff between high validation accuracy (on inliers) and low training error are achieved. This regularization process is aimed at avoiding fitting the incorrect assigned labels in the unsupervised portion of the training data and to achieve disagreement among the base classifiers on the OOD data.", "summary_of_the_review": "A reasonable, simple, and interesting approach for anomaly detection with ensembles. The main weaknesses are the reliability on clean ID data and the lack of a discussion and analysis of the approach limitations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a semi-supervised novelty detection method which tries to identify out-of-distribution samples in the unlabeled data (consisting of in- and out-distribution samples) using a disagreement score of an ensemble. The ensemble is generated by fine-tuning the trained classiifer on the labeled training data plus the unlabeled data which all get a fixed label (which is repeated several times to generate the ensemble). The main idea is that one uses early stopping based on an in-distribution validation set in order to avoid overfitting on the unlabeled points which allows then identification of the out-distribution points via the disagreement score.

The reviewers appreciated the simplicity of the approach and the extensive experimental results. The authors did a good job in trying to answer all questions and concerns of the reviewers. 

However, some concerns remained:
- the setting assumes that the OOD data is fixed which was considered as partially unrealistic and thus evaluation of the OOD detection performance on unseen OOD distributions was requested in order to understand the limitations of the method (this was only partially done by the authors). 
- the theoretical result is for a two-layer network and completely based on previous work. As the authors use much deeper networks later on in the experiments, this result cannot be used to theoretically justify the approach. 
- there remained concerns about the necessary diversity of the ensemble and the early stopping procedure

While I think that the paper has its merits, it is not yet ready for publication. I encourage the authors to to take into account the above points and other remaining concerns of the reviewers in a revised version.
Decision: Reject