Learn what you can't learn: Regularized Ensembles for Transductive out-of-distribution detection
Authors: Alexandru Țifrea
Eric Petru Stavarache
Fanny Yang
Venue:  (2021)
Keywords: out-of-distribution detection
transductive
predictive uncertainty
ensembles
ensemble diversity
outlier detection

Abstract: Machine learning models are often used in practice once they achieve good generalization results on in-distribution (ID) holdout data. To predict test sets in the wild, they should detect samples they cannot predict well. We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios, e.g.\ when OOD data consists of unseen classes or corrupted measurements. This paper studies how such ``hard'' OOD scenarios can benefit from tuning the detection method after observing a batch of the test data. This \emph{transductive} setting is relevant when the advantage of even a slightly delayed OOD detection outweighs the financial cost for additional tuning. We propose a novel method that uses an artificial labeling scheme for the test data and early stopping regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch. We show via comprehensive experiments that our approach is indeed able to significantly outperform both inductive and transductive baselines on difficult OOD detection scenarios, such as unseen classes on CIFAR-10/CIFAR-100, severe corruptions (CIFAR-C), and strong covariate shift ImageNet vs ObjectNet.

Under review as a conference paper at ICLR 2021
LEARN WHAT YOU CAN'T LEARN: REGULARIZED ENSEMBLES FOR TRANSDUCTIVE OUT-OF-DISTRIBUTION DETECTION
Anonymous authors
Paper under double-bling brind review
ABSTRACT
Machine learning models are often used in practice once they achieve good gen- eralization results in-distribution (ID) holdout data. To predict test sets in the wild, they should detect samples they cannot predict well. We show that current out-of-distribution (OOD) detection algorithms for neural networks produce satisfactory results in a variety of OOD detection scenarios, c.g. when OOD data consists of unseen classes corrupted measurements. This paper studies how such "hard" OOD scenarios can benefit from tuning the detection method after observing batch of the test data. This transductive setting is relevant when the advantage of even a slightly delayed OOD detection outweighs the financial cost for additional tuning. We propose a novel method that uses an artificial labeling scheme for the test data and early stopping regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch. We show via comprehensive experiments that our approach is indeed able to significantly outperform both inductive and transductive baselines on difficult se-
OOD detection scenarios, such as unseen classes on SEFAR-IO/CIFAR vere corruptions (CIFAR-C), and strong covariate shift ImageNet vs ObjectNet.
INTRODUCTION
Modern machine learning (MI.) systems can achieve good test set performance and are gaining popularity in many real-world applications from aiding medical diagnosis (Beede al. 2020) to making recommendations for the justice system (Angwin et al. (016)_ In reality however, some of the data points in a test set could come from a different distribution than the training (in-distribution) data. For example, sampling biases can lead to spurious correlations in the training set (Sagawa et al.) 2020). faulty sensor can produce novel data corruptions (Lu et al, 2019), or new unseen classes emerge over time, like undiscovered bacteria (Ren al. 2019). Many of these samples are so different compared to the training distribution that the model does not have enough information to predict their labels but still outputs predictions with high confidence. is important to identify these out-of- titto--distrbbttinn (OOD) samples in the test set and flag them, for example to at least temporarily abstain from prediction (Geifman & El- Yaniv 2017) and involve human in the loop.
To achieve this, Bayesian methods (Gal & Ghahramani 2016 Malinin & Gales 2018) or alterna- tives such Deep Ensembles (Lakshminarayanan et al. 2017) try identify samples on which given model cannot predict reliably and include. Their aim is to obtain predictive models that si- multaneously have low еттоr on in-distribution (ID) data and perform well on OOD detection. Other approaches try to identify samples with low probability under the training distribution, independent of any prediction model, and use, for instance, density estimation (Nalisnick et al. 2019 or statistics of the intermediate layers of neural network (Lee et al. 2018).
Most prior work have reported good OOD detection performance, reaching an almost perfect area under the ROC curve (AUROC) value of nearly 1. However these settings generally consider differ- entiating two vastly different data sets such as SVHN vs CIFAR1O. We show that the picture is very different in lot of other relevant settings. Specifically, for unseen classes within CIFAR10 or for data with strong distribution shifts (e.g. (resized) ImageNet vs ObjectNet (Barbu et al. 2019)), the AUROC of tate-of-the-ar methods often drops below 0.8.

---

Under review as a conference paper at ICLR 2021
Training sample; a Test sample O
ID support; ?
Test set
RETO
Max. Clasif, Discrep.
Mahelenoba Vanilla Ensembles
AUROC
0.9
AUROC gap
0.8
0.7
T000 I
Transductive OOD detection setting
OOD detection hardness
Figure 1: Left: Transductive OOD detection setting. Labeled training set and an unlabeled test set with samples both in- and outside the support of the training distribution. Right: Performance of RETO and some baselines data sets ranked by their difficulty (Appendix [ contains details on the hardness metric). The shaded area represents the gap area under ROC curve (AUROC) between RETO and next best baselines. The gap is wider for hard OOD detection settings.
Almost all of these methods assume setting where at test time, no training is possible and the OOD detection method can only be trained beforehand. This inductive setting allows real-time decision-making and is hence more broadly used. However, in many cases we can indeed do batch predictions, for example when sensor readings come in every second and it is sufficient to make prediction and decision every few minutes (e.g. automatic irrigation system). In this case we have a batch of unlabeled test data available that we want to predict (and be warned about) that we can use together with the labeled training set to detect the OOD points in the set. We call this the transductive OOD setting (related but quite different from transductive classification (Vapnik 1998)). Even in an online setting, transductive OOD could be very useful (see Section 2.1
(How) Can we achieve significantly belter OOD detection performance in the transductive setting?
Even though the transductive setting improves test accuracy in small data settings for tasks such as classification zero-shot learning. it is unclear how to successfully leverage simultaneous availabil- ity of training and test set in the transductive OOD setting which quite distinct from the former problems. A concurrent recent work Yu & Aizawa 2019) tackles this challenge by encouraging two classifiers to maximally disagree on the test set (i.e. to produce different predictions on test samples). However this leads models that disagree a similar degree both ID and OOD data and hence one cannot distinguish between the two, as indicated by the low AUROC in Figurel I We introduce a new method called Regularized Ensembles for Transductive OOD detection (RETO) for overparameterized models, which heavily uses regularization to make sure that the ensemble dis- agrees only on the 0OD samples in the test set, not on the ID samples. In summary, our main contributions in this paper are as follows:
We experimentally identify many realistic OOD scenarios where SOTA methods achieve subpar AUROC below 0.84. We hence argue that the field of OOD detection far from satisfactorily solved and more methods will be proposed that include these (or other) hard OOD cases as benchmarks.
For the transductive OOD detection setting, we propose a new procedure, RETO, that man- ages to diversify the output of ensemble only the OOD portion of the test set and hence achieves significant improvements compared to SOTA methods (see Figure T with a relative gain of least 32%.
REGULARIZED ENSEMBLES FOR TRANSDUCTIVE OOD DETECTION
Our main goal is to detect samples that are outside of the training distribution and focus on classifi- cation tasks. We are only interested in situations where we can obtain model that generalizes well given the training data. If the models do not generalize well in-distribution (ID), then the primary task should be to find better classifier instead. Given classifier with good generalization, the next challenge becomes to ensure that samples on which the model cannot make confident predictions (e.g. samples that too far from the training data) correctly identified. This constitutes the
2

---

Under review as a conference paper at ICLR 2021
Training sample:
Test sample arbitrary label;
Predicted label; Identified OOD
Model
Model
RETO ensemble
Figure 2: Cartoon illustration of ensemble of two linear classifiers that fit both the training set and the test We assign either the blue label (Model I) the red label (Model 2) the whole test set. Linear classifiers are smooth enough
they cannot fit both the correct labels the training set test ID samples. The classifiers disagree only on the OOD points the test set (gray crosses) and agree to predict the correct label on the test ID samples.
arbitrary label
main focus of our work. Recall that in our use case, we have batch of unlabeled test samples at our disposal. This test set includes a mixture of samples drawn from the training distribution and samples we call OOD per our definition in the previous section. The goal is to distinguish between the ID and the OOD samples in the test set.
In this section we propose our method that uses the more numerous training data as counterweight that does not allow sufficiently smooth model fit an arbitrary label on ID test samples, but only on OOD test samples.
2.1 TRANSDUCTIVE OOD DETECTION
In inductive OOD detection setting, one can only tune a method training time, and then with unchanged parameters on any test set. In contrast, in transductive setting, the training data available during test time and it is possible to tune method using both the training set and the unlabeled test set. We stress that no labels are available for the test data, so it is unknown which test samples are indeed anomalous. Moreover, we not assume access to any known OOD samples, unlike some of the inductive methods, which sometimes use OOD data for training or calibration
(Lee et al. 2018 Liang et al. 2018 Malinin Gales, 2018 Cao et al. 2020
When deployed in the context of classification, transductive and semi-supervised learning methods leverage the unlabeled data to obtain low-dimensional representations that are more effective for the prediction task. A key assumption for the setting to be useful is that the data is related in some way, c.g. the unlabeled data comes from the same distribution as the labeled data. On the other hand, transductive OOD detection differs from the usual transductive classification setting, in that the training distribution does not carry information about the OOD data. As a consequence, is not obvious how to adapt existing semi-supervised methods work in this different regime.
Some of the downsides that prevent transductive classification methods from being used more to predict, would need have access to the
broadly that for each test that training data and computational resources. Furthermore, they do not allow predictions on the fly in the online setting. For transductive OOD however, we know that the inductive model predicts reliably in ai-disrribution. Hence we can still predict test points on the fly, and only flag OOD samples with slight delay after receiving batch of test points. example for which all these downsides are not limiting should be quite relatable to the reader. For example, Covid-19 19 test results have crucial role for controlling the spread of the virus. Imagine a machine learning model were to be developed and deployed for reliable fast testing that works well under usual cireumstances. If test pipeline becomes defect, informing the patient of the potentially wrong test result is still crucial, in particular if it is inform negatively tested patient repeat the test to quarantine. In this we would also be willing to allow access to labeled training data and computational resources for fine-tuning precision of utmost importance.
3

---

Under review as a conference paper at ICLR 2021
2.2 THE COMPLETE RETO PROCEDURE
We now provide details on our approach, RETO, outlined in Algorithm
Recall that have access to both labeled training set, and the unlabeled test set. We begin by assigning an arbitrary label (selected from the set of labels of the training data) to all the test samples. We train a classifier on the union of the vorrectly-labeles training set, and the arbitrarily-labeled test set. To find the optimal classifier, we search among functions that are known to generalize well on the ID (training) distribution. If the classifiers are smooth enough, they will not be able to fit both the correct labels of the training set and the arbitrary label on the ID test samples, as illustrated in Figure 2 for linear classifiers. However, they will still fit the arbitrary label on the OOD test samples. Using regularization, we ensure that the models we obtain are not too complex. We search inside a function class of regularized functions, Freg. as discussed in more detail Section 3 We ensemble several such classifiers, where each model fits different label to the test set. We then use a disagreement statistic and flag as OOD all the points in the test set with high disagreement. To avoid training the ensemble from scratch for each new test batch, it is possible to instead start from pre-trained weights and perform a few iterations of fine-tuning, as detailed in Section 4 that we do not calibrate or train our method on any known OOD data.
We stress
Determining OOD samples with RETO. We distinguish between II) and OOD samples using a two-sample statistical test, with the null hypoth- esis: lo x E suppp, for a test sample x and where P denotes the training distribution"" Pre- vious baselines have proposed their own choices of the test statistic, which is discussed in detail in Appendix For RETO, we use:
Algorithm 1: Pseudocode for RETO
Input:
Train set S. Test set T', Ensemble size K,
Test statistic threshold to. Regularized function class Figg. Disagreement metric
Result: O, i.e. the elements of 7 which are 00D
for 4 (U1,-.., yk do // train K models
T + ((z, : : 7}
[
fc + T--arin (5 Frg)
TaN Ting (I) = K(K
Σ drv (L())),()))), i
== 0
for E T do // run two-sample test if disagreement (Fr((t........ Sox lux (1)) > to
the average pairwise total variation distance be- tween the softmax outputs fx(x), E R| of models j {I,..,,K] in the ensemble, where drv is the total variation distance. The null hy- pothesis is rejected for high values of Tv. Ap- pendix[[]contaii] more details about the choice of the test statistic. It follows from the way in which the hypothesis test is stated that true positives are OOD samples that are indeed flagged as OOD, while the false positives are ID samples that are incorrectly predicted as OOD.
then
L
return O
3 WHY RETO CAN WORK WELL
In this section we provide insights as to why RETO can achieve a higher AUROC for hard OOD settings. We frame RETO as a way of learning diverse ensembles. We show that for RETO to perform well, the members of the ensemble must come from a restricted family of function classes. Finally, we argue that early-stopped neural networks are part of this restricted family.
Ensemble- OOD detection The main argument for using an ensemble for OOD detection is that diverse enough set of models will lead to disagreement only on OOD samples. In other words, the models will produce contradictory predictions on OOD inputs, while giving similar predictions for ID data. In order to get diverse set of models, Deep Ensembles (Lakshminarayanan use the stochasticity of the training process. However, the models obtained with this procedure are still very similar and tend to agree on OOD data (Figure 3, especially in hard OOD detection scenarios. Our method uses the additional information available in the unlabeled test set to generate ensembles that are more diverse on the OOD test samples. This approach allows the ensembles to work well even when the ID and OOD data are very similar.
al. 2017)
We denote the test set as T and consider a partition of it into (unknown) test IDD set TID and a (unknown) test OOD set TooD. Recall that we do not know which test samples are outliers: if we
With a slight abuse of notation, we denote by suppy- C X the support of the marginal distribution Px.
4

---

Under review as a conference paper at ICLR 2021
Training sample;
8/ta Predicted label; * Identified as OOD
Vanilla ensemble
RETO -malll func, class
RETO large func. class
Figure 3: Comparison between different kinds of ensembles used for OOD detection. Left: A vanilla ensem- ble will not diverse enough outside training distribution support. Middle: In comparison. the RETO classifiers disagree only on OOD points in the test set (gray crosses) and agree to predict the correct label on the test ID samples. Right: However, if the models in the ensemble are too complex (i.e. not regularized enough), then they can easily fit the arbitrary lahel on the ID test samples as well, thus identifying the whole set OOD.
given T TooD. could just enforce that different models learn different labels T by simply assigning one of the labels c E ) to the whole test set to obtain 7 := {(ti, c) 24 E T), and then training each model in the ensemble given the union of the test ID and test OOD set T = TooD U UiD, without being able to distinguish between the two.
S U T for different But, obviously,
How
encourage an ensemble disagree (only) TooD?
Could we use the same strategy, and assign arbitrary label to the entire test set? A neural network can easily learn random labels (Zhang et al. 2016). Hence, we train a neural network to conver- gence S U T the models will also disagree the test ID data! This is illustrated in Figure Right. How can we enforce models to have contradictory predictions on TooD but to agree on TID?
3.1 KEY FOR TRANSDUCTIVE OOD DETECTION: REGULARIZATION
We remedy this issue with strong regularization of the models in the ensemble. The key intuition is that it is difficutt to fit an arbitrary label on ID data that is near the training samples. The signal in the correctly labeled ID points from the training set prevents an arbitrary label from being easily fit on the ID test samples. This is illustrated in Figure 2 for linear classifiers. Conversely, it is easy to learn the arbitrary label on samples that are far enough from the training data, which are exactly the OOD test samples! For instance, when training neural networks with SGD, the arbitrary label will be fit much faster on the OOD test samples than on the test samples.
What is the right complexity for the models the ensemble?
In the language of statistical testing, the model complexity should be small enough to limit false positives (i.e. ID samples incorrectly flagged as OOD) and large enough to have enough power (i.e. identify correctly OOD samples). In other words, we want the classifiers not fit the wrong labels on test ID samples, but only on test OOD samples. We encourage high power by making the models fit different labels on the test set, and reduce false positives by regularizing the function class just enough. Specifically, we constrain our search to a hypothesis class that is just able to learn the labels on OOD, but not on test samples.
Controlling false positives We now deseribe necessary condition the complexity of the model class, in order to control the false positive rate RETO. First of all, we require our models to generalize well in-distribution, since they need to have enough common ground to agree the test ID samples. If model has poor generalization, then we should first find a better classifier before concerning ourselves with flagging OOD samples which they cannot predict well.
However there may be model classes which generalize well but are still able to fit arbitrary labels on ID. Hence, we consider the of regularized functions with low population error, i.c. F² := {f Frg E [[][)]=]] < <). where Frgg is restricted function class (e.g. functions parametrized by neural networks regularized via early-stopping). The required amount of regularization is captured in the following condition.
5

---

Under review as a conference paper at ICLR 2021
Condition 3.1 (FF cannot fit noisy labels). The probability of point drawn from the ID distribution to be misclassified by a function in F² is at most 6 for small constant >>
In words, the functions in F only disagree on small set with respect to the marginal distribution
of X. As consequence, with very high probability (1 8) we cannot fit set of s random
i.idd in-distribution points with the wrong label. This means the functions in F are smooth enough
to "ignore" wrong labels on in-distribution samples, as illustrated in Figure 3 Middle. Note that unrestricted overparameterized models in the large function class F do not satisfy Condition 3. due to their capability to fit random labels (Figure Right).
Notes on power Note that Condi- tion 3.11B.JJooooo enforces ensemble agreement ID points to limit the amount of false pos- itives. The power (ensemble disagree- ment on OOD) depends very much on the the support of the OOD samples and its relation to Ft (see Figure 4). If the boundary between the OOD and ID set requires higher complexity than Fü, our model class might have too little power as illustrated in Figure
Training sample; a/a
Predicted label;
Identified OOD
Coreactty
The empirical OOD detection perfor- mance of our method indicates that early- stopping regularization finds just the right complexity models that are complex enough for many hard OOD detection problems in image classification tasks that we consider. However, we leave as future work thorough analysis of the trade-off between the function class size and the detection capabilities of RETO.
Fot. class too small
Right fct. class size
Figure 4: Importance of the right amount of regularization for power. Left: If the model class is too small, then we cannot fit the arbitrary label on test OOD points, and the classifiers will agree them, leading to low statistical power. Right: In this situation, the OOD samples can only be detected using larger function class.
3.2 REGULARIZING NEURAL NETWORKS WITH EARLY STOPPING
An example of models that satisfy Condition 3.1 are deep neural networks trained with early stop- ping. The recent results of Yilmaz & Heckel (2019; Arora et al. (2019); Li et al. (2020) suggest that early stopping helps neural networks be more robust to label noise without sacrificing standard accuracy, thus satisfying Condition 3.1 Figure shows the learning curves obtained when fitting a neural network S, Tip and TooD for chosen label the training and test OOD samples fit first and after epoch 50, the predictor starts fitting the wrong label on the test ID set as well. We can also observe that early stopping at the point with the highest accuracy on a validation set (drawn from the same distribution as the training set), captures this phase transition well.
Train (correct labels)
Training accuracy
(arbitrary label)
Test 00D (arbitrary label)
Valication (ID; correct labels)
20 40
60
100
Epochs
Figure 5: Accuracy measured on the correctly-labeled training set and on the arbitraril-llbbled test ID and OOD subsets. Validation accuracy is computed on hold-out set with correctly-labeled ID samples. The training set and the test OOD samples are fit first, while the test ID set reaches high accuracy much later. At the early stopping iteration (after around 50 epochs) the models tend predict the arbitrary label for test OOD samples and the correct label for test ID. The model is Resnet20 trained CIFARIO as ID and SVHN as OOD.

---

Under review as a conference paper at ICLR 2021
4 EXPERIMENTS
In this section we evaluate the OOD detection performance of RETO for deep neural networks on several image data sets, We find that our approach outperforms all baselines on difficult OOD detection settings. In addition, we provide insights into the trade-off between offline detection and the good performance of our algorithm.
4.1 ID VS OOD SETTINGS
We report results on two broad types of OOD detection scenarios:
1. Easy OOD data (most previous benchmarks): ID and OOD samples come from very differ- distributions (e-g. CIFARIO SVHN). These are the settings usually considered in the OOD detection literature on which most baselines perform well.
2. Hard OOD data: We explore two types of more difficult OOD detection tasks (i) The OOD data sampled from "novel" classes: e.g. the first digits of SVHN as training, the last 5 digits OOD). (ii) The test data suffers from semi-strong covariate shift: e.g. the test set contains corrupted samples from the training distribution (e.g. CIFARIO vs CIFARIO-C (Hendrycks & Dietterich 2019 2 or samples that violate the spurious correlations present in the training set (e.g. ImageNet vs ObjectNet Barbu al. 2019 3.
Appendix C provides more insight on OOD detection hardness, while Appendix B presents exam- not too interested in the practical scenario of covariate shift (Shimodairal 2000). where the distributions are so close that domain adaptation techniques could perform well In our hard OOD data sets, domain adaptation also leads to unsar- isfactory results. For instance, Sun et al. (2020) obtain a classification error of 20.4% on CIFARIOG severity compared the 8.3% error achieved on the CIFARIO 10 test set. Alternatively, when do- main adaptation fails, OOD detection can prompt a system to abstain on samples from the shifted distribution to prevent erroneous predictions.
ples of images for the various settings. Note that
Apart from using these canonical data sets, we also compare the performance of our method on more realistic data, namely recently proposed OOD detection benchmark for medical imaging (Cao et al.) 2020). The authors collected suite of data sets that cover the aforementioned categories of difficulty, as detailed in Appendix K
4.2 RETO vs. BASELINES
We compare our method against both inductive and transductive baselines. Importantly, some of the baselines require oracle knowledge of OOD data for training. For example, Outlier Exposure Hendrycks et al.) 2019) uses Tiny Images for training as the set of outliers, irrespective the OOD set used for evaluation. On the other hand, the Mahalanobis baseline (Lee et al. 2018)) is tuned on samples from the same OOD distribution as the one seen at test time. We also present a transductive version of this approach, referred to Mahalanobis-T. T, on which we elaborate in Appendix A
For all the baselines we, use the default hyperparameters suggested by their authors and we do not adjust RETO for any of the OOD settings. We defer the details regarding training the models to Appendix A For evaluation, we use two metrics that are common in the OOD detection literature: the area under the ROC curve (AUROC; larger values are better) and the false positive rate (FPR) at a true positive rate of 95% (FPR @95; smaller values are better).
4.3 MAIN RESULTS
For our method we train ensembles of five ResNet20 (He et al., 2016) networks (results for other architectures are presented in Appendix G). For each model in the ensemble we perform post-hoc
Both CIFARIO-C and CIFAR100-C contain types corruptions, at severity levels. We consider corrupted samples with severity 5.
'ObjectNet (Barbu et al. 2019) contains both novel classes that do not appear in ImageNet, and images from ImageNet classes, with strong distribution shift. We resize both ImageNet and ObjectNet 32x32 images.
'These situations when domain adaptation performs well, are sometimes more challenging for OOD detec- tion, since it means that ID and OOD data more similar. In Appendix G.T we show that RETO maintains its remarkable performance even the more difficult CIFAR 10-C data with severity

---

Under review as a conference paper at ICLR 2021
Table 1: Main results. We report the AUROC and the #5. The best RETO and best baseline metries are highlighted for each setting. For the corrupted CIFAR data sets, we report the average and the worst-casc values, over all corruption types. For all instances, we used a training set of 40,000 labeled samples and a test set of size 20,000 with an equal number of ID and OOD samples.
ODD
DPN
Vanilla Ensembles Inductive
Mahal.
Mihol-T
MCD
RETO (rand init)
RETO (peetrannd)
Transductive
AUROC 1 /EPR0995 L
SVHN
CIFARIO
092/0.32
1.00
0.97 /.112
1.00
0.99/0.02
0:99/000
0.97 /0.15
1.00/
0.9977..5
CIFARLO
SVIIN
081/0,74
0.95/0.15
0.92/0.22
0.97/0.11
0,99/0.01
0.99/000 /.09/00044
1.002..02
1 0.00
1.00/0.00
CI
SVHN
083/0,71
0.77/0.55
0.84/0.52
0.82/0.50
0,98/0.10
0.98
00.2
1.00/ 0000
1.00/0.00
SVHNNO0:1
SVENN[5:9
0.54/0.92
0.87/0.81 0.81
0.92/0.31
0.85 0.45
0.22/0.29
0.91
0.91 /0.19
0.96/
0.94/0.34
CIFAR1I0.41
CIFARI05-9
0.59/0886
0.82
0.80/0.0
0L82
0.79/0.73
0.64/087
0.09
0.11/
0.91 /0.34
CIFAR10000491
CHBAR100550591
0.1/00.6
0.7070.74
0.65
0.74/
0.72/ 0.90
0.72/081
0.70/0.74
0.81 /0.60
CI
CIFAR I0-C. (wvg)
067/0.72
0.86 0.84/0.51 00,40
0.94/0.30
0.88/0.37
0.95/0..1
1.00/0000
1.00
CIFARIO
CIFARIO-C (wund)
0.51/0.64
0.72/0.90
0.60/ 0.90
0.63/0.89
0,78/0.73
0.68/0088
0.60/0.92
1.00 0.00
0.98 1.98/0.14
CIFARI00
CIFARIOG 100-C (aug)
067/0..3
0.7470.64
0.78/0.63
0.76/ 00.63
0.22/
0.84/0445
0.91 /0.35
1.00/0.01
0.99 0.003
CIFARIDO
CIFAR 100-C (wors))
/0,94
/0.88
0.64/0.66
0.62/0.87
0,71/
0.63/087
0.60/0.90
0.58 0.11
0..29
Tiny ImageNet
Tiny ObjectNee
0.51
0.76/
0.82/0.51
0,79 /063
0.75/ 0.74
0.72/0.75
0,99
0.99/
0.98.70.12 7.112
Average
0.67/0.77
0,,52
0.850045
0.84/
0.89
0.85/0042
0.90/...3
0.98 / 0.10
0.16/
early stopping:
we train each model for 100 epochs and select the iteration with the lowest validation loss. For all settings, we used a labeled training set of 40,000 samples, a validation set of 10,000 ID samples and an unlabeled test set of 10,000 ID samples and 10,000 OOD samples. We present results for training the models from random initializations, and for fine-tuning pretrained models (pretraining is always performed the training set for 100 epochs). When using pretrained weights, as few as three epochs of fine-tuning are enough on average to achieve the performance that we report, which is significant cut in computation cost. In addition, Appendix D the dependence on the ensemble size for RETO and vanilla ensemble.
Table D summarizes the main empirical results. For the corruption data sets, the table shows the average of the AUROC and FPR @95 taken over all corruptions, and the value for the worst-case setting. Appendix G contains a more detailed breakdown of these numbers. In addition to being successful at identifying OOD samples, our method also maintains a good classification performance on the training distribution. The validation accuracy of the early stopped ensembles, averaged over all settings, is only 1.4% smaller than that of vanilla ensembles.
The evaluation for the scenarios presented in Table Tis performed on the same test set that was used for training, as usual in transductive learning. In addition to that, the OOD detection performance of RETO extrapolates well to unseen samples from the same distribution 5 In order to show this, experiments in which we compute the AUROC ID and OOD distributions as the one used during training. The AUROC on the hold-out test set is within 0.01 from the one calculated on the test set observed during training.
hold-out test set drawn from the same
For the medical OOD detection benchmark we present the average AUROC achieved by some rep- resentative baselines in Figure 6al We refer the reader to Cao et al. (2020) for precise details on the methods. Appendix K contains more results for the medical settings, as well as additional baselines.
Limitations and trade-offs. Having access to the test set in the transductive setting provides enough information to discriminate well between the ID and the OOD test samples, succeeding when inductive approaches are less effective. In order to bridge the gap between (offline) transduc- tive OOD detection and online anomaly detection we investigate the impact of the size of the test set the OOD detection performance. In addition, we also vary the ratio of OOD samples in the [Toopl Our findings suggest that there is a broad spectrum of values for which RETO maintains a good performance. In the cases when either the size of the test set or the test OOD ratio is small, the OOD detection performance deteriorates to the point where it is comparable to vanilla ensembles, as shown in Figure where we report the gap in AUROC between RETO and vanilla ensemble. When there are only a few very diverse OOD test samples, their contribution to the gradient is small. Moreover, if the number of ID test samples is large, fitting a single arbitrary
test set, i.e. Tiol+17oom
TThis setting is similar for instance to the one in the Mahalanobis baseline, which assumes oracle knowledge of the OOD distribution at training time.
8

---

Under review as a conference paper at ICLR 2021
Racoraa OCE
2005
BETO quad
Test
-09
8775
0005
0.0
0.05
0.25
62
(a) Comparison of RETO with baselines on medical
Praporion on of CO0 test
OOD benchmark
(b) Effect of OOD proportion detection
Figure 6: Left: AUROC averaged over all scenarios the medical OOD detection benchmark. The values for the baselines are computed using code from the paper(Cao et (2020). Right: The gap AUROC between RETO and vanilla ensemble, as the number and proportion of ID (CIFAR10) and OOD (CIFAR 10-C/snow) samples in the test set varied. The AUROCs obtained with an ensemble of ResNer20 models.
label on them is significantly easier. This, combined with the signal in the OOD test data being weaker, means that the arbitrary label can take longer to on the few OOD samples, than on the (numerous) test ID samples, leading to many false positives (i.e. ID samples incorrectly flagged as OOD). The loss in efficacy can be mitigated by either splitting the test set in smaller batches, or by using a different labeling scheme for the test set, the details of which we leave as future work.
RELATED WORK
Transductive learning. Transductive learning [Vapnik] 1998) has been successfully used for prac- tical applications in problems like zero-shot learning (Fu et al. 2015 Kodirov et al. 2015) Wan et al.) 2019). Scott& & Blanchard (2008) proposed to solve the transductive anomaly detection problem sim- ply by discriminating between the ID and OOD distribution with a constraint on the false positive rate. However, is difficult assess the predictive uncertainty of classifier trained on the source set, using this binary classifier. The setting we consider in which one has access to both labeled and an unlabeled data set, reminiscent of the problem of semi-supervised learning. Approaches based on self- trained predictors have recently been proposed by Kumar et al. (2020): Chen et al. 2020
Sun al. (2020) in the context of domain adaptation.
Ensemble diversity. Some limitations of ensemble-based OOD detection approaches have been highlighted in recent years. First, neural networks tend to make overconfident predictions even on test samples far from the training distribution (Hein al. 2019). Moreover, the stochasticity of conventional NN training approaches (e.g. random initialization, SGD) is not sufficient to obtain ensembles that are diverse enough to give good uncertainty estimates on OOD samples (Maennel & Tifrea 2020). Some recent works address this problem by adding explicit regularizers that incen- tivize model diversity, either a transductive setting, like MCD (Yu & Aizawa (0119). or inductively (Bahng et al. 2020), but do not manage detect OOD samples well in hard scenarios.
Bayesian prediction. One of the important appeals of the Bayesian framework is that it directly provides uncertainty estimates together with the predictions, in the form of a posterior distribu- tion. Approaches like MC-Dropout (Gal & Ghahramani] 2016) or Deep Prior Networks (Malinin & Gales 2018) have been proposed the context of OOD detection, but the uncertainty estimates they provide are often inaccurate OOD samples (Ovadia et al. 2019). Even though Bayesian Neu- ral Networks (Neal 1996) have seen some progress in recent years (Graves) [01T Blundell et al.) 2015). sampling efficiently from the posterior over parameters remains important open problem.
CONCLUSIONS
Reliable OOD detection is essential in order for classification systems to be deployed in safety- critical environments. We present a method that achieves state-of-the-an performance in a transduc- tive OOD detection setting, and which, like other approaches, can ultimately be used to abstain on OOD samples. As future work, we propose a more thorough investigation of the influence of the labeling scheme of the test set on the sample complexity of the method, as well as an analysis the trade-off governed by the complexity of the model class of the classifiers.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias: There's software used across the country to predict future criminals. and it's biased against blacks. 2016.
Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, and Ruosong Wang. Fine- Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. volume 97 of Proceedings of Machine Learning Research, pp. 322-332, 2019.
Hyojin Bahng. Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In Proceedings of the 37th International Conference on Machine Learning, 2020.
Andrei Barbu, David Mayo, Julian Alverio, William Luo. Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems 32, pp. 9453-9463. 2019.
Emma Beede, Elizabeth Baylor, Fred Hersch, Anna Iurchenko, Lauren Wilcox, Paisan Ruamviboon- suk, and Laura M. Vardoulakis. A human-centered evaluation of deep learning system deployed in clinics for the detection of diabetic retinopathy. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1-12, 2020.
Charles Blundell. Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. In Proceedings of the 32th International Conference on Machine Learning, 2015.
Tianshi Cao, Chin- Wei Huang, David Yu-Tung Hui, and Joseph Paul Cohen. A benchmark of medical out of distribution detection. arXiv preprint arXiv:2007.04255 2020.
Beidi Chen, Weiyang Liu, Zhiding Yu, Jan Kautz, Anshumali Shrivastava, Animesh Garg, and An- ima Anandkumar. Angular visual hardness. arXiv preprint arXiv: 1912v279,,.02279, 2019.
Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. Self-training avoids using spurious features under domain shift. arXiv preprint arXiv:2006. 10032, 2020.
Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2009.
Yanwei Fu, Timothy M. Hospedales, Tao Xiang, and Shaogang Gong. Transductive multi- zero- shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 211233--2343 2015.
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. volume 48 of Proceedings of Machine Learning Research, pp- 1050-1059, 2016.
Yonatan Geifman and Ran Yaniv.. Selective classification for deep neural networks. In Advances in Neural Information Processing Systems 30, pp. 4878-4888 2017.
Alex Graves. Practical variational inference for neural networks. In Advances in Neural Information Processing Systems 24, pp. 2348-2356. 201
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why ReLU networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common ruptions and perturbations. In Proceedings of the International Conference on Learning Repre- sentations, 2019.
10

---

Under review as a conference paper at ICLR 2021
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In Proceedings of the International Conference on Learning Representations, 2019. 2019.
Elyor Kodirov, Tao Xiang, Zhenyong Fu, and Shaogang Gong. Unsupervised domain adaptation for zero-shot learning. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2452-2460, 2015.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. arXiv preprint arXiv:2002. / 1361, 2020.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 6402-6413. Curran Associates, Inc., 2017.
Yann Lecun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, pp. 2278-2324 1998.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Process- ing Systems 3/, pp. 7167-7177. 2018.
Mingchen Li, Mahdi Soltanolkotabi, and Samet Oymak. Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. Proceedings of Machine Learning Research, pp. 4313-4324, 2020.
Shiyu Liang Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In Proceedings of the International Conference Learning Repre- sentations, 2018.
Alex X. Lu, Amy X. Lu, Wiebke Schormann, David W. Andrews, and Alan M. Moses. The cells of sample (COOS) dataset and benchmarks for measuring out-of-sample generalization of image classifiers. arXiv preprint
2019.
Hartmut Maennel and Alexandru Tifrea. Fourier networks for uncertainty estimates and out-of- distribution detection. 2020.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems 32, pp. 7047-7058, 2018.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don't know? In Proceedings of the International Conference on Learning Representations, 2019.
Radford M. Neal. Bayesian Learning for Neural Networks. Springer-Vertee Verlag, 1996.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances Neural Information Processing Systems 32, pp. 13991 14002. 2019.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize to cifar-10? 2018. Inttps://arxivv http://wwor//bb/////eeee 00451
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do ImageNet classifiers generalize to ImageNet?, 2019.
11

---

Under review as a conference paper at ICLR 2021
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for -usto--distriuution detection. In Advances in Neural Information Processing Systems 32, pp. 14707-14718. 2019.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In Proceedings of the International Conference on Learming Representations, 2020.
Clayton Scott and Gilles Blanchard. Transductive anomaly detection. Technical report, 2008.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of Statistical Planning and Inference, 90:227-244, 2000.
Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via informa- tion. arXiv preprint arXiv:1703.00811 2017.
Yu Sun, Xiaolong Wang, Liu Zhuang, John Miller, Moritz Hardt, and Alexei A. Efros. Test-time training with self-supervision for generalization under distribution shifts. In Proceedings of the 37th International Conference on Machine Learning, 2020.
Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11):1958-11 1970888-99900110 2008
Vladimir N. Vapnik. Statistical Learning Theory. Wiley-Interscience, 1998.
Ziyu Wan, Dongdong Chen, Yan Li, Xingguang Yan, Junge Zhang, Yizhou Yu, and Jing Liao. Transductive zero-shot learning with visual structure constraint. In Advances in Neural Informa- tion Processing Systems 32, pp. 9972-9982, 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST. A novel image dataset for bench- marking machine learning algorithms, 2017.
Fatih Furkan Yilmaz and Reinhard Heckel. Image recognition from raw labels collected without annotators. arXiv preprint arXiv:1910.09055, 2019.
Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distributin detection by maximum classifier discrepancy. Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference (BMVC), 2016.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint
2016.
12

---

Under review as a conference paper at ICLR 2021
A EXPERIMENT DETAILS
A.1 BASELINES
We instantiate all baselines with the hyperparameters suggested by the authors for the respective settings (e.g. different hyperparameters for CIFARIO or ImageNet). For all methods, we use pre- trained models provided by the authors when available and we pre-train our own models when that not the case. When doing our own pre-training, we always use the parameters described in the original paper. The code published for the Mahalanobis method performs hyperparameter search automatically for each of the settings on which we ran it.
k- k-Nearest Neighbors: We take k = 8. For each test sample, we take the average distance to the nearest neighbors in the input (pixel) space, and we use this as the test statistic.
We train an ensemble on the training set according the true labels. For test sample, average the models' probabilities, and use the entropy of the resulting distribution as the test statistic. We use ensembles of 5 models, with the same architecture and hyperparameters as the ones used for RETO.
Vanilla Ensembles (Lakshmiaraaaanan et al. 2017):
Outlier Exposure (Hendrycks et al.) 2019): It makes the model's softmax predictions close to the uniform distribution on the known outliers, while maintaining a good clas- sification performance on the training distribution. We use the WideResNet (Zagoruyko) & KKomodaiis, 2016) for the RGB data sets. For fine-tuning, we use their recommended settings of 10 epochs at learning rate 0.001. For training from scratch, we train for 100 epochs with an initial learning rate of 0.1. When the training dataset is either CIFAR and ImageNet, we use the default WRN parameters of the author's code, namely 40 layers, 2 widen-factor, droprate 0.3. For when the training dataset is SVHN, we use the author's recommended parameters of 16 layers, 4 widen-factor and droprate 0.4. All settings use the cosine annealing learning rate scheduler provided with the author's code, without any modifications.
Bayesian Method: It trains a neural network (Prior Network) to parametrize a Dirichler distribution over the class prob- abilities. We train for 100 epochs an WRN-28- 10 using SGD with momentum 0.9, with an initial learning rate of 0.01, which is decayed by 0.2 at epochs 50, 70, and 90. For MNIST, we use EMINST//etterss as OOD for tuning. For all other settings, we use Tinylmages as OOD for tuning.
Deep Prior Networks (DPN) (Malinin & Gales 2018):
Mahalanobis (Lee et al. 2018): It pretrains models on the training data. For a data point, uses the intermediate representations of each layer as "extracted features".. It then performs binary classification using logistic regression using these extracted features. the original setting, the classification is done on "training" ID vs "training" OOD samples (which are from the same distribution as the test OOD samples). Furthermore, hyperparameter tuning for the optimal amount of noise is performed on "validation" ID and OOD data. We use the WRN-28- 10 architeture, pretrained for 200 epochs. The initial learning rate is 0.1, which is decayed at epochs 60, 120, and 160 by 0.2. We use SGD with momentum 0.9, and the standard weight decay of 5 10-4
Mahalanobis- TransuuutiTaassuutiiieee The methodology proposed by Lee al. (2018) is very dif- ferent from the other settings, where we do not have access to samples which are known to be OOD and from the same distribution as test OOD. Therefore, we propose transductive alternative: carly-stopped logistic regression is used to distinguish between the training set and the test set (instead of ID OOD samples). The carly stopping iteration is chosen to minimize the classification errors on a validation set that contains only ID) data (recall that we do not assume to know which are the OOD samples).
Maximum Classifier Discrepancy (MCD) YYu & Aizawa 2019): It is a transductive method that trains two classifiers at the same time, and makes them disagree on the test data, while maintaining good classification performance. We use the WRN-28-10 10 archi- tecture as suggested in the paper. We did not change the default parameters which came with the author's 's code, so weight decay is 10 and the optimizer is SGD with momentum 0.9. When available (for CIFAR10 and CIFAR 100), we use the pretrained models provided
13

---

Under review as a conference paper at ICLR 2021
by the authors. For the other training datasets, their methodology to generate pre- trained models: We train a WRN-28- 10 for 200 epochs. The learning rate started at 0.1 and dropped by a factor of 10 at 50% and 75% of the training progress, respectively.
A.2 TRAINING CONFIGURATION FOR REGULARIZED ENSEMBLES
When training RETO with certain neural network architecture, we use hyperparameters that give the best test accuracy when training a model on the ID training set. We do not perform further hyperparameter tuning for the different OOD data sets on which we evaluate our approach.
For MNIST, we train 3-layer MLP with ReLU activations. Each intermediate layer has 100 neu- rons. The model optimized using Adam, using a learning rate of 0.001, for 10 epochs.
For CIFAR and ImageNet, we train WideResNet WRN-28-100 10. The model is trained using SGD with momentum 0.9, and the learning rate starts at 0.1, and is multiplied by 0.2 at epochs 50, 70 and 90. The weights have a 12 regularization coefficient of 5e 4. We use batch size of 128 for all scenarios. The hyperparameters have been selected to achieve high accuracy on the CIFAR 100 classification problem, thus obtaining ensemble validation accuracy of 80.8%, where each indi- vidual model has between 77% and 78% acuracy. After that we used the same hyperparameters for all settings. For the fine-tuning scenarios, trained for 10 epochs with constant learning rate of 0.001 for all scenarios.
For the medical datasets, train Densenet-121 the authors do in the original paper. For training from scratch, we do not use random weight initializations, but instead we start with the ImageNet weights provided with tensorflow. The training configuration is exactly the same as for WRN-28-10. except for: we use a batch size of 32 because of GPU memory restrictions, and for fine tuning we use constant learning rate of 10-5
B ID AND OOD DATA SETS
B. DATA SETS
For evaluation, we use the following image data sets:
MNIST (Lecun et al. 1998) and Fashion MNIST (Xiao et al. 2017).
SVHN Netzer et al. 2011).
CIFAR1O, CIFAR100 (Krizhessky 2009). and their corrupted variants (Hendrycks & Dietterich
2019
ImageNet (Deng al. 2009) and ObjectNet (Barbu et al., 2019), both resized 32x32.
14

---

Under review as a conference paper at ICLR 2021
B.2
SAMPLES FOR THE SETTINGS WITH NOVEL CLASSES
MNIST[0:4]
CITRR101044]
label
label
label
label
label
label
label
label
label
label
I
2
3
CHARRO15:9]
label
label
label
label
label
5
6
7
8
9
FashionMNISTO.2.3.1
SVIIN00:4]
kD
FashionMNIST: 1,4,5,6,9
SVHNI5.9
5
36
17
8
(a)
(b)
Figure 7: (a) Data samples for the MNIST/FashionMNNIST splits. FARIO/SVHN splits.
(b) Data samples for the CI-
B.3 SAMPLES FROM OBJECTΝET
ImageNet
ObjectNet
Chairs
Chairs by rotation
background
by
Chairs by viewpoint
Teupots
Tishirts
Figure 8: Samples from ImageNet and ObjectNet taken from the original paper by Barbu et al. 2019).
15

---

Under review as a conference paper at ICLR 2021
B.4
SAMPLES FROM CIFARIO-C
CIFARIO-C severity
Brightness
Contrast
Defocus blur
Elastic
Fog
Frost
Frosted glass blur
Gaussian noise
Impulse noise
Joeg compression
Motion blur
Pixelate
Shot noise
Snow
Zoom blur
Maiiens
CIFAR10-C severity
Brightness
Contrast
Defocus blur
Elastic
Fog
Frost
Frosted glass blur
Gaussian noise
Impulse noise
Joeg compression
Motion blur
Pixelate
Shot
blur
Figure 9: Data samples for the corrupted CIFARIO-C data set.
C OOD DETECTION HARDNESS
Out distribution detection benchmarks can be assessed based on their difficulty. In what follows, we propose simple way to evaluate the hardness of an OOD detection setting and provide empirical evidence that shows that the scenarios we looked at are indeed more complicated than some of the common OOD detection benchmarks.
Consider the task of distinguishing between samples that come from two distributions P,Q with disjoint supports supp suppq- Let us assign labels according to the distribution the points coming from: D {0,, yı) V: -1 if x; E suppp, V/ if E supp2). We search solve the classification problem by searching for a minimizer of the empirical risk inside a function class F.
The intuition for measure of hardness is follows: if it is difficult for binary classifier to separate samples from and Q, then it will also be difficult to detect test samples from Q as OOD, when only training set drawn from P is available.
16

---

Under review as a conference paper at ICLR 2021
To quantify the difficulty of the binary classification problem, the area under the training curve, i.c the curve the training loss as a function of iterations of the optimization algorithm. The larger the area, the more iterations it takes to converge, which in turn indicates that the classification problem is difficult.
Formally, define the hardness of the OOD detection task with respect to function class F as:
HOODD(D;F) := S. L(fiddt,
where ft the model after fraction of training epochs are finished.
For our task, we start with VGG model and train it for 30 epochs. In order to approximate the integral, we take the training loss for the whole data set every 5 epochs, and we average these losses.
Table 2: OOD detection hardness.
ID data set
OOD data set
HOODD
MNIST
FashionMNIST
0.01
FaShionMNIST
MNIST
0.01
1) Easy OOD
SVHN
CIFAR1O
0.05
CIFARIO
SVHN
0.05
CIFARI00
SVHN
0.06
MNIST[0:4]
MNIST[5:9]
0.15
FashiomMNNIST:02...7...
FashionMNNIST:14.5560
0.27
2) Novel classes
SVHN[0:4]
SVHN[5:9]
0.24
CIFAR1O[0:4]
CIFARIO[5:9)
0.45
CIFAR100[0:49)
CIFARI00[S001
0.59
CIFARIO
CIFARIO-C sev5
0.08
3) Hard covariate shift
CIFAR100
CIFARI00-C sev5
0.10
ImageNet
ObjectNet
0.18
CIFARIO CIFAR100
CIFARIO-C sev2 CIFARI00-C sev2
0.18
3*) Easy covariate shift
0.19
Notice that the settings with novel classes and the with hard covariate shift are generally more difficult than the common benchmarks used in the OOD detection literature.
Apart from the three categories of settings that introduced in Section 4 also present numbers for CIFARIO-4 and CIFAR 100-C with lower-severity corruptions, i.e. severity 2. These scenarios usually easier solve with domain adaptation techniques. Nevertheless, in Appendix Gle show that RETO performs well on OOD detection on these settings as well. Even though performing OOD detection is redundant here, the good results of method go show that it still work well, even in difficult situations.
D RESOURCE REQUIREMENTS FOR RETO
Computational cost Our method can work with training each model in the ensemble from scratch (i.e. random initialization), but it also preforms well when fine-tuning network from pretrained weights. This reduces significantly the inference time for each batch of test data. For the settings considered, average, few three epochs of fine tuning enough to achieve the best performance: the training is stopped early, on average, after three epochs, according to the condition on the validation loss.
17

---

Under review as a conference paper at ICLR 2021
Vanilla Ensemble
0.95
RETO
AUROC
0.9
0.85
0.8
0.75
2
4
6
8
9
10
Ensemble Size
Figure 10: Effect of ensemble size on CIFARIO0 vs. SVHN. Both methods are trained from scratch for 100 epochs.
Dependence ensemble size Figure shows that the good performance of RETO does not rely on a large number of models in the ensemble. Unlike vanilla ensembles, our method achieves a high AUROC with few as 2 models. This is because, vanilla ensembles, the networks are diverse 'by chance', due to the stochasticity of the training procedure. On the other hand, in RETO, our training method actively encourages the ensembles be diverse, and hence two models will already disagree OOD data almost as much as five.
E GENERALIZATION TO HOLD-OUT TEST SET
In this section present experiments which show that after training/fine-turr and OOD samples, one can also use our method to detect OOD samples from the same distribution, that have not been seen during training. Concretely, samples to train RETO ensembles (we reiterate that we do not have access to which samples are indeed OOD in the test set). For evaluation, we compute the metrics on separate data set, with 5000 ID and 5000 OOD samples, where the OOD samples come from the same distribution as the samples seen during training. As revealed in Table the performance does not change substantially, when evaluating on the hold-out test set. For the corruption data sets, we report for each metric the average taken over all corruptions (A), and the value for the worst-case setting (W).
test with ID
test set of 5000 ID and 5000 OOD
Table Generalization on held out test set.
data
(normal) AUROC
RETO (holdout)
FPR
CIFARIO
0.99/0.0
CIFARIO
SVHN
1.00/0.00
1,0000000
CIFARI00
SVLI
0.00
1.00/0000
SVHN|0-4]
SVHN[5:9]
0.94/0.34
0:95 0.95/0.22 /0.28
CIFARI0[044]
CIFAR
0.91 /0.34
0.90/0.37
CIFAR100449
10
CIFARIO
CIFARIO-C ((A)
0.6/0.14
0,95 /0.16
CIFARIO-C
CIFARIO
CIFARIO-C spv (A)
1.00
0.00
CIFARIO
CIFARIO-C sev (W)
0.98 00.14
0:97 0.21
CIFAR100
CIFARIOD-C sev 2 (A)
0.94/0.24
0.95/0.33
CIFARIO0
CIFAR Iop-C szv (W)
0.71 70.81
0.71 /.84
CIFARLO0-C
CIFAR100
CIFAR sev (W)
0.96
0.95/0.30
Averag
18

---

Under review as a conference paper at ICLR 2021
F REGULARIZED TRAINING/TEST DISCRIMINATOR FOR TRANSDUCTIVE OOD DETECTION
The paper Scott & Blanchard (2008) suggests that training binary classifier with bounded false positive rate to distinguish between the training set and the test set can successfully separate the OOD samples from the ID samples in the test set. However, this approach does not fall in the OOD detection methods, since it does not provide a good
category of predictive uncertainty-ban classifier of the labeled training set well.
We present what follows set of experiments run to check i similar technique works for the data sets we considered. Early stopping with respect to validation set that contains only ID samples is enough to obtain good OOD detection performance.
For the corruption data sets, the table shows the average of the AUROC taken over all corruptions (A), and the value for the worst-case setting (W).
Table 4: AUROC for an early-stopped binary classifier trained to separate the training set from the test set.
ID data
OOD data
Binary classifier
AUROC 1
SVHN
CIFAR1O
0.99
CIFAR10
SVHN
1.00
CIFAR100
SVHN
1.00
SVHN[0:4]
SVHN[5:9]
0.82
CIFAR10J0:4]
CIFAR
0.83
CIFAR100(0:49]
CIFAR100[50:99]
0.61
CIFARIO
CIFARIO-C sev (A)
0.94
CIFARIO 10
CIFARIO-C sev (W)
0.51
CIFAR10
CIFARIO-C sev (A)
1.00
CIFAR1O
CIFARIO-C sev (W)
1.00
CIFARI00
CIFARI00-C sev 2 (A)
0.83
CIFAR100
CIFAR sev (W)
0.50
CIFAR 100
CIFAR 100-C sev 5 (A)
1.00
C0
CIFAR 100-C sev (W)
1.00
Tiny ImageNet
Tiny ObjectNet
1.00
Average
0.94
19

---

Under review as a conference paper at ICLR 2021
G MORE EXPERIMENTS
G.1 EXTENDED RESULTS WITH RESNET
Table 5: Extended results all setting (including a breakdown for all corruption types). For RETO and vanilla ensembles, we train ResNet20 models for each setting.
ID data
ODD dama
LINN
DPN
Vanilla Ensembles
OB
Mahal
Mahal-T
MCD
RETO
(rand init)
RFTO (pretizined)
AUROC 7/ FPRO995 1
SVHN
CI
0.92/0.53
7.00
0.97/0.12
1.00 0..00
10.02
0.99/ /000
F0.1/
1.00
/0.03
CIFARIO
SVHN
08./0..4
0.95/0.15
0.92//222
0.97
10.11
10.04
0.99/0.0
10.02
1.00
0.00
CIFAR100
SVIIN
0.83/0.71
0.55/
/
0.82
/0,50
0.98
/0,10
0.98 / 0.008
0.97
/0.27
1.00
1.00
1.00/ 0.00
SVHNJ044
SVHN15.91
0.54/0992 0.87/031
0.92 70.31
0.85 70.48
0.92 0..29
0.91 0.57
0.91 00.49
0.96
0.20
0.94/
CIEARIU0::4
CIEARIQ15:91
0,59/0.85
0.82/0.68
0.66
0.82 0.559
0.79/0.73
0.64
0.69
/075
0.97
0.14
0.91
0.33
CIPAR1000-91
(0
0.51/036
0.70/0.34
0.781
0.64
00.00
0.70
0.85
CIFARIO
CIFAR
CE-ARRO-Chrignneen hrightnes 2
0,54/0.93 0.55
70.95
0.53 10.94
0058 7..92
0.55/0.91
0.52 /0.94
0.86
7 0.55
0.68 0081
CIFARID
CIFAR
0.94/0.20 0.69 /069/035
0.62
0.2/00.56
0.69
0.82
1.00
CIFARIO
CIEARIO-C defocus blur 2
0.56/0.91
0.48: 00.95
0.54 /0.93
0.56 0.993
0.67 10.8] 0,58/00 0.90
0. 0.60 0.50
0 0.07
0.94
CIFARIO
CIFAR I0-C clasic
0,56 /..92 0
0.63/0..7
0.65 /0.87
0.71 10.76
0.66 0.85
0.68 00.1
0.99 0.08
0.93 /0.37
CIFARID
CIFAR 10-C fog 2
0.87 /0.52
0.61 0.89
0.53/0093
0.55 10.93 0,81 00.62
0.71
0011000 0.77 0.58 /0.92
1.99 0.06
0.90/0L 0.39
CIFARIO
CIFARIO-C
0,50/0.92
0.86 60
0.69
0.72/0.73
0.59
/0.95
1,06
CIFARIO
CIFARIO-4 frostad gloss Mur 2
/..95
0.98 00.000000000000000000000000//000 0.89/0-43
0.56 (0.50
1..02 0.99 0.04
0.99 /0.04
1.00 9.00
0.00
CI
CIFAR ganssion noise? 2
0.57/00.9
0.97/ 0.13
0.83/
0.85 /0:58
1.00
00.01
0.99 0.002
1.00
10.00
1.00 / 0.00
1.00 0.00
CIFARIO
CIFAR IO-C impulse noise 2
06.85
0
0.60/
0.81
/0.61 1.002
0 0.04
1.00 10,00
1.00 0.00
1.00/ 0000
CIFARIO
I0-C
ipeg compression
/.915
0.65/0
/0
0.73
0.68
10.00
CIFARIO
CIEARIO-C motiun blur
0.60/
0.65 0.38
0.71
00.1
0.74 00777
0.80
0.69
0,79/0.51
0.99 00.02
1.00 0.00
0.99 0.02
CIFARIO
CO-C piaglaie
0.52/0.94
0.82/ 0..6
0.65
0.88
0.66 /.85
0.79
/0.71
0.66 0.86
0.93 /0.73
1.00 0.00
/.00
CIFARIO
CIFAR I0-C shot mrise
0.53/0.92 0.67
0.74//..9
0.75 10.82
0.98
0.00
0.57
0.96 /0.23
1.00 0.00
1.00/ 0.00
CIFARIO
CIEARIO-C snow
0,57/0.90
0.74
10.18
0.73/0.74
0.69 /0.9
1.00
0.99 0.00
CIFARIO
CIFARIO-C zoom blur
0.59 0.56 0696
0.71
/0.77
0.73 10.81
0.33
10.56
0.77 0.66
10000
1,00
2 9.00
1.00/ 0.00
CI
CIFAR 10-C hrightess
0.56 /0.88 0.77 0.77
0.60/
0.90
0.63 /0.89
0.78
/0.75
0.79
0.72
0.60 /0.9
1.00 0.00
0.98 0.14
CIFARLO
CIFARIO conirast 5
10,00
0.98 00008
0.92/0.25
0,96
/0.14
7.00
1.00/0..0 0.01 0 0.04
1.00
10,00
1.00 0.00
1.00/ 0,000,00
CIFARIO
CIFAR defocus blur 5
0.65/0.82
0.90
70.1 0.95/
10.00
CIFARIO
CIPARIO-C clastic
0.59/0.89
0.62
0.80/0.64
0.78 /0,24
0.34
3344/0.00 0.61
0.77
0.92 10.55
1.00
1000-00 9.00
1.00 0.00
CIFARLO
CIFARIO-C fog
0.89 /0.27
0.13
0.79/ 0.66
1.50/0.09
0.98
/0.10
0.76/ 0.61
1.00 0.000
1.00 9.00
1.00/ 00.00
CIFARIO
CIFAR IO-C frost 5
0,70/0.68 0.94 /0.27
0..5
0.84 70.60
0.98
0 0.07
0.79
0.99 10.04
1.00 0.00
1.00 0.000
CIFARIO
CIEARIO frosted glass
0.55/0.92
0.90/0.37
0.89
0.05
0
2.00
10.00
1.00 9.00
1.00 0.00
CIFARIO
CIFARIO-C gaussia moise
0.67/0.75
1.00 00.0)
0.91
70.30
0.96 /0.17
1.00
0.00
1.00 10000
1.00 10.00
1.06 9:00
1,00
/ 0.000
CIFARIO
CIFAR 10-C impulse nuise 5
0.83/0.50
0.99/0.00
0.94 /0.17
0.98 10.09
7.00
10.00
1.00
0.00
1.00 10.00
1.00 0.000
1.00
10.00
CIFARIO
CIFARIO jpeg compression
0,51/0.94
0.73
0.69/
0.81
/0.63
0.61
0.687
10.00
1.00 0.00
1.00
CIFARIO
CO-C
0.66/0.82
0.61
0.55
0.56 70,50
00227
0,86/ 0.45
7.00
10,00
1 9:00
0.00
CIFARIO
CIFARIOCC pixelute
0.56/0.22
0:98: 0.07
0.84/0.35
/0.39
0.38 /0.49
0.9%
7 0.04
0.97 00.12
0.99 00007
1.00
9.00
1.00 0.00
CIFARIO
C-FARIO-C shot noise 5
/..7
0.90
0.30
0.95
/0,20
1.00
1 0.00
1.00/
0.01
1.00
00000
1.00
1.001.000 0.00
1.00
00.00
CIFARLO
CIFAR I0-C
00.8
0041
0.700
0.79
00.72
0.95
1
0.72/ 0.74
0.82
/0.00
1.00
0.00
1.00
00000
CIFARIO
CIEARIO-C zoom blur
0.63/0.84
0.72/055
0.87
70448
0.38 10.43
0011 0.91 0.30
10.00
1.00
9.00
1.09 0.00
CIFARLOO
CIEARIO0 brightness 2
0.53/0.94
0.54 00.93
0.52 0.99
0.52 /0.94
0.35
0..9
0.55
00.4
0052
0,86
9.56
0.7L 00.1
CIFAR100
CIFAR 100-R00--C cunirast 2
8,9//0.27
0.62/0.87 0.87
0.50/ 0:90
0.58 10.90
0.75
/071
0.66
0.83 0.60
0.91
0.96.
0.22
0.92
0044
CIFAR100
CIFARIOD defocus blur
0.56/0.92
0.64
0.55/0.92
0.54
0.70/0.82
0.82
0.68/ 0.50
0.94
0.91
0.60
CIFARLOO
CIEARIO0 clastic
0.55/0.93
0.32/0.97
0.88
0.59
0.89
0.86 0.66/
0.85
0.56
0.92
0,23
0.87
CIFARIOO
CIFARI00-C fog 2
0.82/0.52
0.59 0050
0.55 /093
0.53 /0.9
0.75
0..72
0.73/ 0.73
0.54
0.93
0.95/ 9
0.85
00.6
CIFARI0O
CIFARIOO-C C frost 2
0.53/0.50
0.73 0.76
0.71 0.79
0.65
0088
0.82
/ 0.65
0.71
0.75
0.66
0.89
0.55
0.00
0.99 0.03
CIFARL00
CIFARIOO frasted glass blur
0,50/0.95
0,78: 0..6
0.69
0.72
/0.72 0.98
/ 0.04 0.96
0.17
1.00
10.00
1.00 0.00
1.00 0.00
CIFAR1O0
CIFARIOD-C gaussian noise
0.56/0.89
0.74 00.88
0.81 0055 0.79 10.36
0.99
10.05 0.68 0.80
0.99
10.01
1.09 9.000
0.00
CIFAR10O
CIFARIOO impulse nuise 2
00.8
0.83
0.39 0.82/ 0.39
0.83 00.5
0.59
/ 0.00 0.69 0.75
0.99
00,00
1,00 9.00
0,00
CIFAR100
CIFARI00 C jpey compression 2
0..9
0.63/
0.70 0.79
0.89 0.79
0.71 0.88
0.66 0.89
0.70 0.58
1.00
1.000000 0.01
0.98
00.06
CIFAR100
CIAARIOG mocion blur
0.60/0.89
0.49 0392
0.63
0.67/
0.39
0.29
1.00 9.00
0.57
CIFARLOO
CIEARIOO piselate
0.52/0.54
0.76/0.75
0.63 0.88 0.60 1087
0..4
0 0.91
0.77
0.89
1 9.00
0.96
1.11
CIFARLOO
CIFARIO0 shot noise 2
0.53/0.92
0.77
0.75
0.73
0.73 10.7
0.9430.23
0.60/ 0.85
0.83
/0.79
1,00
9.00
1.00 0,00
CIFAR1O
CIFAR100 C snow 2
0.55/0.9
0.76/ 0.73
0.73
0223/003 0.77 0.09
02500
0.93
0.33
0.61 0.88
0.79/ 0.88
1..99
9.00
0.99
0.05
CIFAR100
CIFARIOO
blur
0,59/0.90
0.41
0054
0.81
0.63
1085
0.8310.44
0.68/
/.90
1.00
9.00
0.99
0.06
CIFARIO0
CIFAR brightess
0.53/0.3
0.73
0.64 0.86
0.62
1088
0.7410.76
0.72/
0.77
0.600
0.00
1.9%
9:11
0.96
0.29
CIFAR1UO
CIAAIOOOC C contrast
7.00 /.00
0.79/0.5
0.78/
0.57
0.82 /0.53
0.99
10.05 0.98 0.11
0.99
/ 00.00
1,00
9.000
1.00 0.00
CIFARI0O
CIFARI00 C defocus blur 5
0.65 /0.8
0.83
0.51
0031/0000 0.63
0.75 0.66
0.16/0.15
0.95
0.19
0.99
10.00
1.00
0.00
1.00
0000
CIFAR100
CIFAR 1O0-C elastic
0.59/0:50
0.73
0
0.09
00.81
0,71
/035
1,00
9.00
0.99
0.07
CIFAR10O
CIFARIOD-C fog 5
0.90/0.25
0.60
0.75 0.20 0.70 0022
0.92
7..00
0.80/ 0.67
0.95
00-46
1.00 9.000
0.59
0..0
CIFARL0O
CIFARIOO frost 5
0.73/0.65
0.66
0.79/
0.72
/0.71
0.94
0.26
0.66 0.77
0.95
/0.45
1,06/
9.00
1.00
0.00
CIFARIDO
CIFARIOU frosted glass blur 5
0,55 0.55/0.93
0.63
0.50/
0.60
0.73 70.72
0.97
/ 0.00
0.70
1.00
0..00
1.00
9.00
1.00
0.00
CIFAR100
CIFARIOG
noise
0.6700.7
0..5/
0.85
0.00
1.00 0.00
10.00
1.00
9.00
0.00
CIFARIO0
CIFAR 100-C impulse noise
0.840448
0.88/0.37
0.86/0.42
/0200
0..00
1.00 0.00
1.00
00,00
1,09
7 9.010
1
0.000
CIFARIOI
CIFARIOI-C jpeg compression 5
/854
0.69
0.79
0.77
0.75 00.69
0.83
0.55
1,00 9.000
0.99
/004
CIFARIOO
CIFARIO0 C motion blur
0.66
0.36
0.76
/
0.70
0.71
/0.74
0.91
0.22
0.77/
0.54
0.87
1.00
1.01
0:99
/003
CIFARTNE
CIFAR 1OU-C: piselite
0.55/0.92
0.
0.90/
0.50
70.01
0.9%
00.00
1,00
0.000
CIFARIOO
CIPARIOD-C shot noise
0.66
0.35
0.47
0.54
/0.44
0..01
1.00/
0.99 01,00
1,00 0.00
1,00 0.00
CIFARIOO
CIFARIOU C snow
0.56
0.80/0.3
0.77
0.73
/0.75
0.50
0.45
0.63/
0.57 0.79/ 0077
0.99 0.06
0.59
0.02
CIFARI00
CIFAR 100-C zoom bluc
0.64/0.85
0.49/088
0.69/0.66
0.74 0.70
0.92
0922/0.2
0229
0.53
0.24
0.92/0022
1,00
/ 9.00
1.00/ 0.00
Tiny ImngeNe:
Tiny ObjectNet
0.91
0,70/066
0.82/ 0.51
/0.63
0.75
/.77
0.72/ 0.75
0.99
10.02
0.99
/ 0.00
0.96 0.12
Average
0.63/
0.76/0.00
0,76/ 0.66/0664 0.76/0.66
0.87 5 0.38 0.74/ 0.86 00.00
0.98 / 9.08
0.96/0.15
20

---

Under review as a conference paper at ICLR 2021
G.2 RESULTS WITH SMALLER TEST SET
Table 6: Experiments with test set of size 1000, with equal number of ID and OOD test samples. For RETO and vanilla ensembles, we train 5 ResNet20 models for each setting.
Vanilla Ensemblls
RETO (pectrained)
ID data
ODD data
LINN
DPN
DE
Mahal.
Mihal-1
MCD
AUROC
/ L
SVEN
CIFARIO
0.92/0.32
1.00/0.00
0.97/0.12
1.00 10.00
0.99/
0.99:0..5
0.95/
1,00/ 9.01
CIFARIO
0.81/
0
0.92/0.22
0.91/0.11
0.99
0.9910.04
1.00/0.02 0.00
1,00
CIFAR100
SVHN
0.83/0.71
0.73
0.84/0.52
0.82/0.30
0.98/0.10
0.98 10.08
0.97
0.99 0,00
SVHN[0-4]
SVHN[5-0]
0.54/0.92
0.870081
0..3/0.31
0.85/0.48
0.92/0..9
0.91/
0.91
0.97 0.14
CIFAR1]j044]
CIFARI0(5:9)
0.59/
0.82/
0.80/
0.82 /0.59
0.79/
0.87
0.69
0.87 00.50
CIFAR10000-40
CIFAR [00[50
0,51/
0,70/0,74
0.78 F0.65
0.74/0.69
0,72/0.80
0.72/088
0.70/0.74
0.79 0U62
CIFARIO
CIFARIO-C brighnnes 2
0,54/0.93
0.59/
0.51 /0.95
0.52 0.94
1.58/0.92
0.55 0.993
0.52/
0.57 0.91
CIFARIO
CIFAR contrast
0.94/0.20
0.88/0.86
0.59/0.11
0:62/ 00.9
0.82/0.56
0,09/0.79
082/0930
0,91 0.39
CIFARIO
CIFARIO-C deticus blur
0.56/091
0.4870.95
0.36/099 0.34/095
0.67 0.88
0.58/090
0.00//0900
0.990 0.60
CIFARIO
CIFARIO-C C Clastic 2
0.56/0.92
0.47/0.97
0.63 00.8
0.65 0.65/ /0.87
0.71 0.766
0..6
0.85
0.68//090
0.
0.84
0.62
CIFARIO
CIFARIOCT fog 2
0.87 F0.52
0.66 0.88
0.53 70.93
0.55 /0.93
0.81 /.62
0.71
10733
0.53/
0:71
0.77
CIFARIO
CIFARIO-C frost
0,50/0.92
0.84/0.60
0..9/
0.8970885 1085
0.90/
0.721003
0
0 0012
CIFARIO
CIEARIO-C frusted glass
0,51/
0.98/0.09
0,89/0,42
0.86/050
0.99
0.99 10.04
0.99
CIFARIO
CIFARIO-C gaussins noise? 2
0.57/0.89
0..7/
0.83 /0.62
0.85 /0.58
1.00 0.01
0.99 00002
L/HI/ 0.000
1,00
CIFARIO
CIFARIO-C impulse noise 2
0 /0.23
0.88
/0.6
1.00 0.02
0.99 1004
1.00 00000
1.00
8800
CIEARIO
CIFARIO ipeg compression
0,51/0.95
0.65/0.92
0,71 0..1
0.77
0.75
0.75
0.68 10.84
0
1.0 00000
CIFARIO
CIPARIO-C motion blar
0.60/0.88
0.6570.8
0.1 1807
0:74/ /027
0 0
0.79 19:61
0.99
CIFARIO
CIFARIO-C pixelase
0.52/0.94
0.8216.67
0,65 70.8
0.06/ 0.85
0.79 0.71
0.06
0.86
095/0.1
0.73
0.96
0.25
CIFARIO
CIFARID
CIFAR I0-C shoe noise
CIFARIO-C
0.57/0.50
0.53/0.92
0.66
0.65/061
0.00
0.75 0.80
0.98
0.96/
0.18
0.06
0.73
0.80
0:57
0.9
1.00
0.99 T
CIFARIO
CIPARIO-C zoom blur
0.59/0.89
0.36/00.6
0.11 07.7
0.73 1981
0 0
0:77
19.66 0.66
629 7000 0
CIFARIO
CIFARIO-C hrightess
0.56/0.88
0.77/00.7
0.60 /1.00
0:63 10.99
0.78
0.78
0.79 19772
00
CIFARIO
CIFARIO
CIFARIO-C contrast
CIPARIO-C defocus blur 5
0.65/0.82
7.00 /0,000
0.82/0.77
0.98
0.88 /0.99
0.92 10.25
0.92/022
0.99/ 0.96/0.74
1.00
0.96/0.1 0.15
(.0)
0.95
1.99
1004
LIM 00:00
1.000 0000
1.00
1,000000 0000
CIFARIO
CIFARIO-C clastic
0.59/0.89
0.62/0.66
0.0/0064
0:78/022 707/
0.64
0.75
00:77
10-55
CIFARIO
CIAARIO-C fog 5
0.87
0.53/
0.79 10.0
0.80/ 10.69
0.98
0.10
0
7000 0:000
0,59 2002
CIFARID
CIFAR I0-C. frast 5
0.70/0.68
0.94/0.27
0.34/033
0.84/ 0..0
0.98
00007
0.90
1.00
CIFARIO
CIFARIO4 frosted glass Mar
0.55/0.92
0.97/0.11
0.37
0.89 89/0
0.05
0.00
058
1.00 2
CIFARIO
CIEARIO-C gaussins noise
0.67
1.00
0.1 00,00
0.96 0017
1.00 0,000
100 19.00
1000
7 800
CIFARID
CIFARIO-- impulsc naise 5
083/0.50
0.90/0.04
0.1/
0.98 10.09
1.00
0.00
1.00 19.00
TII 00:00
1 (.00
CIFARIO
CIFARID
CIFARIO-C
CIFARIO-C joag compression 5
bluz
0.66
/0.3/
0.81/0.71
0.73
0.79 /0.68
0.84/0.59
0.81 0.63
0.96
0.62
0.27
0.86
1.00 00:00
0399
1 0002
000
CIFARIO
I0-C pixelate
0.56/0.22
0.98
0.8//0.5
8.8//0.9
2012 0,7 0.90
0.99
6.99/00 D0TT
0,99
CIFARIO
CIFARIOCC shoe noise
0.66 0..7/
0.99/0.11
0.91 /0.30
0:95 /0
0.00
1.000 1..0
LIK 0.000
1 8.00
CIFARIO
CIFAR I0-C snow
061
0.91/0.41
0.77 70.70
0.79/0.22
0.95 0.25
0.77
0,32 /0.0
0.98
CIFARIO
10-C: hlur
0
0.72/0.90
0.87
0.88 /..8
1000
0000
CIFARIO0
CIPAR buighmess
0.53/0.4
0.54/6.93
0.52/034
0.52/094
0.55
0.55
25552094
0.52
10:55
0.55
CIFAR
8.4/
0.62/0877
0.60 /0.00
0.58/ /0990
0.75
0.75
0.000 1..91
0
CIFARI00
IOAARIDD-CCCae defoems blar 2
0.56/
0.44/0.96
0.55 70.92
0.54/093
0.70/ 0.82
0.69
1.P11.00 10000
0.52/0
10:94
CIFARI00
CIFARIOO-C 2lastiee
0.55/055
0.32/0.9
0.61 /088
0.89
0.86
0.667 0085
0.36/00-3
0,68
CIFAR100
COFARI00-C fog 2
0.82/0.52
0.50/0.90
0.95/0.93
0.53/093
0.75/
0.73/073
0.54/0.3
CIFARIOI
CIFARIDI-C frast 2
0.53/0.00
0.73/0.76
0.71 /0.79
0.65 7083
0,82 0.65
0.71
10755
0..6/0.9
1
CIFARI00
CIFAR 100-C fracted glass blr 2
0,50/0:95
0.78/0.66
0.79/0.61
0.72 /0.72
0.004
0.96 /011
1.00
00,00
1.00 8.00
CIFAR 100
ganssian noise
0.56/0.89
0.74/0.68
/0.50
/0.56
0.007
0.68 /0.80
/0.01
0.00
CIF
CIFARIO0-C impulse noise
/0.85
0.53/0.59
0.2/0.59
0.83/0.5 00.55
0.99 0.00
1078
0.99 0.00
0,99
0.00
CIFAR100
CIFAR 100-C00-C jpeg compression 2
0.51
0.63/0.66
0.70/0.79
0.69 10.79
0.71 00
0.66 0089
0.70 0.88
0.88 0.61
CIFAR100
CIFAR100
CIFAR I00-C mecion blur?
1O0-C piselute
0,52/0.54
0.600..99
0.76/0.75
0.49/0.92
0.63 /0.88
0.68 70.82
0.63/
/0.8
0.47
0.57
0.57 /0.9
0.67
79
0.77/0.99
0.68
0.89
0.56 0.62
0.60
CIFAR100
COFAR shot noise
0.53/0.92
0.70/0.77
0.75/
0.73 /0.22
0.94/0.23
0.8
0, 0.98 0.3/0.79
CIFARID0
IDD-C snow
0.55/0.1
0.76/0.73
/0.7
0.69 /080
0.336
0.61 /0:88
0.79/0.81
1
CIF
CIFAR ID0-C zoom blur
0,59/0.30
0.470094
0.69 /0.81
0.63 0.85
0.33
0.68 1081
0.71 /0:90
0.91 0.55
CIFARIOG
1A0-RRO0-C Crightess 5
0.53/0.91
0.70/0.83
0.64/0.66
0.62/087
0.74
0.72/0.77
00.00.00
0.81
CIFAR
7.00
0.79/0.55
0.83/0.53 0.78/0.57
/..03
0.16/0112
0.99 0.00
0, 1.00
CIFAR100
CIFAR C defocus blar 5
0.65 0.65
0,54/088
0.81 /0.63
/
0.16
/0.9
0.99 / 10.//
0 0.00
CIFARI00
CIFAR 100-C 5
0.59/0.90
0.73/0.78
0.72 70.76
0.69/ /..7
0.78
0.71 0082
0..5
0.89 0.57
CIFAR CO0
100-C: fog
0.90/0.25
0.60/0.60
0.75/0.70
0.70/0.33
0.92
0.90
0.98 0.44
0.34
CIEARIOO-C frust 5
0.73/0.68
0.78/0.6
0.79/0.63
0.72/0.71
0.9450.26
0.6
0.95/045
0, 1.96 0,06
CIFAR100 100
CIFAR frosted glos blur 5
0,55/0.93
0.80/0.63
0.80 0.060
/0.7
F.90
0.76/0.70
1.00/ 0.00
1.00 0,00
CIFARIO0
CIFAR 100-C ganssian moise
0.67/0.77
0.8170.54
/0.4/
0.85 /Q.4
1.000
1.00 10,00
1 01000
1.00 0.00
CO
CIFAR impuise
0.4/0.48
0.68/0.37
0.96/ /.200
1.00
1.00 0.00
0000
CIFAR
CIFAR 100-C jeeg compression
0.59
0.60/0.79
0..7/
0.75 /0.69
0.73/088 0.62
0.54
0.95 0.39
CIFARI00
CIFARIO0 C mocion blur
0.66 0.66/0.83
0.56/00.8
0.76/0.70
0.71 /..7
0.22
0.77 00.6
0.71
0.95 0.00
CIFARI00
CIFAR 100-C C piselate
0,55/0.92
0.95/0.17
0.50/ 0.80
0.90
9.08
0.99 00,00
1,00 9.00
CIFAR
CD-CC shot
0.54/0.50
0.84/0.44
7.00
1.00
0.00
0.00
CIFARL00
CIEAR1O0-C snow
0.56/0.90
0.80/0.63
0.77
0.73 /0.33
0.90
0.63 0.87
0.79 / 0.77
0,90 0.96/ 0.37
CIFAR 100
CIFAR 100 CCC zoom blur 5
0.64/0.85
0.4910.86
0..6
0.74 /.700
0.92/0.29
0.93 1 0.24
0.92 / 0.62
0.98 0.06
Average
0.64/0.79
0.76/ 0.60
0.76 0..6
0.76 0.76/0.6 0.64
0.88 0.37
0.79 0.53
0.85 / 0.41
0.91 0.22
21

---

Under review as a conference paper at ICLR 2021
G.3 RESULTS WITH VGG
Table 7: Results on all setting where we used ensembles of VGGI6 models for RETO and vanilla.
For the corruption data sets, report for each metric the average taken all corruptions (A), and the value for the worst-case setting (W).
RETO
GOD
Ensembles
Mahal-T
(rand init)
(pretniined)
AUROCT 1 HPRA893
SVHN CIFARIO
CIFARIO
0.92/0.32
7.00 0.00
0.97/0.12
7.00 10.00
0.99 0.02
0.02
0.99/0.05
0.97 70.15
0.99 0000
0.99 0.06
SVHN
/0.7
0.95/0.15 0.15
188870.31
0.37
0.99 0.04
0.399/0.04
7.00 F1002
1.00 0.00
1.00 8.00
SVHNj0:4
SVHN 5:91
0..4
0.87/0.81
0.8970.40
0.85 0..55
0.52 0.229
0.91 0.91/0.49 0.31
0.94/ 0..34
0.33/
CIFARIQ(H)]
CIEAR1O05:9
0.59
0.82 0..5
0.74/0..1
0.82 10.59
0,75/0.73
0.64/0.87
0.69/0.75
0.95 / 0.26
0.31/
CIFARIO
CIEAKIO-C sev (A)
1159 00.4
0.73/0.60 0.:9
1.66/0083
0.70/0.00
0.8d 0.47
0.75/0.62 00662
0.22/0.5
0.98 / 00.07
0.34/0.21
CIFARIO
0.58/0..2
0.52/0..4
CIFARIO
CIFARIO-C sev
0.77
0.40
0.80/0.59
0.86 0..6/0.66
0.94/0200
0.000
CIFARIO
CIFARIO-C sev (W)
0.51 0.94
0.72/ 0.50
0.58 0.90
0.63 0.39
0.78
0.68 0.88
0.60/0.5
0.99 /0.01
0.95 0
Average
0.69 0.69/0.33
0.88 0.45
0.8270.88 0.49
0.87 0.47
0.91
0.86 0.39
0.89 /0.35
0.98 / 0.11
0
G.4 RESULTS ON MNIST AND FASHIONMNIST
Table 8: Results on MNIST/FashionMNIST settings. For RETO and vanilla ensembles, we train 3-hidden layer MLP models for each setting.
ID data
OOD dama
KNN
DPN
Vanilla Ensembles
OE
Mahal.
Mahal-T
MCD
RETO (nmi
RETO (pretrained)
FFRE99
MNIST
10.9
1.00/
MNIIT
MNIST
0.98 70,09 7.00 0.00
0.87 10.88
0.68/0.84
0.99 /0.03
0.99.0.00 70004
1.00 0.00
1.00 0,00
1.00/ 0.00
MNISTT044]
MI
0.03
1028
0.99/
Fashion WahionMNNITT0....7 AINiIMNNISTTLA.SGG
0.74/0.8 0.77
0.85
0.64 /0.98 0.66
0.58
0.77/0.80
0.82 10.67
0.78
10770
0.22
0.93/0.38
Average
0.90/ 0.94/ 0.22 0.27/0.70 0,82/ 0.94/0211 0.95 0.94/0.24 0.99/0.07
0.98/ 0.10
For FashionMNIST chose this particular split (i.e. classes 0,2.3,7,8 vs classes 1,4,5,6.9) because the two partitions are more similar to each other. This makes OOD detection more difficult than the 0-4 vs 5-9 split.
G.5 MORE RESULTS FOR OUTLIER EXPOSURE
Table 9: Results for Outlier Exposure, when using the same corruption type, but with higher/lower severity, as OOD data seen during training.
ID data
OOD data
OE (trained sev5)
OE (trained sev2)
AUROC t
CIFAR1O
CIFAR10-C sev 2 (A)
0.89
N/A
CIFAR1O
CIFAR10-C sev 2 (W)
0.65
N/A
CIFAR1O
CIFARIO-C sev 5 (A)
N/A
0.98
CIFARIO
CIFARIO-C sev 5 (W)
N/A
0.78
CIFAR LOO
CIFARIOO sev (A)
0.85
N/A
CIFARION
CIFAR100-CC sev (W)
0.59
N/A
CIFAR 100
CIFAR sev (A)
N/A
0.97
C0F
CIFAR100-C sev (W)
N/A
0.67
Average
0.87
0.98
The Outlier Exposure method needs access to a set of OOD samples during training. The numbers report in the rest of paper for Outlier Exposure are obtained by using the Tinylmages data set as the OOD samples that are seen during training. In this section we explore the use of an OOD data that is more similar to the OOD data observed test time. This is much easier setting for
22

---

Under review as a conference paper at ICLR 2021
the Outlier Exposure method: the closer OOD triin is OOD ust. the easier it will be for the model tuned on uOD to detect the test OOD samples.
In the table below focus only on the settings with corruptions. For each corruption type, the lower severity corruption as OOD and evaluate on the higher severity data and vice versa. We report for each metric the average taken over all corruptions (A), and the value for the worst-case setting (W).
H EXPERIMENTS ON CIFARIOV2
Here we will present our results on distinguishing between CIFARIO (Krizhessky (2099) and CI- FAR10v2 (Recht et (2088), a dataset meant to be drawn from the same distribution as CIFARIO (generated from the Tiny Images collection Torralba et al. (0088) ). Recht et al. (2018) 8) and Recht al. (2019) argue that classifiers originally trained on CIFARIO have statistically significant drop in accuracy when evaluated on CIFAR 10v2. Furthermore, Recht al, 2019 argue that CIFARIO and CIFARI0V2 come from the same distribution by training a binary classifier to distinguish be- tween them, and observing that the accuracy is obtained is very close random (50.1% % for the randomly initialized models, and 52.9% for models with pre-trained weights).
Our experiments show that the two datasets are actually distinguishable, contrary to what previous work has argued. First, our own binary classifier trained on CIFAR10 vs (IFAR10y2 obtains a test accuracy of 67%, without any hyperparameter tuning (the model is ResNet20 trained for 200 epochs using SGD with momentum 0.9. The learning rate is decayed by 0.2 at epochs 90, 140, 160 and 180, We 1600 examples from each dataset for training, and validate using 400 examples from each dataset).
Table 10: OOD detection performance on CIFARIO vs CIFARIOv2
data
UOD data
KNN
DeN
Ensembles
Og
Mahal.
Mahal-T
MCD
(rund ini)
(pretramedd)
AUROC FFR 4
CIFARIO CIFARIO
0.53 /093
0.63/0.9 0.64
0.64/0.0
0.55/0.92
0
0.99/0.90
0.91 0.20
0.76/0.74
Furthermore, OOD experiments (as shown in Table 10) show that most baselines able to distinguish between the two datasets, with RETO achieving the highest performance. The methods which require OOD data for tuning (Outlier Exposure and DPN) use CIFAR 100 for tuning.
23

---

Under review as a conference paper at ICLR 2021
24

---

Under review as a conference paper at ICLR 2021
I DEPENDENCE ON THE TEST SET CONFIGURATION
10000.0.
0.65
0.54
0.67
0.74
0.96
0.98
0.99
1.00
1.00
1.00
(0.79)
(0.80)
(0.78)
(0.77)
(0.77)
(0.77)
(0.77)
(0.77)
(0.77)
(0.76)
0.64
0.70
0.88
0.92
0.97
0.99
1.00
1.00
(0.79)
(0.80)
(0.77)
(0.77)
(0.77)
(0.77)
(0.75)
(0.75)
Test size
0.79
0.66
0.84
0.90
0.97
0.98
0.99
0.99
(0.79)
(0.79)
(0.77)
(0.76)
(0.76)
(0.76)
(0.76)
(0.74)
200.0.
0.74
0.80
0.85
0.92
0.97
0.97
(0.78)
(0.77)
(0.74)
(0.74)
(0.76)
(0.67)
0.81
0.67
0.86
0.87
0.92
0.95
100.0.
(0.77)
(0.76)
(0.77)
(0.75)
(0.70)
(0.75)
0.75
0.78
0.90
0.90
40.0.
(0.72)
(0.69)
(0.77)
(0.61)
0.0005.
0.001.
0.005.
0.01.
0.05.
0.1.
0.25.
0.5.
0.75.
0.9.
Proportion of OOD in test
(a) Initialization from pretrained weights.
0.64
0.45
0.59
0.75
0.97
0.99
1.00
1.00
1.00
1.00
10000.0.
(0.79)
(0.80)
(0.78)
(0.77)
(0.77)
(0.77)
(0.77)
(0.77)
(0.77)
(0.76)
0.55
0.71
0.92
0.96
0.98
0.99
1.00
1.00
2000.0.
(0.79)
(0.80)
(0.77)
(0.77)
(0.77)
(0.77)
(0.75)
(0.75)
Test size
0.76
0.71
0.83
0.93
0.97
0.98
1.00
1.00
1000.0.
(0.79)
(0.79)
(0.77)
(0.76)
(0.76)
(0.76)
(0.76)
(0.74)
0.71
0.81
0.90
0.97
0.99
0.98
200.0.
(0.78)
(0.77)
(0.74)
(0.74)
(0.76)
(0.67)
0.78
0.74
0.87
0.92
0.96
0.99
100.0.
(0.77)
(0.76)
(0.77)
(0.75)
(0.70)
(0.75)
40.0.
0.72
0.79
0.91
0.89
(0.72)
(0.69)
(0.77)
(0.61)
0.0005.
0.001.
0.005.
0.01.
0.05.
0.1.
0.25.
0.75.
0.9.
Proportion of OOD in test
(b) Initialization from random weights.
Figure 11: AUROCs obtained with an ensemble of ResNet20 models as the composition of the test set changed. Only settings with at least 5 OOD samples have been considered. The ID samples from CIFAR1O, while the OOD samples are from CIFARIO- I0-C/ooow.. For comparison, we provide in parentheses AUROC values for vanilla ensemble trained once the training set and evaluated on each test set configuration. In Tia the models are initialized from pretrained weights and fine tuned. In ITble the models are initialized with random weights and trained from scratch.
25

---

Under review as a conference paper at ICLR 2021
J EFFECT OF LEARNING RATE AND BATCH SIZE
0.77
0.71
0.76
0.76
0.77
0.76
0.75
bs 128
0.7
0.73
0.75
0.76
0.75
0.74
Batch size
bs 64
0.71
0.74
0.76
0.74
0.72
0.73
0.72
0.71
0.75
0.74
0.72
0.69
0.71
0.7
bs 16
0.73
0.71
0.73
0.76
0.75
0.69
Ir 0.001
0.005
Ir 0.01
0.05
Ir 0.1
Initial learning rate
Figure 12: AUROCS obtained with an ensemble of WRN-28-10 models, the initial learning rate and the training batch size are varied. For this we used the hardest setting, CIFAR 100:0-5 as ID, and CIFAR100:50-15 100 as OOD.
K MEDICAL OOD DETECTION BENCHMARK
The medical OOD detection benchmark organized as follows. There are four training (ID) data sets, from three different domains: data sets with chest X-rays, with fundus imaging and one with histology images. For each ID data set, the authors consider three different OOD scenarios:
1. Use case 1: The OOD data set contains images from completely different domain, similar to our category of easy OOD detection settings.
2. Use case 2:
The OOD data set contains images with various corruptions, similar to our category of hard covariate shift settings.
3. Use case 3: The OOD data contains images that various selection biases, similar to our category of novel class settings.
during training due to
The authors evaluate number of methods all these scenarios. The methods be roughly categorized as follows:
1. Data-only methods: Fully non-parametric approaches like kNN.
2. Classificr-only methods: Methods that use classifier trained the training set, e.g. ODIN (Liang et al. 2018). Mahalanobis (Lee et al. 2018). RETO falls into this category as well.
3. Methods with Auxiliary Models: Methods that use an utoencoder or generative model, like Variational Autoencoder Generative Adversarial Network. Some of these ap- proaches can be expensive to train and difficult to optimize and tune.
We stress the fact that for most of these methods the authors (known) OOD data during training. Oftentimes the OOD samples observed during training come from a distribution that is very similar the OOD distribution used for evaluation.
26

---

Under review as a conference paper at ICLR 2021
For exact details regarding the data sets and the methods used for the benchmark, refer the reader to the paper Cao et al. (022))
We did not evaluate RETO the histology image data set due to resource limitations; the data set is much larger than the others.
0.89
0.91
AUROC
0.79
0.60 0.90
0.90
0.52
0.85
0.72
0.72
0.74
0.24
0.60
0.61
0.6
0.3
Prob. tirestold
Score Side
Festure len Abscct Ralls
Rocoutt sERCE
Videce-
Ataice Atscc--0001 1001
Suople laper Hera.
AdMOLE AZMOL-K000- Kine-
Vigtest aoat--
Becunet All
ViEacE 100-- -
13m1-
Recanot VAgHel
Secorrt VAEACE
Ressonst AEMIGE
RETO
Figure 13: AUROC averaged all scenarios in the medical OOD detection benchmark (Cao et al. 2020). The values for all the baselines are computed using code made available by authors of the paper Cao al. (2020). Notably, most of the baselines assume oracle knowledge of OOD data
at training time.
In Figures 14
16 present AUROC and AUPR (Area under the Precision Recall curve) for RETO for each of the training data sets, and each of the use cases. Figure [33 presents averages over all settings that considered, for all the baseline methods in the benchmark.
15
100
100
50
AUROC, data Only
clasaller only
austlary
100
50
Score SYM
Prott ttrmshald
Rarondt ALICE
Pastzara km
VINEICE scep.
AlSICE 2as
AEMES Addl
Rerondl VADMSE
VREMSE SREMSE-KOM CIN-
Raccnnst DEMaSE
WASSCE 2000
0000
Raconst
VRE VREMSEE-Ean Ktan
Bloaky clacittr
Reconst VAcarE
Matalarobs
RETO
SETO/pretainee
tayer Naha.
Figure 14: Comparison between RETO and the various baselines on the NIH chest X-ray data set, for use case (top), use case 2 (middle) and use case 3 (bottom).
L TWO-SAMPLE TEST FOR OOD DETECTION
Distinguishing between ID and OOD samples can be cast as two-sample hypothesis test, with Hu : x E supPp and H₃ x & suppp. for a sample z. The various OOD detection methods differ in their choice of the test statistic. For approaches that ensembles of classifiers, the test statistic should reflect the belief that the models have similar outputs on samples, and disagree on OOD samples. For example, [_akshminarayanan al. (2017) propose averaging the softmax outputs of all
27

---

Under review as a conference paper at ICLR 2021
100
50
100
30
AUROC, delaanyy
classifer only classifeer
axciery auolary
100
30
AURICE HERICELARR 1LM0-1
Prob drrechald
MoMes 1001:1
Aeacit sbilp
Score
Recanct AlICe
Aphayg 20t6-
Recaratt MAEBIE
Recoraae Afeest
VADMSE c3t0
Beconnst SACRSE
Feature lnn
Mahatannen
Enary ciassiier
Singlo laver Hana.
RETO
RET (ETOipatrainet)
Figure 15: Comparison between RETO and the various baselines on the PC chest X-ray data set, for use case (top), use case (middle) and use case 3 (bottom).
100
50
100
30
AIIOC, delasnny
classfer only
MIROC aucllry Buolary
100
30
Score SWM
Prob drrechold
oPeN
Festurre kaos
VACHESE AS00
Magusc 5309
VAE K00N
Recancz Alfic
VAFRiC 4p00
Mahabannhaa
Stnggoo layer Maho.
Altossi: Wilk
Diroory clessiter
Bec-onst AGMSE
Necomttt VREMSE
ebou
AERICE L360-
ADMSE Kaits
RETO
REToipatrainet
VAUBICE
Figure 16: Comparison between RETO and the various baselines on the DRD fundus imaging data set, for use case (top), use case 2 (middle) and use case 3 (bottom).
the models in the ensemble and then taking the maximum or the entropy of the resulting probability vector as the test statistic. For a K -odel ensemble and an input æ this can be written as follows:
K
Tmax-p (x) = max
Σ (fa(z)), with (x) e RC the softmax output of the ttth model K
sE(C]
Averaging the softmax vectors loses some information about the model predictions, because differ- ent initial probability vectors can map to the same averaged vector. In our approach, models are more uncertain on ID samples than on OOD samples, which can make the averaged softmax vector
28

---

Under review as a conference paper at ICLR 2021
OOD
00,
OOD average
average
0.
(0,
Figure 17: Averaging the probability outputs of the models in an ensemble can lead to catastrophic information loss, in some cases. The softmax vectors of 3-model ensemble are represented on the 2D probability simplex. For an OOD sample, each model predicts with high confidence the arbitrary label it has seen during training. For the ID sample, the models predict the correct class with moderate confidence. Therefore, the average probability vectors for the ID and the OOD sample are close, which can make it hard distinguish between them.
fall at the same location for an ID and an OOD sample. This makes impossible to distinguish between the two, illustrated in Figure 17
For neural network ensembles, following a standard training procedure of minimizing the cross- entropy loss leads to models that make confident predictions both ID and OOD samples, as shown by Hein et al.] (2019); Maennel Tifrea (020). Consequently, the information lost through averaging is not causing any issues: ID samples, the models will tend to give the same predic- tion, while on OOD) samples the models tend to disagree, giving different predictions with high confidence.
However, in our case, because of early stopping, the training process is halted at different stages for test ID and test OOD samples, indicated in Figure
2017 Chen et al. 2019) analyze the dynamics of op- timizing the cross-entropy loss with SGD. They suggest that there might exist stages: one in which a good decision boundary is found, and another in which the margin is increased between the representations of inputs from different classes. dent predictions on both ID and OOD samples. Thus, early stopping may cause the models to be more uncertain on test ID samples than on test OOD. This is indeed confirmed in Figure 18
Recent papers like Shwartz-Ziv & Tishby
this second stage that also leads to overconfi-
To avoid the problem of information loss described previously, we compute the pairwise total varia- tion distances between the softmax outputs of the models the ensemble, and we take the average of these distances as our test statistic:
2
Ta Tv := K(K
drv (fi(z),
29

---

Under review as a conference paper at ICLR 2021
Vanilla, end training
Num. samples
0.2
0.4
0.6
0.8
Ours, early stopped
10k
Num. samples
Ours, end of training
Num. samples
5k
0.6
Confidence (i.c. probability)
Figure 18: Distribution of confidence for a model trained on the training set alone (Top) and a model trained both the training set and the arbitrarily labeled test set, with early stopping (Middle) and after 100 epochs (Bottom). The models in the vanilla ensemble are confident on both ID and OOD samples. The model trained on training+test is only confident on OOD data early during training.
30


Reviews:
[{"review_id": "vGWS3143ls1", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think the authors did a great job with the overall structure of the manuscript. It is great to see so many figures that illustrate the different settings, also it really helps to have the main research questions exposed right in the corresponding paragraph. \n\nOne superficial thing that could be improved is that often times I found the mathematical notation to be not very ‘convenient’ in that it wasn’t really necessary for a derivation or the like, but it made it really difficult to read the text, as it required jumping back and forth between the occurrences of a term and its definition, while the actual meaning of a term wouldn’t have taken much more space than its mathematical counterpart. \n\nA minor thing: the authors use the abbreviation RETO in the abstract without mentioning the full name. \n\nFigure 1 is great, I really appreciate these types of figures and I believe these comparisons are a great contribution to the community. Yet without details on the experimental settings that led to these results it is rather difficult to assess the value of this contribution. In particular as the figure is a central part of the motivation of the proposed approach, it would be great to at least reference the experimental details in Appendix A.1 - but even after having found this information (which shouldn’t be hidden from the reader for such essential parts), it remains a bit unclear how exactly the parameters were chosen and which experiments were performed exactly. \n\nOne of the main strengths of the paper is the extensive experimental validation. The authors did a great job at comparing with a variety of different methods. \n\nThe authors state that “For simplicity, we consider a noiseless setting, i.e. we assume that there exists a deterministic function f*”. It is perfectly fair to make such an assumption, but it is also important to keep in mind that in many real world application scenarios, especially when ground truth training data was acquired in an automated fashion or in crowdsourced annotation, label noise is not unlikely to violate that assumption. So it could help, for the assessment of the manuscript, to mention when this assumption would be violated in real world settings and how the proposed approach would be affected by such an assumption or a violation thereof.\n\nThe authors also state that “Empirically, this assumption [generalizability] seems to hold true for trained deep neural networks” and refer to Theisen 2020 - I might be missing something, but that reference is focussing on defining the “typical” rather than “worst” case, in particular for linear methods. In light of other findings like those that highlight that Neural networks will learn even noise labels perfectly fine, see e.g. Zhang et al https://arxiv.org/abs/1611.03530 it would be great to substantiate that claim a bit more. \n\nThe authors state that “Yu & Aizawa (2019) consider a similar transductive setting” and argue that that approach has the disadvantage that \"the method produces indistinguishable outputs on ID and OOD samples”. I’m not sure I understand why that distinction is an advantage, after all the ultimate goal would be to perform well on target test data, independent of whether that data is ID or OOD. \n\nA major limitation of the experimental validation is that bayesian methods are excluded from the evaluation. I have to admit I am not the most bayesian person I know, and I really enjoyed reading the Ovadia et al. 2019 NIPS paper, referenced by the authors, and I agree that this is a fair point, but after all this sort of transductive setting could be considered as one of the main strengths of bayesian/probabilistic networks. I would have expected to see at least one method from that field in the comparisons. \n\nOverall the experimental results are very impressive. Yet there are some methodological details that could be clarified, next to the points listed above. Most importantly it felt that the hyperparameters of all methods were chosen in a bit of an ad hoc fashion. I think it would substantially improve the results to employ standard practices for the hyperparameter optimization for all methods, even just grid search or random search would be fine.\n\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "b9l_7dojYHf", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new approach to detect out-of-distribution (OOD) examples in the transductive setting. The idea is to train an ensemble of models that fit the in-distribution (ID) data well, but disagree with each other on OOD examples.\n\nPros\n+ Extensive experiments are conducted to compare the proposed alogrithm with a number of baselines.  The proposed algorithm seems to significantly out-perform the baselines.\n+ The paper is generally well-written and easy to follow.\n\nCons\n- The concept of the F-detectable OOD set OOD(P, F) is somewhat misleading: the name suggests that it is a set of OOD examples, while in fact this set is not necessarily a subset of OOD examples, because when there are multiple empirical minimizer for P, they may disagree on some ID examples.\n- A key assumption (the empirical minimizers of F generalises well) is justified by a miscitation: the paper states that Theisen et al. 2020 shows that the assumption seems to hold true for trained deep neural networks, while that paper only discusses overparametrized linear classifiers.\n- Another key assumption (Assumption 2.1) seems rather strong and unlikely to hold in practice, and its validity seems to be overstated. Specifically, the assumption assumes an empirical minimizer of the training set augmented with ID test examples labelled using an arbitrary label is an empirical minimizer on the training set. This will imply that the ID test examples are not important in the underlying data distribution. While the papers states that \"In Section 3 we provide empirical evidence which shows that Assumption 2.1 holds in many practical scenarios\", there is no direct support for the assumption in Sec 3.\n- The paper states that OOD(P, \\tilde{F}) \\subset OOD(P, F), because \\tilde{F} \\subset F. This is not necessarily true, because the empirical minimizers in \\tilde{F} can disagree on more ID examples than the empirical minimizers in F.\n- Fig. 4 is used to justify that early stopping is an effective regularization method. However, the figure shows that at around epoch 35, there is a sharp drop of the validation accuracy, and early stopping may stop training here, making the network underfit.\n- The paper doesn't seem to describe how the test statistics are converted into predictions of whether the examples are OOD. This is important for reproducibility and is needed for justifying the soundness of the algorithm.\n\nMinor comments\n- Write down ID in full when it's used the first time\n- Fig. 2 is not mentioned in the main text. In addition, the caption doesn't seem to match the figure.\n- Fig. 3: what does zoro bias mean here?\n- Alg. 1: y_{1}, ..., y_{K} have already been used a the training set labels.\n- The \"two sample test\" doesn't really use two samples.\n\n**Post-rebuttal**\n\nThe revised version removed questionable or invalid notations/assumptions/justifications, and there are some defences for previous formal justifications in the rebuttal. IMHO, the rebuttal raises further concerns about the technical quality, and the paper still requires stronger justification for acceptance. I'll lower my score instead.\n\nSpecifically, I find the rebuttal generally confusing, and I disagree with various points in the rebuttal/revised version.\n\n- Condition 3.1 doesn't seem right: by definition, a classifier in \n has a misclassification probability of at most  on the ID data, but Condition 3.1 states that the misclassification rate is ? Importantly, I don't see why this condition is needed as there is no justification on how it is connected to the disagreement test. In addition, \"As a consequence, with very high probability 1 − (1 − δ)s we cannot fit a set of s random i.i.d. in-distribution points with the wrong label\" is quite vauge and doesn't seem right either.\n- \"According to the definition, there exists a class of functions F that is complex enough such that OOD(P, F) is the complement of the support of the training distribution.\": I don't think the definition implies this. In addition, if OOD(P, F) is the complement of the training distribution, then ID examples not in the training distribution are included in OOD(P, F).\n- \"If we defined OOD(P_n, F) with P_n the empirical distribution, then the reviewer would be correct and indeed this set could contain ID samples.\": in theory, OOD(P, F) CAN contain ID samples, unless additional assumptions are made.\n- \"two-sample test\": while I think this is a minor issue to call the test a two-sample test, I still don't think it agrees with the standard usage of the term (which means two random samples are used).\n- \"one may simply pick the time point with the highest validation accuracy, after training for a fixed number of epochs\": I doubt that this can be called \"early stopping\". So I don't think the response addresses my question regarding Fig. 5 (previous Fig. 4). In addition, since \"early stopping\" is used for regularization, this makes it questionable whether \"regularization\" is indeed needed.\n- The test statistic is included in the main paper now, but how about the threshold? In rigorous statistical testing, the threshold can be rigorously calculated, while in this case, it is not clear how the threshold is set.", "extra_scores": {}, "extra_text": {}}, {"review_id": "J8Zp-VFfnxE", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a technique to detect OOD data using disagreement among an ensemble of classifiers. It assumes a transductive setup where OOD data is available during training time.\n\n\n1. The application setting for OOD as stated in the paper is incorrect:\n\nAbstract: \"For settings where the test data is available at training time..\"\n\nIntroduction: \"..Most prior work approaches OOD detection inductively, trying to infer OOD samples after only observing the (ID) training data,..\"\n\nThe main property defining OOD data is that it is never seen during training. The problem fundamentally is that of detecting unknown-unknowns. This is what makes the problem hard and sets it apart. The proposed approach violates this and hence is not justified as an OOD detection algorithm. The paper text (including title) should be changed.\n\nThe proposed technique should instead be positioned as one that helps in discriminating between two known distributions. In that respect, the baseline/competitor algorithms should be selected appropriately. The current choices (except Malalanobis-T) are biased towards the proposed algorithm.\n\n2. Section 2, ID data.: It is assumed for simplicity that a deterministic function f* maps ID data to correct labels. However, this assumption looses generalizability. Considering that real world data has a lot of labeling noise, most such functions as f* would end up memorizing the train data and would have bad performance on ID data in the test set. The paper is not clear what would happen if the train data is noisy.\n\n3. Section 2, Transductive OOD Detection: Here the assumption is that there is lot more correctly labeled train data than test data. The basis for this assumption is not clear and clearly limits the applicability. In most transductive cases, the opposite is true: there is more unlabeled data than labeled, and semi-supervised techniques are used to reduce human labeling effort.\n\n4. Section 2.2, Regularized ensembles: \"..points in T for which at least two models in the...\" -- Does that not make the model fragile? In case the ensemble has 20 members and only one member disagrees with 19 others, would it be justified to label the data as outlier/OOD?\n\n5. Section 2.2: \"...our method indicates that using the regularized function class suffices for many hard OOD detection scenarios.\" -- This is an overstatement considering that results were presented on only a couple of datasets (CIFAR variants, SVHN)\n\n6. Section 3.2, Statistical test for OOD detection: \"The null hypothesis is accepted for high values of Tavg-TV.\" -- I am missing something here: if there is a lot of disagreement among ensembles, then d_TV would be high and as a result for such data Tavg-TV would be also high. Hence, high value of Tavg-TV should probably result in rejecting of null hypothesis instead of accepting.\n\n\n============================\nUpdate after going through the updated paper and discussions\n---------------------------------------------\nThe paper adds better illustrations in the revised paper to explain the technique and the approach seems to be effective even though it is simple. Given the approach, 'transductive' in the title looks okay.\n\nFigure 1, Figure 3, and Figure 4 in the current version suggest that the one-class SVM/SVDD techniques [1, 2] would be important to compare against. Popular anomaly/outlier detection algorithms Isolation Forest [3], LOF [4] would also be relevant here. The paper can be strengthened with these additional baselines. RETO has at least one short-coming over the outlier detector techniques: it needs to wait some time until it can collect enough test data.\n\n[1] Lukas Ruff, Robert A Vandermeulen, Nico Görnitz, Lucas Deecke, Shoaib A Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In ICML, volume 80, pp. 4390–4399, 2018.\n\n[2] Lukas Ruff, Robert A Vandermeulen, et. al., Deep Semi-Supervised Anomaly Detection, ICLR 2020.\n\n[3] Liu, Fei Tony; Ting, Kai Ming; Zhou, Zhi-Hua (December 2008). \"Isolation-based Anomaly Detection\". ACM Transactions on Knowledge Discovery from Data\n\n[4] Breunig, M. M.; Kriegel, H.-P.; Ng, R. T.; Sander, J. (2000). LOF: Identifying Density-based Local Outliers. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. SIGMOD.", "extra_scores": {}, "extra_text": {}}, {"review_id": "EgdUVjmSXo0", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The problem of good predictive uncertainty-based out of distribution (OOD) detection is essential for classification systems to be deployed in safety-critical environments. The authors present a method RETO that achieves state-of-the-art performance in a transductive OOD detection setting. Like other predictive uncertainty-based approaches RETO can ultimately be used downstream on problems like active learning or abstaining on OOD samples in combination with selective classification.\n\nBenchmark data such as CIFAR, SVHN, and MNIST are used to compare conventional and proposed baseline methods, such as k-NN, Vanilla Ensembles OE, Mahal, Mahal-T, and MCD. Experimental results, including those of supplemental materials, show that the proposed method provides good accuracy while reflecting the hardness of the task.\n\nHowever, there is not enough discussion on how the proposed method can achieve such a high level of accuracy compared to the conventional methods; early stopping is used in RETO, but is it promised to reproduce the same level of performance in other tasks?\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Although the rebuttal helped clarify the reviewers' confusion on notational confusion and the motivation of problem setup, all reviewers are still in a position of unable to championing the paper:
- the technical concerns by Reviewer 4 need to addressed
- the paper would have been stronger if baselines such as one-class classification / outlier detection have been compared
- the algorithm also has at least one short-coming over other techniques that it needs to wait some time until it can collect enough test data

We hope the reviews can help the authors strengthen the paper in the next revision. 
Decision: Reject