Paper: Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (ICLR-2024)

Reviewer comment (Experiments):
The ensemble of OOD detection methods seems ad-hoc for this benchmark by evaluating and picking some of the methods that perform relatively well on the benchmark.

Author response:
As stated page 7, at the end of Section 3, we do not assume access to OOD samples to pick which scores to use for ensembling, precisely to avoid the issue mentioned by the referee. Instead, we describe the selection process for the different ensemble versions, which are only based on in-distribution ImageNet validation inputs, and are thus independent from performances on BROAD. We could likely achieve better performance by picking scores based on BROAD, but refrained from it in agreement with the referee’s remark.

Reviewer comment (Theory):
The proposed method of fitting GMM over scores from different OOD detection methods does not make sense to me. For example, in Sec.3, it says "this approach is adept at identifying atypical realizations of the underlying scores, even in situations where the marginal likelihood of each score is high, but their joint likelihood is low." It would be weird if most in-distribution samples can not achieve high likelihood at each score while out-of-distribution samples can, since all these methods aim at measuring if the sample is in-distribution. In other words, what is the advantage of fitting GMM over taking average of different scores?

Author response:
Overall, generative modeling with GMM is better suited because it models exactly the quantity of interest: how likely is it to observe the given set of scores, assuming we are in-distribution ? While the referee’s suggested heuristic discards valuable information.

While the referee says “these methods aim at measuring if the sample is in-distribution”, and it is indeed the claimed goal, our work precisely establishes that due to standardization of evaluation benchmarks in the literature, these methods instead only discriminate between in-distribution and novel classes. Their behavior on other types of distribution shifts is *a priori* uncertain. Some methods have below random AUC performances for certain distribution shifts (see table 2), which demonstrates it is not always the case that a lower score means a higher chance of being in-distribution (instead, it means a higher chance of not being a novel class).

Moreover, the sentence cited by the referee simply points out that generative modeling is able to model information in the joint distribution that is not present in marginal distributions. For instance, EBO and MaxLogits are heavily correlated in-distribution (see Figure 3). Thus, even when the scores of EBO and MaxLogits are both within reasonable range, a lower-than-usual score for EBO and higher-than-usual score for MaxLogits have low joint density, which would not be captured by marginal probabilities.
 
Additionally, nothing indicates that the distribution of scores must be single mode. As clearly shown in Table 7 (appendix), using a mixture of several Gaussians performs better than a single Gaussian. An average of scores might be unadapted if the distribution of each score has different modes.

Finally, even if we rescale scores to balance their variance, depending on the functional form of their distribution they may not trigger false positives at the same frequency, and redundant scores would have exaggerated weights.

Reviewer comment (Evaluation):
The time complexity or the scalability of the proposed method is still of concern. Though the results shows that the time complexity of the proposed ENS-F is acceptable at 25% of the time for a normal inference. However, when comparing to the methods it ensembles (e.g. MSP takes 1% additional time), the time complexity is extremely high.

Author response:
As the referee acknowledged, a 25% overhead is acceptable. While some methods indeed have faster inference, half of the methods have comparable – or even much higher, overhead. Many acclaimed recent works such as ViM and ODIN have overheads larger than 100% and were still found to be of high interest to the community. Besides, several of the fastest methods (MSP, max logits and logits norm), are simple baselines commonly used in the literature and unrepresentative of typical overhead in published methods. We believe Ens-F has, in fact, a smaller overhead than most methods introduced in the past 3 years, and it should thus not be considered a limitation.

Reviewer comment (Experiments):
The proposed method uses a validation set to fit GMM, does it affect the performance? With the use of additional data, is it fair to compare the proposed method with other baseline methods?

Author response:
It is important to note that the training of the GMM uses 45,000 **in-distribution** validation samples, from the validation set of ImageNet (the comparison would indeed be unfair if our ensemble had access to some OOD samples). First, it is unlikely that the use of 45,000 additional images compared to the 1.2 millions samples in ImageNet 2012’s training set would make a significant difference. But most importantly, it is common for the methods in the literature to use validation data for various purposes, such as tuning (CADet), learning parameters (MDS), or calibrating (ViM). Since we use the same validation data as most methods we compare to, we believe it is fair. Instead, it can introduce a slight overfitting of the underlying scores to the validation data on which the GMM is trained, hindering performances, but we find the GMM still largely outperforms underlying methods.

Reviewer comment (Experiments):
Does the proposed ensemble method outperform simple ensemble methods such as taking the average of the scores or the largest score? If so, why would it be?

Author response:
We did not evaluate the heuristics mentioned by the referee. Firstly, because a generative approach seemed better suited due to the elements mentioned in the response to weakness 2. Secondly, because we wanted to refrain from picking the ensemble method based on BROAD performances, which would incur risks of overfitting our specific benchmark, and may lead to bad generalization on unseen OOD datasets.

Reviewer comment (Theory):
The major weakness of the paper is that most OOD detection scores considered in the paper are proposed to only handle novel classes. Expecting such OOD scores to detect adversarial perturbations and corruptions may be **out-of-scope** and unrealistic. In particular, recent work [1] has demonstrated that when OOD samples are not involved during training (the setting considered in this work), it can be **theoretically impossible** to expect common OOD detection methods to work. Despite that detecting adversarial perturbations and corruptions are interesting tasks, directly utilizing post-hoc OOD detection scores is ill-justified and the failure is expected.

Author response:
We agree OOD detection scores considered in the paper are only suited for novel class detection (though some of them generalize well on certain other types of shifts). In fact, **it is exactly our point**: while described as “OOD detection” methods, these are, in fact, “novel class detection” methods. While they are virtually always justified in the literature by the need to address unexpected test-time inputs, they are in fact only evaluated on novel class benchmarks. There is thus a discrepancy between the role that these methods are supposed to play, and what they are actually evaluated on and suitable for. Our work proposes to fix this issue by introducing a more comprehensive evaluation benchmark. The fact that these methods generalize unreliably to other types of shifts is indeed not surprising, since they were not evaluated on them in their original papers. However, it justifies that, in fact, there is a misalignment between the literature and the needs of real-world systems, where a large variety of unexpected inputs can occur. Existing “OOD detection” methods fail to detect certain types of OOD inputs encountered in real-world settings, as demonstrated by our work. In order to move toward useful systems, we need to move toward a more realistic definition of OOD detection, and understand how existing methods perform in such a setting, which is exactly what our submission does. Thus we believe it is not out-of-scope to evaluate OOD detection methods on distribution shifts they were not designed for – but that occur in the type of problem they aim to address.

[1] Is a very interesting work that establishes that it is impossible to discriminate the training distribution from *all* possible distribution shifts. However, it does not mean that it is impossible to discriminate from the specific distribution shifts encountered in practice, which are only a small subset of all possible shifts. In fact, the performance of our ensemble method formally demonstrates that, at least to a reasonable extent, it is possible to simultaneously detect all the distribution shifts considered in BROAD. We argue there is a reasonable middle point to find between only considering detection of one specific and limited type of shift, and trying to discriminate against all possible (in the mathematical sense) shifts. That middle point should be to attempt detection of the shifts reasonably encountered in practice, and our work shows that, though difficult, it is not infeasible.

Reviewer comment (Experiments):
In the multi-label scenario, it seems more reasonable to use object detection models instead of classification models. The failure of OOD detection based on classification models is expected. It would be more interesting to see the performance of OOD detection given bounding boxes.

Author response:
Indeed, using an object detection method would be much better suited for *cocomagenet* samples, which would in that case not even be considered OOD. But the principle of OOD detection is to protect against input types that *are not expected*, thus we can not assume a priori knowledge of these input types to select a better suited model or paradigm. There are many classification models deployed in the real world, which are designed to handle single-class inputs. Due to unexpected use, they may occasionally be presented with inputs that, in fact, have several training classes present. Since such cases are uncommon and out of normal usage, it is often not reasonable to consider an object detection framework just to palliate this eventuality. This is connected to the general discrepancy noted by our work between real-world needs and the literature: the standardization of evaluation benchmarks has led recent OOD detection methods to only consider expected OOD inputs, although their point is precisely to detect inputs that are **not** expected.

Reviewer comment (Theory):
Can authors justify in theory or principle why OOD detection methods are suitable for detecting adversarial perturbations and corruptions?

Author response:
**Q1:** Certain of these methods have been designed for simultaneous detection of adversarial samples (e.g., CADet) while others have been found in prior work to be powerful baselines in detecting adversarial inputs (e.g., ODIN and MDS). However, as argued above, our point is precisely to demonstrate that existing OOD detection methods are in general not suitable to detect other shifts encountered in practice.