  We reconsider the evaluation of OOD detection methods for image recognition.
Although many studies have been conducted so far to build better OOD detection
methods, most of them follow Hendrycks and Gimpel's work for the method of
experimental evaluation. While the unified evaluation method is necessary for a
fair comparison, there is a question of if its choice of tasks and datasets
reflect real-world applications and if the evaluation results can generalize to
other OOD detection application scenarios. In this paper, we experimentally
evaluate the performance of representative OOD detection methods for three
scenarios, i.e., irrelevant input detection, novel class detection, and domain
shift detection, on various datasets and classification tasks. The results show
that differences in scenarios and datasets alter the relative performance among
the methods. Our results can also be used as a guide for practitioners for the
selection of OOD detection methods.
