Deep Neural Networks are powerful models that attained remarkable results on a variety of tasks. These models are shown to be extremely efficient when training and test data are drawn from the same distribution. However, it is not clear how a network will act when it is fed with an out-of-distribution example. In this work, we consider the problem of out-of-distribution detection in neural networks. We propose to use multiple semantic dense representations instead of sparse representation as the target label. Specifically, we propose to use several word representations obtained from different corpora or architectures as target labels. We evaluated the proposed model on computer vision, and speech commands detection tasks and compared it to previous methods. Results suggest that our method compares favorably with previous work. Besides, we present the efficiency of our approach for detecting wrongly classified and adversarial examples. Recently, several studies proposed different approaches to handle this uncertainty [13, 25, 23, 19] . In [13] the authors proposed a baseline method to detect out-of-distribution examples based on the models' output probabilities. The work in [25] extended the baseline method by using temperature scaling of the softmax function and adding small controlled perturbations to inputs [14] . In [23] it was suggested to add another term to the loss so as to minimize the Kullback-Leibler (KL) divergence between the models' output for out-of-distribution samples and the uniform distribution. Ensemble of classifiers with optional adversarial training was proposed in [19] for detecting out-ofdistribution examples. Despite their high detection rate, ensemble methods require the optimization of several models and therefore are resource intensive. Additionally, each of the classifiers participated in the ensemble is trained independently and the representation is not shared among them.
Shaleve et al. REF used multiple semantic dense representations of the target labels to detect misclassified and adversarial examples.
Out-of-Distribution Detection using Multiple Semantic Label Representations
ArXiv
ArXiv
Computer Science Mathematics