Impact statement: Out-of-distribution detection is crucial for robust, reliable and fair machine learning systems. Mitchell et al. [27] and Gebru et al. [13] argue that applying machine learning models outside of the context where they were trained and tested can lead to dangerous and discriminatory outcomes in high-stake domains. We hope that our work will generally contribute to the understanding of out-of-distribution detection and facilitate methodological progress in this area.