  We develop and rigorously evaluate a deep learning based system that can
accurately classify skin conditions while detecting rare conditions for which
there is not enough data available for training a confident classifier. We
frame this task as an out-of-distribution (OOD) detection problem. Our novel
approach, hierarchical outlier detection (HOD) assigns multiple abstention
classes for each training outlier class and jointly performs a coarse
classification of inliers vs. outliers, along with fine-grained classification
of the individual classes. We demonstrate the effectiveness of the HOD loss in
conjunction with modern representation learning approaches (BiT, SimCLR, MICLe)
and explore different ensembling strategies for further improving the results.
We perform an extensive subgroup analysis over conditions of varying risk
levels and different skin types to investigate how the OOD detection
performance changes over each subgroup and demonstrate the gains of our
framework in comparison to baselines. Finally, we introduce a cost metric to
approximate downstream clinical impact. We use this cost metric to compare the
proposed method against a baseline system, thereby making a stronger case for
the overall system effectiveness in a real-world deployment scenario.
