  Early detection of skin cancers like melanoma is crucial to ensure high
chances of survival for patients. Clinical application of Deep Learning
(DL)-based Decision Support Systems (DSS) for skin cancer screening has the
potential to improve the quality of patient care. The majority of work in the
medical AI community focuses on a diagnosis setting that is mainly relevant for
autonomous operation. Practical decision support should, however, go beyond
plain diagnosis and provide explanations. This paper provides an overview of
works towards explainable, DL-based decision support in medical applications
with the example of skin cancer diagnosis from clinical, dermoscopic and
histopathologic images. Analysis reveals that comparably little attention is
payed to the explanation of histopathologic skin images and that current work
is dominated by visual relevance maps as well as dermoscopic feature
identification. We conclude that future work should focus on meeting the
stakeholder's cognitive concepts, providing exhaustive explanations that
combine global and local approaches and leverage diverse modalities. Moreover,
the possibility to intervene and guide models in case of misbehaviour is
identified as a major step towards successful deployment of AI as DL-based DSS
and beyond.
