## 5 Conclusions  
In this study, we presented a detailed study of a CNN trained on mammograms in an explainable way. We trained a CNN classifier on a wide set of clinical mammograms to classify them according to breast density and then we implemented an explanation algorithm to explore the CNN behavior on different input data. We evaluated the CNN performance using different distributions of class labels in the training and test sets, and different pre-processing steps, taking into account the accuracy, precision and recall figures of merit, and the saliency maps obtained with the grad-CAM algorithm. This approach can be extended to other medical images in the attempt to provide clinicians with reliable and explainable AI-based decision support tools. Conceptualization, F.L. ([PERSON]), C.S. and M.E.F. ; Data curation, F.L. ([PERSON]) and C.S. ; Formal analysis, F.L. ([PERSON]) and C.S. ; Funding acquisition, M.E.F. and A.R. ; Investigation, F.L. ([PERSON]) and F.L. ([PERSON]); Methodology, F.L. ([PERSON]) and A.R. ; Project administration, M.E.F. ; Resources, F.L. ([PERSON]); Software, C.S. ; Supervision, F.L. ([PERSON]), M.E.F. and A.R. ; Writing--original draft, F.L. ([PERSON]), C.S., M.E.F. and A.R. ; Writing-review & editing, F.L. ([PERSON]), C.S. and F.L. ([PERSON]). All authors have read and agreed to the published version of the manuscript. This work was carried out with support by INFN (National Institute for Nuclear Physics) in the framework of the experiment AIM (Artificial Intelligence in Medicine). It was also supported by Physics Department of Pisa University and SNS (Scuola Normale Superiore). The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Review Board constituted by the Radiologist responsible of the Diagnostic Senology Operative Unit of AOUP and the full professor of the Diagnostic and Interventional Radiology Department of University of Pisa. Ethics Committee approval was waived because it was not necessary for this retrospective study: the collection of negative mammograms started in 2011. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Image data are not public available, but can be obtained upon request to the authors. A research agreement for their use would be needed. This work has been carried out within the Artificial Intelligence in Medicine (AIM) project (INFN-CSN5, 2019-2021, [[https://www.pi.infn.it/aim](https://www.pi.infn.it/aim)]([https://www.pi.infn.it/aim](https://www.pi.infn.it/aim)), accessed on 21 December 2021). We acknowledge also Fondazione Pisa, Technological and Scientific Research Sector, for the support to the RADIOMA (2017-2020) project ([[https://fondazionepisa.it/progetti-finanziati/radiazioni-ionizzanti-in-mammografia-conoscerle-per-limitarle-sviluppo-di-un-indicatore-di-dose-innovativo-in-vista-del-recepimento-della-nuova-direttiva-europea/](https://fondazionepisa.it/progetti-finanziati/radiazioni-ionizzanti-in-mammografia-conoscerle-per-limitarle-sviluppo-di-un-indicatore-di-dose-innovativo-in-vista-del-recepimento-della-nuova-direttiva-europea/)]([https://fondazionepisa.it/progetti-finanziati/radiazioni-ionizzanti-in-mammografia-conoscerle-per-limitarle-sviluppo-di-un-indicatore-di-dose-innovativo-in-vista-del-recepimento-della-nuova-direttiva-europea/](https://fondazionepisa.it/progetti-finanziati/radiazioni-ionizzanti-in-mammografia-conoscerle-per-limitarle-sviluppo-di-un-indicatore-di-dose-innovativo-in-vista-del-recepimento-della-nuova-direttiva-europea/)), accessed on 21 December 2021). The authors declare no conflict of interest. The following abbreviations are used in this manuscript: