## 4 Discussion  
### Interpretability  
Because AI-based healthcare systems may have an impact--either positive or negative--on people's lives, it is fundamental to dispel the myths around these black boxes in order to  
Figure 3: General perception of the survey respondents stratified by area of expertise and by years of experience.  
increase trustworthiness [56; 57]. There are some works that focus on specific AI techniques in healthcare and interpretability, such as common methods, features, training and testing methodologies, metrics, and current challenges relating to implementation in clinical practice (these aspects could vary in terms of the project aim, disease, and input/output sources) [30; 31]. It is, consequently, no surprise that interpretability has become a prominent subject in AI for healthcare applications in recent years [39; 58; 59; 60; 61; 62]. However, it remains an open problem due to current theoretical gaps between model complexity and explainability; insufficient feedback from all stakeholders throughout the development process (e.g., via evaluations of user satisfaction, explanation goodness, acceptance, and trust in AI-based systems) [17; 35]; and inadequate traceability indicating how models reach conclusions [39; 63]. According to our international survey, a fundamental issue that needs to be addressed is the unequal understanding among professionals: most healthcare professionals do not understand how AI systems operate nor whether these adhere to medical guidelines. However, participants also manifested that the level of granularity is essential. While a certain degree of clarity of the whole process--equations, biography, medical guideline, and algorithms--seems reasonable, providing excessive details may be confusing and of no practical value. We propose the following recommendations for better interpretability:  
* _Involvement of all stakeholders throughout all phases of the software development life cycle._ The development of a healthcare system leveraging AI is a multidisciplinary process that requires continuous communication between system creators and those affected by their use (e.g., healthcare professionals, policy-makers, and patients) [17; 64]. This ensures explanations are clear and sufficient, real interpretability needs are addressed timely, and limitations are known by the target population.
* _Interpretability over complexity_. Complexity is entangled with interpretability: predictions made by a complex model are more difficult to comprehend and explain than a simpler one. Since trust in AI systems can only be improved by improving interpretability, the use of simpler yet interpretable methods should be considered for healthcare applications over black boxes [39; 64].
* _Uncertainty quantification_. Real-world data are far from perfect; they contain missing values, outliers, and invalid data. Inputting it into models trained and tested on well-selected datasets may result in flawed verdicts [65]. Quantifying uncertainty for a certain input-output pair thus allows developers and users to understand whether they can trust a model's prediction, whether the cause of the problem is in the input data or the AI system's inability to handle such a case. It should be noted that the latter could be employed to improve model robustness.
* _Evaluation beyond accuracy_. _\"[...]_ AI research should not be limited to reporting accuracy and sensitivity compared with those of the radiologist, pathologist, or clinician\" [66]. Performance evaluation should be accompanied by a thorough and multidisciplinary assessment of interpretability [67]. Evaluations striving to explain the reasoning behind a certain prediction can improve our understanding of healthcare. Assessing interpretability and explainability requires maturation [39].