## 5 AI Regulations, Verification, and Ethical Problems
### Verification of the Model
#### 5.2.2 Explainability  
Explainability [145] is a relatively new term that lacks a single, definitive definition. In [146], the following definition is presented: \"Explainability is associated with the notion of explanation as an interface between humans and a decision maker that is, at the same time, both an accurate proxy of the decision maker and comprehensible to humans\". In [147], the definition of explainability is as follows: \"An AI system is explainable if the task model is intrinsically interpretable\".  
The explainability of AI models in dermatology refers to the ability to understand and interpret the behaviour of algorithms that analyse skin images and make diagnostic decisions [129; 148; 149; 150; 151]. This is a key aspect in medical applications, as it allows doctors and patients to trust AI's decisions and understand why a model produces a given diagnosis.  
Model explainability, especially in medicine, is crucial because it increases doctors' and patients' trust in automatic diagnoses. AI models can be a powerful tool to support doctors, but their decisions must be understandable and verifiable in order to be accepted in clinical practice. As shown in the previous section, AI models can make mistakes. If a doctor understands the basis for the model's decision, they can detect potential errors more quickly. Moreover, a key aspect in the context of explainability is also meeting legal regulations. The FDA [120] and the EU's AI Act [119] require explainable models in medicine to ensure compliance with ethics and patient safety. Another example of a phenomenon that can be addressed by using explainability methods is bias reduction. Models can be susceptible to biases resulting from the uneven representation of patient groups. Explainability helps to detect and eliminate all of the above-mentioned problems.  
There are several popular methods of explaining AI decisions in dermatology. These include, among others, attention heatmaps (saliency maps [152], Grad-CAM [153], LIME [72], SHAP [154]), which highlight the areas of the skin image that were crucial for the model's decision. They help doctors to understand whether the model focused on a mole or, for example, on the skin background. Another method is a comparison with similar cases. Models can explain their decisions by indicating similar cases from a database, e.g., \"This mole is similar to others that turned out to be melanoma\". In feature-based models (e.g., classic ML algorithms), one can check which features (colour, shape, texture) had the most significant impact on the diagnosis. This method is called model feature analysis. Counterfactual explanations demonstrate how a minor change in the image (e.g., a change in the colour of a mole) can impact the model's decision, which helps to clarify its operation.  
A clinical trial (reader study) [155] of 76 dermatologists evaluated the impact of explainable AI (XAI) on melanoma diagnosis. Dermatologists diagnosed dermoscopic images of skin lesions with the support of an XAI model that provided explanations for their decisions, and their activity was tracked using eye tracking. The explainable AI model improved dermatologists' sustainable diagnostic accuracy by 2.8 percentage points compared to the standard black-box model. Furthermore, the presence of XAI explanations increased doctors' confidence in AI decisions and revealed patterns of higher cognitive load (more eye fixations) in cases of diagnostic discrepancies.  
However, explainability methods are expensive. The cost is influenced by, among other aspects, the complexity of neural networks. Moreover, modern models, such as CNNs and Transformers, are challenging to understand, even for experts. There are still no standards for explainability, and different methods can provide different explanations, complicating their interpretation. Another problem is the limited availability of data, as models are often trained on small datasets, which can lead to misinterpretations.