# Intelligent System Using Data to Support Decision-Making
## 2 Materials and Methods
* Separability--the metric states that if there are two distinct cases, they must also have distinct interpretations. The metric assumes that the model has no degrees of freedom, that is, all features used in the model are relevant for prediction. If we want to measure the separability metric, we choose a subset S from the test dataset that does not contain duplicate values and then obtain their interpretations. Then, for each instance of \(s\) in \(S\), we compare its interpretation with all other interpretations of instances in \(S\), and if such an interpretation does not have a duplicate, then we can say that it satisfies the separability metric. Using this metric, we will reveal how well the model explanations are separated between different classes. Higher values indicate a better ability of the model to distinguish between classes based on the explained attributes. * Stability--this metric states that instances belonging to the same class must have comparable interpretations. The authors first grouped the interpretations of all instances in test dataset using the K-means algorithm, so that the number of clusters is equal to the number of labels in the data. For each instance in the test dataset, they compared the cluster label assigned to its interpretations after clustering with the predicted class label of the instance; if they match, the interpretation satisfies the stability metric. This metric could help us evaluate the consistency of model explanations across training datasets. The higher the stability, the more consistent and reliable the explanations. * Understandability--helps us evaluate whether the explanations are easy for doctors to understand. It is related to the ease with which an observer understands the explanation. This metric is crucial, because no matter how accurate an explanation may be, it is useless if it is not understandable. It is related to how well people understand the explanations [44]. According to [PERSON] et al. [45], understandability is understood as the confidence that a domain expert can understand the behavior of algorithms and their inputs, based on their everyday professional knowledge. The doctor assigns a value from 1 to 5 based on the Likert scale, with 1 being the lowest and 5 being the highest. New proposed metrics:
* User experience--evaluates how well the model explanations fit into the overall user interface and interaction between individual system functionalities. A good user experience improves user engagement and trust. This metric was created based on an understanding of how usability is evaluated using the ISO/IEC 25002 [46] quality assessment metrics. The doctor assigns a value from 1 to 5 based on the Likert scale. * Response time (Effectiveness)--assesses how quickly and effectively the model generates explanations. Higher efficiency reduces the time needed to explain the model. Doctors will evaluate this metric based on a Likert scale from 1 to 5, where \(\odot\) If the explanation generation time is up to 30 s, the doctor will assign a value of 5. \(\odot\) If the explanation generation time is between 30 and 60 s, the doctor assigns a value of 4. \(\odot\) If the explanation generation time is more than 60 s, the doctor assigns a value 3. \(\odot\) If the explanation generation time is between 2 and 5 min, the doctor assigns a value 2. \(\odot\) If the explanation generation time is more than 5 min, the doctor assigns a value 1. This combination of metrics could provide confirmation of trustworthy, high-quality, and understandable explanations in the field of explanatory model evaluation, with the aim of achieving the shortest possible time for generating such models, increasing trust in the models, reducing clinician decision-making time, and understanding new factors that may influence predictions. We used a dataset containing 3848 COVID-19 patient records from the Department of Infectology and Travel Medicine, Faculty of Medicine, L. Pasteur University Hospital (UNLP) Kosice, Slovakia.