# Intelligent System Using Data to Support Decision-Making
## 3 Results
### CDSS-EQCM Testing  
We performed on-site manual acceptance testing at UNLP Kosice with five infectious-disease physicians, using a set of predefined scenarios. 1. Test Scenario 1: Evaluating the Interpretability of Explainable Models Using Evaluation Methods. 2. Test Scenario 2: Displaying the Best Option from the Previous Evaluation. 3. Test Scenario 3: Verification of the Application's Functionality and Transparency. Testing occurred in two phases: the first phase uncovered the need for additional data preprocessing, leading to refinement; the second phase validated the fully integrated system locally. The doctors performed the second phase of all three test scenarios, from which we obtained a relevant evaluation of the explainable methods. The doctor reviews all models explained using SHAP and evaluates all the metrics for the SHAP explanation. The metric evaluation results are saved and displayed in Table 3. We continuously consulted with physicians throughout development. Testing was carried out in several phases; after each phase, we gathered their feedback and integrated all recommendations. Following the final round of testing, we concluded that the system shows strong potential and outlined further enhancements for future work. When the doctor decided to evaluate using the TOPSIS method an evaluation mechanism was displayed, along with a choice of metric preferences required for multi-criteria decision-making and the Saaty matrix, see Table 4. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Separability** & **Stability** & **Accuracy** & \
\begin{tabular}{c} **Response** \\ **Time** \\ \end{tabular} & **Understandability** &
\begin{tabular}{c} **User** \\ **Experience** \\ \end{tabular} \\ \hline Separability & 1.00 & 0.20 & 0.14 & 0.20 & 0.14 & 0.14 \\ \hline Stability & 5.00 & 1.00 & 0.14 & 0.11 & 0.11 & 0.14 \\ \hline Accuracy & 7.00 & 7.00 & 1.00 & 3.00 & 0.33 & 3.00 \\ \hline Response time & 5.00 & 9.00 & 0.33 & 1.00 & 3.00 & 5.00 \\ \hline Understandability & 7.00 & 9.00 & 3.00 & 0.33 & 1.00 & 5.00 \\ \hline User experience & 7.00 & 7.00 & 0.33 & 0.20 & 0.20 & 1.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Final table—Saaty matrix. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**ML Method** & \
\begin{tabular}{c} **Response** \\ **Time** \\ \end{tabular} & **Understandability** &
\begin{tabular}{c} **User** \\ **Experience** \\ \end{tabular} & **Separability** & **Stability** & **Accuracy** \\ \hline RF LIME & 1.00 & 0.80 & 1.00 & 1.00 & 0.76 & 0.98 \\ \hline RF SHAP & 1.00 & 1.00 & 1.00 & 1.00 & 0.92 & 0.98 \\ \hline DT LIME & 1.00 & 0.80 & 0.80 & 1.00 & 0.65 & 0.97 \\ \hline DT SHAP & 1.00 & 1.00 & 1.00 & 1.00 & 0.90 & 0.97 \\ \hline LR LIME & 0.80 & 1.00 & 0.80 & 1.00 & 0.75 & 0.95 \\ \hline LR SHAP & 0.80 & 1.00 & 1.00 & 1.00 & 0.92 & 0.95 \\ \hline k-NN LIME & 0.60 & 0.80 & 0.60 & 1.00 & 0.84 & 0.88 \\ \hline k-NN SHAP & 1.00 & 1.00 & 1.00 & 1.00 & 0.95 & 0.88 \\ \hline SVM LIME & 0.80 & 0.80 & 0.60 & 1.00 & 0.55 & 0.95 \\ \hline SVM SHAP & 1.00 & 1.00 & 1.00 & 1.00 & 0.94 & 0.95 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Resulting table of metrics evaluated by the doctor. Then, the doctor selected the preferred metrics indicating which of the two is more important, and a pairwise comparison is performed. After evaluation, the models table was displayed with weights based on the geometric mean calculation in Table 5, along with a table of metric values and a list of models ranked in descending order in Table 6. The model selected by the doctor as the best was saved to the database. In our case, the doctor chose the model identified as the best by the Borda method (Random Forest with SHAP visualization). The testing revealed that all the actual system responses were identical to the expected system responses. The aim of this testing was to identify any potential system errors and gather feedback from the domain expert, which would allow us to continue our research and consider potential system improvements.