# An Explainable Artificial Intelligence Approach for Detecting Empathy in Textual Communication
## 6 Results and Discussion
### Classification Results  
Tables 4 and 5 show the results of all the cross-validation dataset pairs. We can see that there is a clear distinction between each classifier involved. Additionally, we can spot trends in these datasets that are held in multiple circumstances. For example, we can point out that the performances of PBC4 cip, Random Forest, and MLP usually obtain the highest scores in both metrics. Meanwhile, the algorithm that tended to perform worse was the Gaussian Naive Bayes classifier. These trends would be relevant on the statistical analysis. The performance of the metrics across the cross-validation process can be seen in Figure 7. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Fold**} & **Random** & \multirow{2}{*}{**KNN**} & **Naive** & \multirow{2}{*}{**DT**} & **MLP** & **PBC4 cip** \\  & **Forest** & & & **Bayes** & & \\ \hline
1 & 0.6199 & 0.5614 & 0.5251 & 0.5665 & 0.5930 & 0.6530 \\
2 & 0.6173 & 0.5603 & 0.5278 & 0.5372 & 0.6000 & 0.5850 \\
3 & 0.6170 & 0.5104 & 0.5431 & 0.5219 & 0.6180 & 0.6380 \\
4 & 0.5831 & 0.5242 & 0.5438 & 0.5341 & 0.5853 & 0.6190 \\
5 & 0.6170 & 0.5720 & 0.5551 & 0.5069 & 0.6014 & 0.6290 \\ \hline
**avg** & **0.6108** & **0.5456** & **0.5389** & **0.5333** & **0.5995** & **0.6248** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Cross-validation AUC results. We can infer that there is a difference between Random Forest, MLP, and PBC4 cip when compared to the rest of the classifiers: The former group performs better than the latter. It is important to note that this behavior can be seen using both of our metrics, with a difference in magnitude. This was interesting, as both metrics support the three classifiers being clearly better than the others. This would have to be analyzed using statistical tools in order to see if this difference was significant. To test if these algorithms presented a statistically significant difference in comparison to PBC4 cip, we carried out statistical analysis using the Friedman test [95, 96, 97, 98]. This was selected since it maintains the same statistical power than the sign test for normal and non-normal distributions [95] and has been used previously for the comparison of machine learning algorithms [14, 96, 98, 99]. For the AUC metric, the average ranks obtained by the Friedman test can be observed in Table 6. The Friedman statistic was 20.0857, with a \(p\)-value of 0.0012. Therefore, we rejected the null hypothesis of the test, which states that there was no significant difference in between the classifiers [99]. Once this was done, it was necessary to perform a post-hoc test to find the comparisons that presented a difference [98]. Since PBC4 cip presented the highest rank, we could make a 1 xN post-hoc test [98]. The Finner post-hoc test is recommended due to its power and comprehensibility [14, 99]. Therefore, we decided to use it. The results from the Finner post-hoc test are presented in Table 7. We can see that the adjusted \(p\)-values suggest that there was a significant difference between PBC4 cip and KNN, DTC, and GNB. Meanwhile, there was no difference between the pattern-based classifier and RF or MLP. This supported the previous tests. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Fold**} & **Random** & **KNN** & **Naive** & **DT** & **MLP** & **PBC4 cip** \\  & **Forest** & & **Bayes** & & & \\ \hline \(1\) & 0.5681 & 0.5609 & 0.4155 & 0.5670 & 0.5657 & 0.5939 \\ \(2\) & 0.5712 & 0.5452 & 0.3999 & 0.5333 & 0.5574 & 0.5291 \\ \(3\) & 0.5675 & 0.5155 & 0.4505 & 0.5211 & 0.5663 & 0.5695 \\ \(4\) & 0.5648 & 0.5430 & 0.3867 & 0.5413 & 0.5465 & 0.5855 \\ \(5\) & 0.5718 & 0.5476 & 0.4415 & 0.5022 & 0.5684 & 0.5615 \\ \hline
**avg** & **0.5687** & **0.5425** & **0.4188** & **0.5330** & **0.5609** & **0.5679** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Cross-validation CEM results. Figure 7: Metrics across the first phase of experimentation. With regard to the CEM metric, the Friedman statistic obtained was 18.6, with a \(p\)-value of 0.002281.