# An Explainable Artificial Intelligence Approach for Detecting Empathy in Textual Communication
## 6 Results and Discussion
### Extracted Patterns  
For the second phase of experimentation, we first obtained features that presented a significant contribution when using PBC4 cip. Each of the five groups for cross-validation was fed into the Feature selection module of WEKA. In order to minimize the possible negative impact, and increase the chances of a better performance, we decided on using the Accuracy as the metric on which the module would perform the evaluation. After the module processed the data, we were able to obtain a ranked list of attributes. The average maximum impact across all validation groups was found to be 34.04%. In contrast, the average minimum impact was 0.255%. Due to this difference, we decided to use a cut-off point of 10%. All features that presented an impact of under 10% would be dropped for this experiment. We expected to possibly improve the patterns found by the algorithm without producing any significant negative impact on its performance. Following the selection of attributes, we implemented the classification algorithm on the data. Secondly, we carried out the processing of the dataset with regard to the emotional context. Each emotional label present in the feature was replaced by the corresponding features in accordance with [PERSON]'s emotion model. The model was in the form of a vector with eight binary values, accompanied with an integer value between 1 to 3. Each of the binary values would represent the presence of one of [PERSON]'s main emotions. Meanwhile, the integer value would represent the intensity of the emotions found in the binary vector, with 1 representing high intensity and 3 representing low intensity. The binary values would represent the emotions in the following order: \"joy\", \"trust\", \"fear\", \"surprise\", \"sadness\", \"disgust\", \"anger\", and \"anticipation\". Once we had processed the datasets in order to add the features of [PERSON]'s emotion model, we only needed to apply the classification algorithm. Both modifications of the text representation yielded similar performance. The resulting values obtained from the classification tasks seemed to support the hypothesis \begin{table}
\begin{tabular}{c c c c} \hline \hline
**i** & **Hypothesis** & **Unadjusted \(p\)** & \(p_{avg}\) \\ \hline
1 & RF vs. GNB & 0.0002 & 0.003004 \\
2 & PBC4 cip vs. GNB & 0.00132 & 0.0132 \\
3 & GNB vs. MLP & 0.006841 & 0.047886\% \\ midrule 4 & RF vs. KNN & 0.027992 & 0.279918 \\
5 & RF vs. DT & 0.027992 & 0.279918\% \\ midrule 6 & PBC4 cip vs. KNN & 0.090969 & 0.545814 \\
7 & PBC4 cip vs. DT & 0.090969 & 0.545814 \\
8 & KNN vs. GNB & 0.12819 & 0.769141 \\
9 & GNB vs. DT & 0.12819 & 0.769141 \\
10 & KNN vs. MLP & 0.236724 & 0.946894 \\
11 & DT vs. MLP & 0.236724 & 0.946894 \\
12 & RF vs. MLP & 0.310494 & 1.241978 \\
13 & PBC4 cip vs. MLP & 0.61209 & 1.241978 \\
14 & PBC4 cip vs. RF & 0.61209 & 1.241978 \\
15 & KNN vs. DT & 1 & 1.241978 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Unadjusted and Adjusted \(p\)-value for the post-hoc analysis (CEM). that, even though some features were modified, there was no significant negative impact in the classification of empathy. This was seen for both modifications. The results for the classification, along with the original text representation, using PBC4 cip can be seen in Figure 8.  
We performed the Paired \(t\)-test [102; 103; 104] for the cases using PBC4 cip, in order to see if there was a significant difference when applying a different set of features. The results are found in Table 10. What we found by performing these tests is that the different features used in this research for finding useful patterns did not negatively impact the performance of the PBC4 cip algorithm in any manner. This was desirable, since we did not expect the use of a different emotion model system or the reduction of features to create an adverse effect in the classification of empathy. In each of the implementations of PBC4 cip, a series of patterns were obtained, which were later used for the classification of the utterances.