# An Explainable Artificial Intelligence Approach for Detecting Empathy in Textual Communication
## 4 Empathy Measurement
### Empathetic Conversations (EC)
These volunteers would participate in the evaluation of conversations through the Delphi method, using an empathy metric compatible with the one found in the database. Finally, this was to be analyzed using statistical tools. The structure and description of the steps performed to validate the database can be appreciated in Figure 2.  
First, it was necessary to obtain a sample of the database. We used the stratified random sampling technique [48; 49; 50]. This was because the database presented a large representation of emotion labels as one of its defining features [37]. We considered a confidence level of 95%, a degree of variability of 0.5, and a level of precision of \(\pm\)5% [51; 52; 53]. For our population, we have the conversations in ED. We are studying conversations that present an inaccurate label (e.g., conversations with any number instead of 5, when the actual empathy is 5). Thus, we define our population with an attribute of interest as \"conversations with any error in measurement\". We do not know the actual number of such conversations. Therefore, we decided to take a conservative approach. This led to a proportion of 0.5. With these parameters, we obtained a sample size of 384. We rounded this number to 400 for simplicity. With this process, we obtained a sample of ED with balanced emotional context. We required the selection of participants that presented a high empathetic ability, to compare their evaluation scores to the Empathy/Symptathy score in the database sample. We searched for volunteers in various online communities and social media, as well as former students of Instituto Tecnologico y de Estudios Superiores de Monterrey. When the volunteers were contacted, their written consent was requested. All ethics standards were followed in accordance with our institution. The volunteers were asked to take the EQ. We considered those that presented a score of 35 or higher for our research. We were able to procure 10 volunteers that matched our criteria. These volunteers agreed to evaluate the conversations in the database sample. However, they were only willing to evaluate 200 conversations each. Therefore, we decided to separate the participants into two anonymous groups of 5 considering their EQ score, their native language, and gender expression. The groups presented an average EQ of 55.2 and 57.2, respectively. The EQ of each participant can be seen in Table 1. The average EQ of both groups are higher than the mean for the control group used to validate the EQ [18]. As such, we believe both groups are capable of identifying and acting according to empathy. Figure 2: Database evaluation process. The groups would independently evaluate each 200 conversations of the 400 conversations present in our sample. This would be carried out using the Likert scale present in the Empathy/Sympathy score. However, the ED database did not present a complete Likert scale, as a correspondence to each of the points present in it was not explicitly stated. Therefore, we needed to infer the variations in each of the levels of the scale as equidistant points, in accordance with [PERSON] et al [38]. The Likert scale which was presented to the participants of both groups will now be referred to as Empathy score (ES). The levels are described as follows:  
1. Not empathetic at all;
2. A little empathetic;
3. Somewhat empathetic;
4. Empathetic;
5. Very much empathetic. This meant that the amount of empathy was whether the responses show understanding of the feelings of the person talking about their experience, regardless of the role of listener or talker. The evaluation was done at the conversation level, not at the utterance level, and this was pointed out to the volunteers. The Delphi technique is a reliable iterative method for eliciting and refining group judgments [54, 55, 56, 57]. It has been used for validating the suitability of applied machine learning models by using the opinion of experts [14].