## 2 Related Work
#### Evaluation Metrics  
Sensitivity or recall (Equation (1)) is the fraction of actual positive cases that are correctly predicted as positive or pertaining to the class. It is also known as the true-positive rate [38; 39; 40]. Precision (Equation (2)) is the fraction of correctly predicted positive cases from all cases predicted as positive [38; 40; 39]. The F1-score, also known as the Dice similarity coefficient, is the harmonic mean of precision and recall, providing a balance between the two [23; 38; 39]. For this paper, the micro (Equation (3)), macro (Equation (4)), and weighted (Equation (5)) variants were calculated to compare their performance on the same classification task.  
\[\text{Recall}=\frac{TP}{TP+FN} \tag{1}\]  
\[\text{Precision}=\frac{TP}{TP+FP} \tag{2}\]  
where TP is the true positive, FP is the false positive, and FN represents the false negative.  
Micro F1-score: This metric considers all classes by using the total true positives, false negatives, and false positives:  
\[F1_{\text{micro}}=\frac{2\times Precision_{\text{micro}}\times Recall_{\text{ micro}}}{Precision_{\text{micro}}+Recall_{\text{micro}}} \tag{3}\]  
**Macro F1-score:** This metric calculates the F1-score independently for each label and then averages them, giving each class the same weight:  
\[F1_{\text{macro}}=\frac{1}{N}\sum_{i=1}^{N}\frac{2\times Precision_{i}\times Recall _{i}}{Precision_{i}+Recall_{i}} \tag{4}\]  
**Weighted F1-score:** This metric calculates the F1-score for each class as in the macro F1-score, but the average is weighted by the number of true instances each class has:  
\[F1_{\text{weighted}}=\sum_{i=1}^{N}w_{i}\frac{2\times Precision_{i}\times Recall _{i}}{Precision_{i}+Recall_{i}} \tag{5}\]where \(w_{i}\) is the proportion of true instances for label i. The weighted F1-score considers the support or weight of each class, \(w_{i}\) (Equation (6)):  
\[w_{i}=\frac{TP_{i}+FN_{i}}{\sum_{j=1}^{N}(TP_{j}+FN_{j})} \tag{6}\]  
where TP\({}_{i}\) and FN\({}_{i}\) are the true positives and the false negatives for class i, and the denominator is the sum of all the true positives and false negatives for all classes.  
These evaluation metrics were selected because they are commonly used in assessing the performance of emotion analysis models (see Table 1), utilizing emotion-annotated datasets ranging from news headlines to book passages, movie subtitles, and messages from Twitter and Reddit [36; 41; 42; 43; 44]. It is important to note that some researchers still rely solely on accuracy [41; 43] or the micro F1-score [42]. Using these measures to evaluate performance can result in overly optimistic outcomes or complicate comparisons across different models and datasets [36; 45; 46].  
In this paper, we consider several variants of the F1-score as well as precision-recall curves. The goals of this paper are to improve the evaluation of text emotion classification models by applying different performance metrics and to compare the effectiveness of micro, macro, and weighted F1 metrics in reflecting the performance of emotion classification models. The objective of the study is to demonstrate the comparative usefulness of the different F1-score variants in the evaluation of text emotion classifiers for imbalanced datasets. The outcomes of our analysis are presented in the results section.