## 5 Conclusions  
This study confirms findings from the existing literature on the importance of selecting appropriate evaluation metrics, particularly different types of F1-scores, in multilabel emotion classification tasks. Specifically, it highlights that relying on a single metric does not provide a comprehensive picture of classifier performance. In this paper, the XED distilled lexicon applied to the GoEmotions dataset reached the benchmark value of 0.53 at a threshold of 0.3. This was the only experiment among the non-aggregated datasets to achieve this score while applying the lexicon to a different dataset than the one from it was originally derived. It was also found that the configuration of the emotion system used in the annotated dataset matters. This was demonstrated by aggregating emotions into fewer categories, which improved model performance metrics, especially for macro F1-scores. In this case, the XED Aggregated distilled lexicon applied to the GoEmotions Aggregated dataset and the GoEmotions Aggregated distilled lexicon applied to the XED Aggregated dataset both achieved results above 0.53 in the micro and weighted F1 for certain thresholds. The main limitation of this study is that only the [PERSON] emotion classification system was used. Furthermore, only unigrams were considered when using the distilled lexicons, which means that the context of the messages was not taken into consideration, impacting the classifier's performance. For future research, we suggest further exploring the influence of class imbalance and category aggregation on emotion classification models. Specifically, different emotion systems, such as [PERSON]'s, dimensional, or componential models, should be used and compared with the results obtained from using [PERSON]'s system. In conclusion, we consider that presenting the three F1-score variants--weighted, macro, and micro F1-scores--when evaluating the performance of a classifier offers a more comprehensive picture of the model in imbalanced scenarios than when relying on a single metric. We recommend these metrics to be used together with the F1-scores per class for multilabel classification tasks, as this could avoid the issues caused by averaging them and assigning weights. The following supporting information can be downloaded at: [[https://www.mdpi.com/article/10.3390/app14219863/sl](https://www.mdpi.com/article/10.3390/app14219863/sl)]([https://www.mdpi.com/article/10.3390/app14219863/sl](https://www.mdpi.com/article/10.3390/app14219863/sl)). Conceptualization, M.C.H.L., J.B. and J.S. ; data curation, M.C.H.L. ; formal analysis, M.C.H.L. ; funding acquisition, J.B. and J.S. ; investigation, M.C.H.L. ; methodology, M.C.H.L., J.B. and J.S. ; project administration, M.C.H.L. ; resources, M.C.H.L., J.B. and J.S. ; software, M.C.H.L. ; supervision, J.B. and J.S. ; validation, M.C.H.L., J.B. and J.S. Data Availability Statement: The data presented in this study are available on Hugging Face. The GoEmotions dataset was retrieved from [[https://huggingface.co/datasets/google-research-datasets/go_emotions](https://huggingface.co/datasets/google-research-datasets/go_emotions)]([https://huggingface.co/datasets/google-research-datasets/go_emotions](https://huggingface.co/datasets/google-research-datasets/go_emotions)), and it is described in [PERSON] et al. (2020) [37], accessed on 24 October 2023. The XED dataset was obtained from [[https://huggingface.co/datasets/Helsinki-NLP/xed_en_fi](https://huggingface.co/datasets/Helsinki-NLP/xed_en_fi)]([https://huggingface.co/datasets/Helsinki-NLP/xed_en_fi](https://huggingface.co/datasets/Helsinki-NLP/xed_en_fi)), and it is detailed in [PERSON] et al. (2020) [36], (accessed on 24 October 2023). These datasets are publicly available and have been used in previous research for sentiment analysis and emotion detection. For this paper, Grammarly was used for grammar correction, and ChatGPT was employed to improve the clarity and flow of the text, for the basic debugging of code, and for brainstorming.