Emotion classification on code-mixed text messages via soft prompt tuning (2023)

Abstract: Emotion classification on code-mixed text messages is challenging due to the multilingual languages and non-literal cues (i.e., emoticons). To solve these problems, we propose an innovative soft prompt tuning method, which is lightweight and effective to release potential abilities of the pre-trained language models and improve the classification results. Firstly, we transform emoticons into textual information to utilize their rich emotional information. Then, variety of innovative templates and verbalizers are applied to promote emotion classification. Extensive experiments show that transforming emoticons and employing prompt tuning both benefit the performance. Finally, as a part of WASSA 2023, we obtain the accuracy of 0.972 in track MLEC and 0.892 in track MCEC, yielding the second place in both two tracks.

## introduction emotion plays an important role in social relationships, decision making, etc. emotion analysis aims to enable machines to learn the emotions contained in textual information, such as conversations, ecommerce reviews and personal blogs @xcite . with the surge of social media, emotion classification receives increasing attentions and brings huge commercial and social implications. 1 text message is a common form of communication that exists in twitter, youtube, etc. some text messages may contain typos, e.g., "habe" (have), and emotional words, e.g., "hahhhhh", while others may include visual cues such as icons, e.g., ":-)", and emoji. these features bring rich emotion information, but are typically not included in the vocabulary of pre-trained model like bert (bidirectional encoder representation from transformers), t5 (text-to-text transfer transformer) @xcite , etc. in addition, codemixing is a critical challenge in emotion classification, which means that two or more languages 1 * these authors contributed equally to this work. @xcite , such as english and roman urdu, are contained in a single piece of text. although code-mixed text is widely used, research on emotion classification for code-mixed corpus is still scarce @xcite since it is more difficult to recognizing emotions in code-mixing languages than in a monolingual language. fine-tuning is a widely-used method to improve performance of pre-trained language models (plms), however, it is memory-consuming and time-consuming. compared to fine-tuning, prompt learning is more lightweight, thus has received increasing attentions @xcite . t5 introduces a unified framework that converts any language problem into a text-to-text format. by introducing different prompts, the model can then adapt to different tasks, e.g., language translation and emotion classification. gpt-3 (general pre-trained transformer-3) @xcite proposes context learning, where models can be applied directly in zero-shot or few-shot tasks without further finetuning. utilizing language prompts helps to probe knowledge in plms and obtains better performance @xcite . prompt learning has been widely applied in tasks of natural language understanding @xcite and generation (zheng and huang, 2021), and even has been adopted in some vision tasks @xcite and multi-modality tasks @xcite . in this paper, we present our effort in emotion classification on code-mixed text messages. the contributions are summarized as follows: ## results and discussion we adopt cm-mec-21 corpus @xcite in this work, which includes 11650 mcec messages and 11603 mlec messages. we combine mcec and mlec text messages for both training and evaluating. the accuracy based on the merged dataset for both tracks are summurized in ## conclusion in this paper, we propose the innovative prompt tuning for emotion classification on code-mixed text messages. we first transform emoticons into textual information to utilize their rich emotional information. then, variety of innovative templates and verbalizers are applied to promote emotion classification. the results show that transforming emoticons benefits the final results. more importantly, even without model fine-tuning, optimizing the prompts yields efficient classification. finally, we obtain the accuracy of 0.972 in track mlec and 0.892 in track mcec, yielding the second place in both two tracks. in future work, we will pay more attention to combining model fine-tuning with prompt learning, and further explore the capacity of plms on emotion classification.