TECHSSN 1 at S em E val-2024 Task 10: Emotion Classification in H indi- E nglish Code-Mixed Dialogue using Transformer-based Models (2024)

Abstract: The increase in the popularity of code mixed languages has resulted in the need to engineer language models for the same . Unlike pure languages, code-mixed languages lack clear grammatical structures, leading to ambiguous sentence constructions. This ambiguity presents significant challenges for natural language processing tasks, including syntactic parsing, word sense disambiguation, and language identification. This paper focuses on emotion recognition of conversations in Hinglish, a mix of Hindi and English, as part of Task 10 of SemEval 2024. The proposed approach explores the usage of standard machine learning models like SVM, MNB and RF, and also BERT-based models for Hindi-English code-mixed data- namely, HingBERT, Hing mBERT and HingRoBERTa for subtask A.

## introduction code-mixed hindi and english, also referred to as 'hinglish', has gained widespread usage, especially in the realm of social media. with the increasing prevalence of code-mixed languages like hinglish, there arises a necessity to analyze and understand this linguistic material. while language models designed for individual languages like english or hindi (ly, 2022) are quite robust and effective, they often struggle to perform well with code-mixed languages. this difficulty stems from the colloquial nature of the conversations in code-mixed dialogue, with no formal grammar rules. traditional machine learning models perform well on code-mixed data only when the nature of the classification task is simple, like in the form of sentiment analysis (classification into positive, neutral, and negative emotions). task 10 of semeval 2024 @xcite ) contains emotions from the extended ekman model @xcite , which contain emotions that are more complex to discern and distinguish between like contempt versus anger. this paper explores the usage of both classical machine learning models as well as transformerbased bert models, specifically designed for hinglish data. sentiment analysis in code-mixed data is done in a plethora of ways, spanning from machine translation to corpus processing based on sentence structure. @xcite introduced a framework employing a pipeline for the conversion of hinglish to english, offering a structured approach to the task. similarly, sinha and @xcite present a method for translating hinglish to both english and hindi, leveraging hindi and english morphological analyzers and implementing cross-morphological analysis to achieve accurate conversion. ensemble learning for identifying emotions in contextual texts was proposed by @xcite . additionally, @xcite proposed a lexiconbased solution for recognising emotions in tamil texts. das and singh (2023) embraced a deep learning paradigm, implementing convolutional neural networks (cnn), long short-term memory (lstm), and bi-directional long short-term memory (bi-lstm) for sentiment analysis. meanwhile, @xcite conclusively identified a combination of tf-idf vectorizer, gain ratio-based feature selection, and a radial basis function neural network (rbfn) as the optimal pipeline for sentiment analysis of hinglish data. @xcite utilized m-bert and the transformers framework, diverging from traditional methods. @xcite employed diverse techniques for sentiment analysis of hinglish, leveraging various embeddings such as count vectorizer and word2vec across different machine learning algorithms including svm, knn, and decision trees. a similar work by @xcite focused on recognizing emotions using gaussian process and decision trees. however, the task of emotion classification poses a much greater challenge compared to the simpler task of sentiment analysis. it necessitates the utilization of specific techniques to process and balance data across a broader spectrum of classes. this paper attempts to utilize both traditional and transformer based approaches for hinglish emotion classification. ## dataset the semeval 2024 task 10 dataset @xcite comprises 8056 samples, featuring fields such as id, speaker, utterance, and emotion. the id uniquely identifies each episode of the conversation, while the speaker field denotes the person speaking. the utterance field represents the dialogue, expressed in hinglish, and the emotion field indicates the corresponding emotion conveyed in the utterance. adding on, the validation dataset contains 1354 samples while the test dataset contains 1580 samples. ## data preprocessing in the domain of code-mixed emotion recognition, preprocessing the utterances is essential for effective model training. the emotion column, representing a spectrum of eight distinct emotions-'disgust', 'contempt', 'anger', 'neutral', 'sadness', 'fear', and 'surprise'-is encoded using a label encoder for standardized representation. code-mixed data inherently presents spelling ambiguities, demanding robust normalization techniques. for example, the word 'friend' in hindi could be spelled as 'dost', 'dhosth', 'dhost' etc. spelling correction is done using a phonetic similarity assessment. for each word, a phonetic code is computed and identifies feasible correction candidates from a dynamically created phonetic dictionary. the levenshtein distance metric is used to evaluate the dissimilarity between the input word and potential corrections. this procedure is applied to all the utterances, on each word. the resultant corrected words are subsequently merged to form a spell-corrected utterance. a dictionary of all the speakers is also created, and the speaker names present in the utterances are removed, along with numbers and symbols. ## conclusion the future scope of this work entails improving and enhancing the proposed models to handle a wider variety of data. the unstructured nature of hinglish poses a challenge to the model's performance. by understanding the nuances, fine-tuning can be implemented to enhance the model's efficacy. additionally, the work can be extended to encompass the classification of other types of emotions apart from the traditional ekman model and refined to undertake tasks such as sarcasm or humor detection.