LIDOMA @ D ravidian L ang T ech: Convolutional Neural Networks for Studying Correlation Between Lexical Features and Sentiment Polarity in T amil and T ulu Languages (2023)

Abstract: With the prevalence of code-mixing among speakers of Dravidian languages, DravidianLangTech proposed the shared task on Sentiment Analysis in Tamil and Tulu at RANLP 2023. This paper presents the submission of LIDOMA, which proposes a methodology that combines lexical features and Convolutional Neural Networks (CNNs) to address the challenge. A fine-tuned 6-layered CNN model is employed, achieving macro F1 scores of 0.542 and 0.199 for Tulu and Tamil, respectively

## introduction in recent years, there has been a significant surge of interest in sentiment analysis on social media platforms for dravidian languages. the linguistically diverse and multicultural environments in which these languages are spoken have contributed to the prevalence of a linguistic phenomenon known as code-mixing. code-mixing refers to the occurrence of multiple languages within a single document or utterance (e. @xcite . this phenomenon is particularly prominent in written texts, where non-native scripts and hybrid words combine elements from more than one language. the shared task on sentiment analysis in tamil and tulu, proposed by dravidianlangtech at ranlp @xcite @xcite , aims to address the challenges associated with sentiment analysis in code-mixed text. this shared task seeks to introduce a new gold standard corpus specifically designed for sentiment analysis in the context of tamil-english and tulu-english code-mixing language. moreover, their dataset @xcite also has class imbalance problems depicting real-world scenarios. the main focus of the proposed approach is to identify sentiment polarity in code-mixed comments and posts extracted from social media platforms. these comments and posts often contains more than one sentence, making the sentiment analysis task more complex. in order to tackle the proposed shared tasks, this paper presents an approach that utilizes lexical features and convolutional neural networks (cnns) @xcite . lexical features have demonstrated a strong correlation with various pragmatic phenomena, including sentiment analysis tasks such as hope and hate speech detection @xcite . they have also been effective in other pragmatic tasks such as user preferences predictions in entertainment domains @xcite . additionally, lexical features have shown significant relevance in sentiment analysis when code-mixing is involved, as demonstrated in (e. @xcite with kannada and english languages. on the other hand, cnns have proven to be effective in detecting relevant features associated with sentiments across different classes @xcite , which is the reason why they were employed on this work. the presented model consists of a 6-layered cnn with the following structure: the first layer generates an embedding from a bag of words vectorization. the second and third layers are convolutional layers designed to learn the lexical features that have the strongest relationship with the labeling of each sample, which in this particular case are positive, negative, neutral, and mixed feelings. the fourth and fifth layers help prevent overfitting and reduce the dimensionality of the output by fine-tuning the lexical feature extraction using max pooling @xcite . finally, the sixth layer utilizes a sigmoid ac-tivation function to relate the learned features with the binary golden label. the achieved f1 scores were 0.542 for the tulu-english dataset and 0.199 for the tamil-english dataset. the structure of this paper is as follows: in section 2 it is described some state-of-the-art works on sentiment polarity detection. in section 3, the methodology is detailed. in section 4, it is provided a brief description of both datasets, and the experimental workflow is outlined. in section 5, it is discussed the results of the experiments. finally, in section 6, the paper is concluded. ## related work sentiment polarity analysis is considered one of the pioneering tasks in computational sentiment analysis. one of the earliest approaches in this field are the general inquirer @xcite , which is a 1961 ibm system capable to perform content analysis for behavioral sciences, most particular pattern detection in text for categorizing words according to their semantics, related to positive or negative sentiments. in 1997, a most focused approach was proposed with the system smokey @xcite , designed to detect abusive messages by using a rule-based approach to identify offensive language and contexts. following on the line of negative sentiment detection, in @xcite , the authors proposed a lexicon-based approach for hate speech detection. their approach focused on analyzing the sense in which selected words were used in sentences to identify hateful or offensive content, making the task close similar to word sense disambiguation. however, they discovered that this hypothesis is vulnerable when faced with incomplete datasets, especially in cases where a word only appears in one type of speech. on the other hand, in the domain of positive speech, a notable line of research is the peace speech line initiated in @xcite , where the authors primarily focused on analyzing peace-oriented discourse, particularly in the context of a conflict between pakistan and india. furthermore, in (chakravarthi, 2020), the authors focuses more towards the themes of equality, diversity, and inclusion. notably, chakravarthi also organized a series of shared tasks @xcite , where team lidoma utilized a convolutional neural network (cnn) to address the specified task @xcite . this model is a variation to the model presented in this paper. about code-mixing detection, several computational approaches have been done to address the task in languages from india. for instance, in @xcite , the authors worked on code-mixing between hindi and english, presenting a methodology for language identification in a dataset comprising facebook, twitter, and what-sapp messages. in @xcite , the authors proposed a shared task at semeval-2020, in which team limsi upv @xcite proposed a recurrent convolutional neural network architecture to address the task. in @xcite , the authors expanded this line by incorporating urdu into the analysis and utilizing transformer models with attention mechanisms, specifically employing bert models. in @xcite , the authors considered code-mixing involving saraiki and bengali. they employed recurrent neural networks and word vectorizations to address the task of language identification in code-mixed texts. in @xcite , the author proposed a setting that aligns closely with the shared tasks mentioned earlier, but with a focus on english-hindi and english-bengali code-mixing. additionally, she introduced an index to measure the level of mixing within the corpora, providing insights into the degree of code-mixing present in the data. furthermore, in (e. @xcite , the authors proposed an n-gram-based approach to tackle the task of language identification in kannada-english code-mixed texts. ## methodology diving further in the structure outlined in the introduction, the overall followed procedure is explained now, along with the used hiperparameters. ## results after the 30-epoch training, the model achieved a macro f1 score of 0.516 in the tulu evaluation set, and 0.199 in the tamil evaluation set. the most important factors for these results were the notable differences between kannada, tamil, tulu and latin alphabets, in which this network was designed, and the nature of the labelling: regardless previous experiences where variations of this cnn was employed, the datasets employed for this task includes the categories of neutral and mixed feelings, while in the other sentiment analysis tasks the labelling was binary in terms of a single polarity, another important factor was the balance of the dataset. as shown in figure 2 , there is a high imbalance in the dataset which led to a general low performance in the proposed methods, being macro f1-score of 0.32 the best for tamil and 0.542 the best for tulu, not so far of our results. ## conclusions in this paper, it was presented the lidoma submission for the shared task on sentiment analysis in tamil and tulu, proposed by dravidian-langtech at ranlp2023. they employed cnn's, who have proven being effective in sentiment polarity tasks. the proposed methodology involved the conversion of labels into categorical values, then basic preprocessing of the samples and finally the training of a 6-layered cnn. the findings highlight the complexities involved in handling non-balanced datasets along with the merge of polarities within the mixed feelings cathegory. future work will focus on adapt the cnn architecture to deal better with mixed cathegories, along with adding more steps of preprocessing adapted to kannada, tamil and tulu alphabets. also, it is possible to add the use of atention mechanisms to enhance results in this and other similar datasets.