FII - UAIC at S em E val-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text Using CNN (2020)

Abstract: The “Sentiment Analysis for Code-Mixed Social Media Text” task at the SemEval 2020 competition focuses on sentiment analysis in code-mixed social media text , specifically, on the combination of English with Spanish (Spanglish) and Hindi (Hinglish). In this paper, we present a system able to classify tweets, from Spanish and English languages, into positive, negative and neutral. Firstly, we built a classifier able to provide corresponding sentiment labels. Besides the sentiment labels, we provide the language labels at the word level. Secondly, we generate a word-level representation, using Convolutional Neural Network (CNN) architecture. Our solution indicates promising results for the Sentimix Spanglish-English task (0.744), the team, Lavinia_Ap, occupied the 9th place. However, for the Sentimix Hindi-English task (0.324) the results have to be improved.

## introduction the explosive growth of social media (sm) platforms offers a new type of user, multilingual, that can alternate two languages or more in the same conversation @xcite . in linguistics, it is call @xcite or language alternation. any virtual communication channel creates the possibility for people from different countries to share their sentiments without restrictions, about anything @xcite , often using multiple languages. this explosion of sentiments has aroused the interest of many researchers for sentiment analysis (sa) for code-mixed sm message. research on sa has focused, especially, on understanding the dynamics of sentiment in sm, most of them choosing twitter because it provides free api very useful for data retrieval goal. that facility allows the developer to find real time tweets from different multilingual users. actually, the analysis of tweets, which includes language alternation, is a challenging natural language processing (nlp) issue. code-mixing (cm) has several challenges to apply nlp techniques, such as word-level language identification or semantic processing @xcite . the goal of this paper is to implement a model for cm content on twitter, which imply two objectives: first, the classification of positive, negative, and neutral tweets by generating word-level representation, using convolutional neural network (cnn). the spanish-english code-mixed content has become ubiquitous on the internet, creating the need to process this form of natural language. a codemixed sentence retains the underlying grammar and script of one of the languages it is comprised of. the legitimate question of this survey is: can we achieve results comparable with those of our peers using more standard, less customized techniques? the rest of the paper is structured as follows: section 2 describes other works related to sentiment analysis for code-mixed social media text, section 3 presents the dataset and methods of this study, section 4 briefly relates the results we have obtained, followed by section 5 with the conclusions. ## background recent research regarding sa for code-mixing social media is becoming more attractive and challeng-ing @xcite . the use of sentiment resources @xcite has proven to be a neces-sary step for training and evaluating systems that implement sa, which also include finegrained opin-ion mining @xcite . a relevant work @xcite ) is based on lexicon expansion techniques by adding synonymy and antonymy relations provided by @xcite . in @xcite an opinion lexicon was devel-oped, compounded by a list of positive and negative opinion words or sentiment words for english (around 6,800 words) and spanish (around 1,500 words). a similar approach has been used for building wordnet @xcite which expands six basic categories of emotions. several experiments have been performed on social media texts including code-mixed data. the first step toward information gathering from these texts is to identify the languages present. until now, several language identification experiments or tasks have been performed on several codemixed language pairs such as spanish-english @xcite , french english @xcite , bengali-english @xcite . many shared tasks have been organized for language identification of code-mixed texts, studied with promising results in other papers @xcite @xcite , as well. language identification in code-switched data 5 was one of the shared tasks, which covered four language pairs such as spanish-english, modern standard arabic and arabic dialects, chinese-english, and nepalese english. in the case of indian languages, mixed script information retrieval @xcite shared task at fire-20156 was organized for eight code-mixed indian languages such as bangla, gujarati, hindi, kannada, malayalam, marathi, tamil, and telugu mixed with english. ## dataset and method this section contains details about dataset built as part of semeval-2020 task 9 "sentiment analysis for code-mixed social media text" and the study methodology, which was the basis for solving it. ## results below, the official results for each individual subtask using the development and test sets are presented. we report precision (p), recall (r) and f1-score (f1), for each baseline on all classes. sub-task 1_spanish-english sub-task 2_hindi-english: our best result over the test dataset for span-eng achieved a p of 82% for the positive sentiments, an r of 0.979 and an f1-score of 0.892, but it is lower for hi-eng. one of the reasons for the modest results for the hi-eng dataset is that some en words (e.g. 'costly') can be written in hindi with different spelling variations. note that, 30% of tokens were eliminated from the english-spanish dataset, mostly filler words, such as stop words or misspelt words. the dataset is unbalanced -there are 37161 occurrences of english words throughout the entire dataset, with 57801 spanish words, 150 ambiguous-labeled ones and tagged as 94 "mixed". ## conclusions in this paper, we present description of the system that we have used in semeval task 9. with our model, we were able to achieve fifth position in sentimix spanglish-english task in evaluation. it combines cnns networks, which prove to be very effective of training process of sa. we conduct several experiments on a real world code-mixed social media dataset and we have found that preprocessing steps played a huge role in increasing f1, especially for english-spanish. for hindi-english, it is necessary a large and diverse data collection. in order to improve the results, an approach regarding representing the word-vector space could be a good solution.