NLP - CIC at S em E val-2020 Task 9: Analysing Sentiment in Code-switching Language Using a Simple Deep-learning Classifier (2020)

Abstract: Code-switching is a phenomenon in which two or more languages are used in the same message. Nowadays, it is quite common to find messages with languages mixed in social media. This phenomenon presents a challenge for sentiment analysis. In this paper, we use a standard convolutional neural network model to predict the sentiment of tweets in a blend of Spanish and English languages. Our simple approach achieved a F1-score of 0:71 on test set on the competition. We analyze our best model capabilities and perform error analysis to expose important difficulties for classifying sentiment in a code-switching setting.

## introduction the phenomenon of combining two or more languages in the same message is known as code-switching or code-mixing @xcite . code-switching is an indicator of bilingual competence @xcite , and it is also motivated by social and cultural factors such as social status, race, age, etc. @xcite . although this phenomenon has been studied extensively in linguistics @xcite @xcite @xcite @xcite , it is still challenging for machines to process mixed natural languages. code-switching is notoriously present on social media posts and chats such as twitter, facebook or whatsapp ; consequently making it more difficult to process the sentiment expressed in such contents. in this work, we present a convolutional neural network (cnn) system to predict the sentiment of a given code-mixed tweet. the sentiment labels are either positive, negative, or neutral, and the languages involved are english and spanish. our best model utilizes only spanish word embeddings from tweets @xcite and does not require manual feature engineering. ## related work sentiment analysis is a widely studied task in monolingual, multilingual and cross-lingual settings. for instance, monolingual opinion mining in a multilingual context @xcite , multilingual sentiment analysis (balahur and turchi, 2014), and cross-lingual polarity detection @xcite . language-independent approaches to sentiment analysis include the use of emoticons @xcite or emoticons and noisy labels @xcite . sentiment analysis has not been extensively studied on code-switched content. a possible reason is the paucity of large annotated data covering several language pairs or combinations. the sentimix shared task is an effort to address this problem on the spanish-english and hindi-english language pairs for sentiment analysis. analysing opinions in tweets that blend spanish and english is a difficult task @xcite . @xcite obtained an accuracy of 59.34% using lexical, syntactic, and n-gram features. they concluded that the task is challenging because of the presence of noise, difficulties with language identification and pos tagging, and the lack of annotated code-mixed lexicons and a large dataset. another line of work related to code-switching is the development of contextual and static multilingual or cross-lingual text representations which cover multiple languages in the same vector space. examples include laser @xcite , muse @xcite , and multilingual bert 1 . these representations can be used to encode inputs for deep learning models. the effectiveness of these text representation approaches on code-switched texts remains an open question. ## methodology we describe the spanglish dataset and our submitted models. ## results and discussion we obtained a precision score of 0.807 and a recall score of 0.647 on the competition, thus a 0.71 f1 score was calculated for the competition rank 5 . moreover, according to the competition guidelines, we also report class-wise f1 scores for test set, in which we achieved a lower score 6 . under this metric, our aforementioned architecture achieved results shown at ## conclusions code-switching is an interesting problem holding an important presence in social media, which combined with informal writing style increases the challenges for social media processing such as sentiment analysis. we experimented with the sentimix spanglish dataset using cnn model and only spanish embeddings. we achieve a precsion, recall, and f1 score of 0.80, 0.64, and 0.71 respectively. our analyses suggest that a deep learning model can be easily biased by the presence of cue words such as vulgar expressions for sentiment analysis. we found that this occurs mostly when the cue word is in english. this observation requires a deeper analysis. we also highlight the need to address complex language usage such as informality and sarcasm. furthermore, we also pointed out that subjectivity in the annotation of sentiment labels is a problem that deserves to be addressed. we plan to test contextual multilingual embeddings (e.g. bert) and leverage the language tags and other non-linguistic constructs such as hashtags and emojis.