JUNLP at S em E val-2020 Task 9: Sentiment Analysis of H indi- E nglish Code Mixed Data Using Grid Search Cross Validation (2020)

Abstract: Code-mixing is a phenomenon which arises mainly in multilingual societies. Multilingual people, who are well versed in their native languages and also English speakers, tend to code-mix using English-based phonetic typing and the insertion of anglicisms in their main language. This linguistic phenomenon poses a great challenge to conventional NLP domains such as Sentiment Analysis, Machine Translation, and Text Summarization, to name a few. In this work, we focus on working out a plausible solution to the domain of Code-Mixed Sentiment Analysis. This work was done as participation in the SemEval-2020 Sentimix Task, where we focused on the sentiment analysis of English-Hindi code-mixed sentences. our username for the submission was “sainik.mahata” and team name was “JUNLP”. We used feature extraction algorithms in conjunction with traditional machine learning algorithms such as SVR and Grid Search in an attempt to solve the task. Our approach garnered an f1-score of 66.2% when tested using metrics prepared by the organizers of the task.

## introduction india has a linguistically diverse population due to its long history of foreign acquaintances. english, one of those borrowed languages, became an integral part of the education system and hence gave rise to a population who are very comfortable using bilingualism in their day to day communication. due to such language diversity and dialects, frequent code-mixing is encountered during conversations. further, due to the emergence of social media, the practice has become even more widespread. the phenomenon is so common that it is often considered as a different (emerging) variety of the language, e.g., benglish (bengali-english) and hinglish (hindi-english). this phenomenon poses a great challenge to the existing domains of natural language processing (nlp) such as sentiment analysis as primarily the language technologies, such as parsing, parts-of-speech (pos) tagging, etc., are built for english. furthermore, labeled/annotated data of such category are hard to come by and hence leads to misfiring when using straight-forward machine learning algorithms. in this work, we participated in semeval-2020 sentimix task 1 and attempted to solve the chore of sentiment analysis of english-hindi code-mixed sentences. initially, our approach includes the use of feature extraction algorithms on the data, procured by the organizers. thereafter, we used support vector regression coupled with grid search algorithm to classify the code-mixed sentences to its respective sentiment class. this approach, when tested using the metrics prepared by the organizers, returned an f1-score of 66.2%. the rest of the paper is organized as follows. section 2 briefly the quantifies the english-hindi codemixed data procured by the organizers of the task. section 3 provides a descriptive literature of our proposed approach. this will be followed by the results and concluding remarks in section 4 and 5. ## data the english-hindi code-mixed data that was used to train our model was collected from twitter using the twitter api, by searching for code-mixed hindi keywords @xcite . the sentiment labels are positive, negative, and neutral. besides the sentiment labels, the language labels for every word of the code-mixed sentence were also provided. the word-level language tags were eng (english), hin (hindi), and o (other) for symbols, mentions, and hashtags. the organizers provided a trial and a training data set and after adding both, we could gather 17,000 code-mixed instances. we further divided this data into two parts; (i.) 15,000 instances as training data and (ii.) 2,000 instances as validation data. ## methodology our approach included converting the given tweets into a sequence of words and then run the grid search cross-validation algorithm on the processed tweet. initially, the tweets were pre-processed using methods as done by @xcite to remove the following: the last step consisted of taking advantage of the pascal casing of hashtags (e.g. #coronavirus). a simple regex can extract all words. this extraction results in better performance mainly because words in hashtags, to some extent, may convey sentiments of hate. they play an important role during the model-training stage. ## results the metric for evaluating the participating systems was as follows. the organizers used f1 averaged across the positives, negatives, and the neutral. the final ranking was based on the average f1 score. our submitted system garnered an f1 score of 66.2%. the detailed results are shown in ## conclusion in the current work, we attempted to solve the problem of sentiment analysis of code-mixed english-hindi data, while participating in the semeval shared task. our system was based on using traditional machine learning algorithms coupled with beam search cross-validation. our system, when evaluated by the organizers garnered an f1 score of 0.662. there was an option of developing an unconstrained system, but we only used the provided data to develop the system. as future work, we would like to increase this data, use state-of-the-art neural network architectures on this data, taking into advantage the concept, matrix and embedded language, sentiwordnet, and other nlp features.