NITS - H inglish- S enti M ix at S em E val-2020 Task 9: Sentiment Analysis for Code-Mixed Social Media Text Using an Ensemble Model (2020)

Abstract: Sentiment Analysis refers to the process of interpreting what a sentence emotes and classifying them as positive, negative, or neutral. The widespread popularity of social media has led to the generation of a lot of text data and specifically, in the Indian social media scenario, the code-mixed Hinglish text i.e, the words of Hindi language, written in the Roman script along with other English words is a common sight. The ability to effectively understand the sentiments in these texts is much needed. This paper proposes a system titled NITS-Hinglish to effectively carry out the sentiment analysis of such code-mixed Hinglish text. The system has fared well with a final F-Score of 0.617 on the test data.

## introduction with the advent of social media, india alone stood at 326.1 million users in 2018. a large portion of the population is fluent in both english and hindi henceforth it is regular to see the use of code-mixed hinglish language on social media platforms. in the world of natural language processing(nlp), sentiment analysis of the code-mixed hinglish text is a great challenge. the task at hand is the semeval-2020 task 9 @xcite : sentiment analysis for code-mixed social media text for hinglish. code-mixed hinglish tweets are provided as training, validation, and test sets for model training, validation, and testing respectively. the proposed framework nits-hinglish-sentimix, an ensemble model wherein different models have been consolidated to improve the general f-score of the framework. the referenced group model is a mix of a basic lstm (long short term memory), an lstm+convolution, a bilstm (bidirectional lstm), and a cnn (convolution neural network) model. this paper is composed as follows, section 2 is a concise writing overview of related works. section 3 talks about the strategy which covers bits of insights into the dataset given, alongside the pre-processing, the details on the model architecture. section 4 talks about the training and results acquired and the paper concludes with section 5. ## literature survey in recent years, a lot has changed in the field of sentiment analysis. this section discusses a few of the works which helped us build insight into the problem and come up with a new solution. the initial approach towards sentiment analysis includes systems powered by traditional machine learning techniques like the work by @xcite explores the performance of svm for the task of multinomial classification of documents. the approach used the wordnet synonym graph with good and bad as anchor points to find out the mutual relationship of words between the documents and the sentences.these relationships were then used for sentiment prediction with the help of svm. graph cut techniques were also applied to improve classification accuracy. the work by @xcite proposes a real-time system for sentiment analysis of tweets made of many steps but only 2 steps were of prime importance i.e pre-processing and sentiment prediction. pre-processing includes tokenization with a tokenizer that handled the urls, common emoticons, phone numbers, html tags, twitter mentions and hashtags, numbers with fractions and decimals, repetition of symbols and unicode characters dexterously and ensured no noise in the processed data. the statistical classifier used for sentiment analysis or prediction was a naïve bayes model on unigram features. the architecture and the methods used were generic and hence the domain of this model could be extended. the paper proposed by @xcite focuses on the classification of offensive tweets written in the hinglish language. they approach the problem of sentiment analysis or classification of the tweets in the heot (hindi-english offensive tweet) dataset using the concepts of transfer learning where the proposed model consists of convolutional neural networks (cnn) pre-trained english tweets followed by retraining on hinglish tweets. their paper also examined various cnn based models for the task but leaves out other deep learning models based on lstm and bilstm which are expected to show a high affinity towards semantic-based tasks like sentiment analysis. the works proposed by @xcite helped in manually annotating the data as it provides an insight into how humans annotate data. the study was conducted on a dataset specifically prepared for this purpose called the mcgill twitter sentiment analysis (mtsa) dataset containing 7,026 tweets. translating code-mixed social media comments is attractive and challenging research. work by (singh and solorio, 2018) suggested a translation model on code-mixed facebook comments. the proposed system was based on two approaches, one is language identified and without using language identifier. the hindi-english code-mixed model has achieved an improved result over the baseline model. code-switching is an alternation of spoken language i.e. utterance or conversation in the multilingual community. in various natural language processing tasks like named entity tasks, parts of speech tagging, sentiment analysis, machine translation, and conversational system code-switching played a major role. @xcite surveyed a current code-switching dataset and categorized them. ## methodology in this section, a detailed description of the dataset used is given along with the detailed steps employed for the pre-processing of the text data and the details of the proposed model architectures. ## results each of the independent models were trained for 200 epochs with a batch size of 128, vocabulary size of 20000, text sequence length of 50 with sparse categorical loss and learning rate of 0.01. the results have been summarised in the