IIT G andhinagar at S em E val-2020 Task 9: Code-Mixed Sentiment Classification Using Candidate Sentence Generation and Selection (2020)

Abstract: Code-mixing is the phenomenon of using multiple languages in the same utterance. It is a frequently used pattern of communication on social media sites such as Facebook, Twitter, etc. Sentiment analysis of the monolingual text is a well-studied task. Code-mixing adds to the challenge of analyzing the sentiment of the text on various platforms such as social media, online gaming, forums, product reviews, etc. We present a candidate sentence generation and selection based approach on top of the Bi-LSTM based neural classifier to classify the Hinglish code-mixed text into one of the three sentiment classes positive, negative, or neutral. The proposed candidate sentence generation and selection based approach show an improvement in the system performance as compared to the Bi-LSTM based neural classifier. We can extend the proposed method to solve other problems with code-mixing in the textual data, such as humor-detection, intent classification, etc.

## introduction code mixing is one of the most frequent styles of communication in multilingual communities, such as india. this pattern of communication on various platforms such as social media, online gaming, online product reviews, etc. makes it difficult to understand the sentiment of the text. sentiment classification of the code-mixed text is useful in the scenarios of socially or politically driven discussions, fake news propagation, etc. some of the major challenges with the text in the code-mixed language are: the sentence in the example misses a question mark(?) apart from other necessary modifications to make the structure of the sentence correct. ## dataset we use the dataset @xcite provided by the task organizers for building our system @xcite . each sentence in the dataset has a sentiment label as positive, negative, or neutral. ## experiments the availability of code-mixed embedding is a challenging task due to the scarcity of large scale codemixed corpora. we are using the glove embedding @xcite for the english words, and we train the embedding on the phinc dataset @xcite for the romanized hindi words. we are using the code-mixed sentences from phinc to train the code-mixed embedding. initially, we train the system using bi-lstm based neural architecture. the architecture of the bi-lstm classifier has the embedding layer followed by the bi-lstm layer and then two dense layers and, finally, the softmax prediction for the three sentiment classes. for prediction on the test set, we pre-filter the sentence based on the list of abusive words. if a sentence contains any words from this list, we label that sentence as negative. in the pre-filtering process, we identify 123 sentences in the test set containing one or more of the abusive words from the list. post pre-filtering step, we generate 15 candidate sentences for each of the remaining test instance using the candidate sentence generation (csg) procedure. we then select the best sentiment prediction for the sentence using the candidate sentence selection (css) procedure. algorithm 1 shows the csg procedure. algorithm 2 shows the css procedure. figure 2 shows the flow diagram of the proposed approach. in the csg procedure, we try to confuse the model with nearly similar sentences with additional phrases. we generate five similar sentences to the original code-mixed sentence for each of the three buckets (positive, negative, and neutral). we detect the degree of confusion in the sentiment prediction using the css procedure. we also keep track of the degree of confusion by sentences in each bucket. if the degree of confusion is significantly higher, we change the previous prediction by the model using the rules (as discussed in algorithm 2). phrases positive good enough, sure thing, undoubtedly, theek hai, even so, indubitably, tathaastu, of course, savaida, certainly, sakaaraatmak, gladly, affirmative, apanee marjee, abhee to, sabase adhik aashvast, amen, good, yakeenan, bahut achchha, theek, precisely, by all means, beyond a doubt, surely, yeah, unquestionably, very well, exactly, positively, khushee se, har tarah se, most assuredly, definitely, achchha negative nowhere, nahin kar sakate, koee bhee nahin, nahin, wouldn't, won't, nahin hai, nahin karana chaahie, nahin kiya ja saka, kabhee naheen, never, don't, neither, couldn't, nothing, doesn't, koee nahin, barely, mushkil se, kuchh bhee to nahin, wasn't, shouldn't, scarcely, nahin karata hai, hardly, nahin hoga, kaheen bhee nahin, no, not, na, nobody, can't, shaayad hee, no one, none ## results and analysis to evaluate the system performance, we use accuracy, precision, recall, and f-score as the evaluation metric. load the set of positive and negative phrases p. ## conclusion and future work we present a bi-lstm based sentiment classifier for the classification of code-mixed hinglish sentences. we also propose a candidate sentence generation and selection based approach on top of the bi-lstm based classifier to improve the system performance. up to a certain extent, the proposed approach is able to detect the ambiguous labels in the dataset. we can extend the proposed method to solve other challenges relevant to code-mixing, such as sarcasm detection, fake-news identification, intent classification, etc.