“Something Something Hota Hai!” An Explainable Approach towards Sentiment Analysis on I ndian Code-Mixed Data (2021)

Abstract: The increasing use of social media sites in countries like India has given rise to large volumes of code-mixed data. Sentiment analysis of this data can provide integral insights into people’s perspectives and opinions. Code-mixed data is often noisy in nature due to multiple spellings for the same word, lack of definite order of words in a sentence, and random abbreviations. Thus, working with code-mixed data is more challenging than monolingual data. Interpreting a model’s predictions allows us to determine the robustness of the model against different forms of noise. In this paper, we propose a methodology to integrate explainable approaches into code-mixed sentiment analysis. By interpreting the predictions of sentiment analysis models we evaluate how well the model is able to adapt to the implicit noises present in code-mixed data.

## introduction in the hindi movie jab we met, the lead actress kareena kapoor has a famous line in which she says, "main apni favourite hoon," which translates to "i am my favourite." while this dialogue remains iconic in indian pop culture even 14 years after the film's release, it also represents the phenomenon of code-mixing that is very common in multilingual countries like india. code-mixing is a phenomenon of embedding linguistic units such as phrases or words of one language into another language @xcite . english words, for instance, are often mixed with hindi sentences to form what is colloquially known as hinglish. over the past decade, social media sites like twitter and facebook have seen an exponential rise in users from countries like india. this has led to an accumulation of a large volume of code-mixed data on these platforms. therefore, analysing and processing this data has become imperative to provide * all authors have contributed equally to the work. users with the best possible experience on these sites. an important technique used to analyse such large amounts of data is sentiment analysis, which labels a given text as positive, negative, or neutral. its applications include product management, customer satisfaction review, hate speech detection and censoring of abusive and derogatory speech. the rise in the use of deep learning models for sentiment analysis has led to more accurate predictions @xcite @xcite .these algorithms behave like black boxes, making it difficult for practitioners to understand model predictions. explainable ai provides a set of methods to help us understand and interpret a model's decisions. specifically for the task of sentiment analysis of textual data, these techniques allow us to understand how a word or phrase influences the sentiment of the text. this paper discusses the implications and importance of model-agnostic explainable methods for code-mixed data. we observe that due to the unavailability of nlp tools for hindi-english code-mixed text and the noisy nature of such data, several popular methods for sentiment analysis are not applicable @xcite . therefore we first conduct a survey of models on the sail codemixed dataset @xcite and interpret their decisions with lime and shap. ## preliminaries various techniques have been used to achieve accurate results in sentiment analysis. a simplistic and computationally inexpensive approach is the random forest. a random forest is generated by bagging multiple uncorrelated decision trees but may fail to capture context and complexities. we look at convolutional neural networks and long short term memory(lstm) to overcome this problem. word embeddings, which are numeric representations for each word within a given sentence, are used to enable contextual learning over these networks. convolutional neural networks are a class of neural networks commonly used in computer vision. they also show promising results in nlp tasks such as sentiment analysis. when applied to the input embedding matrix, the convolution operation captures the local features and relationships between the words of the sentence, outperforming feed-forward neural networks and machine learning algorithms such as random forests @xcite . although cnns are great for capturing local features, they lose information about the order of words in a given sentence. the loss of sequential information significantly hampers performance, and recurrent neural networks such as lstms and grus are better suited for this. lstms are designed to infer sequential data, making them some of the most viable neural network architectures for sentiment analysis. their ability to deal with vanishing gradients and their relative insensitivity to gap length gives them an advantage over other sequential models. however, sentence structure goes beyond just sequential comprehension, and therefore a structured model that would allow word association becomes necessary for performance improvement @xcite . bidirectional encoder representations from transformers (bert) utilizes the bidirectional training of transformers, a popular attention model for learning. the architecture is built for the objective of masked language modelling and achieves considerably high performance compared to other word embedding techniques. the masked language model enables a fusion of both left-to-right and right-to-left contexts and makes the transformer bidirectional. bert also achieves next sentence prediction (nsp), where the model receives a pair of sentences as inputs and predicts whether the second sentence is in conjunction with the first @xcite . xlm-roberta, a variant of bert is a transformer-based cross-lingual language model trained on 100 different languages @xcite . ## conclusion and future work code-mixed data is an integral part of communication in multilingual communities and their culture. the application of state-of-the-art interpretable methods on this assortment of data will pave a path towards the adoption of the same during real-world implementation. our use of lime and shap, which quantify local and global model explanations, allows us to display their importance and relevance in sentiment analysis of code-mixed data. our work demonstrates that locally sensitive explanations provided by lime, give higher mean absolute error when the number of deletions increases. here, essential deletions refers to the deletion of polarizing words which influence the classifier's output. we also observe the model's ability to pick up polarizing hindi words, even in a code-mixed context. shap can be applied to produce high-quality global interpretations and extract model sensitivity. on the other hand, lime can be used for locally sensitive explanations, allowing developers to focus on individual users' sentiments. for future explorations, we would like to explore these techniques on more datasets and compare their performance. we also look to include other explainable methods and develop a specific metric to measure the global impact of hindi words in a code-mixed corpus. we would also like to investigate other techniques such as lexical normalization to reduce noise and measure its impact on our predictions. the trends of biases in code-mixed data can also be observed through explainable methods. we believe that our work serves as a valuable resource for code-mixed ml practitioners, developers, and researchers. the integration with explainable methods paves a new path towards future research and development.