Span Extraction Aided Improved Code-mixed Sentiment Classification (2022)

Abstract: Sentiment classification is a fundamental NLP task of detecting the sentiment polarity of a given text. In this paper we show how solving sentiment span extraction as an auxiliary task can help improve final sentiment classification performance in a low-resource code-mixed setup. To be precise, we don’t solve a simple multi-task learning objective, but rather design a unified transformer framework that exploits the bidirectional connection between the two tasks simultaneously. To facilitate research in this direction we release gold-standard human-annotated sentiment span extraction dataset for Tamil-english code-switched texts. Extensive experiments and strong baselines show that our proposed approach outperforms sentiment and span prediction by 1.27% and 2.78% respectively when compared to the best performing MTL baseline. We also establish the generalizability of our approach on the Twitter Sentiment Extraction dataset. We make our code and data publicly available on GitHub

## introduction with the rapid growth of social media networks and the democratization of internet technology, massive amounts of text-based user-generated content is being produced everyday. it is essential to understand the opinion and sentiment of users from these textual posts. in the past decade the nlp research community has made several advancements in the field of language based sentiment analysis. however most of these advances are in high-resource languages like english. in contrast there are limited resources for sentiment analysis for indian languages. in the context of the indian sub-continent, the user-generated content on social media is unique because is not in any one particular language, rather a single utterance may consist of words, phrases and phonemes from multiple different languages. this phenomenon is code-mixing and is widely observed in multi-lingual communities such as india. although recent advances have been made in developing sentiment analysis text corpora and methods for indian languages such as hindi and bengali there has been little progress for truly lowresourced languages such as tamil, a dravidian language which is spoken by well over 70 million people worldwide. @xcite ) is a seminal work on creating corpora for sentiment classification of tamil-english code-mixed text. while sentiment classification is well researched; sentiment span extraction on the other hand @xcite is a rather new nlp task which involves the extraction of supporting phrases from text in the form of a sequence of contiguous words, which reflect the sentiment of the sentence. these support phrases can be used to further perform finegrained analysis of the sentiment to understand the opinion and feelings of the user. similar approach has also been applied to toxicity analysis from text @xcite . in this paper we present our hypothesis that solving sentiment span extraction as an additional task can help the model learn better semantic representations of the text which in-turn will improve sentiment classification. we explore this hypothesis for code-mixed tamil texts. firstly we develop a novel tamil-english code-mixed sentiment extraction dataset to support the task of sentiment span extraction. we obtain this dataset by extending the dravidiancodemix dataset @xcite by adding gold-standard human-annotated sentiment span labels to it. to the best of our knowledge this is the first code-mixed sentiment span extraction dataset. the proposed dataset will facilitate further research in this direction and helps improve sentiment classification performance in a low-resource setting in a language spoken by millions around the globe where annotated data is scarce. secondly we experiment with various single-task learning and multi-task learning models to evaluate our hypothesis. further inspired from @xcite we explore a methodology based on transformer architecture which explicitly models the interactions between the two tasks of sentiment prediction and sentiment span extraction in a unified framework (refer to fig. 1b . extensive experiments and ablation study establish the efficacy of this proposed approach, we also demonstrate that this model generalizes well to a similar english dataset for sentiment analysis. to the best of our knowledge, we are the first to explore the modelling of the two tasks together for improving performance on sentiment classification. moreover, our framework performs better than the generic multi-task learning setup which acts as one of our baselines. to summarize, the following are our main contributions • we propose a novel dataset consisting of 2152 user-generated comments along with goldstandard human-annotated sentiment-span labels. ## proposed approach in this section we describe our proposed approach. it takes as input a piece of text x and predicts the sentiment of the x and the span within x that display this sentiment. fig 5 depicts the architecture of the proposed model. it consists of a text encoder that provides contextual representation of x at both sentence and word level. it then uses a task interaction module (tim) to learn the interactions between the two tasks of sentiment classification and sentiment span extraction. ## experimental results in this section we present the results (averaged over 5 independent runs) on our test set and perform comparative analysis followed qualitative and error analysis. for comparison we use the following standard metrics -accuracy, macro averaged f1 score, precision, recall for sentiment prediction task and f1 score, exact match and jaccard similarity for the span prediction task. ## conclusion and future work in this work we explore the use of sentiment span cues towards improving code-mixed sentiment prediction. we first curate a novel manually annotated dataset to support code-mixed sentiment span extraction. we then propose a novel methodology based on transformer architecture to explicitly model the the mutual interactions between sentiment prediction and sentiment span extraction tasks. empirical evaluation along with an extensive ablation study suggests the efficacy of our proposed model and its design choices. we also establish the generalizability of the proposed model by demonstrating its performance on the twitter sentiment extraction dataset.