K an CMD : K annada C ode M ixed Dataset for Sentiment Analysis and Offensive Language Detection (2020)

Abstract: We introduce Kannada CodeMixed Dataset (KanCMD), a multi-task learning dataset for sentiment analysis and offensive language identification. The KanCMD dataset highlights two real-world issues from the social media text. First, it contains actual comments in code mixed text posted by users on YouTube social media, rather than in monolingual text from the textbook. Second, it has been annotated for two tasks, namely sentiment analysis and offensive language detection for under-resourced Kannada language. Hence, KanCMD is meant to stimulate research in under-resourced Kannada language on real-world code-mixed social media text and multi-task learning. KanCMD was obtained by crawling the YouTube, and a minimum of three annotators annotates each comment. We release KanCMD 7,671 comments for multitask learning research purpose.

## introduction a surge in the active users on social media has given rise to people's engagement in expressing their opinions in the form of comments and reviews on social media platforms such as facebook, youtube and twitter @xcite @xcite . we see a lot of informal texts which do not follow any grammatical rules, sometimes code-mixed and even written in non-native scripts @xcite @xcite . these texts can be offensive and may be directed towards an individual or community to show their dissent. thus offensive language identification is essential for social media platforms to minimise these activities @xcite . sentiment analysis is used to interpret and classify emotions from text data after analysing it with various existing techniques @xcite . it has received a lot of interest in industry and research for identifying customer satisfaction on products and services, but there is no dataset available for code-mixed kannada language. code-mixing refers to the pairing of linguistic units from two or more languages into a single conversation @xcite . code-mixed sentiment analysis and offensive language identification when succeed trained on code-mixed data while regular monolingual sentiment analysis might prove ineffective due to the large variations in the text @xcite . kannada language (iso 639-3:kan) belong to dravidian language family (chakravarthi et al., 2019b; chakravarthi, 2020), spoken predominantly by the people of karnataka state in south india. kannada language uses kannada script also called caranese which is derived from bhattiprolu brahmi script @xcite . it is a phonemic adugida written from left to right @xcite . we observed six combinations among code-mixed sentences such as no-code-mixing only kannada written in kannada script or kannada written in the latin script, inter-sentential code-mixing, code-switching at morphological level, intra-sentential code-mixing, inter-sentential and intra-sentential mix. most comments were written in kannada script either with kannada grammar with english lexicon or english grammar with kannada lexicon. some comments were written in kannada script with english expressions in between. figure 1 illustrate the different level of code-mixing in our dataset. ## related work sentiment analysis has become one of the primary areas of research with applications across many trades and industries(such as finance, online marketing, political science) @xcite . over the last 20 years, social media networks have contributed immensely to the availability of rich data sources for analysis of sentiment @xcite . this combined with efforts directed towards the compilation of sentiment lexicons @xcite have resulted in this branch of natural language processing maturing out. in the early years of research, n-grams were used for classification of sentiments carried by the datasets. recently, these methods have been replaced by neural model architectures. however, sentiment analysis in kannada @xcite has not achieved this. aggression identification in social media @xcite and offensive language identification @xcite shared task has been conducted to improve the research in this area. offensive language identification dataset was released for greek @xcite . however, offensive language identification has not been made for the kannada language. for language identification (lid) systems in code-mixed languages, a kannada-english dataset containing english, kannada and several word-level code-mixed words was created by sowmya @xcite . a stance detection system was employed to detect stance in kannada social media code-mixed text using sentence embeddings. machine learning models such as logistic regression and a distributed memory model for sentence vectors were among the models to experiment @xcite . distributed representations of texts through neural networks method has experimented for sentiment analysis on kannada-english code-mixed dataset, which had three tags, positive, negative and neutral @xcite . however, the dataset for kannada was not easily available for research purpose. following @xcite , we downloaded the youtube comments for kannada and annotated. in our research, we release code-mixed dataset for under-resourced kannada for sentiment analysis and offensive language identification as multi-task learning dataset. ## dataset construction we create a dataset for two tasks, namely, sentiment analysis and offensive language identification. we collected comments from youtube using youtube comment scrapper 1 . we collected comments from 18 videos on different topics ranging from movie trailers, current trends about the ban on mobile apps in india, india-china border issue, mahabharata, and transgenders. we used these keywords to find the video and then from the videos we collected the comments. this was collected between feburary, 2020 and august, 2020. ## data statistics and analysis after performing annotations using google forms for both of the tasks, sentiment analysis and offensive language detection, all of the responses were converted into .csv format and then combined to a single dataset containing all the annotations. our goal is to analyse the multitask dataset and perform experiments with several machine learning algorithms to establish benchmark results. the second task performed on the dataset was offensive language detection. it was categorised into six groups such as not offensive, offensive untargeted, offensive targeted individual, offensive targeted group, offensive targeted other, other languages. since the same code-mixed dataset was used for this task, the dataset statistics would be the same here. out of 7,671 posts, 4338 of them were not considered to be offensive, which was the most frequent category in this task. we similarly split the dataset to what was done for the task of sentiment analysis. ## benchmark systems in order to provide a simple baseline, we applied several traditional machine learning algorithms such as logistic regression (lr), support vector machine (svm), multinomial naive bayes (mnb), k-nearest neigbours (knn), decision trees (dt), random forest (rf) separately, for both of the tasks, sentiment analysis and offensive language detection on kancmd, the code-mixed kannada-english dataset. ## conclusion in this paper, we presented kancmd, a multi-task learning dataset for sentiment analysis and offensive language identification in under-resourced kannada language. the dataset consists of 7,671 youtube comments annotated by a minimum of three annotators and had 0.73 for sentiment analysis annotation, and 0.78 for offensive language identification in terms of kripendorffs alpha inter-annotator agreement. we believe this dataset will allow future work in under-resourced kannada language to progress in multi-task learning of the code-mixed real-world data. we have created computational models to set the benchmark for this dataset. we aim to promote research in the kannada language and to encourage future investigations into multi-task learning for under-resourced languages in general and how it can be used to improve performance for under-resourced languages.