Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of M arathi (2021)

Abstract: The widespread presence of offensive language on social media motivated the development of systems capable of recognizing such content automatically. Apart from a few notable exceptions, most research on automatic offensive language identification has dealt with English. To address this shortcoming, we introduce MOLD, the Marathi Offensive Language Dataset. MOLD is the first dataset of its kind compiled for Marathi, thus opening a new domain for research in low-resource Indo-Aryan languages. We present results from several machine learning experiments on this dataset, including zero-short and other transfer learning experiments on state-of-the-art cross-lingual transformers from existing data in Bengali, English, and Hindi.

## introduction the presence of hate speech, cyber-bullying, and other forms of offensive language in online communities is a global phenomenon. even though thousands of languages and dialects are widely used in social media, most studies on the automatic identification of such content consider english only, a language for which datasets and other resources such as pre-trained models exist @xcite . in the past few years researchers have studied this problem on languages such as arabic @xcite , french @xcite , and turkish (c ¸öltekin, 2020) to name a few. in doing so, they have created new datasets for each of these languages. competitions such as offen-seval @xcite and trac @xcite provided multilingual datasets, which enabled the use of data augmentation methods (ghadery and moens, 2020), multilingual word embeddings (pamungkas and patti, 2019), and cross-lingual contextual word embeddings @xcite to tackle this problem. in this paper, we revisit the task of offensive language identification for low resource languages, focusing on marathi, an indo-aryan language spoken by over 80 million people, most of whom live in india. even though marathi is spoken by a large population, it is relatively low-resourced compared to other languages spoken in the region. we collect and annotate the first marathi offensive language identification dataset to date and we train a number of monolingual models on this dataset. finally, we explore state-of-the-art cross-lingual learning methods to project predictions to marathi from bengali, hindi, and english. we address two research questions in this paper: rq1: what is the impact of the dataset size in monolingual and cross-lingual models for offensive language identification? while the marathi dataset is relatively small, cross-lingual transfer learning methods allow us to take advantage of larger available datasets in other languages. rq2: what is the influence of language similarity in cross-lingual predictions for offensive language identification? previous work used english as the base language to make predictions in lower resourced languages. in this paper we use two indo-aryan languages, bengali and hindi, to project predictions into marathi. our main contributions are the following: ## related work the problem of offensive content online has been widely studied using computational models. researchers have trained system to recognize various types of such content such as cyberbulling, hate speech, and many others. in terms of computational approaches, early studies approached the problem using feature engineering and classical machine learning classifiers, most notably svms @xcite , while more recent work applied deep neural networks combined with word embeddings @xcite . with the development of large pre-trained transformer models such as bert and xlnet @xcite , several studies have explored the use of general pre-trained transformers in offensive language identification @xcite @xcite as well retrained or fine-tuned models on offensive language corpora such as hatebert @xcite . while the vast majority of studies address offensive language identification using english data @xcite , several recent studies have created new datasets for various languages and applied computational models to identify such content in arabic @xcite , @xcite , french @xcite , german @xcite , greek @xcite , hindi @xcite , italian @xcite , portuguese @xcite , slovene @xcite , spanish (plazadel arco et al., 2021), and turkish (c ¸öltekin, 2020). a recent trend is the use of pre-trained multilingual models such as xlm-r @xcite to leverage available english resources to make predictions in languages with less resources @xcite @xcite . this is made possible by the availability of the aforementioned datasets as well multilingual datasets made available at shared tasks such as hasoc 2019 @xcite , trac 2018 and 2020 @xcite , and two tasks at semeval: hateval 2018 @xcite and offenseval 2020 @xcite . ## datasets we present mold and four other datasets used in this work: the bengali dataset @xcite used in the trac-2 shared task @xcite -henceforth be, the hindi dataset @xcite used in the hasoc 2019 shared task-henceforth hi, and the english datasets used in offenseval, semeval-2019 task 6 and semeval-2020 task 12-henceforth en-olid @xcite and en-solid @xcite , respectively. to annotate mold, we followed olid's annotation scheme for english which has been replicated in solid and in datasets in greek @xcite , turkish (c ¸öltekin, 2020) and many other languages. olid's taxonomy comprises the following three levels: level a: offensive language identification: offensive (off) vs. non-offensive (not) level b: categorization of offensive language: targeted insult or thread vs. untargeted profanity. level c: offensive language target identification: individual vs. group vs. other. this hierarchical taxonomy represents multiple types of offensive content in a single annotation scheme (e.g. targeted insults to an individual are often cyberbullying and targeted insults to a group are often hate speech) making it a great fit for crosslingual learning applied to low-resource languages like marathi. we used olid level a labels to annotate mold and we map these labels to those included in the bengali and hindi datasets. ## conclusion and future work this paper introduced mold, the first offensive language dataset for marathi. we evaluated the performance of several machine learning models trained to identify offensive content in marathi. our results show that applying cross-lingual contextual word embeddings substantially improved performance over monolingual models. furthermore, we showed that xlm-r with transfer learning from hindi outperforms all of the other methods we tested. the results obtained by our models confirm that closely related languages provide an advantage in our transfer learning experiments, answering our rq2. this is likely due to the fact that hindi and marathi are typologically related and also because these languages are in a situation of language contact sharing cultural background. to the best of our knowledge, this paper is the first to address the question of language similarity in cross-lingual learning for offensive language identification. with respect to our rq1, our results show that the difference in performance between transfer learning strategies from olid and from solid is minimal. solid is more than eight times larger than olid, suggesting that beyond a certain point, more instances do not necessarily yield significant performance improvements in transfer learning. finally, we believe that the findings presented in this paper can open a wide range of avenues to offensive language identification applied to other low resource languages, particularly from the indo-aryan family. mold is the official dataset for marathi at the hasoc 2021 foot_2 shared task on hate speech and offensive content identification in english and indo-aryan languages. we are expanding the annotation of this dataset to the levels b and c of olid's annotation taxonomy. this will provide us with the opportunity to test computational models to identify the type and target of offensive posts in marathi. as future work, we would like to evaluate the performance of transfer learning from dravidian languages spoken in india such as tamil and telugu to analyze the interplay between language similarity and cultural overlap in cross-lingual offensive language identification as in @xcite .