L 3 C ube- M aha H ate: A Tweet-based M arathi Hate Speech Detection Dataset and BERT Models (2022)

Abstract: Social media platforms are used by a large number of people prominently to express their thoughts and opinions. However, these platforms have contributed to a sub stantial amount of hateful and abusive content as well. Therefore, it is impor tant to curb the spread of hate speech on these platforms. In India, Marathi is one of the most popular languages used by a wide audience. In this work, we present L3Cube-MahaHate, the first ma jor Hate Speech Dataset in Marathi. The dataset is curated from Twitter, anno tated manually. Our dataset consists of over 00 distinct tweets labeled into four major classes i.e hate, offensive, pro fane, and not. We present the approaches used for collecting and annotating the data and the challenges faced during the pro cess. Finally, we present baseline classi fication results using deep learning mod els based on CNN, LSTM, and Transform ers. We explore mono-lingual and multi lingual variants of BERT like MahaBERT, IndicBERT, mBERT, and xlm-RoBERTa and show that mono-lingual models per form better than their multi-lingual coun terparts. The MahaBERT model provides the best results on L3Cube-MahaHate Corpus.

## introduction in the past decade, there has been an expeditious rise in the popularity of online social media platforms all over the globe. people have become more open to sharing their opinions without thinking excessively. this often leads to the spread of hate or offensive speech thereby causing violence and cyberbullying. hate speech is a kind of abusive language directed towards a community that is underprivileged in terms of race, gender, ethnic origin, disability, etc., or can be an insult or threat to an individual @xcite . the users often defy the boundaries of freedom of speech without even realizing it by posting harmful messages and comments @xcite . therefore it is today's need to neutralize these activities from proliferating further. in this work, we consider hate speech detection in the marathi language, a regional language in india, spoken by over 83 million people across the country @xcite . despite being one of the popular languages in india, work in the area of hate speech detection in marathi is extremely limited @xcite @xcite as compared to other languages @xcite @xcite . even general text classification in marathi has received limited attention @xcite . in this paper, we present, l3cube-mahahate corpus, the largest publicly available hate speech dataset in marathi. the dataset is collected from twitter, tagged with four fine-grained labels which are defined as follows-hate (hate): a twitter post abusing a specific group of people or community based on their religion, race, ethnic origin, gender, geographical location, etc. stimulating violent behaviors. ## related work hate speech detection is considered to be a highly critical problem and a lot of attempts have been made to control it. a significant amount of work can be seen in english text analysis. but recently, efforts have been made towards widening the research in regional languages like marathi as well. @xcite presented the marathi offensive language dataset (mold), with nearly 2,500 annotated tweets labeled as offensive and not offensive. it is considered the first dataset for offensive language identification in marathi. also, they evaluated the performance of several traditional machine learning models and deep learning models (e.g. lstm) trained on mold. @xcite collected over 8200 hostile and non-hostile hindi text samples from multiple social media platforms like twitter, facebook, whatsapp. hostile posts were further extended into fake, defamation, hate, and offensive. a total of 8192 posts were collected and tested on various machine learning models using mbert encoding. a hindi-english code-mixed corpus was constructed in @xcite using the tweets posted online for the duration of five years. tweets were scrapped using twitter python api by selecting certain hashtags and keywords from political events, public protests, riots, etc. after removing noisy samples a dataset of 4575 code-mixed tweets was created. the experiments were performed with svm and random forest algorithms along with character and word n-gram features. in @xcite authors presented a dataset containing over 16000 marathi tweets, manually tagged in three classes namely positive, negative and neutral. they also provided a policy for tagging sentences by their sentiment. analysis was performed on cnn, bilstm, and bert models. @xcite collected hate phrases identified by hatebase.org and then used those phrases to collect english tweets from twitter using twitter api. the final set of 25k tweets was annotated by crowdflower workers with labels hate, offensive and neither. this dataset was then tested on logistic regression, naive bayes, decision trees, random forests, and linear svms. in @xcite , the authors evaluated the effect of filtering the generated data used for data augmentation (da). this demonstrates up to 7.3% and up to 25% of relative improvements on macro-averaged f1 on two widely used hate speech corpora. @xcite proposed a hypothesis that there exists a relation between fake messages or rumors and sentiments of the texts posted online. the experiments were performed on the standard twitter fake news dataset and showed good improvement on the same. @xcite provided an annotated corpus of hate speech with the context information. this evaluates by using logistic regression and neural network models for hate speech detection around 3% and 4%, and it improves to 7% by combining these two models together. @xcite presented mimct to detect offensive(hate or abusive) hinglish tweets from the proposed hinglish offensive tweet dataset. demonstrated the use of the multi-channel cnn-lstm model for sentiment analysis. ## conclusion in this paper, we have presented l3cubemahahate -a hate speech dataset containing 25000 distinct samples equally distributed in 4 classes. this is the first major dataset in the domain of hate speech. we also provide the binary version of the dataset of over 37500 samples. we further perform experiments to obtain baseline results on various deep learning models like cnn, lstm, bilstm, and transformerbased bert models such as indicbert,