Does Machine Translation Impact Offensive Language Identification? The Case of I ndo- A ryan Languages (2025)

Abstract: The accessibility to social media platforms can be improved with the use of machine translation (MT). Non-standard features present in user-generated on social media content such as hashtags, emojis, and alternative spellings can lead to mistranslated instances by the MT systems. In this paper, we investigate the impact of MT on offensive language identification in Indo-Aryan languages. We use both original and MT datasets to evaluate the performance of various offensive language models. Our evaluation indicates that offensive language identification models achieve superior performance on original data than on MT data, and that the models trained on MT data identify offensive language more precisely on MT data than the models trained on original data.

## introduction social media platforms have seen rapid growth in popularity in recent years. several types of content are shared across these platforms such as product reviews, discussions on topics ranging from entertainment to politics, and general commentary on topics like health, social issues, etc. however, some of this content shared on social media platforms can include offensive and toxic language, misinformation, etc., and can be harmful to individuals. offensive content can include pejorative language @xcite , cyberbullying @xcite , hate speech @xcite , targeted insults @xcite , and many others. machine-learning approaches have been used to moderate such content and mitigate its spread @xcite . the content posted on these social media platforms is written in several languages and dialects. while most of the research on offensive language identification has been grounded around high-resources languages like english, german, and spanish @xcite @xcite , recently, this research has been extended to low-resource languages, including the indo-aryan languages @xcite . models such as transformers have achieved state-of-the-art performance on this task in most of the languages @xcite . various social media platforms translate content between languages using machine translation models @xcite . however, the non-standard nature of the user-generated content, the mt content is often mistranslated @xcite . as most of the models and high-and low-resource languages are trained on the original instances from social media platforms, mt of such content can be challenging for offensive language detection models, especially in low-resource languages. furthermore, there has not been much research on offensive language detection of machine-translated content. in this paper, we fill this gap by evaluating the performance of state-of-the-art offensive language identification models on mt content in three low-resource indo-aryan languages: hindi, marathi, and sinhala. these three languages have been spoken by millions of speakers each and have been widely used on social media platforms, despite their low-resource nature. the presence of language experts to validate the translations was another factor influencing the language choice. this work address the following research questions: to answer these questions, we compare the performance of offensive language identification models, trained and evaluated on translated and non-translated data for the three aforementioned languages in various scenarios. the main contribution of this paper is a comprehensive evaluation of state-of-theart offensive language identification models on machine-translated content in indo-aryan languages. we evaluate the performance of finetuned bert-based models in four scenarios. the remainder of this paper is structured as follows: section 2 presents related work on mt and social media, section 3 presents the datasets used in this paper, section 4 presents the models used in our experiments and section 5 presents the results of the experiments and a comprehensive discussion and analysis. finally, section 6 concludes this paper and presents avenues for future work. ## related work multiple studies have focused on offensive language identification of multilingual content @xcite @xcite . while there have been various competitions organized for the task @xcite @xcite . transformer models such as bert @xcite have achieved state-of-the-art performance in these datasets. studies have also been conducted to understand the generalizability of the offensive language models @xcite . @xcite introduces a subsequent task that identifies offensive spans and their associated targets. the language understanding capabilities of the recent large language models (llms) have highlighted their ability to identify offensive content. these models, when evaluated with in-context learning approaches on english offensive language datasets, achieve comparable performance to the fine-tuned bert-based models @xcite . however, these studies do not consider the machine-translated content. mt is however used to augment offensive language datasets with more instances to train machine learning models @xcite . @xcite translate english tweets to arabic using google machine translation api and train models to classify offensive tweets. @xcite develop machinetranslated datasets for five high-and lowresource languages and use these to understand the impact of mt on offensive language detection. lohar et al. (2018) and saadany et al. (2021 ) explored the impact of mt on sentiment analysis task, while a naive bayes model was trained for sentiment analysis of english reviews translated to greek and italian @xcite . other works that utilized mt for sentiment analysis include a metric to assess the sentiment similarity between mt content and reference content () to identify the mistranslations of sentiments. while @xcite propose a neural machine translation(nmt) model that utilizes a two-step approach which first generates a sentiment label and then uses this sentiment label to train an nmt model. the work by @xcite translates arabic to english using three nmt models, where two of these models are sentiment sensitive. ## data the original datasets used in the paper are presented in ## approach the following sections define the models and hyperparameters used in our experiments. ## results and discussion in this section, we present the results of the different models evaluated in the four aforementioned training and testing scenarios. ## conclusion in this work, we presented an evaluation of the impact of mt on offensive language detection from english to three indo-aryan languages; hindi, marathi, and sinhala. unlike the previous work that used mt to augment the existing datasets in low-resource languages, in this work, we study the effects of mt on offensive language detection. we answer the two research questions posed in the introduction. for rq1, the orig-trans experiments show that the models trained on the source language dataset and evaluated with the translated datasets misclassify the offensive text. this can be attributed to the cultural stereotypes and translation quality, including mistranslations, irrelevant translations, etc. words that are considered offensive in one language may not be offensive in another, and the models trained on the translated language datasets may not identify such offensive words, leading the models to misclassify the text. the trans-trans and orig-trans experiments answer the rq2. the results indicate that models trained on the translated data outperform the models trained on the original data when evaluated with the translated datasets. this behavior can be attributed to the translation patterns learned during training, which, in turn, improves performance. as mentioned in section 5.4, the translations generated by the automatic machine translation models consist of several inaccuracies and errors. such errors are propagated to the offensive language models trained using this translated data. hence, content can be misclassified, limiting the purpose of deploying the models for social media moderation. moreover, cultural stereotypes and languagespecific contextual differences might prompt the models to misclassify the text. these factors need to be considered when training online content moderation models. the inaccuracies and errors from mt are propagated to the offensive language detection models that are trained on the translated data. this can cause the models to misclassify the content, limiting the use of such models for social media content moderation. furthermore, cultural stereotypes, along with language-specific contextual differences, might prompt misclassification. factoring these issues during model training can improve the performance of the online content moderation models.