SSN -Nova@ LT - EDI 2024: POS Tagging, Boosting Techniques and Voting Classifiers for Caste And Migration Hate Speech Detection (2024)

Abstract: This paper presents our submission for the shared task on Caste and Migration Hate Speech Detection: LT-EDI@EACL 20241 . This text classification task aims to foster the creation of models capable of identifying hate speech related to caste and migration. The dataset comprises social media comments, and the goal is to categorize them into negative and positive sentiments. Our approach explores back-translation for data augmentation to address sparse datasets in low-resource Dravidian languages. While Part-of-Speech (POS) tagging is valuable in natural language processing, our work highlights its ineffectiveness in Dravidian languages, with model performance drastically reducing from 0.73 to 0.67 on application. In analyzing boosting and ensemble methods, the voting classifier with traditional models outperforms others and the boosting techniques, underscoring the efficacy of simper models on low-resource data despite augmentation.

## introduction the deep-seated phenomenon of caste discrimination in india has endured over time, with recent advancements reflecting breakthroughs in challenging these deeply ingrained biases. despite contemporary endeavors to disentangle from the shackles of caste-based prejudices, the phenomenon still persists, exerting influence on diverse facets of individual lives @xcite . in the era of expanding social media platforms, marked by attributes like user anonymity, widespread accessibility, and the fostering of online communities and discourse, the identification and surveillance of hate speech rooted in caste discrimination pose a significant societal challenge. while machine learning models for hate speech detection have made significant strides in the western context, @xcite there is a glaring gap when it comes to adapting these models to the nuanced dynamics of casteism in india. casteism, a concept uniquely embedded in the social fabric of south asian communities, introduces complexities that are not adequately addressed by current research and detection mechanisms. unlike hate speech patterns prevalent in the west, caste-based discrimination in india operates within a distinct socio-cultural context, marked by intricate layers of subtext and nuanced contextual variations @xcite . the dearth of research tailored to this phenomenon unique to the indian subcontinent hinders the effectiveness of existing models in capturing the intricacies of this societal issue, especially in the sphere of social media. it is crucial to acknowledge that the linguistic, cultural, and historical dimensions of casteism necessitate a more nuanced approach to hate speech detection, one that transcends the limitations of generic models designed for western contexts @xcite . our paper is structured as follows -section 2 explores other publications pertaining to text classification tasks in low resource languages, section 3 provides an analysis of the distribution of the dataset, section 4 highlights the methodology undertaken for our proposed model and section 5 analyses the performance metrics of the solutions and provides a conclusion. ## related work though work has been done in text classification for low-resource languages in the recent past, it is apparent that the lack of annotated datasets has continually limited the scope of research in the field, with @xcite making notable strides in this regard. this inherent drawback has severely affected the applications of widely adopted methods, including pos tagging, on morphological learning in dravidian languages @xcite . hence, data augmentation, with an emphasis on backtracking, poses as an attractive solution to aid in combating such data issues @xcite . with respect to classifiers utilised, research has been focused on transformer and deep learning models @xcite . however, little light has been shed on the efficacy of ensemble approaches with traditional machine learning models @xcite , which have proved to outperform state-of-the-art technology that requires large quantities of annotated data @xcite , a vision that remains to elude research in dravidian languages. ## dataset analysis the labels given for the data were "caste/migration hate speech" and "non-caste/migration hate speech". the data distribution is provided below in ## results and analysis the impact of the pos tagging was elementally validated by juxtaposing the performance of the svm classifier. analysis found that the macro-average f1 score of the model significantly decreased with the implementation of pos tagging from 0.73 to 0.67. this could be attributed to the morphosyntactic intricacies of dravidian languages. this discrepancy stems from the profound dissimilarity in grammatical structures and semantics inherent to dravidian languages, diverging significantly from the syntactic patterns prevalent in the latin alphabet. nonetheless, pos tagging heavily relies on annotated corpora for discerning patterns and relationships between words and their corresponding pos tags. hence, the model's ability to generalise effectively could be hindered due to the limited dataset, lack of grammar and semantic standardisation paired with the significant number of outof-vocabulary words that may not be adequately covered in training data. the augmentation of data, ostensibly believed to augment the efficacy of the model, yielded only a marginal enhancement in model performance, manifesting as a modest 1-2 percent improvement. our hypothesis posits that the inherent simplicity of the operative models acts as a constraining factor, impeding their capacity to effectively leverage the augmented dataset. nonetheless, despite marginal improvements, the augmented dataset was systematically employed for subsequent exploration and analysis. ## conclusion our approach aimed to leverage data augmentation through back translation to address the issue of sparse datasets in low-resource dravidian languages. however, the implementation did not yield signifcant improvements in model performance. tangentially, part-of-speech (pos) tagging is exceptionally valuable in natural language processing, providing crucial insights into the grammatical structure of sentences, enabling accurate syntactic analysis, and facilitating downstream tasks like sentiment analysis and machine translation. despite having such crucial applications, pos tagging remains ineffective on dravidian languages, highlighting the exigency for nuanced linguistic models attuned to the unique intricacies of non-latin script languages. on analysis of different boosting and ensemble methods, the voting classifier incorporating traditional models proved to outperform the other models, highlighting the efficacy of simpler models on low-resource data despite data augmentation. on probing deeper, it was found that between the xg-boost and adaboost as well, the simpler of the two models proved to perform significantly better.