KEC - AI - NLP @ LT - EDI -2024:Homophobia and Transphobia Detection in Social Media Comments using Machine Learning (2024)

Abstract: Our work addresses the growing concern of abusive comments in online platforms, particularly focusing on the identification of Homophobia and Transphobia in social media comments. The goal is to categorize comments into three classes: Homophobia, Transphobia, and non-anti LGBT+ comments. Utilizing machine learning techniques and a deep learning model, our work involves training on a English dataset with a designated training set and testing on a validation set. This approach aims to contribute to the understanding and detection of Homophobia and Transphobia within the realm of social media interactions. Our team participated in the shared task organized by LTEDI@EACL 2024 and secured seventh rank in the task of Homophobia/Transphobia Detection in social media comments in Tamil with a macro- f1 score of 0.315. Also, our run was submitted for the English language and secured eighth rank with a macro-F1 score of 0.369. The run submitted for Malayalam language securing fourth rank with a macro- F1 score of 0.883 using the Random Forest model.

## introduction in the contemporary digital landscape, social media platforms serve as pivotal mediums for communication, education, and information sharing. among these platforms, youtube stands out as a prominent social networking and video-sharing hub, enabling users to create accounts, share videos, and interact through comments. however, the prevalence of abusive comments, particularly targeting transgender and homosexual individuals, poses a significant challenge to the well-being of platform users. the escalating use of online communication has raised concerns about the dissemination of slander, hate speech, and cyberbully, with negative 1 https://codalab.lisn.upsaclay.fr/competitions/16056 consequences for individuals and societal harmony. slander, characterized by false spoken statements that harm individuals or groups, is increasingly acknowledged for its detrimental impact @xcite . such negative comments not only inflict psychological harm but also contribute to the proliferation of animosity, division, and discontent in online spacesmishna et al. @xcite . major social media platforms like youtube, facebook, instagram, and twitter have responded by implementing policies and protocols to address and mitigate hateful content. our study aims to scrutinize and identify offensive comments within an english dataset, treating the detection of abusive comments as a text classification problem. focused on machine learning and deep learning methodologies, our research excludes the use of transfer learning models and does not involve the integration of machine learning and deep learning approaches. the objective is to train and compare various models to determine the optimal approach for identifying hate comments in english. ## literature review research in the field of abusive language detection spans various approaches and methodologies, as evident in several notable papers. @xcite emphasize the challenges faced in arabic abusive language detection, including dialects and informal language. @xcite introduce a novel approach using graph convolutional networks (gcns) to capture syntactic and semantic dependencies for effective abusive language identification. addressing gender bias in abusive language detection, @xcite propose a method incorporating gender information into the training process, showcasing its effectiveness in reducing bias while maintaining overall performance. ibrohim and budi (2019) focus on multi-label hate speech detection in indonesian twitter, analyzing various approaches, including feature-based, deep learning, and ensemble methods. narang and brew (2020) present an approach utilizing syntactic dependency graphs for abusive language detection, achieving superior performance compared to baseline models. @xcite introduce hatebert, a retraining approach for bert tailored for english abusive language detection, demonstrating its superiority in precision, recall, and f1-score. @xcite investigate racial bias in hate speech datasets, highlighting potential biases in annotation processes and emphasizing the need for fair evaluations. @xcite introduce hurtbert, combining bert with lexical features for enhanced abusive language detection performance. @xcite propose a zero-shot abusive language detection using emoji-based masked language models, demonstrating competitive performance. chakravarthi (2020) contribute hopeedi, a multilingual dataset for hope speech detection, aiming to facilitate research on positive discourse in social media. overall, these works offer diverse insights and methodologies, advancing the understanding and detection of abusive language in various linguistic and societal contexts. ## dataset description the goal of this shared task on homophobia and transphobia comment detection is to detect and reduce abusive comments on social media that target homosexual and trans-gender individuals. the dataset used here is shared by the shared task @xcite . the primary goal of this project is to develop methods for detecting and classifying instances of hate speech in english language. the homophobia and transphobia comment detection data set is made up of english comments retrieved from the youtube comments area @xcite . the data set consists of a comment and its related label from one of the three labels: non-anti-lgbt+ content, homophobia, transphobia. smote, which stands for synthetic minority over-sampling data augmentation technique, is a widely used technique in the field of machine learning specifically in the context of handling imbalanced datasets.imbalanced datasets occur when the classes have significantly different numbers of instances, leading to a bias in the model's performance towards the majority class. ## performance evaluation after submitting the run using the random forest model, it proved beneficial for various languages. analyzing the results in the random forest model excelled in handling high-dimensional data, managing noisy and missing values, and mitigating overfitting concerns through feature subsampling and bootstrap aggregating. notably, the dataset's class distribution was not uniform, with two crucial classes having very few instances. despite this challenge, the random forest model demonstrated exceptional performance. the contrasting deep learning model, reliant on significant computational resources and extensive parameter tuning, fell short, resulting in comparatively lower accuracy and f1 score. in figure 2 , the presented confusion matrix provides a comprehensive overview of the performance of the random forest model when applied to the malayalam dataset. this emphasizes the importance of selecting an appropriate modeling technique tailored to the dataset's characteristics, leading to improved predictive performance. ## conclusion the study concentrates on detecting homophobic and transphobic comments in youtube discussions, comparing the performance of various models in this task. strikingly, deep learning models did not demonstrate superior results when trained and evaluated on english data. instead, machine learning models outperformed deep learning in effectiveness. it's crucial to note that our study did not make use of contextualized embeddings like bert or gpt, which have shown potential in enhancing language model performance. acknowledging this limitation, we propose that future research should explore the implementation of contextualized embeddings using deep learning techniques, such as bert or gpt. the absence of these advanced embeddings may have limited the effectiveness of the models used in our study. incorporating such embeddings holds promise for significantly improving the detection of homophobic and transphobic comments in youtube discussions. additionally, we did not explore transfer learning with other models in our current stage. still, we emphasize the possibility of integrating these models in our future work, indicating a pathway for ongoing exploration and enhancement in identifying such comments on youtube.