Quartet@ LT - EDI 2024: Support Vector Machine Based Approach For Homophobia/Transphobia Detection In Social Media Comments (2024)

Abstract: Homophobia and transphobia are terms which are used to describe the fear or hatred towards people who are attracted to the same sex or people whose psychological gender differs from his biological sex. People use social media to exert this behaviour. The increased amount of abusive content negatively affects people in a lot of ways. It makes the environment toxic and unpleasant to LGBTQ+ people. The paper talks about the classification model for classifying the contents into 3 categories which are homophobic, transphobic and nonhomophobic/ transphobic. We used many traditional models like Support Vector Machine, Random Classifier, Logistic Regression and KNearest Neighbour to achieve this. The macro average F1 scores for Malayalam, Telugu, English, Marathi, Kannada, Tamil, Gujarati, Hindi are 0.88, 0.94, 0.96, 0.78, 0.93, 0.77, 0.94, 0.47 and the rank for these languages are 5, 6, 9, 6, 8, 6, 6, 4.

## introduction social media platforms of the current century have evolved into a way people communicate with each other. social media is about having conversations, building communities and connecting with the audience. it has become an integral part of everyone's lives. it is not just a marketing tool or a way of broadcasting news. it not only allows you to hear what people say about you but gives the space for you to share your own opinions on the matters happening and helps us in influencing people in both positive and negative ways. this can have a very significant impact on people and the decisions they make. one of the consequences of the rapid increase in the number of social media users is the increase in the increase in the inappropriate use of social media by the users.workshops and collaborative tasks held recently have stimulated projects regarding the identification of hate speech, toxicity, misogyny, sexism, racism, and abusive content @xcite . the convenience of accessing information and being a great source of great conversations, it also makes cyber bullying and hate speech possible. since it allows us to share our view points on everything, hate speech on transsexual and homosexual people are very common. transphobia is when people have a deep-rooted negative prejudice about being transgender or non-binary. homophobia is the aversion or hatred towards people who are homosexual or gay. this has a negative consequence on people belonging to these minority gender groups. despite a greater acceptance of sexual variations and same-sex marriage in many places, homophobia and transphobia still exist widely and is sustained by many religious, political and individual practises. many studies have presented that around 8 to 9 out of 10 people are subjected to hate speech online the percentage of transgender and homosexual people in it is significantly high @xcite . homophobia and transphobia don't end with conversing. it can take a physical form of violence too. violence is becoming too common on social media platforms and it influences people negatively. violence in the form of murder, beating or even sexual violence such as molestation is becoming too common @xcite . social media has a great part in this. it is a powerful tool that can easily influence many people. many hateful comments such as "gay people should be killed mercilessly", "transgender people should be stoned" are becoming too common and greatly influence the current generation. numerous workshops and collaborative efforts are currently focused on identifying abusive content as well @xcite . recognising these homophobic and transphobic comments on social media automatically can make it very easy for us to block these immediately @xcite . this tool can flag all the homophobic and transphobic comments and can make the environment inclusive. it can reduce harm and harassment directed at individuals solely based on their sexual orientation or gender identity. numerous research efforts are underway to identify abusive content in various local languages as well @xcite . it helps in influencing the social media users positively and helps in reducing homophobia and transphobia around the world. ## related works sentiment analysis is a field in which constant works and researches are being carried on. they have many applications on social and e-commerce platforms. @xcite has addressed this classification problem by applying the well established deep learning models, including convolutional neural network (cnn), long short-term memory (lstm) with glove embedding, and transformerbased models like multilingual bert @xcite and indicbert @xcite . results obtained show that the indicbert outperforms all the models and was finally used. @xcite solved the challenge of data imbalance by introducing a solution involving data augmentation and ensemble modeling. they fine tuned various large language models, including bert, roberta @xcite and hate-bert @xcite . a weighted majority vote is applied to aggregate their predictions. abdul kareem (2023) has employed transformer based models widely, as it provides better results than the traditional machine learning models. implementation includes roberta @xcite , distilbert @xcite , and mbert @xcite , with a comparative analysis against previous studies. the results showed that distilbert provided better results than roberta and mbert. a hypothesis that states that the performance of the models on a newly constructed dataset with limited data will be improved by data augmentation via pseudo labeling through transliterating the code mixed text to the parent language. performance of several models were run and tested by @xcite . the task done by bhandari and goyal (2022) involved multi class classification to identify homo-phobia or transphobia in youtube comments. the pipeline comprised a transformer-based classification head and data augmentation for oversampling the english dataset, detailed subsequently. multiclass classification system done by @xcite , utilizes a bert based model identifying homophobia and transphobia across english, spanish, hindi, malayalam and tamil. retraining xlm roberta @xcite with relevant social media data, including script-mixed samples, improved performance, especially in malayalam. transformer based models are sensitive to register and language-specific retraining, enhancing classification across various conditions. ## task and data description the shared task on homophobia/transphobia detection in social media comments at lt-edi@eacl 2024 foot_0 was created with the task of detecting homophobia, transphobia and non-lgbtq+ content on youtube comments. the task concentrates more on regional languages so homophobic and transphobic comments were given in tamil, telugu, malayalam, english, kannada, gujarati, hindi, marathi languages (hom, 2024). ## methodology we used many traditional models to test our model from it. we ran all the dataset through logistic regression, support vector machine, random forest classifier, k-nearest neighbour. by noticing the accuracy and the f1 score of each output, we determined if the model was over or under fitting and by comparing all the metrics, we selected the best model out of all the available options. support vector machine (svm) produced the best output in majority cases. ## results in every case, the support vector machine model has demonstrated a better performance compared to all other models. the evaluation was conducted by assessing the classification report of training,development and test datasets. the outcomes of support vector machine (svm) are presented in ## conclusion in conclusion, our study delved into the comprehensive evaluation of traditional machine learning models for text classification, employing an array of techniques from logistic regression to sophisticated models like support vector machine (svm) and transformer-based approaches. rigorous preprocessing, including lower casing, removal of punctuation, emojis, and stop words, ensured data quality. the tf-idf vectorizer facilitated effective feature extraction, translating textual data into numerical representations. notably, svm consistently outperformed other models in terms of accuracy and f1 score across diverse datasets. while our traditional models exhibited commendable performance, it is imperative to acknowledge the evolving landscape of deep learning and advanced embeddings, suggesting avenues for future exploration and refinement of models to capture intricate language nuances and patterns.