SSNCSE _ NLP @ LT - EDI - ACL 2022: Homophobia/Transphobia Detection in Multiple Languages using SVM Classifiers and BERT -based Transformers (2022)

Abstract: Over the years, there has been a slow but steady change in the attitude of society towards different kinds of sexuality. However, on social media platforms, where people have the license to be anonymous, toxic comments targeted at homosexuals, transgenders and the LGBTQ+ community are not uncommon. Detection of homophobic comments on social media can be useful in making the internet a safer place for everyone. For this task, we used a combination of word embeddings and SVM Classifiers as well as some BERT-based transformers. We achieved a weighted F1-score of 0.93 on the English dataset, 0.75 on the Tamil dataset and 0.87 on the Tamil-English Code-Mixed dataset.

## introduction human beings have constantly tried to create an identity for themselves, and with the world becoming increasingly progressive, they have more freedom of choice in many spheres of life, including gender expressions and sexuality. @xcite however, the understanding of these concepts continues to gradually evolve, and despite various major social advancements in the last few years, lgbtqia+ people face discrimination on the grounds of sexual orientation and gender identity. although social media has provided this minority with a platform to express themselves by sharing their experiences and build a strong, healthy community, there has been an increasing amount of general toxicity on the internet @xcite . there has also been a spread of transphobic and homophobic comments through these online forums, due to the easy access to anonymity they provide, which ensures that these violators are never held accountable @xcite . the need for the detection and filtering of such acerbic content in user-created online content is thus at an all-time high. however, the manual detection and flagging of certain words might be time-consuming and ineffective in the long run. the tendency of tamil speakers to use code-mixed transliterated text also poses a challenge to the task. in this paper, we examine various approaches for the classification of tamil code-mixed comments into three categories, namely, homophobic, transphobic and non-anti-lgbt+ content as a part of the shared task homophobia/transphobia detection @ lt-edi-acl2022 @xcite . after tackling the data imbalance using sampling techniques, feature extraction using count vectorizer and tf-idf was done along with various classifiers. another approach involved the usage of transformer models to classify the text. the same has also been analysed for english and tamil datasets. the remainder of the paper is organized as follows. section 2 discusses related works according to this task. section 3 analyses the given datasets. section 4 outlines the methodology followed for the task. the results are presented in section 5 and finally, a conclusion is delivered. ## related work the first formal defense of homosexuality was @xcite . the 20th century witnessed many ups and downs in the progress of social acceptance of sexual minorities. various studies on the existence of different sexualities have been conducted such as @xcite , @xcite , @xcite and @xcite , and it has been observed that there has been a positive shift in the attitude of the general public towards homosexuality @xcite . more recently, the lgbtq+ movement has picked up and has gained many followers through social media. several people have worked on the task of using machine learning to identify and filter out hurtful comments, thus aiding in the battle against homophobic/transphobic sentiments. some of the early works in this field include @xcite and @xcite , in which offensive language is identified in multiple indian languages as well as some foreign languages. in (pereira, 2018), homophobia was predicted in portuguese tweets using supervised machine learning and sentiment analysis techniques. a wide range of techniques was utilised in this study, some of which include naive bayes, random forest and support vector machines. the models were combined using voting and stacking, with the best results being obtained through voting using 10 models. @xcite presents an expertlabelled dataset and various machine learning models for the identification and classification of homophobia and transphobia in multilingual youtube comments. in @xcite , sentiment analysis and offensive language detection were performed for dravidian languages in codemixed text, which are super-sets of the homophobia/transphobia detection task. in this paper, an experimentation of a number of machine learning algorithms such as svm, mnb, knn, decision tree, random forest and some bert-based transformers, was done. in our work, we have put forward a comparison of some of the most popular models for this area of research and estimated the top three models for each language in the datasets given for this task. ## methodology as part of our experimental setup, various classifier models were applied to the processed data after extracting the necessary features from it. for each dataset, three models that worked best for the language under consideration were chosen to predict the classification results for comments collected in that language. for reference, the models under consideration for the english dataset have been listed in ## observations it was found that certain models or combinations of models outperform others for each dataset under scrutiny. the performance results for each chosen model are presented in the tables given below. this task is evaluated on the macro averages of three performance metrics -precision, recall and f1-score. the scores achieved for this homophobia detection task are tabulated below in