J udith J eyafreeda@ LT - EDI -2023: Using GPT model for recognition of Homophobia/Transphobia detection from social media (2023)

Abstract: Homophobia and Transphobia is defined as hatred or discomfort towards Gay, Lesbian, Transgender or Bisexual people. With the increase in social media, communication has become free and easy. This also means that people can also express hatred and discomfort towards others. Studies have shown that these can cause mental health issues. Thus detection and masking/removal of these comments from the social media platforms can help with understanding and improving the mental health of LGBTQ+ people. In this paper, GPT2 is used to detect homophobic and/or transphobic comments in social media comments. The comments used in this paper are from five (English, Spanish, Tamil, Malayalam and Hindi) languages. The results show that detecting comments in English language is easier when compared to the other languages.

## introduction homophobic and/or transphobic comments is a form of hate speech directed towards lgbtq+ community. with the increase in internet and use of social media, the use of derogatory comments have increased considerably. these comments cause mental health issues for a lot of people within the lgbtq+ community @xcite . thus identification of these comments is necessary to improve the well being of the community. this is a specific case of offensive language or hate speech detection. the task in this paper, is to identify and classify text into 3 classes (sub task 1) or 7 classes (sub task 2) [detailed in section 2]. the language concerned in this task are english, tamil, malayalam, hindi and spanish. some text are code-mixed text. code-mixing is the process of mixing more than one language in a text. @xcite and @xcite have devel-oped a dataset and methods for sentiment analysis for code-mixed data for the dravidian languages of tamil and english. the task in this paper is a multi class classification problem. in this task, there are more than 2 predefined classes and each text can be placed in only one of the predefined class. several multi class classification approaches have been proposed previously like in @xcite , @xcite . however, considering the languages and context, all the methods might not be suitable for the task at hand. @xcite have proposed a embedding for the language tamil. other forms of pre processing for dravidian languages have been proposed by @xcite al. (2021 1 and 2 shows the number of comments in each set for the different languages and different sub tasks. recent efforts on classification of offensive text involve the use of neural networks. within this context, @xcite compare four models: an interpretable machine learning model (naive bayes), a model-agnostic explanation method (lime), a model-based explanation method (lrp), and a self-explanatory model (lstm with an attention mechanism), showing that complex models perform better than simpler ones. most work done within this area focuses on the english language, however there are language processing challenges when different languages are used. @xcite @xcite presents an improvement of word sense translation for under-resourced languages. @xcite proposed a multiclass classification method, where several machine learning algorithms have been adapted to the task of sentiment analysis and based on the accuracy of the algorithms on the development set the best suited technique is chosen for the language and the task. (andrew, 2021) suggests few machine language approaches to classify texts from codemixed dravidian languages. (andrew, 2022) uses a cnn approach for the classification of emotion in youtube comments for the dravidian language of tamil. in this paper, the data from various languages are pre-processed with using methods described in (andrew, 2021) and @xcite . this is then used along with a gpt model for classification. ## proposed system in this work gpt2 is used for classification of homophobic and transphobic comments. the model is finetuned on the training dataset for each task and every language. for languages other than english, the text is replaced with the ipa equivalent, this approach has been inspired from (andrew, 2021) and @xcite . the categories are in english language, thus ipa equivalent character need not be substituted. pre-processing: similar to (andrew, 2022), a few steps of pre-processing is performed to get the accurate representation of the text. this involves the following: gpt2 gpt, generative pre trained models, is a neural network based architecture which uses transformers. these use a self-attention mechanism allowing to focus on different parts of the input text during the various stages on processing. gpt-2 model has 1.5 billion parameters and has been trained on 8 million web pages in a selfsupervised fashion. @xcite provides a detailed description of the model. the inputs are sequences of continuous text of a certain length and the targets are the same sequence, shifted one token (word or piece of word) to the right. the model uses internally a mask-mechanism to make sure the predictions for the token i only uses the inputs from 1 to i but not the future tokens. this allows the model to learn the inner representation of the language, which can then be used to extract features for downstream tasks. gpt2 for classification: python has a range of packages that allow the use of gpt models such as hugging face's transformers, nltk, and textblob. in this paper, this python package is used for classification of text into classes of sentiments. the training data for each language is used to fine tune these models with the different classes (varying for the two sub classes) and for each language. 5 evaluation the performance of the classification system is measured in terms of macro averaged precision, macro averaged recall and macro averaged f-score across all the classes (for both sub tasks). the scikit-learn foot_0 package is used for this purpose. tables 3 and 4 show the results of the sub tasks 1 and 2 respectively. to recap, the sub task 1 is to classify the text into 3 classes (non-anti-lgbt+ content, homophobia and transphobia) and the sub task 2 is to classify the text into 7 classes (counter-speech, homophobic-derogation, homophobic-threatening, hope-speech, transphobic-derogation, transphobic-threatening, none-of-the-above). although the models are specifically fine tuned for each languages and sub tasks, some fine tuned models perform better than the others. from