ABLIMET @ LT - EDI - ACL 2022: A Roberta based Approach for Homophobia/Transphobia Detection in Social Media (2022)

Abstract: This paper describes our system that participated in LT-EDI-ACL2022- Homophobia/Transphobia Detection in Social Media. Sexual minorities face a lot of unfair treatment and discrimination in our world. This creates enormous stress and many psychological problems for sexual minorities. There is a lot of hate speech on the internet, and Homophobia/Transphobia is the one against sexual minorities. Identifying and processing Homophobia/ Transphobia through natural language processing technology can improve the efficiency of processing Homophobia/ Transphobia, and can quickly screen out Homophobia/Transphobia on the Internet. The organizer of LT-EDI-ACL2022- Homophobia/Transphobia Detection in Social Media constructs a Homophobia/ Transphobia detection dataset based on YouTube comments for English and Tamil. We use a Roberta -based approach to conduct Homophobia/ Transphobia detection experiments on the dataset of the competition, and get better results.

## introduction at present, the internet is full of various hate speeches, including racial discrimination, religious hostility, mutual hostility between political groups, and discrimination against sexual minorities. the discrimination against sexual minorities is called homophobia/transphobia. homophobia and transphobia are two concepts that are similar and different. homophobia refers to unwarranted fear, hatred, and unfair treatment of homosexuals, and transphobia refers to disgust and discrimination against transgender people. homophobia/ transphobia will bring serious psychological stress to lgbtq people, making them unable to participate in social activities normally, and even causing them serious mental illness. therefore, the quick and efficient identification and screening of homophobia/transphobia on the internet will help to clean up cyberspace, build a healthy and harmonious internet community, and help more people realize the unfair treatment of lgbtq groups. shared task on homophobia/transphobia detection in social media comments at lt-edi 2022-acl 2022 is a classification task. the organizer of the competition constructed a homophobia/transphobia detection dataset in english, tamil, and tamil-english based on youtube comments. the model needs to determine whether the target data contains homophobia/ transphobia. and if so, which type of homophobia/ transphobia is included. we used roberta @xcite as our pre-trained language model and fine-tuned it for the task. in our experiment, we process the target data by the pre-trained language model, the output was normalized firstly by a layer normalization module， then we use two fully-connected layers and between them there is a layer normalization operation. we use the cross-entropy as our loss function, and optimize it by adamw @xcite . through the above steps, we complement the identification and classification of homophobia/transphobia in the target data. we participated in all of the english, tamil, and tamil-english homophobia/transphobia detection subtasks, and we use a version of roberta pretrained on the corresponding linguistic data for each language. by training the model on train data and validating it on development data, we achieved better results on test data. specifically, we achieved a 0.57 macro f1-score on the english subtask and ranked 1st among all participating teams, achieved a 0.75 macro f1-score on the tamil subtask and ranked 5th among all participating teams, achieved a 0.53 macro f1 score on the tamil-english subtask and ranked 6th among all participating teams. ## background in this section, we introduce the relevant background of the shared task on homophobia/ transphobia detection in social media comments at lt-edi 2022-acl 2022, including the details of the task and the related research on homophobia/transphobia detection. ## system overview in this section, we will introduce our approach to the task, the multi-label homophobia/transphobia detection task, which we solve using a finetuning approach of the pre-trained language model. specifically, we process the target data by the pre-trained language model, the output was normalized firstly by a layer normalization module ， then we use two fully-connected layers and between them with a layer normalization operation. the model architecture is shown in figure 1 . input is the target data to be processed, and the pre-trained language model (plm) processes the input data and outputs the result to the layer normalization module. to prevent internal covariate shift, we follow the pre-trained language model and first fully connected layer with a layer normalization module. the pretrained language model we use is roberta-base for english subtask, tamil-roberta for tamil, and tamil-english subtasks. the output tensor size of the pre-trained language model is 768, while our target label is only 3, the difference between the two is large, so we connect two full connection layers behind the pre-trained language model layer, where the output tensor size of the first fully connected layer is 64, and the output tensor size of the second fully connected layer is 3,the number of target labels. for the loss function, we use the cross-entropy provided by the pytorch @xcite framework, and use adamw @xcite as optimizer, which is an improved version of the adam (kingma and ba, 2017) optimizer. due to the huge parameters of the pretrained language model, it is easy to overfit when using the adam optimizer, while adamw uses l2 regularization to reduce overfitting, which can significantly improve the generalization ability of the model. ## experimental setup in this section, we will introduce the relevant design, parameter settings, and experimental environment of the experiments. in terms of hardware, we use a laptop with a gtx 1650 graphics card for model training. on the software side, we use pytorch @xcite and huggingface @xcite library to code the tasks. ## results we use the roberta -based approach to train the model on the training datasets of the task, validate the model on the development set, and use the trained model to predict the label of the test set. repeated experiments with different epoch values, we found that when the epoch is 8, the trained model has the best validation results on the development set. so we train our model for 8 iterations, predict the labels of the test set with the model, and submitted the run results in all of the three languages. details of results on test data sets and ranks are shown in as we can see from from table3 we also can see that the f1-scores of tamil and tamil-english subtasks are 0.75 and 0.53 respectively, with a wide difference. to address the reason for the problem, we averaged the f1-scores of all teams for the two subtasks separately, then subtracted the average of f1scores of the tamil-english subtask from the average of f1-scores of the tamil subtask, the difference between the two is 0.18. then we subtracted the f1-score of our model on the tamil-english subtask from the f1-score of our model on the tamil subtask, the difference between the two is 0.22. so far we found that the f1-score of the tamil subtask is generally higher than the f1-score of the tamil-english subtask, so we think that the difference between the f1scores of the two subtasks is related to the features of the datasets for tamil and tamil-english subtasks. to make a comparison between the test results of the models trained on balanced data sets (balanced by using randomoversampeler class) and the test results of the models trained on the original unbalanced train data sets after the competition. we train the models using the original unbalanced train data sets and test them by using test data sets with labels. the results are shown in ## conclusion we use a roberta-based approach for homophobia/transphobia detection tasks and achieved better results in our experiments. although the results on the tamil and tamil-english subtasks are not so ideal, the results on the english subtask show that our approach is effective for the homophobia/transphobia detection tasks. although this competition has come to an end, there are still some directions we can continue to study in the future. for example, we can use prompt learning to process this task, by converting this task into cloze form and designing reasonable templates and verbalizers, we can fully make use of the knowledge of pretrained language models, and may get better results. besides, by converting the homophobia/ transphobia detection task into a text generation task, and then using text generation models like gpt @xcite , t5 @xcite to solve the task, we may get unexpected results. in future research, we will continue to study the directions mentioned above, and strive to achieve better homophobia/ transphobia detection performance.