MUCS @ LT - EDI 2023: Homophobic/Transphobic Content Detection in Social Media Text using m BERT (2023)

Abstract: Homophobic/Transphobic (H/T) content includes hate speech, discrimination text, and abusive comments against Gay, Lesbian, Bisexual, Transgender, Queer, and Intersex (LGBTQ) individuals. With the increase in user generated text in social media, there has been an increase in code-mixed H/T content, which poses challenges for efficient analysis and detection of H/T content on social media. The complex nature of code-mixed text necessitates the development of advanced tools and techniques to effectively tackle this issue in social media platforms. To tackle this issue, in this paper, we - team MUCS, describe the transformer based models submitted to “Homophobia/Transphobia Detection in social media comments” shared task in Language Technology for Equality, Diversity and Inclusion (LT-EDI) at Recent Advances in Natural Language Processing (RANLP)-2023. The proposed methodology makes use of resampling the training data to handle the data imbalance and this resampled data is used to fine-tune the Multilingual Bidirectional Encoder Representations from Transformers (mBERT) models. These models obtained 11th, 5th, 3rd, 3rd, and 7th ranks for English, Tamil, Malayalam, Spanish, and Hindi respectively in Task A and 8th, 2nd, and 2nd ranks for English, Tamil, and Malayalam respectively in Task B.

## introduction social media platforms provide a means for users to express their views, ideas, reviews, comments, opinions, and emotions, freely and instantaneously without any barriers of the language and content. this has given raise to the creation and sharing of useful content as well as unhealthy posts, such as offensive, abusive, and hatred content, targeting a person, a group, or a community @xcite . h/t content is one such content that expresses the hatredness towards lgbtq community on their sexual orientation or gender identity @xcite . lgbtq individuals face various forms of textual violence and discrimination such as, hate speech, cyberbullying, exculsion and isolation, online shaming, and misgendering in online environment or on social media platforms. they also become targets of threats and abuse, leading to significant mental health issues @xcite . hence, identifying and removing h/t content on social media platforms is a crucial aspect in order to promote equality, diversity, and inclusion in the society. by implementing these measures, it is possible to create a safer online environment for the lgbtq community and support their well-being and mental health @xcite . identifying h/t content in social media text poses challenges due to the complex nature of code-mixed text prevalent on these platforms @xcite . social media text often includes the mixing of local or regional languages such as hindi, malayalam, tamil, etc., with english, at sub-word, word and sentence level leading to code-mixed content. @xcite @xcite . this code-mixing makes the identification and analysis of h/t content more difficult, as traditional language processing models may fail to accurately interpret and classify such mixed-language texts. to effectively address this challenge, natural language processing (nlp) techniques need to be explored for code-mixed text, taking into consideration the linguistic nuances and variations in different languages. by developing robust algorithms and models to handle code-mixed h/t text, it is possible to identify and combat h/t to address the challenges of h/t content identification in social media text, in this paper, we -team mucs, describe the models submitted to "homophobia/transphobia detection in social media comments" shared task 1 at ranlp-2023 2 . the shared task consists of two subtasks: i) task a -a comment-level polarity classification task to identify h/t content in english, tamil, hindi, malayalam, and spanish languages, with 3 labels (non-anti-lgbt+, homophobia, and transphobia) and ii) task b -to identify h/t content in english, tamil, and malayalam texts, with 7 labels (none-of-the-above, hope-speech, counterspeech, homophobic-derogation, homophobic-threatening, transphobic-derogation, and transphobic-derogation) @xcite . sample comments from the datasets provided by the organizers of the shared task for task a and task b are shown in the rest of the paper is structured as follows: section 2 contains related works and section 3 explains the methodology. section 4 describes the experiments and results and the paper concludes in 1 https://codalab.lisn.upsaclay.fr/competitions/11077/ 2 http://ranlp.org/ranlp2023/ section 5 with future work. ## related work several researchers have explored h/t content detection, offensive language identification and hate speech and offensive content detection in various languages and few of the relevant ones are described below: hegde and shashirekha (2022) describe the learning models to perform sentiment analysis (sa) and h/t content detection in code-mixed dravidian languages as task a (malayalam and kannada) and task b (tamil and romanized tamil (tamil-english)) respectively. using conventional preprocessing of converting emojis to text and removal of digits and stopwords, these models make use of dynamic meta embedding (dme) to train long short term memory (lstm) model for sa and h/t content identification in code-mixed dravidian languages. these models obtained macro f1 scores of 0.61 and 0.44 for malayalam and kannada languages respectively in task a and 0.58 and 0.74 for tamil and tamil-english texts respectively in task b. @xcite presented tl approach for fine-tuning cross lingual language models robustly optimized bert (xlm roberta) model in zero-shot learning framework for the detection of h/t contents in english and tamil-english and obtained macro f1 scores of 0.89 and 0.85 for english and tamil-english texts respectively. ashraf et al. ( 2022 ) proposed the machine learning (ml) models (support vector machines (svm), random forest (rf), passive aggressive two distinct models: cooli-ensemble -a voting classifier with three estimators (multi layer perceptron (mlp), extreme gradient boosting (xgb) and logistic regression (lr)) and cooli-keras -a keras dense neural network architecture model, described by @xcite aims to classify code-mixed texts in kannada-english, malayalam-english, and tamil-english language pairs into six predefined categories and malayalam-english language pair into five categories for identifying offensive content. character and word sequences extracted are vectorized using countvectorizer foot_0 and the relevant feafrom the literature review, it is clear that identification of h/t content in low-resource languages like, tamil, malayalam, and hindi are rarely explored. hence, there is lot of scope in this direction for further research. ## methodology the proposed methodology includes preprocessing, resampling, and classifiers construction using mbert models to address the challenges of task a and task b of the shared task. the framework of the proposed methodology is visualized in figure 1 and the steps involved in the methodology are given below: ## experiments and results statistics of the dataset provided by the organizers of the shared task for the identification of h/t content in social media text for task a and task b are shown in tables 3 and 4 respectively @xcite . from the tables, it is clear that the datasets provided by the organizers are highly imbalanced. to overcome this, oversampling is carried out for both the tasks using oversampling methods provided by the sklearn library foot_1 . the feature vectors. after loading the pretrained mbert model with its default parameter values, the model is frozen to prevent further updates to its weights. classificationmodel 6 -a transformerbased classifier is employed to make predictions and the hyperparameters and their values used in the model are shown in in the minority classes, the model becomes too specialized on the minority class and fails to generalize well to unseen data resulting in over-fitting. ## conclusion and future work this paper describes the models submitted by our team -mucs, to the shared task "homophobia/-transphobia detection in social media comments" at ranlp 2023 for the identification of h/t content in social media text. tl model with mbert are proposed for both tasks a and b along with oversampling. these models secured 11 th , 5 th , 3 rd , 3 rd , and 7 th ranks for english, tamil, malayalam, spanish, and hindi respectively in task a and 8 th , 2 nd , and 2 nd ranks for english, tamil, and malay-alam respectively in task b. data augmentation techniques for handling imbalanced classes with effective feature extraction techniques will be explored in future.