cantnlp@ LT - EDI -2024: Automatic Detection of Anti- LGBTQ + Hate Speech in Under-resourced Languages (2024)

Abstract: This paper describes our homophobia/transphobia in social media comments detection system developed as part of the shared task at LT-EDI-2024. We took a transformer-based approach to develop our multiclass classification model for ten language conditions (English, Spanish, Gujarati, Hindi, Kannada, Malayalam, Marathi, Tamil, Tulu, and Telugu). We introduced synthetic and organic instances of script-switched language data during domain adaptation to mirror the linguistic realities of social media language as seen in the labelled training data. Our system ranked second for Gujarati and Telugu with varying levels of performance for other language conditions. The results suggest incorporating elements of paralinguistic behaviour such as script-switching may improve the performance of language detection systems especially in the cases of under-resourced languages conditions.

## introduction the purpose of this shared task was to develop a multiclass classification system to predict instances of homophobia/transphobia in social media comments across different language conditions @xcite . the ten language conditions were: english (eng), spanish (esp), gujarati (guj), hindi (hin), kannada (kan), malayalam (mal), marathi (mar), tamil (tam), tulu (tcy), and telugu (tel). the main contribution of this paper is that we extend on the work using spatio-temporally retrained transformer-based language models in @xcite . we have expanded on the synthetic scriptswitching approach by incorporating real-world (or organic) samples of script-switching during domain adaptation in the development of our multiclass classification model using pretrained language models. ## methodology in this section, we provide an overview of our system development methodology. we took a transformer-based language model approach to develop our system. we used xlm-roberta as the base plm for our system @xcite . the embeddings in xlm-roberta were trained on two terabytes of web-crawled data for over 100 language including nine of the ten language conditions of interest (with the exclusion of tcy). a significant advantage of transformer-based plms is the ability for domain adaptation as discussed in section 2.1. this means we can retrain the default language embedding models with additional language data without the need to train resource-intensive plms from scratch. we tested different forms of script-switching in order to understand the impacts of script-switching on our classification system. we then used the plms developed in section 2.1 to fine-tune our multiclass classification model as discussed in section 2.2. based on the weighted macro f 1 for each language condition, we submitted the results from the best performing multiclass classification system to the organisers. ## results based on the average macro f 1 score of the candidate models as shown in our final rank for each language conditions are as follows: for eng we came tenth equal out of ten teams; for esp we came fourth out of five teams; for guj we came second out of six teams; for hin we came fourth equal out of seven teams; for kan we came fourth out of eight teams; for mal we came seventh out of nine teams; for mar we came fifth out of six teams; for tam we came fifth out of eight teams; for tcy we came third equal out of four teams; and finally for tel we came second out of nine teams. foot_3 ## discussion the use of synthetic and organic script-switched language data during domain adaptation increased the performance for all language conditions from the baseline model with the exception of eng, hin, and tcy. we expected the eng and esp language conditions to perform poorly with our proposed methodology as there were very few instances of script-switching, but the poor performance of tcy was unexpected. we hypothesise the poor performance in tcy was due to the limited number of observation (as shown in figure 1 ) the higher than expected usage of latin-based script for tcy in the labelled training data (as shown in figure 2 ). this will require robust statistical analysis beyond the scope of the current paper. we also posit the poor performance of eng and hin was a result of the class imbalance between instances of homophobic, transphobic and the nonanti-lgbtq+ content as demonstrated in in contrast to the method proposed in @xcite , we did not include any methods to counter the class imbalance in the training data nor did we include random noise injection to expand the minority classes. it was shown that random over sampling of minority classes did not significantly improve the performance of the detection models. ## conclusion the main contribution of the current paper is the proposal to use synthetic and organic scriptswitching examples of during domain adaptation to improve the down-stream performance for underresourced languages. we demonstrated that our methodology improved the model performance for guj, kan, mal, mar, and tam even though the improvement was only marginal. even though our homophobia/transphobia detection system did not rank first for any of the ten language conditions, we were pleased with the performance of our detection system which supports the inclusion of paralinguistic information. for esp, tam, and tel as they were not included in the final rank list due to human error from the organising committee during submission.