D etective R e DAS ers at HSD -2 L ang 2024: A New Pooling Strategy with Cross-lingual Augmentation and Ensembling for Hate Speech Detection in Low-resource Languages (2024)

Abstract: This paper addresses hate speech detection in Turkish and Arabic tweets, contributing to the HSD-2Lang Shared Task. We propose a specialized pooling strategy within a soft-voting ensemble framework to improve classification in Turkish and Arabic language models. Our approach also includes expanding the training sets through cross-lingual translation, introducing a broader spectrum of hate speech examples. Our method attains F1-Macro scores of 0.6964 for Turkish (Subtask A) and 0.7123 for Arabic (Subtask B). While achieving these results, we also consider the computational overhead, striking a balance between the effectiveness of our unique pooling strategy, data augmentation, and soft-voting ensemble. This approach advances the practical application of language models in low-resource languages for hate speech detection.

## introduction hate speech and offensive language on social media pose significant challenges, affecting individuals and communities globally. these concerns are exacerbated by the anonymity afforded by online platforms, leading to more aggressive behaviors @xcite . addressing hate speech is crucial for protecting vulnerable and marginalized populations from discrimination and racism. the issue is particularly profound in low-resource languages like arabic and turkish, where cultural and linguistic diversity adds additional complexity to detection. conventional approaches in hate speech detection, which often rely on standard tooling libraries, may opt to remove emojis due to the unavailability of specific language support. this shortcoming is especially pronounced in a social media text, characterized by its brevity and unconventional language, where special characters like emojis have an influential impact on performance. in response to these challenges, we implemented support for arabic and turkish in the emoji package, a functionality previously absent. hate speech detection research has traditionally focused on english @xcite , with a recent shift towards multilingual contexts, including hate speech against immigrants and women @xcite . current efforts are increasingly addressing the challenges in low-resource languages like arabic and turkish through new frameworks, datasets, and shared tasks @xcite @xcite . however, data scarcity and class imbalance in these languages still present considerable challenges, necessitating ongoing research and development. we make the following key contributions and improvements over previous work: (1) a new pooling strategy that significantly improves classification of hate speech in turkish and arabic, contributing to higher macro f1 scores, (2) an evaluation of a cross-lingual data augmentation technique to broaden and enrich the training datasets, enhancing the model's ability to generalize by focusing on language-specific challenges in hate speech contrary to @xcite ) that solely relies on transfer learning from resource-rich to less-resourced language models, and (3) an implementation of a soft-voting ensemble framework to further boost model performance, as evidenced by the achieved macro f1 scores. ## task and dataset description in the hsd-2lang shared task @xcite , we focused on two main tasks: subtask a for hate speech detection in turkish tweets and subtask b for limited arabic tweets. subtask a involves analyzing a dataset of 9, 140 turkish tweets, categorized across topics such as anti-refugee sentiment, the israel-palestine conflict, and anti-greek discourse, with both hateful and non-hateful tweets. subtask b presented the challenge of detecting hate speech in a smaller, imbalanced dataset with 82 "hateful" and 778 "not hateful" arabic tweets, primarily centered on anti-refugee sentiment. ## proposed framework the key elements of our approach to hate speech detection for subtasks a and b include emoji conversion and bidirectional translation between turkish and arabic datasets. we selected convberturk foot_0 (schweter, 2020) and arabertv02-twitter foot_1 (antoun et al., 2020) as our baseline models for turkish and arabic texts, respectively. to tackle the limited and imbalanced data in subtask b, we merged the translated turkish dataset from subtask a with subtask b dataset. we applied a similar strategy for subtask a, incorporating the translated arabic tweets from subtask b. our research introduces an innovative sequence representation technique, going beyond the conventional use of the [cls] token. this method combines the mean and max values from the last hidden layer with the [cls] token, each processed through separate linear layers with tanh activation and dropout. the outputs are then concatenated and fed into a final linear layer for classification as "hateful" or "not hateful". subtask a employed a soft-voting ensemble of five convberturk models in our application. in contrast, subtask b utilized a single arabertv02-twitter model. in the upcoming sections, we provide a comprehensive overview of the methodologies we employed in our project. these include a detailed description of how we preprocessed the data, consolidated the datasets, converted emojis, translated across turkish and arabic, pooled sequence representations, and finally, our training procedures. ## results and discussion in subtask b, the ensemble models exhibited a pronounced sensitivity to data augmentation. the ensemble with [cls] token pooling and no data augmentation achieved the highest test score of 0.7179. conversely, when data augmentation was introduced, the same ensemble approach reduced test performance to 0.5915. similarly, the ensemble model with concatenated pooling reflected this trend, where the non-augmented approach yielded a robust score of 0.7123 on the test set, compared to a lower 0.6027 with data augmentation. for single models in subtask b, the concatenated pooling type with data augmentation resulted in a test score of 0.6000, indicating that the single models were less affected by augmentation. however, this score was still outperformed by the non-augmented ensemble model, highlighting the nuanced impact of augmentation strategies on model performance. the intricate dynamics of the impact of data augmentation are underscored in subtask b, where its application does not enhance model effectiveness. this difference is particularly notable when comparing the performance of single models against ensemble configurations. the test scores suggest that ensemble models, especially with non-augmented concatenated pooling, are robust across both subtasks. the discrepancy in performance between the concat and [cls] methods within ensemble configurations highlights the effectiveness of our pooling strategy. these findings emphasize the need for careful consideration when applying data augmentation, as it may not always be beneficial and depends on the specific task and model architecture. ## conclusion in conclusion, our paper introduces an innovative approach combining data augmentation, pooling strategy, and a soft-voting ensemble framework for effective hate speech detection in turkish and arabic, languages typically underrepresented in computational linguistics. we successfully enriched the training sets with a broader spectrum of examples by leveraging cross-lingual translation through google translator. this approach yielded impressive f1-macro scores of 0.6964 and 0.7123 in turkish and arabic, respectively, demonstrating broad potential in diverse linguistic contexts. the effectiveness of our strategy in low-resource languages opens new avenues for future research, potentially addressing more nuanced aspects of hate speech detection and expanding to other underrepresented languages.