Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Features (2023)

Abstract: The increasing ubiquity of language technology necessitates a shift towards considering cultural diversity in the machine learning realm, particularly for subjective tasks that rely heavily on cultural nuances, such as Offensive Language Detection (OLD). Current understanding underscores that these tasks are substantially influenced by cultural values, however, a notable gap exists in determining if cultural features can accurately predict the success of cross-cultural transfer learning for such subjective tasks. Addressing this, our study delves into the intersection of cultural features and transfer learning effectiveness. The findings reveal that cultural value surveys indeed possess a predictive power for cross-cultural transfer learning success in OLD tasks, and that it can be further improved using offensive word distance. Based on these results, we advocate for the integration of cultural information into datasets. Additionally, we recommend leveraging data sources rich in cultural information, such as surveys, to enhance cultural adaptability. Our research signifies a step forward in the quest for more inclusive, culturally sensitive language technologies.

## introduction cross-lingual transfer has been successful in various nlp tasks @xcite @xcite particularly in scenarios where low-resource languages lack sufficient training data @xcite . to ensure successful cross-lingual transfer, leveraging prior knowledge becomes crucial in selecting the most suitable transfer language and dataset. most research studies predominantly rely on data-dependent and language-dependent features to predict the optimal language to transfer from @xcite . however, the same language is often spoken in vastly different cultures, and conversely, similar cultures may speak very different languages. in subjective tasks such as offensive language detection (old), relying solely on measures of linguistic proximity and data-specific features for predicting the optimal transfer language/dataset may be insufficient. it is crucial to consider cultural differences. cultural sensitivity and adaptation pose a great challenge in nlp tasks @xcite @xcite . despite the capabilities of language models like chatgpt, their cultural adaptation to culturally diverse human societies is limited @xcite . in an increasingly globalized and ai-driven world, this gap needs to be addressed to create and sustain an effective communication environment @xcite . taking steps towards this direction, we focus on the success of cross-cultural transfer learning in old. offensive language can vary greatly depending on cultural backgrounds. while most multilingual old datasets are constructed by filtering a predefined list of offensive words @xcite @xcite , certain offensive words are culturally specific. for example, old models trained on american cultural contexts may struggle to effectively detect offensive words like "m*adarchod" and "pr*sstitute" in indian texts @xcite . previous studies @xcite @xcite have highlighted the challenges of cross-lingual transfer in old, emphasizing the significance of accurately predicting the optimal language/dataset for transfer before the implementation. given this context, we propose six countrylevel features that quantify the variations in cultural values across different countries. additionally, we introduce a feature that measures the distance of offensive words between languages. we address three research questions regarding crosscultural/cross-lingual transfer learning for old: rq1. can we predict transfer learning success? rq2. which types of features (cultural or offensive word distance) are most informative? rq3. do the same features contribute to transfer learning success in old and other nlp tasks? our experiments show the substantial influence of integrating cultural values in predicting language transfer for a culture-loaded task, old. we also find that exploring offensive word distance in specific domains benefits the prediction of the optimal transfer datasets. notably, we observe that the majority of current datasets lack sufficient cultural information, posing a hindrance to the advancement of cross-cultural nlp research. we adopt harm-check @xcite in appendix a for handling and presenting harmful text in research. optimal transfer language prediction. a major challenge in cross-lingual learning is choosing the optimal transfer language. language proximity is not always the best criterion, since there are other linguistic properties that could lead to better results such as phonological or syntactic distances @xcite . other factors that can influence transfer language selection, such as lexical overlap, have shown mixed findings. some studies report a positive correlation with the cross-lingual model performance @xcite @xcite , while others do not support this finding @xcite @xcite . to automate the process of selecting transfer languages, there have been attempts using a ranking metamodel that predicts the most optimal languages @xcite @xcite @xcite @xcite . these mostly rely on data-dependent and language-dependent features, without taking into account cultural background differences during the optimal transfer language prediction process. ## conclusion we explored the feasibility of predicting transfer learning success, identifying informative features for transfer learning effectiveness, and examined the generalizability of these features across different nlp tasks. we have gained valuable insights into the potential of incorporating cultural sensitivity in language technologies: dimensions from cultural values surveys consistently improve transfer success prediction, and offensive language distance features provide a further improvement. we recommend annotation projects to consider not only the balance between genres and age groups but also encompass cultures from different regions. including documentation with information about the cultural background of the annotators enables better interpretation of annotations considering biases and cultural nuances, and promotes a comprehensive and inclusive approach to research and analysis fostering the transparency, usability, and suitability of the dataset. this study highlights the need for culturally informed datasets and the importance of including cultural factors in cross-lingual transfer learning for old. future work will augment the cross-cultural transfer learning strategy by incorporating cultural features directly into models. further analysis will also conduct a socially situated multi-cultural human evaluation on language model predictions to address subjective nuances that cannot be identified in reference-based evaluation on isolated sentences. finally, we will further explore the correlation between features like offensive word distance, geography, and culture, to gain insights into their relationships and effectiveness in capturing offensive language patterns.