Adopting Ensemble Learning for Cross-lingual Classification of Crisis-related Text On Social Media (2024)

Abstract: Cross-lingual classification poses a significant challenge in Natural Language Processing (NLP), especially when dealing with languages with scarce training data. This paper delves into the adaptation of ensemble learning to address this challenge, specifically for disaster-related social media texts. Initially, we employ Machine Translation to generate a parallel corpus in the target language to mitigate the issue of data scarcity and foster a robust training environment. Following this, we implement the bagging ensemble technique, integrating multiple classifiers into a cohesive model that demonstrates enhanced performance over individual classifiers. Our experimental results reveal significant improvements in adapting models for Arabic, utilising only English training data and markedly outperforming models intended for linguistically similar languages to English, with our ensemble model achieving an accuracy and F1 score of 0.78 when tested on original Arabic data. This research makes a substantial contribution to the field of cross-lingual classification, establishing a new benchmark for enhancing the effectiveness of language transfer in linguistically challenging scenarios.

## introduction cross-lingual transfer learning, which involves transferring models from one language to another or from one task to another, has gained significant attention in the field of natural language processing. this approach is particularly valuable in scenarios where task data in the target language is scarce, posing a limitation to training machine learning models for specific tasks such as classification. in such cases, machine translation has emerged as an effective solution to bridge the language gap, enabling acceptable performance in transfer learning @xcite . furthermore, the use of ensemble techniques in the context of cross-lingual classification has shown promise in achieving better performance and generalisation across multiple languages. ensemble models, by combining multiple base models, can effectively capture diverse aspects of the data and mitigate the impact of language variations, thereby enhancing the robustness of cross-lingual classification systems. the significance of transfer learning lies in its ability to utilise the wealth of data available in high-resource languages to benefit low-resource languages, thus enabling access to various nlp tasks. data augmentation techniques, like machine translation, serve to amplify this effect by artificially expanding the dataset in the target language, which allows for a richer and more diverse linguistic feature set that models can learn from, leading to improved performance and reliability in crosslingual applications. the potential of ensemble learning in addressing the challenges of cross-lingual classification cannot be overstated. by leveraging the strengths of multiple learners, ensemble learning introduces a level of diversity that single models alone cannot achieve, significantly enhancing performance and generalisation capabilities across languages. this diversity is particularly crucial in cross-lingual scenarios, where linguistic and semantic disparities between languages can pose substantial barriers to effective model transfer. ensemble methods can mitigate these barriers by combining predictions from multiple models, thereby reducing the risk of misclassification due to language-specific nuances or translation inaccuracies. moreover, ensemble learning can adaptively focus on difficultto-classify instances, ensuring that the aggregated model is not only more accurate but also more robust to the variability inherent in cross-lingual data. consequently, the application of ensemble learning in cross-lingual classification opens up new avenues for building more resilient and adaptive nlp systems that can better serve the needs of a linguistically diverse world. in addressing the challenge of cross-lingual transfer learning in situations where training data is non-existent, our work introduces an effective approach that significantly improves the efficacy of model transfer to the arabic language, a language markedly different in structure and lexicon from english. a key aspect of our contribution is investigating viable strategies, including the integration of machine translation with a bagging ensemble approach, for classifying disaster-related social media posts in arabic using solely english data. this technique demonstrates potential for broad application across various languages and domains, offering a solution for scenarios with limited data availability in the target language, provided there's access to extensive data in a resource-rich language. ## related work with growing interest in cross-lingual text classification, the challenge persists due to linguistic variations and data scarcity across languages. ensemble classification models, employing multiple weak classifiers and combining their predictions through consistency functions like voting, have been widely used in monolingual tasks but less explored in cross-lingual contexts. techniques such as bagging, adaboost, random forest, and gradient boosting have shown promise across various domains @xcite . among the limited literature on cross-lingual applications, funnelling and its advanced iteration, generalised funnelling (gfun), stand out for incorporating calibrated posterior probabilities and additional feature vectors to enhance classification performance on multilingual datasets. however, they assume the availability of training data in all target languages @xcite . earlier studies, such as those by (kilimci and akyokus, 2018) and @xcite , demonstrate the effectiveness of ensemble models in monolingual settings, suggesting potential for cross-lingual adaptation. beyond ensemble models, research has explored leveraging linguistic similarities through character-based embeddings, joint training, and embedding alignment to address cross-lingual text classification. techniques such as instance-weighting have also been employed to assign larger weights to source instances sharing common features with target samples during training. this approach aims to utilise resource-rich data while accommodating the specifics of the target language @xcite . while the foundation for cross-lingual text classification is robust, marked by a variety of methodologies from ensemble learning to linguistic feature exploitation, challenges remain in data availability, computational demands, and language diversity. our work contributes to this ongoing effort by adapting an ensemble approach designed to enhance the effectiveness of cross-lingual classification, especially for under-resourced languages. this approach seeks to build on the existing body of research, pushing the boundaries of what is achievable in the realm of cross-lingual text classification. ## discussion the presence of data imbalance posed a significant challenge and had a noticeable impact on the model's performance. to address this issue, we made a trade-off between better classification and the potential loss of data. as a mitigation strategy, we introduced an additional layer to calculate class weights, accounting for the class imbalance. the class weight calculation function was designed to dynamically assign weights based on the distribution of instances in each class. if the data is already balanced, the function assigns equal weights to all classes. however, for classes with fewer instances, the function assigns higher weights, effectively prioritising those classes during training. by incorporating this mechanism, we aimed to balance the training process and alleviate the negative impact of data imbalance on the model's performance. this approach is particularly useful in scenarios where the dataset is heavily imbalanced, as it allows the model to focus more on the underrepre- figure 3 : the confusion matrix of the ensemble classification results achieved through majority voting of three classifiers. 0, 1, and 2 correspond to irrelevant, storm, and disease classes, respectively. with a closer look at the individual scores, we notice that the mis-classification of class 0 (i.e., irrelevant) has affected the macro-average f1 resulting in a score of 0.70. however, the model performs relatively well in classifying other classes. the diverse nature of the irrelevant posts makes it challenging for the model to accurately classify them, and they are often mis-classified into other classes, mostly class 1 (storm). figure 3 displays the confusion matrix, which provides a visualisation of the classification performance for each individual class. ## conclusion and future work in this work, we have presented a practical solution for transferring models across languages when confronted with limited or nonexistent training data. our experimentation involved the application of a bagging ensemble technique, with each experiment employing a distinct approach. by combining training data from both english and its arabic translation, and partitioning (bagging) this combined dataset into separate splits, we observed a noteworthy enhancement in prediction performance compared to existing methodologies. looking ahead, our future work will explore alternative ensemble approaches to tackle the same challenge. additionally, extending the scope of our approach to a wider set of languages and tasks holds promising potential for further advancement.