Bias Detection in Media: Traditional Models vs. Transformers in Analyzing Social Media Coverage of the Israeli- G aza Conflict (2025)

Abstract: Bias in news reporting significantly influences public perception, particularly in sensitive and polarized contexts like the Israel-Gaza conflict. Detecting bias in such cases presents unique challenges due to political, cultural, and ideological complexities, often amplifying disparities in reporting. While prior research has addressed media bias and dataset fairness, these approaches inadequately capture the nuanced dynamics of the Israel-Gaza conflict. To address this gap, we propose an NLP-based framework that leverages Nakba narratives as linguistic resources for bias detection in news coverage. Using a multilingual corpus focusing on Arabic texts, we apply rigorous data cleaning, pre-processing, and methods to mitigate imbalanced class distributions that could skew classification outcomes. Our study explores various approaches, including Machine Learning (ML), Deep Learning (DL), Transformer-based architectures, and generative models. The findings demonstrate promising advancements in automating bias detection, and enhancing fairness and accuracy in politically sensitive reporting.

## introduction bias detection in news reporting has become a crucial area of research, especially given its significant impact on public opinion and political polarization. in today's digital age, where information spreads rapidly through online platforms, news outlets are essential in shaping how people perceive events. however, media coverage often reflects underlying ideological or geopolitical biases, which can influence how audiences interpret the news. detecting and understanding these biases is key to promoting ethical journalism and ensuring news reporting remains balanced and impartial. recent advancements in natural language processing (nlp) have introduced powerful new tools for identifying subtle biases in news articles. machine learning models, in particular, have made significant strides in uncovering these hidden biases. yet, these methods face unique challenges regarding sensitive and complex topics like the israel-gaza conflict. the language used in such coverage is heavily influenced by historical, cultural, and political factors, making it difficult for existing models to detect biases effectively. a more nuanced approach is needed to tackle this-one that goes beyond general bias detection methods and considers the conflict's specificities. while much of the existing research on bias detection in news has focused on more general forms of bias, such as political slant or ideological bias, the israel-gaza conflict presents a different set of challenges. many studies have looked at these issues in languages like english, but often neglect the complexities of covering sensitive geopolitical topics. additionally, the lack of annotated datasets focused on this conflict makes it even harder to develop effective bias detection tools. this paper addresses these gaps by creating a specialized nlp framework to detect and annotate biased coverage related to the israel-gaza conflict. our work is based on the foundational project "bi-asfignews" @xcite , which collected data on the israeli-gaza conflict. "biasfignews" is a comprehensive multilingual corpus of 12,000 facebook posts annotated for bias and propaganda. the corpus includes posts in arabic, hebrew, english, french, and hindi, covering various events during the israeli war on gaza from october 7, 2023, to january 31, 2024. our main contributions include: by tackling the unique challenges of bias detection in conflict reporting, we hope to contribute to the development of more ethical journalism and improve the quality of media coverage in sensitive areas. to achieve our goals, we use a comprehensive approach that includes data collection, cleaning, and pre-processing, followed by model development using various machine learning algorithms. our first step is to create a multilingual annotated dataset scraped from social media platforms, focusing on news posts about the israel-gaza conflict. after addressing issues such as data imbalance, we apply advanced nlp techniques -such as transformer and sequential models like t5 and bi-lstm -and explore a variety of machine learning algorithms, including svm, random forest, and xgboost. through experiments, we will benchmark these models and assess their performance in detecting bias, with a particular focus on how well they generalize across different languages and types of bias. the rest of the paper is organized as follows: section 2 will cover the related work, section 3 will present our proposed materials & methods, section 4 will present the results & discussion, section 5 will conclude the proposed work and discuss the recommendations of our future work and finally, section 6 will represent the faced limitations in our work. ## related work this section reviews prominent studies on bias detection in nlp, focusing on their methodologies, challenges, and limitations. while existing work has explored media and language bias, few studies address the specific complexities of geopolitical conflicts like the israel-gaza conflict, especially in multilingual and culturally nuanced contexts. evans et al. @xcite investigate how human biases influence nlp models, particularly in hate speech detection. their work utilizes datasets to train the emotion-transformer model based on distilbert. while combining datasets improves bias detection for specific categories, they highlight persistent challenges in addressing imbalances in multi-target bias tasks. rodrigo-gines et al. @xcite conduct a systematic review categorizing types of media bias and distinguishing it from misinformation and disinformation. they emphasize the limitations of existing datasets and methods, calling for improved detection techniques to ensure accuracy and reliability in bias detection. khattak et al. @xcite explore bias in customer interaction datasets, focusing on ethical data handling and fairness. they underscore the importance of mitigating bias during data preparation and advocate for enhanced methods to reduce biases in training datasets while ensuring compliance with gdpr. despite these advancements, existing studies lack a focus on media bias in the unique context of geopolitical conflicts, such as the israel-gaza conflict. the limited exploration of multilingual corpora, particularly in arabic, and challenges with imbalanced data emphasize the need for specialized frameworks tailored to such sensitive and polarized scenarios. ## materials and methods this section provides a detailed overview of the dataset description and the proposed model pipeline, including data cleaning and preprocessing, handling imbalanced classes, the embedding models, and the classification models used. ## results & discussion the models exhibit varying performance, with the accuracy and f1-scores for each summarized in the table shows that in machine learning algorithms, the random forest classifier has the highest performance with an accuracy of 93%, an f1score of 93.23%, a precision of 93%, and a recall figure 4: f1-score figure 5: precision figure 6: recall figure 7: performance metrics visualization of 93%. xgboost followed closely with an accuracy of 92%, an f1-score of 93%, a precision of 92%, and a recall of 92%, indicating strong performances along all metrics. similarly, catboostclassifier achieved good performance with an accuracy of 88%, an f1-score of 88%, a precision of 88%, and a recall of 88where for the deep learning models, the lstm model achieved an accuracy of 84%, an f1-score of 84%, a precision of 86%, and a recall of 82%. bi-lstm and bi-gru with attention achieved lower results: bi-lstm with an accuracy of 38.4%, an f1-score of 33%, a precision of 33%, and a recall of 33%, and bi-gru with attention with an accuracy of 14.18%, an f1-score of 10.2%, a precision of 8.9%, and a recall of 14.18%. on the other hand, the generative models t5-small also had a low accuracy of 15.65%, an f1-score of 10.10%, a precision of 7.62%, and a recall of 15.65%. the generative model silma, based on prompt engineering, performed with an accuracy of 20%, an f1-score of 18%, a precision of 15.32%, and a recall of 20%. among the traditional models, svm achieved an accuracy of 81%, an f1-score of 81%, a precision of 81%, and a recall of 81%, which were reasonable but not high as compared to the ensemble methods. logistic regression also achieved a similar performance with an accuracy of 79%, an f1-score of 79%, a precision of 79%, and a recall of 79%. gaussian naive bayes showed an accuracy of 68%, an f1-score of 69%, a precision of 71%, and a recall of 68%, while decision tree had a moderate performance with an accuracy of 41%, an f1-score of 42%, a precision of 49%, and a recall of 41%. arabert, a pre-trained language model specific to the arabic language, achieved an accuracy of 58%, an f1-score of 56%, a precision of 54%, and a recall of 58%. while its performance outperformed the decision tree model and some of the deep learning models. it is shown that generative and transformer-based models such as silma and t5 performed worse than traditional machine learning (ml) models. this is due to many reasons, including that traditional ml models often benefit from feature engineering, where manually selecting and transforming relevant features can lead to better performance. additionally, traditional ml models are designed for specialized tasks like classification, making them more effective for these specific problems compared to generative models optimized for generating new data. moreover, traditional ml models have built-in inductive biases that make them well-suited for certain tasks, such as random forest being particularly adept at constructing multiple decision trees during training and outputting the mode of the classes, whereas transformers may require more data and computational resources to achieve similar results. figures 3 , 4 ,5,6 illustrate the results of the machine learning, deep learning, generative and transformer-based models for this work. ## conclusion in this paper, we deal with the critical task of detecting bias in news reporting on the conflict between israel and gaza. applying an arabic corpus of texts, advanced preprocessing methods, and several machine learning models, we have arrived at a robust framework for the detection of bias applicable to the nuanced and politically charged context of news reports on conflict situations. among the tried methods, ensemble methods such as random forest and xgboost showed better performance; thus, they are more suitable for this challenging classification. our results indicate that although there are inherent difficulties arising from data imbalance, language-specific challenges, and subtle bias indicators, a good combination of data augmentation strategies such as borderline-smote along with state-of-the-art machine learning techniques can improve the detection of bias considerably. it is part of the larger aim of ensuring ethical journalism and offers a scalable methodology for media coverage analysis in sensitive geopolitical situations.