B ias G anda at FIGNEWS 2024 Shared Task: A Quest to Uncover Biased Views in News Coverage (2024)

Abstract: In this study, we aimed to identify biased language in a dataset provided by the FIGNEWS 2024 committee on the Gaza-Israel war. We classified entries into seven categories: Unbiased, Biased against Palestine, Biased against Israel, Biased against Others, Biased against both Palestine and Israel, Unclear, and Not Applicable. Our team reviewed the literature to develop a codebook of terminologies and definitions. By coding each example, we sought to detect language tendencies used by media outlets when reporting on the same event. The primary finding was that most examples were classified as “Biased against Palestine,” as all examined language data used one-sided terms to describe the October 7 event. The least used category was “Not Applicable,” reserved for irrelevant examples or those lacking context. It is recommended to use neutral and balanced language when reporting volatile political news.

## introduction this shared task aimed to detect bias and double standards in the news coverage of the gaza-israel war of 2023-2024, striving for a deeper understanding of these events. it seeks to build a cooperative community and educate the next generation of researchers in the field of natural language processing (nlp) by establishing a comprehensively annotated shared corpus. this paper adheres to the required title case format provided by the fignews24 team and aligns with the shared task's goal of framing the israel-gaza war of 2024. to achieve this objective, extant research was reviewed. for example, @xcite conducted research to detect linguistic features of bias in a database derived from wikipedia. similarly, @xcite developed annotation guidelines to assist annotators in understanding and detecting biased language. additionally, @xcite introduced a framework utilizing masked-language models to detect biased language across different news outlets. they tested their framework on articles about five highly popular and diverse topics from ten different news outlets to analyze the language used to convey news. this project does not only aim to uncover biased views in news coverage, but it also contributes to the ongoing research and development of nlp field methodologies. specifically, @xcite employed quantitative methods, including a content-based approach that directly detected biased language from published sources by focusing primarily on the tone and word choices used when describing the same event across different news outlets. @xcite , in addition, emphasized the importance of word choice and labeling (wcl) in shaping audience perceptions of reported news. he defined wcl as: "when journalists refer to the same semantic concept by using different terms that frame the concept differently and consequently may lead to different assessments by readers, such as the terms 'freedom fighters' and 'terrorists,' or 'gun rights' and 'gun control'" (p. 79). hamborg also explored various methods to reveal media bias to news consumers, aiming to mitigate the biased language impact in future reporting of similar events. he further suggested that most biased news reports originate from and are controlled by large media corporations within specific countries, citing the usa as an example: "in the us, for example, six corporations control 90% of the media" (79). @xcite further provided biasganda at fignews 2024 shared task: a quest to uncover biased views in news coverage al manar al wardi, foot_0 blqees al busaidi, foot_1 malath al-sibani 1 hiba salim mohammed al-siyabi, 1 najma al zidjaly 1 ## team composition and training the team consisted of four female omani annotators, native arabic speakers and secondlanguage english learners, aged 18-24. one annotator is also learning french. all were undergraduate students at sultan qaboos university's department of english and translation, with specializations in translation, literature, and education. supervision was provided by dr. najma al zidjaly, an associate professor in the department of english and translation at sultan qaboos university, who offered guidance throughout the project. for example, she conducted an online meeting to elucidate the definitions of propaganda and bias, highlighting the distinctions between the two terms, and provided instruction on annotation echniques. we engaged in discussions with her via a whatsapp chat group, where she clarified some annotation process aspects, such as creating a codebook and approaching specific phrases and texts. likewise, the organizers of fignews24 offered support in the form of email announcements and correspondence as well as feedback for our draft. moreover, annotators engaged in lengthy conversations to share insights and solve problems. workload distribution was equitable, and we held frequent meetings to strategize and review our progress. we wrote the first draft of our guidelines collaboratively and then proceeded to annotate the required batches of fignews24's database individually. throughout the annotation process, guidelines were edited each time we had a meeting to discuss annotation problems and their solutions. ## task participation and results given that our team originally comprised six members, we initially assigned each three members a specific task (e.g., bias), while the remaining three were assigned to another task (e.g., propaganda). however, due to the departure of two teammates, our team was reduced to only four members, necessitating a revision of our original strategy. thus, we decided to focus solely on completing the bias subtask. throughout the annotation process, we maintained a consistent approach and adhered to a shared guideline to ensure uniformity in annotations, despite the complex nature of the data. effective communication and cooperation among team members were key factors that contributed to the quality of our work. when encountering annotation-related challenges, such as ambiguities, we would refer to the guidelines/codebook and collaboratively develop solutions. the incorporation of examples and comparisons with related work significantly aided our annotation process, thereby enhancing overall team performance. ## discussion two types of bias were identified in our analysis: framing bias and epistemological bias. framing bias, as described by @xcite , occurs when a subject is presented from a specific perspective or "frame." an instance of this bias in our dataset is the following text: "palestinians walk among the rubble, as they inspect houses destroyed in israeli strikes during the conflict, amid the temporary cease-fire between #hamas and #israel, at khan younis refugee camp in the southern #gaza strip on monday." this text is biased against both palestine and israel because the term "conflict" is used to minimize the severity of the damage, whereas palestinian and israeli media often refer to the current situation as a "war." epistemological bias, as defined by @xcite , involves presenting a subject in a way that presupposes propositions to be true or false, subtly implied, asserted, or hedged in the text. in our dataset, several biased texts exhibit this type of bias, such as "watch: on the first day of the israel-hamas truce, palestinian men attended friday prayers inside a mosque wrecked by israeli airstrikes." this text is biased against israel because the word "wrecked" implies an outcome; an unbiased term would be "bombed" or "destroyed." in some instances, bias is detected in a text that is neither against palestine nor israel, but against other parties. in such cases, the text is classified as "biased against others. the second finding, as shown in in contrast, hebrew sources exhibit the lowest percentage of texts categorized as "biased against israel." this outcome can be attributed to the predominance of israeli media outlets among hebrew sources, unlike other languages where media outlets are more diverse and multinational. for instance, arabic news encompasses not only reports from arabic-speaking countries but also those from foreign news agencies, including israeli spokespersons. approximately more than a third of all texts (37.39%) are categorized as unbiased, with arabic texts displaying the highest percentage of unbiased content. this phenomenon can be attributed to our methodology, which considered reported speech and quotes as biased or unbiased based on how the media outlet presented them, rather than the content of the quote itself. for example, a report like "imam hassan chalghoumi strongly criticizes the atrocities committed by isis, hamas, including the killing of children and women and rape, stressing that there is no cure in islam. we must say that this is forbidden and is not permissible according to islamic law! listen to him" is considered unbiased because the media outlet neutrally presents the speaker's views, despite the speaker's stance against palestine. as an analysis of our results from the bias subtask evaluation reveals several key findings. to begin with, we annotated 2,200 data points in total, which shows our active participation, making a fine enough contribution to the bias subtask. despite not having the highest number of data points, our