Ceasefire at FIGNEWS 2024 Shared Task: Automated Detection and Annotation of Media Bias Using Large Language Models (2024)

Abstract: In this paper, we present our approach for FIGNEWS Subtask 1, which focuses on detecting bias in news media narratives about the Israel war on Gaza. We used a Large Language Model (LLM) and prompt engineering, using GPT-3.5 Turbo API, to create a model that automatically flags biased news media content with 99% accuracy. This approach provides Natural Language Processing (NLP) researchers with a robust and effective solution for automating bias detection in news media narratives using supervised learning algorithms. Additionally, this paper provides a detailed analysis of the labeled content, offering valuable insights into media bias in conflict reporting. Our work advances automated content analysis and enhances understanding of media bias.

heading: 1 Introduction
text: Social media platforms and digital newspapers/articles are seen as a crucial source of information that nearly substituted conventional news media platforms on TV, radio, etc. Despite widely acknowledged bias in news media sources, many individuals perceive information from such online platforms as reliable sources for major or current events/conflicts (Spinde, 2021). People trust social media news as they present updates almost instantaneously, having a big impact on society and public opinion. In this paper, we introduce the FIGNEWS 2024 shared task (Framing the Israel War on Gaza) Zaghouani et al. (2024), which aims at exploring news media narratives and biases on the Israel war on Gaza (2023 to 2024).
The goal of this task is to annotate and label a corpus that contains content from news headlines, articles, and social media posts into different bias categories. The labeling is based on a set of determined bias categories. This task intends to equip future Natural Language Processing (NLP) researchers with the skills to be able to automate
the annotation process by using supervised learning algorithms on media narratives.
The Israel war on Gaza has brought the world’s attention to media posts and news due to its nature of providing instantaneous, global updates. Recent studies have emphasized the significance of addressing biases in news and social media through automation. For instance, Lim (2020) talks about the difficulties in identifying bias in news stories by referring to published datasets. Another study focuses on the development of a comprehensive methodology that automatically annotates news articles based on their bias (Raza, 2024). The study uses human-written guidelines for generating prompts, with algorithms and LLM APIs for processing and annotating. The purpose of this was to annotate a corpus and train the model to enhance the accuracy of bias detection.
Furthermore, Spinde (2021) introduced a feature-based method for identifying terms in news articles that cause bias. In his paper, Törnberg (2024) proposes a set of guidelines to enhance the use of LLMs for text annotation using ChatGPT as the main prompt construction basis. Moreover, in another study, Hou (2024) explored the use of a Generative Pre-trained Transformer in automating the social media annotation process by fine-tuning a model with expert-labeled guidelines to train the model. This model was developed to create a zerogap potential between human and automated annotation.
In this paper, we we developed a robust framework and utilize OpenAI’s GPT-3.5 Turbo API to automate the process of detecting and analyzing media bias concerning the Israel war on Gaza. By annotating sentences based on our written prompt with the LLM model, we aim to streamline and enhance the accuracy of media bias detection. Additionally, we present detailed guidelines developed to ensure consistency during both manual and automated annotation processes.
590

heading: 2 Annotation Methodology and Examples

heading: 2.1 Development of Annotation Guidelines
text: The news content provided in the FIGNEWS 2024 shared task (Zaghouani et al., 2024) presents the Israel war on Gaza through the lens of diverse news outlets across the globe with their different perspectives and languages; many presenting biases. With the aim of identifying these biases, we established annotation guidelines to ensure consistency and quality of the labels within our team.
To develop the guidelines, our team first defined the task and its scope clearly. This involved understanding the provided seven categories of the bias labels and articulating the objective of the task. We also explored existing guidelines and frameworks in the literature (Zhou (2022); Pyatkin (2023). Below are examples of two of our seven labels, along with their definitions and examples. The remaining guidelines are detailed in the Appendix.
• Unbiased: A statement is unbiased when it presents information objectively, without favoring any particular viewpoint or agenda. This includes information with balanced representation, factual accuracy, a neutral tone, and the absence of personal bias.
Example: "Israel Prime Minister Benjamin Netanyahu’s office said Sunday its security cabinet had approved the state of war he vowed a day earlier."
• Biased against Palestine: A statement is biased against Palestine when it shows prejudice or favoritism against Palestinian interests, perspectives, or rights.
Example: "Israel is expected to launch a ground invasion into Gaza after Hamas terror attacks."
Once the categories were defined, we went through an iterative process of reviewing and revising the definitions by running a pilot test. This pilot test involved applying the guidelines to 100 statements manually with the aim of improving clarity, addressing ambiguities, and ensuring comprehensive coverage of labeling scenarios. Our understanding of the terminology and timeline specific to this war and its history was a prerequisite to the labeling process.
We also completed the inter-annotator agreement following these guidelines to ensure a consistent and mutual understanding of the definitions.

heading: 2.2 Data Annotation Process
text: The annotation process employed supervised learning using a large language model (LLM) for label classification tasks. Initially, 100 statements were manually labeled with detailed comments to serve as in-context learning examples. The GPT-3.5 Turbo API from OpenAI (Ye, 2023) was utilized for this purpose. The prompt engineering technique used involved carefully crafting a prompt that included explicit definitions and examples to guide the model in accurate labeling statements.

heading: 2.2.1 Prompt Engineering Technique
text: The prompt provided to the model included a clear explanation of the task, specifying the different bias categories and providing definitions for terms commonly found in the statements (see Appendix). During initial experiments, the model struggled with context-specific terms like Hamas, IDF, or Biden, often mislabeling them. To mitigate this, we included explicit definitions for terms such as Hamas and Netanyahu in the prompt, enhancing the model’s understanding and bias identification. Examples for each bias category were included to guide the model in classifying similar statements. The structured prompt ensured the model received consistent instructions, aiding in accurate and reliable label classification.

heading: 2.2.2 Validation and Testing
text: To ensure the accuracy and consistency of the manual annotations, an inter-annotator agreement (IAA) was conducted. Outputs from the model were reviewed through random checks to ensure accuracy, with ongoing adjustments and context provision until satisfactory results were achieved. Initial testing revealed ambiguities due to a lack of context understanding. Enhancements included detailed context in prompts and attention to spelling to avoid common errors (e.g., confusing "Hamas" with "Hummus"). The model initially misclassified ’unbiased’ labels as biased against Israel or Palestine; thus, more examples and definitions were provided until accurate labeling was achieved. The model’s prompt was iteratively refined, providing additional context and correcting errors until satisfactory results were achieved. The model’s performance was validated using 40% of the manually labeled data. Ambiguities and errors identified during initial testing were addressed by enhancing the prompt with more detailed context and examples. Random checks of annotations confirmed 99% ac-
curacy and consistency.

heading: 2.2.3 Ethical Considerations
text: In employing ChatGPT, it was observed that the model initially exhibited a bias towards Israel. To counter this, the model was instructed to label texts based on the provided context alone. The temperature of the model was set to 0.4 to reduce the likelihood of hallucinations. All manually labeled data was checked for accuracy and consistency, with errors followed up through discussions to reach mutual agreement on the labels. The team ensured the model’s outputs were unbiased and accurate, using a balanced prompt and conducting frequent review sessions to maintain consistency and ethical soundness in all annotations.

heading: 2.3 Inter-Annotator Agreement (IAA) Analysis
text: Inter-annotator agreement (IAA) provides insights into the degree of consensus among the annotators. Each annotator used the defined guidelines, as mentioned in 2.1, to label their assigned statements. Each member took part in defining the guidelines, as this step was essential in providing the annotators with alignments to have an accurate representation of IAA.
The IAA score was calculated using the Fleiss Kappa, and the score shown to be 0.256 which indicates that the degree of agreement across the three members was at a reasonable level of agreement. It also shows that the team had some consistency, and since the complexity of the news headlines and social media posts can be subjective.

heading: 3 Team Composition and Training
text: Our team consists of three female members, all currently pursuing their master’s degree at Hamad Bin Khalifa University Qatar from the College of Science and Engineering. While two of the members hail from South Asia and speak Hindi/Urdu, the third member is from a GCC country and speaks Arabic as her native language. All the members fall within the age range of 18-24. We all used the English translations of the statements to label them, except for one member who used Arabic to analyze the Arabic statements. In terms of the contribution, all the members contributed equally to the labeling (100 main and 200 IAA). While some of the manually labeled statements we labeled together as a team, others were equally split among us and checked by the other members for consistency and
accuracy. Our study involves two types of training. The first involves training the team members, and the second involves training the model. Both were done as a team of three from the beginning of this project. All team members trained for the labeling together by establishing guidelines together based on the literature and further developing the guidelines based on the application of the guidelines on the statements and revising. The model was trained with the statements we labeled as a team and manually checked for accuracy. Any errors were then reflected in the prompt with additional explanations.

heading: 4 Task Participation and Results
text: The performance of the team was improved in various stages by having a well-organised plan and strategy, which was initialized by previous studies and by using the methodologies that was initiated in the studies. As stated previously, we developed the prompt using LLMs in conjunction with GPT3.5. This was done in order to ensure that the annotation process is automated, which led to a better accuracy. The prompt consisted of specific examples for each label (biases categories), which helped enhanced the output. The results of our model are depicted in Table 1. After reviewing the results, we enhanced the model by reorganizing and adding additional examples to reduce ambiguity. This step demonstrated that the results will be improved as the prompt will include valuable information.

heading: 46.3 29.3 61.2
text: This work highlighted the complex process of recognizing bias in social media postings with a diverse language used and cultural aspect. Moreover, our methodology used a similar approach to this this study Raza (2024) as we defined guidelines and generated a prompt to be trained to the LLM model. However, in this study the the authors used different ways to evaluate the annotation’s quality, including expert assessment, consensus rating, and tagging, however,for our approach we did not use this kind of quality assurance.

heading: 5 Discussion
text: Analyzing the labels gives us insights into how different media represent the Israel war on Gaza. Based on the 1800 statements labeled from the first two batches, we deduce the following:
The summary graph in Figure 1 indicates that around 40% of the analyzed statements were biased against Palestine, while 19% were biased against Israel, and only 29% were unbiased. This indicates a disproportionate representation of the war in media, which is problematic as it not only distorts the public perception of the war but is also unethical to provide unbiased reporting. This underscores the critical need for unbiased journalism.
As seen from Figure 1, the topmost labels were Biased against Palestine, Biased against Israel, and Unbiased. The word cloud of the top 20 words were analyzed in these three labels to gain insights into the prominent themes and biases. In the word cloud representing bias against Palestine as shown in Figure 2, the term "Hamas" is the most prominent. This prevalence indicates that most of the biases against Palestine were linked to Hamas, suggesting that the portrayal of Hamas significantly contributes to the negative bias towards Palestine. Words such as "terrorist" and "attack" frequently co-occur with "Hamas," reinforcing this negative perception.
In contrast, the word cloud for bias against Israel features terms such as "Israel," "Gaza," and "Palestinian." Figure 3 depicts the prominence of "Gaza" and "Palestinian" alongside "Israel" suggests that the negative bias against Israel often revolves around its actions and policies towards Gaza and the Palestinian people. Words like "war," "attack," and "killed" also appear frequently, indicating that the discourse often highlights violent interactions and casualties associated with Israel.
For the unbiased category, Figure 4 shows a balanced representation of terms related to both sides of the conflict, such as "Israel," "Hamas," "Gaza," "war," and "people." The terms "live," "world," and "us" suggest a broader, more neutral discussion that encompasses various perspectives without showing explicit favor or prejudice towards either party.The portrayal of Hamas and Israel’s actions in Gaza significantly drives bias against Palestine, while the unbiased category provides a comprehensive view that considers multiple aspects of the conflict.

heading: 6 Conclusion
text: In this paper, we presented our methodology for automating the annotation/labelling of the bias categories to the dataset provided by the shared task. The team automated the annotation process using the prompt construction - LLMs in conjunction with ChatGPT-3.5 by providing clear guidelines and examples to ensure consistency in the labelling of bias categories across different sets of sentences from several news headlines and media platforms. The methodology for automating the annotation process had limitations, including the model’s time-consuming nature (1 hour for 500 statements), which limited the number of examples
we could add. We reduced the number of examples to 10 and also incurred the cost of running the model due to its closed-source nature. In future work, we aim to improve the IAA and improve the prompt to get a better accuracy.

heading: A Appendix
text: A.0.1 Prompt You are an expert in labeling. You will be given a set of statement in English which you would have to label.
Help classify statements into one of the following bias categories, using the given labels:
Unbiased: A statement is unbiased when it presents information objectively, without favoring any particular viewpoint or agenda. This could be information with a balanced representation, factual accuracy, neutral tone and absence of personal bias.
Biased against Palestine: A statement is biased against Palestine when it shows prejudice or favoritism against Palestinian interests, perspectives, or rights.
Biased against Israel: A statement is biased against Palestine when it displays prejudice or favoritism against Israeli interests, perspectives, or rights.
Biased against both Palestine and Israel: A statement is biased against Palestine and Israel when it demonstrates prejudice or favoritism against both Palestinian and Israeli interests, perspectives, or rights.
Biased against others: A statement is biased against other when it exhibits prejudice or favoritism against parties or perspectives not directly related to Palestine or Israel but in the context of the Palestine or Israel conflict.
Unclear: A statement is unclear in terms of bias when it does not clearly exhibit bias for or against any particular party or perspective in the context of the Palestine or Israel conflict. Not Applicable: A bias is not applicable to a statement that does not pertain to the Palestinian and Israeli conflict situation.
For your context, below are some definitions mentioned between ***
*** Hamas: Hamas stands for the Islamic Resistance Movement and in Arabic means “zeal”.So if there is any mention of hamas it is basically talking about Palestine and people of Palestine.
Israel-Hamas : Any statement that mentions Israel-Hamas basically is a balanced view and is just referring to the war which means it doesn’t indicate any bias on its own and is just talking about the war in general terms from both sides.
Benjamin Netanyahu : He is an Israeli politi-
cian who has been serving as the prime minister of Israel since 2022, having previously held the office from 1996 to 1999 and again from 2009 to 2021. So if there is any mention of Benjamin Netanyahu it is basically talking about Israel as Benjamin Netanyahu is the prime minister of Israel.
Gaza: The Gaza Strip , or simply Gaza, is a polity and the smaller of the two Palestinian territories (the other being the West Bank) and is a part of palestine. So if there is any mention of gaza it is basically talking about Palestine.
IDF: Israel Defense Forces (IDF), armed forces of Israel, comprising the Israeli army, navy, and air force.So if there is any mention of Israel Defense Forces or IDF it is basically talking about Israel.
*** Learn from few examples provided between ### to classify statements into the following categories based on bias: ’Bias against Palestine’, ’Bias against Israel’, ’Bias against both Palestine and Israel’, ’Bias against others’, ’Unbiased’,’Not Applicable’, and ’Unclear’. Each example below belongs to one of these categories, providing guidance on how to classify similar statements.
### Bias Types: 1. Biased against Palestine: Guideline: Statements that emphasize negative actions or portray Palestine in a negative light, suggesting bias against Palestine.
Examples: —————————————- English MT: Israel is expected to launch a ground invasion into Gaza after Hamas terror attacks.
Comments: Israel is "launch a ground invasion" whereas Hamas gives a "terror attack." The difference in the use of words to describe both attacks shows bias against Palestine.
—————————————- English MT: US President Joe Biden says both Hamas and Russia’s Vladimir Putin “want to completely annihilate a neighboring democracy.” https://trib.al/UAfb8eA: https://www.bl oomberg.com/news/articles/2023-10-20/bi den-casts-russia-hamas-as-parallel-thr eats-to-democracy. Comments: Saying Hamas wants to "completely annihilate a neighboring democracy" shows bias against Palestine.
—————————————- English MT: Israel-Palestine War LIVE: US to
send military ships, aircraft closer to Israel The United States will send multiple military ships and aircraft closer to Israel as a show of support, Defense Secretary Lloyd Austin said, with Washington believing Hamas’ latest attack may have been motivated to disrupt a potential normalising of Israel-Saudi Arabia ties. Israel Palestine IsraelPalestine.
Comments: The US shows support to Israel only by sending "ships and aircraft closer to Israel." Also, believing "Hamas’ latest attack may have been motivated to disrupt a potential normalising of Israel-Saudi Arabia ties" provides a blame with no evidence, showing a bias toward Israel and spreading propaganda.
—————————————- English MT: LIVE: President Joe Biden Responds to Hamas Terrorist Attack on Israel... Comments: Clearly blames Palestine with the statement "Hamas Terrorist Attack on Israel." —————————————- English MT: They Said We Were Hamas ’They said, “We’ll kill you, you’re Hamas." These Palestinian men, working in a factory in the West Bank, were arrested by Israeli forces, detained for 23 days, and subjected to torture and interrogation. Amnesty International reports a surge in Israeli authorities’ use of administrative detention of Palestinians in the occupied West Bank since October 7th.
Comments: Shows that Israel imprisoned people and blamed them for being Hamas even if it wasn’t true. This is shown by "reports a surge in Israeli authorities’ use of administrative detention of Palestinians."
—————————————- English MT: “They beat and assaulted all the girls.” Palestinian teenager Zeina Abdo was jailed for posting a Palestinian flag on social media. She says Israel retaliated against Palestinian detainees after Hamas’ attack on October 7.
Comments: A statement from Zeina Abdo, who was jailed for posting a Palestinian flag on social media. Stating that she was jailed for something so minor shows bias against Palestine.
—————————————- English MT: A Controversial Video Goes Viral on Social Media: Women Laugh and Take Selfies in the Background of a Kidnapping, Displaying Disrespectful Gestures. Israel HamasWar IsraelUnderAttack Gaza Palestinians IsraelPalestineWar Gaza IsraelFightsTerror IndiaStandWithIsrael tela-
viv Hezbollah FPJ. Comments: The tweet itself doesn’t provide much context but the hashtags like IsraelUnderAttack, IsraelFightsTerror, IndiaStandWithIsrael show bias towards Israel. The lack of context and support for Israel is propaganda that makes the reader believe it is the Israelis that were being kidnapped.
—————————————- English MT: The head of the FBI says the bureau is "working around the clock" to stop potential terrorist attacks in the U.S. by individuals or groups inspired by Hamas’ deadly assault on Israel.
Comments: The phrase "Hamas’ deadly assault on Israel" shows a bias against Palestine and propaganda.
—————————————- English MT: ROTTEN’: Rob Schmitt tackles the latest round of shocking visuals to stem from modern liberal, pro-crime ideology which has allowed Hamas supporters to deface and endanger a Biden-run America.
Comments: The phrase "pro-crime ideology which has allowed Hamas supporters to deface and endanger a Biden-run America" attacks Palestine, indicating propaganda and bias against them.
—————————————- English MT: DEEPLY CONCERNING’: Congressman Jared Moskowitz says he "continues to be disappointed" in fellow Democrats who are trusting Hamas’ information about a blast in the parking lot of a Gaza hospital over the US intelligence community’s assessment.
Comments: Clear bias against Palestine with the use of the phrase "continues to be disappointed" as it doesn’t want people to trust Hamas’ sources and only trust US intelligence sources, and propaganda because of ’DEEPLY CONCERNING’ which creates a very biased opinion against Hamas and also lies regarding their source.
—————————————- 2. Biased against Israel: Guideline: Statements that emphasize negative actions or portray Israel in a negative light, suggesting bias against Israel.
Examples: —————————————-
English MT: CNN’s Dana Bash pressed Israeli Prime Minister Benjamin Netanyahu on the hospital situation in Gaza, efforts to get hostages released and whether he takes responsibility for failing to prevent the surprise October 7 attack by
Hamas on Israel. https://cnn.it/3FTuEWn: https://www.cnn.com/middleeast/live-new s/israel-hamas-war-gaza-news-11-12-23/ index.html. Comments: One-sided tone with the comment "whether he takes responsibility for failing to prevent the surprise October 7 attack".
—————————————- English MT: WIN AGAINST TERROR: Strikes in Gaza killed dozens of Hamas targets throughout the night, Israel says. https://trib.al/LNycYK D: https://www.foxnews.com/live-news/oc tober-22-israel-hamas-war. Comments: The phrase "WIN AGAINST TERROR" indicates support for Israel targeting Hamas.
—————————————- English MT: Israeli jets launched massive strikes on targets in Gaza, and Prime Minister Benjamin Netanyahu said Israel is “at war” after Hamas attacked Israel by air, sea and land on Saturday. The militant group told Al Jazeera its military operation was in response to decades of Israeli atrocities against Palestinians. IsraelGaza OperationIronSwords MiddleEastConflict cgtnamerica.
Comments: The tweet shows a bias towards palestine as "its military operation was in response to decades of Israeli atrocities against Palestinians." That’s why Prime Minister Benjamin Netanyahu statement on how Israel is “at war” after Hamas attack is in quotations, Isreal was already attacking Palestine way before.
—————————————- English MT: Iran’s president called on Muslim countries to sever ties with Israel as the latest Israel-Hamas war continued into its 11th day. Watch! Iran president MuslimCountries Israel Gaza Hamas Palestine war bombing civilians dead blast Airstrike bomb devestation killed military armed hostage WorldNews.
Comments: Iran’ president enourages other countries to cut ties with israel, showing a bias against Israel.
—————————————- English MT: In an interview with Mark Regev, Senior Adviser to Israeli PM Benjamin Netanyahu, Mehdi Hasan asks hard hitting questions, pointing out the various inaccuracies and disinformation promulgated by IDF on Twitter. etribune Israel Hamas Palestine Calendar IDF Tweets MehdiHasan MarkRegev.
Comments: The mention of hard-hitting questions and pointing out inaccuracies implies a crit-
ical stance towards the IDF’s communication tactics.
—————————————- English MT: Is Israel really rounding up ‘Hamas fighters’ or is it just humiliating Palestinian civilian men and boys and broadcasting it to the world to dehumanise them? — #AJOpinion by Marc Owen Jones aje.io/qkwepo: https://www.aljazeer a.com/opinions/2023/12/10/fact-or-ficti on-is-israel-really-rounding-up-hamas. Comments: This statement suggests that Israel is "humiliating Palestinian civilian men and boys and broadcasting it to the world to dehumanise them," showing a bias. It also shows how Israel is spreading propaganda by "broadcasting." This sentence, itself, is making us question Israel’s motives and creates a propaganda.
—————————————- English MT: Israeli warplanes pounded the Gaza Strip on Friday after talks to extend a week-old truce with Palestinian militant group Hamas collapsed, wounding and killing more civilians and forcing hundreds to flee their homes. Al Awda hospital is currently one of the last operable hospitals in northern Gaza. In this episode, a CGTN stringer speaks to Dr Mohamed Obaid. He said the pressure has mounted as patients keep coming here since other hospitals are not functioning anymore. Our most important hurdle is "the lack of medical staff, and there’s also a severe lack in oxygen for operations," he said. Gaza IsraelHamas.
Comments: The last operable hospitals in northern Gaza and hundreds fleeing from their homes is caused by the Isreali warplanes, showing a bias. This does not show propaganda but acknowledge the situation with a normal tone.
—————————————- English MT: UN Secretary-General Antonio Guterres says Hamas’ October 7 attack on Israel ’did not happen in a vacuum’ as the Palestinian people are subjected to ’suffocating occupation.’
Comments: The Secretary-General states how Isreal has created a suffocating occupation and the Hamas attack was a response to that, therefore showing a bias towards Palestine.
—————————————- English MT: Russia’s FM Sergei Lavrov says it is unacceptable for Israel to use Hamas’ offensive on October 7 as justification for the collective punishment of Palestinians in Gaza. LIVE updates: https://aje.io/hetbk: https: //www.aljazeera.com/news/liveblog/2023
/12/10/israel-hamas-war-live-no-safe-p lace-in-gaza-as-severe-hunger-spreads/. Comments: this statement belittles Hamas’s attacks and portrays Israel as the one more at crime and at fault with the quoted phrase ’collective punishment’.
—————————————- 3. Biased against both Palestine and Israel: Guideline: Criticizes both Israel and Palestine for their actions. Portrays both sides as being equally at fault or aggressive. Avoids favoring one side over the other.
Examples: —————————————-
English MT: WARNING: GRAPHIC CONTENT Israeli forces encircled Gaza City, the Gaza Strip’s main city, in their assault on Hamas, but the Islamist militant group resisted their drive with hitand-run attacks from underground tunnels https: //reut.rs/3FLF0HH: https://www.reuters. com/world/middle-east/gaza-says-israels -strikes-refugee-camp-kill-more-than-1 95-people-2023-11-02/. Comments: Shoes bias towards both the groups as Israel assaulted Hamas and Hamas hit-and-run attacks. This does not show any propaganda.
—————————————- English MT: Kremer is likely the first Israeli to play in an MLB playoff game, a distinction he will earn against the backdrop of Israel’s ongoing war against Hamas.
Comments: This statement puts an individual’s achievement against a negative connotation that involves both countries at war. It also evokes emotions in the reader’s mind, acting as a propaganda.
—————————————- English MT: College campuses across the United States have been at a fever pitch in the past week over statements from students, professors and administrators related to Hamas’ attack on Israel and the ensuing war in Gaza.
Comments: Talks about both, bias against palestine based on phrase ’the ensuing war in Gaza’ and bias against israel ’Hamas’ attack on Israel’, not propaganda because it just mentions the situation in us right now.
—————————————- 4.Biased against others: Guideline:Criticizes entities other than Israel or Palestine. Shifts blame to third parties for the conflict.
Examples:
—————————————- English MT: The commander of Iran’s Islamic Revolutionary Guard Corps Quds Force visited Lebanon on Nov 1 to ‘coordinate with Hezbollah’ amid the ongoing Israel-Hamas war in Gaza, as per a report by al-Jadeed. Commander Esmail Qaani’s ‘visit’ comes amid warnings by Iran and Lebanonbased Hezbollah of opening a second front against Israel if it continues its ground invasion of Gaza. Is Tehran acting on its warning of ‘reaction’ from the Axis of Resistance in the wake of an Israeli ground offensive in Gaza which started last week? Watch for details. israelhamaswar israelhamaswarupdate hezbollah iran axisofresistance gaza groundoffensive israel hamas war netanyahu israelgazawar middleeast geopolitics worldnews israelpalestineconflict.
Comments: Use of quotation creates a sense of suspicion, creating propaganda and showing bias against lebanon and iran.
—————————————- English MT: Aid Trucks Move to Gaza via Rafah Border | Face to Face rizwanahmed gaza israel hamas Face to Face is an endeavor by Dr. Syed Rizwan Ahmed Adv and Snap Wave Production to come up with Freelance journalism news analysis Youtube channel for impartial analysis on happening around Nation . Dr.Syed Rizwan Ahmed is a socioreligiouslegalpolitical_commentator known for his candid,frank,bold,nationalist views. For Enquiry :- Facetofacelucknow@gmail.com.
Comments: It presents Dr. Syed Rizwan Ahmed as a commentator known for his "candid, frank, bold, nationalist views," indicating a specific perspective that may not necessarily align with impartial analysis.
—————————————- English MT: LIVE | On WorldatWar with Mohammed Saleh: + Hamas strikes Israel with 7,500 rockets from Land, Air Sea + Russia developed a nuclear powered missile with unlimited range + More than 116 people killed in a drone strike on a military college in Syria + Mali braces for a two-frontal assault by Islamists Tuareg rebels.
Comments: Does not give information that can be verified.
—————————————- 5.Not Applicable: Guideline: Statements that are irrelevant to the conflict or offer vague, generalized statements that don’t relate to specific biases.
Examples:
—————————————- English MT: "When freedom of speech becomes freedom of hate it becomes a whole different thing."
Comments: no context related to any bias against palestine or israel or other groups , general vague statement.
—————————————- English MT: After years of the Democratic Party idolizing Congresswoman Rashida Tlaib as one of the party’s "Squad" rockstars, now, a few Democrats are scrambling to dissociate with the purveyor of disinformation. Oddly, Meta, YouTube, and the rest of extreme-Left Big Tech have yet to silence and deplatform Tlaib for spreading disinformation.
Comments: no context related to any bias against palestine or israel or other groups , general vague statement.
—————————————- 6. Unbiased: Guideline: Statements that objectively describe events without showing explicit favor or prejudice towards any party in the Israel-Palestine conflict.
Examples: —————————————-
English MT: Yemen’s Houthis have waded into the Israel-Hamas war raging more than 1,000 miles from their seat of power in Sanaa, declaring they fired drones and missiles at Israel in attacks that highlight the regional risks of the conflict https: //reut.rs/40jSi7A: https://www.reuters. com/world/middle-east/yemens-houthis-e nter-mideast-fray-hardening-spillover-f ears-2023-10-31/. Comments: It is just stating something that happened (news).
—————————————- English MT: Isreal - Hamas Conflict | Face to Face. Comments: A short title with not much context —————————————- English MT: Videos show how armed men from Gaza stormed a music festival in southern Israel, killing an estimated 260 people on Saturday. The videos, which have been verified by The New York Times, captured panicked concertgoers fleeing through fields and the surrounding countryside, where many hid for hours. Some of the hostages taken over the weekend by Hamas attackers were abducted from the festival. https://nyti.ms/45 jl21i: https://www.nytimes.com/live/202
3/10/10/world/israel-hamas-war-gaza/vi deos-show-the-beginning-of-the-attack-o n-a-music-festival-in-southern-israel. Comments: It is just stating something that happened (news).
—————————————- English MT: IDF releases audio recording about misfired rocket causing Gaza hospital blast rizwanahmed IsraelWarZone IsraelHamas Israel RocketAttack Palestine GazaStrip IsraelVsPalestine Hamas IsraelGazaWar Gaza BreakingNews.
Comments: Stating what happended with a balanced tone.
—————————————- English MT: Israel Prime Minister Benjamin Netanyahu’s office said Sunday its security cabinet had approved the state of war he vowed a day earlier.
Comments: This tweet is simple telling us what happened without showing any bias in the tone.
—————————————- English MT: Syrian President Bashar al-Assad has agreed to send a Russian-made missile defence system to Lebanon’s Hezbollah with the help of Russian mercenary group Wagner, as per a CNN report citing sources familiar with US intelligence. A similar report published earlier by the Wall Street Journal citing US officials said that Wagner plans to provide anti-aircraft systems to Hezbollah but Assad’s alleged role wasn’t mentioned. The reported delivery comes as the armed group Hezbollah and Israel continue to exchange fire across the Israeli-Lebanese border. Watch for more details. israelhamaswar israelhamaswarupdate nasrallah syria basharalassad wagner russianmissile hezbollah usintelligence gaza groundoffensive israel hamas war netanyahu israelgazawar middleeast geopolitics worldnews israelpalestineconflict israelhamaswartoday.
Comments: This tweet mentioned texts from different sources, showing different views, indicating it’s not propaganda. The tone of the text itself is also unbiased.
—————————————- English MT: Supporters of Israel gather for demonstrations in the nation’s capital as the IsraelHamas war rages on.
Comments: The tweet does not show support for either side.
—————————————- English MT: Palestine-Israel conflict has resumed in the Gaza Strip for the third day. As of
Monday, October 9, the death toll on both sides soared over 1,200. Gaza-based Palestinian armed movement Hamas launched a military operation and fired thousands of rockets at Israel on October 7. The military operation caught Israel off guard, bringing heavy battles to Israel’s streets for the first time in decades. Hamas claimed the attack was a response to the recent escalating Israeli assaults in the West Bank and Jerusalem, as well as decades of atrocities against Palestinians. Israel pounded the Gaza Strip from Sunday night into Monday, and ordered a complete siege on the Gaza Strip, halting deliveries of food, fuel, and supplies to its 2.3 million Palestinians. Houses, apartment buildings, and mosques were all among the targets hit overnight. According to the UN, Israeli missiles also raided health facilities inside Gaza and UNRWA Schools. The death toll, including civilians, women, and children, is likely to rise by minutes as indiscriminate rockets continue due to the ongoing conflict. PalestineIsraelConflict Palestine Israel Gaza GazaStripe MiddleEast.
Comments: The text provides a balanced view, considering both side without showing paritality towards one.
—————————————- English MT: “They were very gracious, this must be said.” One of the Israeli captives released by Hamas on Monday has been talking about her experience .
Comments: no bias content towards any group, just talks about the released hostage’s statement.
—————————————- English MT: BREAKING: Two American hostages held by Hamas have been released and are back in Israel. The two women released, Natalie and Judith Raanan, are members of former NBC News Tel Aviv bureau chief Martin Fletcher’s family. Martin calls it “a miracle.”.
Comments: no bias content towards any group, just talks about the released hostage’s statement.
—————————————- 7. Unclear: Guideline: Stataments that lack sufficient context, leaving their stance open to interpretation. Examples: —————————————- English MT: Hamas was founded over 30 years ago – but what does the movement actually want?. Comments: There is a question that can be interpreted in many ways. There is not enough context provided to understand the meaning.
—————————————- English MT: LIVE: President Biden Addressing the Nation on U.S. Response to Israel-Hamas Russia-Ukraine Wars...
Comments: Limited context, just says Biden responds to Israel-Hamas without any further details.
—————————————- English MT: And I crumpled to the ground and I said, ’They are in the midst of killing our son. Comments: We dont’t have any context on this. We don’t know which side this person belongs to. —————————————- English MT: Multiple fatalities have been reported. Comments: There is no context to the text. We don’t know who it is about. —————————————- English MT: Here is a recap of what happened on the second day of the truce deal between Israel and Hamas .
Comments: Just a title with limited context. —————————————- English MT: The family of 16-year-old Noiya Sharabi said she was murdered in the attack, along with her 13-year-old sister and mother.
Comments: Doesnt say who ’Noiya Sharabi’ in the sense where she is from and who murdered her.
—————————————- ### Please provide with detailed explanations. Your aim is to provide with the most useful context while ensuring the classifications are accurate. and then Please classify the statements using their labels as below:
• Biased against Palestine
• Biased against Israel
• Biased against both Palestine and Israel
• Biased against others
• Not Applicable
• Unbiased
• Unclear
Please ensure that the output is provided only in the following format:
{{statement: "", explanation: ".....", label: ""}} It is important to use the exact format above,
including the double curly braces and commas.
Provide the explanation and label for each statement, ensuring that they accurately reflect the bias present in the statement.

heading: Future Work
text: In future work, we aim to improve the IAA and improve the prompt to get a better accuracy