Eagles at FIGNEWS 2024 Shared Task: A Context-informed Prescriptive Approach to Bias Detection in Contentious News Narratives (2024)

Abstract: This research paper presents an in-depth examination of bias identification in media content related to the Israel-Palestine war. Focusing on the annotation guidelines and process developed by our team of researchers, the document outlines a systematic approach to discerning bias in articles. Through meticulous analysis, key indicators of bias such as emotive language, weasel words, and loaded comparisons are identified and discussed. The paper also explores the delineation between facts and opinions, emphasizing the importance of maintaining objectivity in annotation. Ethical considerations, including the handling of sensitive data and the promotion of multipartiality among annotators, are carefully addressed. The annotation guidelines also include other ethical considerations such as identifying rumors, false information, exercising prudence and selective quotations. The research paper offers insights into the annotation experience, highlighting common mistakes and providing valuable guidelines for future research in bias identification. By providing a comprehensive framework for evaluating bias in media coverage of the IsraelPalestine war, this study contributes to a deeper understanding of the complexities inherent in media discourse surrounding contentious geopolitical issues.

text: Proceedings of The Second Arabic Natural Language Processing Conference, pages 656–671 August 16, 2024 ©2024 Association for Computational Linguistics
This research paper presents an in-depth examination of bias identification in media content related to the Israel-Palestine war. Focusing on the annotation guidelines and process developed by our team of researchers, the document outlines a systematic approach to discerning bias in articles. Through meticulous analysis, key indicators of bias such as emotive language, weasel words, and loaded comparisons are identified and discussed. The paper also explores the delineation between facts and opinions, emphasizing the importance of maintaining objectivity in annotation. Ethical considerations, including the handling of sensitive data and the promotion of multipartiality among annotators, are carefully addressed. The annotation guidelines also include other ethical considerations such as identifying rumors, false information, exercising prudence and selective quotations. The research paper offers insights into the annotation experience, highlighting common mistakes and providing valuable guidelines for future research in bias identification. By providing a comprehensive framework for evaluating bias in media coverage of the IsraelPalestine war, this study contributes to a deeper understanding of the complexities inherent in media discourse surrounding contentious geopolitical issues.

heading: 1 Introduction
text: The world recently witnessed an unprecedented escalation of the war between Palestine and Israel
which resulted in contentious international media coverage of which warrants close scrutiny. The element of subjectivity depending on various factors, such as proximity and exposure to certain narratives, linguistic barriers, etc. also confers an additional challenge. The FIGNEWS 2024 shared task (Zaghouani et al., 2024) addresses this area of research by adopting a collaborative approach, leveraging on a diverse pool of researchers and machine translation technologies to create a corpus of global news media narratives surrounding the Israel-Palestine war that would serve as a foundation for further research in Natural Language Processing. There are two separate subtasks, one tailored to bias annotation and the other to propaganda annotation. As a relatively new team, we decided to focus on the bias annotation alone in our contribution.
The shared task emphasized creation of annotation guidelines, a pivotal step of the NLP pipeline that influences the viability and efficacy of any model. Thus, we sought to develop comprehensive prescriptive annotation guidelines that accounted for the contextual sensitivities of diverse and conflicting discourse. This paper details the rationale behind the creation of the annotation guidelines in addition to team composition and training, data annotation process, task participation and result.
Existing research that leveraged shared tasks have predominantly focused on data tasks such as Named Entity Recognition (Strauss et al., 2016), language identification (Solorio et al., 2014), and information extraction (Nédellec et al., 2013), where prescriptive standards could be established and applied. There have been exceptions such as literary annotations (Reiter et al., 2017), which
656
indicate the potential for shared tasks with respect to efforts that involve higher degrees of variability and subjectivity. Given the inherent diversity in any representative dataset for a global, polarized discourse such as the Israel war on Gaza, it is thus prudent to approach corpus creation in a diverse, collaborative fashion.

heading: 2 Annotation Methodology and Examples

heading: 2.1 Development of Annotation Guidelines
text: The starting point for the shared task was the seven annotation labels identified in the shared task website1. We began with discussing the definition and criteria for each label, using the examples provided (Appendix A) as the “ground truth”. We emphasized identifying and describing distinct, mutually exclusive traits that were attributable to each label. Based on our consensus, a decision tree was created to visualize the annotation scheme (Appendix C). This served as a tool to standardize the thought process during the annotation and to facilitate efficient discussion.
Once the initial annotation guidelines were established, a random sample of 20 texts was generated from the main dataset. We collectively discussed the texts, categorizing them as main or edge cases for the decided label. The substantiation for this categorization was used to refine the guidelines. This process was repeated until the team encountered at least one main and edge cases for each of the seven labels (Appendix D). We then asynchronously annotated the main dataset based on the guidelines. All texts that seemed to warrant amendment or reevaluation of the guidelines were highlighted and raised for discussion during weekly recalibration sessions, and the outputs of said discussion was accordingly reflected in the annotation guidelines.
The random sample was crucial for quality control of the annotation guidelines which increased annotation consistency among the team members. In a later stage, each language was annotated by more than one team member to maintain objectivity, ensure high quality control standards and consistency.
1Website: https://sites.google.com/view/fignews

heading: 2.2 Data Annotation Process
text: Since the team is spread across three countries, the shared task was conducted virtually via Google Sheets, Google Docs, with weekly discussion and recalibration meetings on Google Meets and otherwise asynchronous efforts.
The team started with first with annotating InterAnnotator Agreement (IAA) dataset then moved to the main dataset. Comments and references were made directly in the relevant comments section. Notable texts were highlighted to indicate need for further discussion.
Before beginning or continuing annotations, each researcher would read through and reorient themselves with the latest guidelines and examples. Wherever pertinent, the researcher would revisit previous annotations to ensure consistent application.
The researcher began each annotation by reading the text. The first step taken was to determine the relevance of the contents to the Palestine-Israel war, by virtue of any reference to said war. If no relevance was detected, the text was classified as not applicable. If relevance was established, the subjectivity of the text was examined. If the text is conflicting or ambiguous, it was classified as unclear. If no subjectivity was detected in the text, then it was classified as unbiased. If the text was determined to be subjective, the direction of subjectivity was then analyzed, be it against Palestine, Israel, both Palestine and Israel, or others. This process is depicted in the decision tree mentioned in the previous section.
We frequently consulted the examples of main and edge cases established via discussion of the random sample before finalizing the annotation. Any ambiguity or uncertainty was highlighted for discussion during the next recalibration meeting. Then, we would discuss the source of ambiguity or contention and if needed, consult existing literature to reach a consensus decision on the annotation. This would subsequently be added to the edge cases and reflected in the guidelines.

heading: 2.3 Inter-Annotator Agreement Analysis
text: The IAA Analysis was initiated once the initial annotation guidelines were established. The IAA annotations were completed in 4 phases of 50 texts each. The researchers relied closely on the examples of main cases and edge cases in the
annotation decisions, noting any points of confusion or contention without making specific reference to the IAA text. Once each researcher had completed 50 IAA annotations, the average CohenKappa score would be calculated. If the score fell below 0.7, the researchers would revisit the annotation guidelines and go through another 20 randomly selected texts from the main dataset together, and subsequently revisit their individual IAA annotations. If the score was above 0.7, the researchers would begin annotating the subsequent 50 texts. This process was repeated until the annotation of the IAA texts was complete. This process was guided by the prescriptive annotation paradigm as elaborated by Röttger et al. (2021), whereby inter-annotator divergence signaled incorrect application or inadequate guidelines, which thus prompted further refinement.

heading: 3 Team Composition and Training
text: Our team was composed of researchers with interests in linguistics, computational linguistics, and media studies. Each member comes from a different academic background and demographic (Egypt, Lebanon, Tunisia, and Malaysia), although three members of the team were not residing in their home country at the time of the shared task, with the annotator from Lebanon based in Egypt, the annotator from Tunisia based in Italy and the annotator from Malysia based in Qatar.
The team of researchers were relatively new to annotation, with only one researcher having had practical exposure to data annotation. The first few meetings prioritized knowledge transfer, with team members sharing their relevant expertise, i.e., on the annotation process and rationale of quantifying consistency via the separation of datasets into the Main and IAA portions, the linguistic considerations of assessing bias, etc. The team also shared online resources on annotation, such as the Annotation Instructions Tutorial (QCRI, 2024), in preparation for the annotation task.
The team worked together in a collaborative virtual environment through weekly synchronous meetings and asynchronous group chat. Unfortunately, due to external constraints one researcher was highly limited in their contributions, hence the remaining three team members
2 Results: FIGNEWS_2024_RESULTS-27June2024 - Google Sheets
coordinated on an ad hoc basis to ensure completion of the task.
The team coordination included delegating tasks, group discussions, taking notes of the team’s discussions and important points, and having indepth discussions of confusing items. For instance, the delineation between “unclear” and “not applicable.” Some items had historical references that needed to be researched to understand their context and their relation to current events and therefore be able to decide the appropriate annotation label.

heading: 4 Task Participation and Results
text: Of the 1,800 texts annotated, our team found that 1,064 (59.1%) texts were classified as unbiased, 473 (26.3%) as biased against Palestine, 87 (4.8%) as not applicable, 121 (6.7%) as unclear, 38 (2.1%) as biased against others, 10 (0.6%) as biased against Israel, and 7 (0.4%) as biased against both Palestine and Israel (Figure 1), which was similar to the overall findings2 (Figure 2).
We found considerable presence of bias within the texts (29.4%), and bias against Palestine was much than significant against that Israel or others.
Among the texts biased against Palestine, sources in Hebrew were far more prominently biased than its English, Hindi, Arabic, and French counterparts (Figure 3). Despite our efforts, we had considerable discrepancy both within the team and across other teams with, Kappa scores of 55.5 and 25.6 respectively. The detailed results of the Bias subtrack can be found in Appendix B.
Certain factual associations were determined to be biased due to highly prejudiced associations. For instance, the armed wing of the political party Hamas, which is also the elected government of Gaza, is frequently labelled as terrorists, and conflated with the terrorist group ISIS. Given this biased subtext, references such as “Hamas-run Health Ministry” were determined to be biased against Palestine, since the association can be construed to be made with the intention of undermining said public office.
When factual information was conveyed, there was considerable use of weasel words, such as “loud and clear” and polarizing adjectives and phrases which emphasized and introduced nonneutral content, an indicator of bias which was concurred by Herzig et al. (2011). Due to these contextual nuances, external training and theoretical exposure to annotation practices alone was insufficient and had to be supplemented with frequent discussions based on examples from the dataset. This prevented inconsistent application of the established annotation scheme.
Although the term ‘genocide’ has very negative connotations and could be construed to imply bias, the term objectively describes current events of the Israel-Palestine war. Since the plausibility of
genocide has been affirmed both by relevant legal and scholarly experts, we decided that using the term did not construe a bias against Israel, and in fact, avoiding the term was more likely to construe a bias against Palestine.
Certain texts also contained multiple biases and could justifiably fall into multiple categories. In such cases, the team compared the degree of bias and opted for the bias that was most prominent in the text. The alternative would be to choose the ‘Unclear’ label in these instances; however, this would not serve the purpose of bias identification since bias is clearly detected within said texts, only in multiple directions.

heading: 5 Discussion
text: This shared task establishes a promising alternative to generating crowdsourced annotations for Natural Language Processing that addresses some key concerns in annotation guidelines and practice, particularly in highly sensitive and complex subject areas. For instance, Parmar et al. (2022) expounds upon instruction bias, where Natural Language Understanding is driven by benchmarks where annotators generate examples in a top-down approach, based on annotation instructions. This can be equated to the “ground truth” mentioned in previous sections. However, in refining the annotation guidelines based on a large dataset derived from real world sources, the shared task ensures that the collected data is robust and representative of real-world content.
In view of the widespread conflation of Palestinians with Hamas or other political figures, we equated bias against Hamas and political leaders as bias against Palestinians. This was done to meaningfully assess the sentiments of the text in accordance with existing views and does not reflect the team’s opinions on the plurality of Palestinian political views. Similarly, the assessment of fact versus opinion was based on up-to-date knowledge of current events. This introduces some variability to the annotations, since some content may be proven or disproven at different points in time, for example, headlines claiming genocide released on October 8th, 2023, could reasonably be construed as biased against Israel, but an identical headline published in January 2024 should certainly be viewed differently. This is impactful given the extent of disinformation surrounding the IsraelPalestine war. Notably, this may explain the slight divergence in our team’s findings compared to the
overall findings regarding the presence of bias against Israel. In discussion, we concluded that many texts that were initially labelled as biased against Israel were in fact unbiased due the factual content, based on external research and factchecking. Consequently, the resulting annotations are bound to recency, and efforts to update them based on current information are paramount to preserve the accuracy and integrity of the data. Since the shared task focused on news media content, the team also considered ethical reporting and journalism standards in ascribing bias (Appendix E).

heading: 6 Conclusion
text: This paper proposes a comprehensive framework for bias detection in news media narratives surrounding the Israel-Palestine war. Given the objective of the annotation guidelines to provide a robust prescriptive framework for bias detection, the development thereof took on an iterative approach, that via a systematic annotation process, mitigated the risk of inter-annotator subjectivity. The annotation process revealed the subtle complexities of detecting bias, highlighting the role of diverse human perspectives, robust discussion, and up-to-date information to mitigate annotator bias in such data tasks. We believe that these resources and the insights gained from the production thereof will inform the understanding of the unique challenges of analyzing conflicting discourse in highly contentious topics.

heading: A Criteria and “Ground Truth for Annotation Labels
text: Label Category Guidelines / Criteria Example of Main Cases based on Ground Truth provided by FIGNEWS organisers (Zaghouani et al., 2024) Unbiased The content is relevant and does not exhibit bias
• The content makes specific reference to the Gaza-Israel war and mentions the involvement of actors on both sides as well as external stakeholders from a neutral standpoint. • Content is not subjective. Absence of weasel words, favorable repetition, and words that explicitly convey opinion or emotions. • Subjective content is attributed to an external source or labeled as an opinion. "In the ongoing IsraelPalestine conflict, recent events have escalated tensions. Yesterday, Israeli forces conducted operations in response to rocket attacks from Gaza. Both sides have reported casualties. International leaders are calling for restraint and a return to peace talks."
Biased against Palestine The content is relevant and exhibits bias against Palestine The content makes specific reference to the Gaza-Israel war from a non-neutral (subjective) standpoint that disfavours Palestine - containing weasel words*, favorable repetition, and/or words that explicitly convey opinion or emotions. "Once again, Palestinian aggression has disrupted peace in the region. Palestinian extremists, ignoring efforts for peace, launched unprovoked attacks on innocent Israeli civilians. Israel's response, though portrayed as harsh by some, is a justified measure to protect its citizens." Biased against Israel The content is relevant and exhibits bias against Israel
The content makes specific reference to the Gaza-Israel war from a non-neutral (subjective) standpoint that disfavours Israel - containing weasel words*, favorable repetition, and/or words that explicitly convey opinion or emotions. Or it includes false information. **consider difference between factual and emphatic sentences "In a typical display of excessive force, Israeli troops have yet again targeted Palestinian areas, causing numerous civilian casualties. This aggression, under the guise of selfdefense, highlights the ongoing oppressive tactics Israel employs against Palestinians."
Biased against both Palestine and Israel The content is relevant and exhibits bias against Palestine and Israel The content makes specific reference to the Gaza-Israel war from a non-neutral (subjective) standpoint that disfavours both Palestine and Israel - containing weasel words*, favorable repetition, and/or words that explicitly convey opinion or emotions. "In the latest chapter of their endless and futile conflict, Israeli and Palestinian forces have once again engaged in senseless violence. Both sides continue to commit atrocities, showing a
Label Category Guidelines / Criteria Example of Main Cases based on Ground Truth provided by FIGNEWS organisers (Zaghouani et al., 2024) complete disregard for peace or human life." Biased against others The content is relevant and exhibits bias against other actors/stakeholders
The content makes specific reference to the Gaza-Israel war from a non-neutral (subjective) standpoint that disfavours actors apart from Palestine and Israel - containing weasel words*, favorable repetition, and/or words that explicitly convey opinion or emotions. "In the shadow of the Israel-Palestine conflict, external actors, particularly Iran, are exacerbating tensions. Iran's covert support for extremist groups shows its intent to destabilize the region, disregarding the catastrophic impact on both Israeli and Palestinian civilians."
Unclear The content is potentially relevant but does not offer sufficient specification to establish the presence or absence of bias
The content vaguely references the Gaza-Israel war without identifying specific moments, actors, or actions.
"Recent developments in the Middle East have seen an increase in hostilities. The situation in the region is complex, with various factors contributing to the current state of affairs. The international community remains divided on the issue."
Not Applicable The content is not relevant or off-topic.
The content does not pertain to pivotal moments in the Gaza-Israel war and hence is not directly relevant to the corpus.
"In other news, the annual technology conference in Tel Aviv has unveiled groundbreaking advancements in cybersecurity. Industry leaders from around the globe gathered to showcase innovations that promise to shape the future of digital security."

heading: B Bias Subtrack Evaluation Results
text: Quantity Quality Centrality
Batches Total IAA Within Team Main B1+B2 Across Teams
Team Annotators Main IAA Data Points Kappa Acc Macro F1 Avg F1 Bias* Kappa Acc Macro F1 Avg F1 Bias*
Bias Bluff Busters 4 2 8 2,600 43.3 56.3 48.5 69.3 14.4 28.7 21.0 61.7
BiasGanda 4 2 4 2,200 31.0 51.5 31.5 66.4 26.0 43.6 29.5 64.2
BSC-LANGTECH 2 2 4 2,200 51.0 65.5 39.8 81.5 26.5 46.6 29.2 60.2
Ceasefire 3 2 6 2,400 26.6 46.3 29.3 61.2 24.2 42.0 27.2 66.0
DRAGON 4 15 60 19,500 35.7 75.5 41.0 43.2 19.7 41.1 21.9 59.7
Eagles 4 2 8 2,600 55.5 75.4 48.4 68.5 25.6 46.0 25.4 55.3 Groningen Annotates Gaza 7 2 14 3,200 43.5 56.8 39.8 69.9 25.3 28.9 25.7 56.4
JusticeLeague 3 2 6 2,400 64.4 83.7 63.8 73.6 19.9 43.3 19.6 46.5 Narrative Navigators 7 2 4 2,200 39.4 56.5 45.5 70.5 28.0 44.5 30.5 66.6
NLPColab 21 15 30 16,500 78.8 85.3 76.1 94.3 27.7 42.4 30.8 67.3
Sina 10 12 24 13,200 61.4 81.4 55.4 75.1 11.8 38.7 17.2 48.1
SQUad 2 4 8 4,400 23.4 40.8 27.2 66.8 -0.2 8.5 5.8 56.2 The CyberEquity Lab 5 3 15 4,200 48.1 71.6 39.5 70.5 25.0 46.5 24.1 58.0 The Guideline Specialists 2 2 30 4,800 28.6 51.3 34.9 84.4 21.0 36.8 24.5 66.2 The Lexicon Ladies 3 2 4 2,200 37.2 53.0 35.4 73.4 29.1 44.1 33.1 66.3
UoT1 4 3 12 3,900 44.9 58.2 48.7 71.5 26.8 42.5 29.0 64.6
Average 5.3 4.5 14.8 5,531 44.5 63.1 44.1 71.3 21.9 39.0 24.7 60.2
Total 85 72 237 88,500

heading: C Decision Tree Depicting Annotation Scheme
text: Does the text contain clear reference to the Gaza-Israel war?
Yes
Does the text have conflicting/unclearly
directed bias?
Yes - Unclear No
Is the text subjective (has presence of indicators of bias)?
Yes
Is the bias directed at multiple parties?
Yes
Is the text biased against Palestine and Israel?
Yes - Biased against both
Palestine & Israel
No - The text is labelled according
to the dominant bias exhibited
No
Is the text biased against Palestine or Israel?
Yes
Is the text biased against Palestine?
Yes - Biased against Palestine
No - Biased against Israel
No - Biased against others
No - Unbiased
No
Does the text contain vague reference to the Gaza-Israel war?
Yes - Unclear
No - Not Applicable

heading: D Examples and Justifications
text: Label Examples and Justification Unbiased Ref 41
They Said We Were Hamas 'They said, “We’ll kill you, you’re Hamas. These Palestinian men, working in a factory in the West Bank, were arrested by Israeli forces, detained for 23 days, and subjected to torture and interrogation. Amnesty International reports a surge in Israeli authorities' use of administrative detention of Palestinians in the occupied West Bank since October 7th. Justification:
• Use of quotations and attributing subjective content to external sources (Palestinian detainees) • Provides factual information (dentifies the source of information - Amnesty International)
Ref 23 US President Joe Biden says both Hamas and Russia’s Vladimir Putin “want to completely annihilate a neighboring democracy.” https://trib.al/UAfb8eA:=:https://www.bloomberg.com/news/articles/202310-20/biden-casts-russia-hamas-as-parallel-threats-to-democracy Justification:
• Use of quotations and attributing subjective content to external sources (Joe Biden)
Ref 7 Protest in Aligarh Muslim University in support of Hamas. इज़रायल पर हमले से हाहाकार, AMU में �ो ंहमास की जयकार? #aligarhmuslimuniversity #upnews #tv9upuk #breakingnews #cmyogi #akhileshyadav #upgoverment #bjp #bsp #samajwadiparty #newsupdate #Israel vs palestine #Israel palestine war #israel palestine news #protest AMU israel consulate # Protest in AMU #israel palestine occupation #israel launches air strikes in gaza #AMU Protest For Hamas Justification:
• Specifies relation to Israel-Palestine war (Hamas) • References objective events (motivation behind protest in AMU)
Ref 66 Israel northern border heats up after Hezbollah rocket fire from Lebanon #LIVE #Israel #Palestine #Gaza #Hamas #Hezbollah #GolanHeights #Syria Justification:
• Specifies relation to Israel-Palestine war in accompanying text (#Palestine #Gaza) • References objective events (reaction to Hezbollah rockets in Israel)
Ref 4785 From 13 million dollars in donations, Spain increased to 54 million dollars to finance UNRWA. While around ten countries, including
Label Examples and Justification France, have decided to suspend funding on the basis of accusations of support for Hamas by the Israeli government, Spain has decided to triple its funding, to continue to come in aid of the 2 million Gazans for whom the need for support is vital in the midst of the genocide carried out by Israel Justification: Non-neutral language (highlighted) determined to be unbiased and objective based on larger context of events (ICJ recognition of plausibility of Israel violating genocide convention) Biased Against Palestine Ref 21 SENDING A CLEAR MESSAGE: Americans joined together in DC for the ‘March for Israel’ to show their support in the fight against Hamas terrorists. https://trib.al/iZyroGg:=:https://www.foxnews.com/us/dutydemonstrators-pack-national-mall-solidarity-israel Justification: Non-neutral content not attributed to external source (highlighted) - Usage of term ‘terrorists’ to describe Hamas Ref 794 Hamas delays release of Israeli hostages on second day of its truce with Israel in dispute over aid and Palestinian prisoners. Follow live updates. Justification:
• Non-neutral content not attributed to external source (highlighted) - attribution of delay to Hamas
Framing of Israeli captives as ‘hostages’ and Palestinian captives as ‘prisoners’.
Biased against Israel Ref 4040 Photos of the funeral ceremony of the martyrs of the Orthodox Church in Gaza bombed yesterday by the Israeli air force #GazaUnderAttack #Palestine Justification: Non-neutral content not attributed to an external source (highlighted) - Term ‘martyrs’ alludes to religious/ideological persecution that is unsubstantiated, i.e. the religious identities of the victims were not known hence could not be the motivation for the bombing. Ref 2946 Since we witness the eradication of a people under the pretext of obeying the urgency of saving another, is it even possible to hope for a part of humanism, of respect for life? , in the face of this savage persecution which falls every day, incessantly, without restraint, without conscience on Gaza.... This song, still tragically relevant today, was composed by my friend Tewfik Benghabrit and sung in June 2022 in the town of Umm Qeis (Jordan), in fact almost at the gates of a Palestine that has been wandering for decades…. His words resonate more than ever with the long suffering of a people whose only wrong would be to continue living, whose painful intensity of cry is matched only by their determination to shout their right to exist. .
Label Examples and Justification Justification:
• Use of emotive language (highlighted) Use of non-neutral content (highlighted)
Biased against both Palestine & Israel Ref 31 Asla Abu Kadir, the lawyer and Israeli immigrant, who expressed support for the Hamas terrorist attack on social media, was arrested and brought today to a hearing on the extension of her detention at the High Court. In addition, the police appealed to the Bar Association to exhaust the disciplinary proceedings against her and revoke her law practice license alongside the criminal proceedings. Israeli Immigrants They won't be able to be lawyers! I don't know about you, I'm personally excited! I hope you are at least like me Raz Atia Justification:
• Use of non-neutral content directed at both Palestine (Hamas terrorist attack) and Israel (Israeli immigrants)
Ref 128 Beaz Levy is my neighbor, married to my cousin and also enlisted in Order 8. A smart guy who wrote things that are very important to me. Give them a moment: life is complex. Yair Golan, hated by the Israeli right, sacrificed his soul to die to save Jews from slaughter. hero of Israel. Life is complex. Settlers, hated by the Israeli left, threw themselves into the inferno and fell by the dozens in the battles for the protection of the Otaf kibbutzim. Life is complex. Netanyahu's term, Mr. Defense, ends with the biggest security disaster in Israel's history. Life is complex. Biden, the slandered American president in Israel, the man perceived as lame and lacking in personality, turns out to be a true friend and support in times of need. Life is complex. Those who thought it was possible to have fun with the refusal and the cracking of the reserve formation, were forced to enlist with 150% commitment. Life is complex. Those who fully trusted the right-wing government received a complete disaster. Life is complex. Those who until yesterday did not even agree to destroy terrorists' houses, are today calling for the razing of Gaza. Life is complex. Whoever banished the ultra-Orthodox found out that they are here, volunteering for the most terrible civilian tasks. The one who declared that we are no longer brothers, discovered to his horror that we are brothers to the slaughter. Those who thought that the disengagement, those who trusted in intelligence, those who trusted cameras and smart fences, those who believed that the settlers, those who said that the leftists, those who estimated that the Arabs... life is complex. Concepts are broken. Fundamental beliefs are undermined. Worldviews evaporate. Everything changes in these moments, and the change goes down to the roots of the beliefs of Israeli society. World changes are coming. Who doesn't see it. Those who are still stuck on their previous perceptions as they are. Those who find in the events just another proof of their method. He who remains right as yesterday, knows everything as last year. Those whose views are not currently undergoing a version update. Who doesn't stop for a moment to think. Well, didn't understand anything. He is yesterday's man. Don't be yesterday's people! Don't go back to the old arguments like a dog back to Kio. Don't be too quick to jump to conclusions for the other party. Don't be happy for Id. Don't wear me out in a skirmish. Do not praise the sober ones, because
Label Examples and Justification you too will be forced to sober up. Do not announce the collapse of the right, do not announce the collapse of the left. We all collapsed. We all fell. The right was wrong. The left was wrong. We all failed. Time for far-reaching changes in Israeli society. Justification:
• Use of emotive language (highlighted) • Use of non-neutral content against both Israel and Palestine
(highlighted) Ref 4775 Tonight at 17:00, on the daily emergency broadcaster: * We will host Rabbi Uri Sharki and discuss the question - on the one hand, how do we preserve the spirit of the nation that is revealed in all its glory - and on the other hand, fearlessly warn against the evil ways of the Israeli leadership. * We will discuss the significance of President Biden's visit and the subordination of the decision-making process in Israel to the American interest. * We will try to understand how the decisions are made - how Netanyahu's "head works". Almost two weeks have passed since the Gaza pogrom. Our children are still in cages, our daughters are still in dormitories. No real and required response. All the lazy generals are already asking to attack - and only Netanyahu is preventing it. Why? We will discuss this on the daily emergency broadcaster - this evening at five. Justification:
• Use of emotive language (highlighted) • Use of non-neutral content against both Israel and Palestine
(highlighted)
Biased against others Ref 948 It's amazing to watch how some in the Democratic Party are finally waking up and realizing that they belong to the party of hate and disinformation. Rep. Ritchie Torres (D-NY) said Rep. Ilhan Omar's (DMN) opposition to funding the Iron Dome “would have led to even more dead Israelis and more dead Palestinians” if she had prevailed. He also stated that the humanitarian crisis of the war was started by Hamas killing Israeli civilians. Justification:
• Non-neutral content not attributed to external source (highlighted) • Use of emotive language (highlighted) • Identifying external party as main subject of content (the
Democratic Party) • Specific mention of Israel-Palestine, but not as the direct subject
of main content Ref 114 LOUD AND CLEAR: Biden's refusal to call for a cease-fire in the Israel-Hamas conflict may cost him key voters. https://trib.al/2hE3rL4:=:https://www.foxnews.com/media/young-peopleturn-genocide-joe-cease-fire-stance-biden-absolutely-sucks
Label Examples and Justification Justification:
• Identifying external party as main subject of content (Biden) • Specific mention of Israel-Palestine, but not as the direct subject
of main content Non-neutral accompanying context (highlighted)
Unclear Ref 58 A Controversial Video Goes Viral on Social Media: Women Laugh and Take Selfies in the Background of a Kidnapping, Displaying Disrespectful Gestures. #Israel #HamasWar #IsraelUnderAttack #Gaza #Palestinians #IsraelPalestineWar #Gaza #IsraelFightsTerror #IndiaStandWithIsrael #telaviv #Hezbollah #FPJ Justification:
• Specific reference not in the main text but in accompanying context (hastags #IsraelPalestineWar) • Main text does not identify specific actors in a non-neutral way • Accompanying context alludes to non-neutral framing
(#IsraelFightsTerror, #IndiaStandWithIsrael) Ref 52 This Diwali, all the lamps of our family in #Gaza are in the name of love, peace, happiness, humanity and light
Label Examples and Justification Justification: No specified content Ref 86 Multiple fatalities have been reported. Ref 86 unity!! For the first time in history, after the signature of 85 MKs, the way was paved for the removal of Ofer Kasif from the Israeli Knesset. Justification: No specified relation to the current events of Israel-Palestine war Justification: No specification related to Gaza or Israel
E Identification of Bias
• Emotive language, emphatic language, and non-neutral content
o Terms that incited emotion and/or empathy or portrayed a particular side in a better/worse light were taken as indicators of bias.
• Weasel words
o phrases that create an impression that something specific and meaningful has been said when only a vague or ambiguous claim has been communicated
o E.g.: “it is widely believed”
• Favorable repetition
o repetitive usage of words that in typical usage do not have prior polarity, but when used in a repetitive manner, create a favorable depiction of a sentence’s topic and/or it eventually create a generalization about a certain group.
o E.g.:
o For instance, the repetitive usage of the word “terrorist”.
• Loaded comparisons
o Use of selective vocabulary in order to create distinct connotations for different groups
o E.g.: Using ‘Israeli hostages’ and ‘Palestinian prisoners’ to imply the innocence of Israeli captives and the criminality of Palestinian detainees
o This should be considered in tandem with the functional use of terms i.e. the term ‘hostage exchange does not carry the same biased connotations.
• Selective quotations
o Although clear attribution of opinions or quotes constitutes unbiased content, the team realized that certain content with excessively extreme one-sided quotes revealed an editorial bias by an omission of alternate perspectives and perceived legitimacy.
o Understandably, this is a grey area; so edge cases were highlighted for discussion and the final decision was achieved by consensus.
• Exercising Prudence
o In order to prevent inaccuracies and false attributions of bias, the team elected to treat edge cases with higher levels of scrutiny, choosing to opt for ‘Unclear’ when the text did not confidently meet our established thresholds for bias.
• Facts Vs. Opinions
In order to put objective criteria for bias identification, there was a need to distinguish between facts and opinions. For instance, providing something that contains false information would be categorized as biased. Opinions justifying genocide or a mass murder would be categorized as biased.
• Rumors
o The mention of rumors or crimes committed without a piece of evidence is categorized as bias.

heading: Future Work
text: Industry leaders from around the globe gathered to showcase innovations that promise to shape the future of digital security. "