MOKA : Moral Knowledge Augmentation for Moral Event Extraction (2024)

Abstract: News media often strive to minimize explicit moral language in news articles, yet most articles are dense with moral values as expressed through the reported events themselves. However, values that are reflected in the intricate dynamics among *participating entities* and *moral events* are far more challenging for most NLP systems to detect, including LLMs. To study this phenomenon, we annotate a new dataset, **MORAL EVENTS**, consisting of 5,494 structured event annotations on 474 news articles by diverse US media across the political spectrum. We further propose **MOKA**, a moral event extraction framework with **MO**ral **K**nowledge **A**ugmentation, which leverages knowledge derived from moral words and moral scenarios to produce structural representations of morality-bearing events. Experiments show that **MOKA** outperforms competitive baselines across three moral event understanding tasks. Further analysis shows even ostensibly nonpartisan media engage in the selective reporting of moral events.

## introduction many news media frame their stories to further a particular ideological viewpoint @xcite , often employing moral values rather than explicitly partisan language to subtly affect readers @xcite @xcite . however, existing nlp methods, including llms, face significant challenges in discerning moral values. past work has shown that these limitations may be due to lack of context @xcite , lack of moral reasoning capabilities @xcite , and the complexity of moral stances @xcite . detecting moral values is even harder for non-partisan news outlets which deliberately avoid explicit moral language event participants are annotated per wikipedia pages if applicable. in each event record, the event trigger is a single word in an event span , and it might embody multiple moralities. moral event extraction is challenging due to several reasons: implicit participants (e.g. samesex couples in event record 1) may not be mentioned in the target sentence, and understanding the relations among the participants is necessary to correctly infer the morality. in their reporting, but may express moral values indirectly by selecting which morally-laden events to report. thus there remains an imperative need for nlp tools that can decipher moral values latent in narrated events and interactions among entities. in particular, news articles tell complex stories that contain multiple people and events along with interactions among them. the participants in the events, the ordering of them, and the selection of events themselves have been shown to be useful for crafting impactful news articles @xcite @xcite . in this work, we study morality and moral reasoning at the event level, enabling fine-grained structural analysis, capturing the nuances of relationships between participants performing moral actions, and uncovering the deeper layers of ethical dimensions intrinsic to news narratives. to this end, we first propose the concept of moral events, which capture the interaction among moral participants, such as moral agents and moral patients @xcite , as demonstrated in figure 1 . we then study the problem of structured moral event extraction, which enables finegrained analysis of how the choice of events in news articles and the context in which the events occur together carry moral implications, form effective news stories, and sway readers' perceptions. following prior work, we employ moral foundations theory (mft; @xcite @xcite , which posits five moral foundations, each containing two polarities of virtue and vice, e.g., care/harm. mft has been widely used in analyzing both mainstream news @xcite and social media content @xcite @xcite , but often in superficial ways based on explicit moral language, without utilizing the larger context and external knowledge to better understand interactions between participants in moral actions. to illustrate the challenges in moral event understanding, figure 1 shows a sentence containing multiple events with moral values but little in the way of explicit moral language. the correct identification of event record 1 must take into account a longer context (e.g., the title) and background knowledge (that the defense of marriage act governs same-sex couples) to identify the patient who is affected by the invalidated event. identifying the morality of invalidated requires knowing that although it usually carries a negative connotation and might imply harm on a surface level, here it is actually beneficial to the patient and in fact embodies fairness and care towards same-sex couples. our paper makes the following contributions. first, we define a new schema of moral events, grounded in mft and linguistics. we then propose moral event extraction (mee): given unstructured text, detect morality-bearing event triggers, extract participants, and infer embodied moralities. second, to solve mee, we curate a large dataset, moral events, consisting of moral event annotations of news articles from diverse us media outlets. this dataset is unique in that annotations are conducted on multiple news articles about the same story, allowing us to analyze differences in how news outlets of different ideologies report moral events. moral participant annotations go beyond surface mentions and syntactic constraints, capturing implicit participants in moral actions. we propose moka, a generative framework for mee with moral knwoledge augmentation. capitalizing on the recent success of retrieval augmentation @xcite @xcite , moka integrates moral knowledge derived from varying granularities, moral words and moral scenarios. additionally, to support moka pre-training, we crawl a bank of 344k morality-bearing examples, morality bank, leveraging validated morality lexicons @xcite . extensive experiments highlight the usefulness and robustness of moka over strong baselines, including sota event extraction models and chatgpt (gpt-3.5turbo). the results show that external moral knowledge is essential for lms to excel at mee and ethics-related moral reasoning in general. further analysis of moral event reporting in news reveals substantive findings, including (1) left-right asymmetries where right-to-left moral events are more prevalent than the reverse regardless of underlying moral values or outlet ideology, and (2) a tendency of centrist media to focus primarily on moral events enacted by right-leaning entities. ## moral events curation we define a new structured schema for a moral event which represents a moral action, visualized in figure 1 . a moral event consists of moral agents, moral patients, a morality-bearing event span and event trigger, embodied morality, and event status. we list major concepts below, and refer readers to appendix a for full descriptions. a moral action is performed or enabled by moral agents and affects moral patients. an event trigger is usually a single word within an event span that can best characterize a moral action. this span embodies one or more moralities in mft @xcite . note that moral patients may be implicit: they do not have to be mentioned in a target sentence. to study the linguistic phenomenon, we further annotate event status which describes the factuality of an event @xcite , i.e., whether an event is actual, intentional or speculative @xcite . annotation process. we create our dataset, moral events, using the following process. we first sample 87 news stories from seesaw @xcite , where each story contains 3 articles on the same event but reported by media of different ideologies. to supplement this set with recent news, we further collect a new set of news article triplets from allsides.com focusing on important issues in 2021 and 2022, e.g., abortion, gun control, and public health. we extract text from these articles using newspaper 2 , and clean all articles by removing boilerplate text and embedded tweets. next, moral events are annotated by native english speakers (at least two for each article). each annotator has access to all three articles in a story to maintain a non-biased view. we list the major steps of annotating a single article, with a detailed annotation protocol in appendix b. we ensured the quality of the annotations at multiple steps in the collection process (appendix c). also see table a1 for annotation agreements. statistics. moral events include 474 news articles from 158 stories, published by 63 different media outlets (26 left, 18 center, and 19 right). on average, each article contains 11.6 event annotations. the articles cover 38 salient topics reported from 2012 to 2022, and includes 1, 952 distinct entities (see table a2 ). we annotate diverse en-2 https://github.com/codelucas/newspaper/ 3 other includes religions (e.g., people of faith) and topics (e.g., homeland security) among others. tity types: people (62.4%), organization (20.4%), geo-political (9.6%), and others (7.6%). ## the moka models we now define the task of moral event extraction (mee), which extracts structured moral events from unstructured texts. similar to mainstream event extraction, we decompose mee into the sub-tasks of event detection and event argument extraction, but with a focus on morality. to tackle these tasks, we develop a new framework, moka (figure 2 ), which incorporates external moral knowledge into pre-trained language models at two levels: lexical-based moral word knowledge ( §4.1) and example-based moral scenario knowledge ( §4.2). after two-stage pre-training, moka is fine-tuned on downstream tasks ( §4.3). we instantiate moka with flan-t5-large, though it is compatible with models of other architectures. ## further analyses we further investigate the use of moral language in news media through the lens of selective reporting of moral events. we validate past work showing how different ideologies focus on different moralities (rq1), and go beyond that to show how the selective reporting of moral narratives reveals more subtle and asymmetrical forms of bias (rq2). rq1: does moral language usage correlate with media ideology? figure 3 shows that more extreme outlets unsurprisingly tend to use more moral language overall, whereas the centrist media use it least. the most frequent moral foundation is care/harm, in line with findings on social media text (figure 4 ; @xcite . both news media and social media use relatively little sanctity/degradation, which measures religious purity and disgust and is rarely reported in the news. however, news media use a far higher proportion of authority/subversion than social media because much of the news focuses on politicians and other ruling figures. in contrast, social media covers more fairness/cheating, due to its greater focus on explicit morality as seen in aita reddit forum which has a special focus on personal ethical violations @xcite . finally, within these general tendencies, we also find that left-leaning media focus more on care/harm, while right-leaning media focus more on authority/subversion, in line with mft @xcite . rq2: how is media bias revealed by the selective reporting of agent-morality-patient narratives? moral narratives are fundamentally constructed out of three elements: an agent, a patient, and an action with an associated morality. to understand how ideology and morality shape the news, we must examine these three elements jointly. to measure agent and patient ideologies, all entities that appear in at least two news articles were coded by a domain expert for their partisan leaning on a binary left/right scale, yielding 197 coded entities and 1, 253 associated events. figures a1 and a2 show the correlations between agent-patient relationship and outlet ideologies for the two most prevalent foundations, care/harm and authority/subversion. this reveals rich differences between left, right, and center media that do not fall into the simple partisan symmetries that have been posited previously @xcite @xcite . within care/harm (figure a1 ), the left media report relatively more right-harm-left events than the right media do, and vice versa. interestingly, an asymmetry is observed that media across different ideologies all report more right-harm-left events than the reverse (i.e. left-harm-right). that applies to centrist outlets as well, which show a pronounced tendency of reporting more care/harm where the right entity is the agent. l e f t m e d i a c e n t e r m e d i a r i g h t m e d i a c a r e / h a r m l o y a l t y / b e t r a y a l a u t h o r i t y / s u b v e r s i o n f a i r n e s s / c h e a t i n g s a n c t i t y / d e g r a d a t i o n 53.4% 51.3% 48.9% 9.4% 7.4% 8.8% 21.9% 24.5% 26.0% 14.4% 16.3% 15.0% 1.0% 0.6% 1.2% 0.0 0.1 0.2 0.3 0.4 0.5 0.6 figure 4: employed moral foundation distribution by media outlets of different ideologies. for authority/subversion (figure a2 ), we find both left and right outlets report more authority from right-to-left, while centrist media are once again more focused on right-agent events overall. these asymmetries are even more notable with subversion, where we see right media reporting (disapprovingly) on left entities subverting the right but also (approvingly) on right subverting the left, while centrist media also report more right-subverts-left events. to summarize, mainstream media strive for balance in ideological language, entities, and even expressed values, but when we examine agent-valuepatient triplets, ideological differences become evident. we found both important left-right asymmetries, and the distinctive behavior of centrist media, which overwhelmingly focuses on right agents. these illustrate the importance of eventlevel morality analysis in political news. ## conclusion we studied the task of moral event extraction-a novel reasoning task with the objective of, given unstructured text, producing structural representations for morality-bearing events including their triggers, participating entities, and embodied morality. to support this study, we curate a new dataset, moral events, including 5, 494 structured annotations. we propose moka, a moral reasoning-enhanced event extraction framework with moral knowledge augmentation. specifically, we employ retrieval augmentation by integrating moral knowledge at varying granularities, derived from moral words and moral scenarios. further analyses reveal the effectiveness of using moral events to discern ideological biases even when outlets report seemingly objective events.