Incorporating Factuality Inference to Identify Document-level Event Factuality (2023)

Abstract: Document-level Event Factuality Identification (DEFI) refers to identifying the degree of certainty that a specific event occurs in a document. Previous studies on DEFI failed to link the document-level event factuality with various sentence-level factuality values in the same document. In this paper, we innovatively propose an event factuality inference task to bridge the sentence-level and the document-level event factuality semantically. Specifically, we present a Sentence-to-Document Inference Network (SDIN) that contains a multi-layer interaction module and a gated aggregation module to integrate the above two tasks, and employ a multi-task learning framework to improve the performance of DEFI. The experimental results on the public English and Chinese DLEF datasets show that our model outperforms the SOTA baselines significantly.

## introduction document-level event factuality identification (defi) predicts the factual property of an event from the view of a document, i.e., describing whether an event is evaluated as a fact, a counterfact, or a possibility. it is essential for many nlp applications, such as rumor detection @xcite and sentiment analysis @xcite . based on @xcite , @xcite summarized document-level event factuality into the following five categories: certain positive (certainly happens, ct+), possible positive (possibly happens, ps+), certain negative (certainly not happens, ct-), possible negative (possibly not happens, ps-), underspecified (factuality is uncommitted, uu). different from sentencelevel event factuality identification (sefi) which determines the factuality based on a single sentence in which the event is located, defi is a more challenging task and needs to synthesize the semantics of sentence-level mentions of the document. figure 1 illustrates the relationship and differences between sentence-level and document-level event factuality. the document-level event "tear up textbooks" is denoted by the event trigger (the main word that most clearly expresses the occurrence of an event) "tear" and is mentioned in sentences s1, s2, s4, s5. in s1, since the trigger "tearing" is negated by the negative cue "banned", its sentencelevel factuality is ct-. similarly, the factuality of "tear" in s4 is ps-according to the speculative cue "unlikely". on the contrary, "tearing" in s2 and s5 are not affected by any negative or speculative information, and they have the same factuality, i.e., ct+. although those four event mentions (the sentence containing the event trigger) have various sentence-level factuality values, from the perspective of the document, the document-level factuality value of the event "tear up textbooks" is unique, which is determined as ct-. from figure 1 , we can observe that the factuality of a specific event at the document-level and the sentence-level semantics may be not always consistent, making it challenging when leveraging the sentence-level factuality values. all previous work did not explore the relationship between sentencelevel and document-level event factuality in depth, and usually let models learn global information from the document automatically, often resulting in noisy semantics. in this paper, we re-consider the document-level factuality from a new perspective, i.e., associating document-level factuality with its core event mentions that has the same factuality (e.g., s1 in figure 1 ), while weakening the other event mentions with different factuality values that may impose a negative effect (e.g., s2 and s5 in figure 1 ). for this purpose, we exploit the sentence-level and document-level factuality to propose a novel event factuality inference task, which can semantically bridge the above two-level factuality to determine whether an event mention is core or not. taking figure 1 as an example, we can assign s1, s2, and s4 with three inference labels (i.e., support, refute, and conjecture), representing how they contribute to inferring the document factuality. the inference task uses the event mentions with inference labels for training, making it feasible to measure the importance of event mentions for overall factuality identification. specifically, we propose a sentenceto-document inference network (sdin) with two modules, where one is a multi-layer interaction module for solving the inference task and can provide useful features for defi through a multi-task learning framework, and the other is a gated aggregation module that can selectively aggregate semantic and factual features in event mentions for factuality identification. overall, our main contributions can be summarized as follows: ## related work in sentence-level event factuality identification (sefi), @xcite and @xcite used rule-based methods, and then many studies @xcite @xcite used machine learning-based methods, which relied on the annotated information. deep learning methods @xcite @xcite have also been widely used in recent years and have achieved significant results. compared with sefi, document-level event factuality identification (defi) is still in the exploratory stage. qian et al. (2019) gave the full definition of defi and constructed a document-level event factuality corpus (dlef) on the basis of sentence-level annotations, in which both sentencelevel and document-level factuality of event triggers were annotated. based on the dlef dataset, qian et al. (2019) and huang et al. (2019) proposed lstm-based methods to solve the new defi task. zhang et al. (2021) used cross-domain corpora to train a bert-crf model for detecting negation and speculation scope, and introduced scope features into defi. cao et al. (2021) proposed an uncertain local-to-global network (ulgn), which integrated the local uncertainty as well as the global structure. zhang et al. (2022b) proposed a novel heterogeneous semantics-syntax-fused network (hs 2 n) for defi, which integrated semantic and syntactic information and considered inter-andintra sentence interaction. recently, some work has broadened the direction of defi research. qian et al. (2022a) constructed a larger dlef-v2 corpus to expand the dlef. a complete sentence is used in dlef-v2 instead of a trigger word to represent an event. in addition, dlef-v2 only annotates the document-level factuality related to the event, and no longer performs sentence-level annotations, which supports end-to-end work. based on dlef-v2, qian et al. (2022a) proposed a novel end-to-end reinforced multi-granularity hierarchical attention network to learn information at different levels of granularity from tokens and sentences hierarchically. qian et al. (2022b) proposed a new framework formulating defi as machine reading comprehension (mrc) tasks considering both span-extraction and multiple-choice. distinct from these mentioned efforts, zhang et al. (2022a) constructed a new evidence-based document-level event factuality corpus eb-dlef, and proposed a pipeline approach to solve the new introduced evidential sentence selection task and event factuality identification task. zhang et al. (2023) introduced a twostage data augmentation strategy from text to graph sentence factuality document factuality ct+ ps+ ct-ps-uu ct+ sp cj rf cj uk ps+ cj sp cj rf uk ct-rf cj sp cj uk ps-cj rf cj sp uk uu uk uk uk uk uk ## methodology we propose a sentence-to-document inference network (sdin) for inference and identification tasks, which mainly contains two modules of multilayer interaction and gated aggregation. the detailed model structure is shown in figure 2 . ## conclusion in this paper, we innovatively employ the relationship between the sentence-and document-level event factuality to construct an event factuality inference paradigm, which can bridge these two-level factuality semantically. moreover, we propose a sentence-to-document inference network (sdin), which is a multi-task learning framework incorporating a multi-layer interaction module and a gated aggregation module to integrate the task of document-level event factuality inference and identification. experimental results on the english and chinese dlef datasets demonstrate the significant improvements of our proposed model sdin, in comparison with the sota baselines. in future work, we will focus on end-to-end document-level event factuality identification.