Uncertain Local-to-Global Networks for Document-Level Event Factuality Identification (2021)

Abstract: Event factuality indicates the degree of certainty about whether an event occurs in the real world. Existing studies mainly focus on identifying event factuality at sentence level, which easily leads to conflicts between different mentions of the same event. To this end, we study the problem of document-level event factuality identification, which determines the event factuality from the view of a document. For this task, we need to consider two important characteristics: Local Uncertainty and Global Structure, which can be utilized to improve performance. In this paper, we propose an Uncertain Local-to-Global Network (ULGN) to make use of these two characteristics. Specifically, we devise a Local Uncertainty Estimation module to model the uncertainty of local information. Moreover, we propose an Uncertain Information Aggregation module to leverage the global structure for integrating the local information. Experimental results demonstrate the effectiveness of our proposed method, outperforming the previous state-of-the-art model by 8.4% and 11.45% of F1 score on two widely used datasets.

## introduction event factuality refers to the degree of certainty about whether events actually occur or not in the real world. generally, event factuality can be classified into five categories @xcite : certain positive (certainly happening, denoted as ct+), certain negative (certainly not happening, ct-), possible positive (possibly happening, ps+), possible negative (possibly not happening, ps-) and underspecified (events' factuality cannot be identified, uu). for example, in the sentence "an economist thinks that the tax rate probably increases soon", the event "increases" may happen. therefore, an event factuality identification (efi) model should event: united states reaches an agreement with mexico text: according to politico.com, the united states probably reaches (ps+) an agreement with mexico on the new trade deal before december, 2017. however, mexican economy minister ildefonso guajardo denied that they plan to reach (ct-) any agreement with the u.s. on the trade deal talks. the government has not been informed that any agreement will be reached (ct-) yet, said another two mexican officials. some media speculate that they will possibly reach (ps+) an agreement. document-level event factuality: ct-figure 1 : an example document with both sentenceand document-level event factuality. the factuality between sentence-and document-level may be different. existing efi studies mainly focus on sentencelevel efi, i.e., judging event factuality based on an individual sentence in which the event is located. in recent years, various neural models have been proposed for sentence-level efi, and achieve stateof-the-art performance @xcite @xcite . despite these successful efforts, sentence-level efi suffers from an inevitable restriction in practice: it easily leads to conflicts between different mentions of the same event. take figure 1 as an example, the "reach" event is mentioned multiple times in a document, which has various factuality values in different sentences. the factuality of the event "reach" in s2 is ps+ according to the speculative word "may", while in s3, its factuality is ct-due to the negative word "denied". according to our statistics on the english and chinese event factuality datasets @xcite , 25.7% (english) and 37.8% (chinese) of instances have the problem of event factuality conflict at sentence level for the same event, which is not negligible. fortunately, the event factuality can be uniquely determined from the perspective of a document, which is able to naturally address the problem of sentence-level event factuality inconsistency. therefore, it is necessary to move efi forward from sentence level to document level. however, identifying document-level event factuality is non-trivial. as shown in figure 1 , the factuality between document-level and sentence-level may be quite different. in this scenario, documentlevel event factuality cannot be deduced from each sentence-level factuality separately, but depends on the comprehensive semantic information of the entire document. to this end, we first learn the local information, and then integrate local representations to the global representation for prediction. in this process, we need to consider two important characteristics: local uncertainty and global structure, which can be leveraged to improve performance. in the following, we will introduce the two characteristics and give the reasons why they are critical for document-level efi. local uncertainty: as illustrated in figure 1 , different sentences (i.e., local information) describe different cognitive individuals' judgements towards the event factuality. however, the degree of uncertainty of these judgements is different. for example, as direct participants in the "reach" event, mexican officials (in s4) can judge the event factuality with lower uncertainty (i.e., higher confidence) than other cognitive individuals (e.g., a journalist in s2). apparently, the information of s4 is more important than that of s2 when predicting the document-level event factuality. it would be better if we could explicitly model the uncertainty of local information. therefore, the first challenging problem is how to model the uncertainty of local information. global structure: when integrating local information, utilizing global structure (i.e., document structure) could yield a better global representation. the global structure is manifested in two aspects: positional structure and semantic structure. for positional structure, as shown in figure 1 , the content of the document is roughly organized in chronolog-ical order, which can reflect the evolution of events. for semantic structure, there is a semantic correlation between local information. for instance, the content of s2 is the support of the view about the event factuality in s1, while the content of s3 is the denial of that in s1. there is no doubt that capturing the global structure enables a better understanding of documents. thus, the second challenging problem is how to leverage the document structure for integrating local information. in this paper, we propose a novel method termed as uncertain local-to-global network (ulgn) to address aforementioned problems. specifically, to model the uncertainty of local information, we propose a local uncertainty estimation module. it utilizes a probability distribution to represent the local information, rather than a deterministic feature vector. for ease of modeling, we adopt gaussian distributions. namely, the local information is now parameterized by a mean and variance. the former acts like the normal feature vector as in the conventional model, whereas the latter measures the feature uncertainty. the higher the uncertainty of the local information is, the larger its corresponding variance is. to leverage the global structure for synthesizing local information, we devise an uncertain information aggregation module. the module first constructs a global graph based on the document structure, and then employs an uncertain graph convolution layer to aggregate the local information. it considers the uncertainty of local information via variance-based attention. experimental results on two widely used datasets demonstrate that our method substantially outperforms previous state-of-the-art models. overall, the main contributions of this work can be summarized as follows: ## methodology we propose an uncertain local-to-global network (ulgn) for document-level efi. figure 2 schematically visualizes our approach, which consists of three major components: (1) local uncertainty estimation ( §2.1), which represents the local information by using a probability distribution; (2) uncertain information aggregation ( §2.2), which leverages the global structure to integrate the local information; ## conclusion in this paper, we propose a novel uncertain local-toglobal network (ulgn) for document-level event factuality identification. to model the uncertainty of local information, we propose a local uncertainty estimation module to represent the local information with a probability distribution. to leverage the global structure, we devise an uncertain information aggregation module to integrate the local information. experimental results on two widely used datasets indicate that our approach substantially outperforms previous state-of-the-art methods.