Discriminative Reasoning with Sparse Event Representation for Document-level Event-Event Relation Extraction (2023)

Abstract: Document-level Event Causality Identification (DECI) aims to extract causal relations between events in a document. It challenges conventional sentence-level task (SECI) with difficult long-text understanding. In this paper, we propose a novel DECI model (SENDIR) for better document-level reasoning. Different from existing works that build an event graph via linguistic tools, SENDIR does not require any prior knowledge. The basic idea is to discriminate event pairs in the same sentence or span multiple sentences by assuming their different information density: 1) low density in the document suggests sparse attention to skip irrelevant information. Our module 1 designs various types of attention for event representation learning to capture long-distance dependence. 2) High density in a sentence makes SECI relatively easy. Module 2 uses different weights to highlight the roles and contributions of intra- and inter-sentential reasoning, which introduces supportive event pairs for joint modeling. Extensive experiments demonstrate great improvements in SENDIR and the effectiveness of various sparse attention for document-level representations. Codes will be released later.

## introduction event-event relation extraction (ere) is the task of identifying the relation between two events in texts. as shown in figure 1 , for any pair of events 1 , e.g., (services, downtime), it shall make the classifications for which relation type it holds. clearly, event pairs may be in the same sentence (sere) @xcite @xcite , or scattered across the entire document (dere) @xcite . in practice, dere can benefit a wider range of applications, such as knowledge graph construction @xcite and future event forecasting [2] services on', "egypt's", 'three telecom operators were disrupted in parts of alexandria on tuesday afternoon , and some users in cairo experienced slower internet connections due to "downtime". ## methodology sendir aims to learn high-quality event representations to facilitate both intra-and inter-sentential reasoning. as shown in figure 2 , our framework has four main components: encoder to encode a document into vector, sparse event representation learning (ser) that further learns event representations based on document embeddings, discriminating intra-and inter sentential reasoning (dir) that conducts joint inference based on each pair of event representations, and classification to make final predictions. ## conclusion in this paper, we exploit a novel discriminative reasoning with sparse event representation for dere. it can learn high-quality event representation and facilitate inter-sentential reasoning for document-level understanding. experimental results show that our method is effective and significantly better than competitive baselines, improving inter-sentence cases without harming intrasentence event pairs. the extensive analysis also provides interesting insights about various language biases for sparse long-text representation learning. in the future, we will combine various sparse assumptions for high-quality document representations and incorporate graph reasoning.