Multi-hop Evidence Retrieval for Cross-document Relation Extraction (2023)

Abstract: Relation Extraction (RE) has been extended to cross-document scenarios because many relations are not simply described in a single document. This inevitably brings the challenge of efficient open-space evidence retrieval to support the inference of cross-document relations,along with the challenge of multi-hop reasoning on top of entities and evidence scattered in an open set of documents. To combat these challenges, we propose Mr.Cod (Multi-hop evidence retrieval for Cross-document relation extraction), which is a multi-hop evidence retrieval method based on evidence path mining and ranking. We explore multiple variants of retrievers to show evidence retrieval is essential in cross-document RE.We also propose a contextual dense retriever for this setting. Experiments on CodRED show that evidence retrieval with Mr.Cod effectively acquires cross-document evidence and boosts end-to-end RE performance in both closed and open settings.

## introduction relation extraction (re) is a fundamental task of information extraction @xcite ) that seeks to identify the relation of entities described according to some context. it is a key task integral to natural language understanding @xcite for inducing the structural perception of unstructured text. furthermore, it is also an essential step of automated knowledge base construction @xcite and is the backbone of nearly all knowledge-driven ai tasks @xcite @xcite @xcite . previous works have limited the context of re within a single sentence @xcite @xcite , a bag of sentences @xcite @xcite , or a single document (yao et al., 1 our code is public available at the github repository: https://github.com/luka-group/mrcod this figure shows a 3hop evidence path for the triplet ("a saucerful of secrets", "genre", "progressive rock"), which consists of three passages scattered across two documents from wikipedia. arrows in the figure show the bridging entities that link the passages in this evidence path. in addition, humans also consolidate different steps of a complex event by referring to multiple articles, such as inferring the process of an event from multiple news articles @xcite or instructional events @xcite . to facilitate research in cross-document re, @xcite constructed the first human-annotated crossdocument re dataset, codred, to serve as the starting point of this realistic problem. unlike sentence-or document-level re tasks, cross-document re takes a large corpus of documents as input and poses unique challenges @xcite . first, because inferring relations based on the whole corpus is inefficient and impractical, evidence retrieval, which involves extracting evidential context from a large corpus, is crucial for cross-document re. second, relations in cross-document re are usually described by multi-hop reasoning chains with bridging entities. fig. 1 shows an example of a 3-hop evidence path in codred, which spans three related passages in two documents. in this example, the relation between 'a saucerful of secrets" and "progressive rock" is described by a reasoning chain containing four bridging entities (marked in grey). on average, cross-document multi-hop reasoning through 4.7 bridging entities is needed in codred to infer relations @xcite . besides, previous work @xcite has shown that intra-document multi-hop reasoning is effective for document-level re. therefore, evidence retrieval needs to consider the bridging entities for multi-hop reasoning in cross-document re. against these challenges, we propose a dedicated solution mr.cod (multi-hop evidence retrieval for cross-document relation extraction), which extracts evidence from a large corpus by multi-hop dense retrieval. as illustrated in fig. 2 , mr.cod is composed of two steps: evidence path mining and evidence path ranking. in evidence path mining, we first construct a multi-document passage graph, where passages are linked by edges corresponding to shared entities. then, we use a graph traversal algorithm to mine passage paths from head to tail entities. this step greatly reduces the size of candidate evidential passages. in evidence path ranking, we rank the mined paths based on their relevance. we explore different dense retrievers widely used in open-domain question answering (odqa) as scorers for evidence paths and further propose a contextual dense retriever better suited for multi-hop relation inference. finally, the top-k evidence paths are selected as input for downstream relation extraction models. mr.cod is flexible and can be used with any models designed for long-context re. the contributions of this work are two-fold. first, we propose a multi-hop evidence retrieval method for cross-document re and show that highquality evidence retrieval benefits end-to-end re performance. second, we explore multiple widely-used retrievers in our setting and further develop a contextual dense retriever for multi-hop reasoning. our contributions are verified in both closed and open settings of codred @xcite . we observe that mr.cod outperforms other evidence retrieval baselines and boosts end-to-end re performance with various downstream re methods. ## related works we discuss two topics of research that are closely relevant to this study. ## methods in this section, we describe mr.cod, a multi-hop evidence retrieval method for cross-document re. we will introduce the preliminaries ( §3.1), pro-posed evidence retrieval ( §3.2 - §3.4), and downstream re methods we explored ( §3.5). ## experiments this section presents an experimental evaluation of mr.cod for evidence retrieval and end-to-end re performance. we introduce the experimental setup ( §4.1), main results ( §4.2), and ablation study on incorporated techniques ( §4.3). ## conclusion we study efficient and effective ways to extract multi-hop evidence for cross-document re and propose mr.cod. mr.cod extracts evidence paths from an open set of documents and ranks them with adapted dense retrievers as scorers. to overcome the gap between retrieval in odqa and evidence retrieval for re, we develop a contextual dpr that augments sparse queries with passage context. extensive experiments show high-quality evidence retrieved by mr.cod boosts end-to-end cross-document re performance. future works include extending mr.cod to more retrieval methods, such as generative dense retrievers.