Reward-based Input Construction for Cross-document Relation Extraction (2024)

Abstract: Relation extraction (RE) is a fundamental task in natural language processing, aiming to identify relations between target entities in text. While many RE methods are designed for a single sentence or document, cross-document RE has emerged to address relations across multiple long documents. Given the nature of long documents in cross-document RE, extracting document embeddings is challenging due to the length constraints of pre-trained language models. Therefore, we propose REward-based Input Construction (REIC), the first learning-based sentence selector for cross-document RE. REIC extracts sentences based on relational evidence, enabling the RE module to effectively infer relations. Since supervision of evidence sentences is generally unavailable, we train REIC using reinforcement learning with RE prediction scores as rewards. Experimental results demonstrate the superiority of our method over heuristic methods for different RE structures and backbones in cross-document RE. Our code is publicly available at https://github.com/aailabkaist/REIC.

## introduction the task of relation extraction @xcite aims to identify relations between a pair of target entities in a given text @xcite @xcite . this task is fundamental in natural language processing (nlp) and provides crucial information for applications such as information retrieval @xcite and question answering @xcite . most existing re methods are limited to scenarios where the entity pair is within a single sentence @xcite or a single document @xcite @xcite . however, many scenarios require extracting relations across multiple documents, where the target entity pair may not coexist * equal contribution within the same document. therefore, recent efforts to address these challenges have led to the proposal of cross-document re by @xcite , and research in this area has received considerable attention @xcite @xcite . unlike traditional re research, cross-document re involves extracting relational facts from largescale long documents. for example, the docred dataset @xcite , developed for documentlevel re, contains an average of 198 words per document. in contrast, the codred dataset @xcite , designed for cross-document re, has an average of 2,416 words per document. given the substantial length of documents in cross-document re tasks, extracting document embeddings from pre-trained language models, a step common to all methods, poses considerable challenges. this is because pre-trained language models are limited by the maximum number of tokens they can process; for example, bert @xcite typically has a limit of 512 tokens. a simple approach adopted in several studies is to use text snippets around the target entities as input to pre-trained language models, as shown in the dashed boxes in the middle panel of figure 1. often, this text snippet is determined by either word proximity or document structure without any adjustment. however, if only a subset of sentences around the entities is extracted from long documents, there is a risk of missing important sentences that are crucial for re. for example, as seen in the left panel of figure 1 , using snippets around the entities fails to capture the relation 'platform' between the entity pair @xcite . in this paper, we introduce reward-based input construction (reic), the first learning-based input sentence selection module specifically designed for cross-document re. the right panel of figure 1 displays the sentences selected by our model from the example. by identifying relational evidence in june 2013, jonathan riddell of kubuntu announced that kubuntu did not plan to switch to mir. he stated; a few months ago canonical announced their new graphics system for ubuntu, mir. in september 2013, an intel developer removed … linux was the first operating system kernel run the x86-64 architecture in long mode. other distributions, such as ubuntu, are available in one version compiled for a 32-bit architecture and another compiled for a 64-bit architecture. sentences through the selection module, the re module infers the relation between the target entities using the reasoning chain illustrated by the gradient-colored arrows in figure 1 . our approach is to develop a sentence selection module that computes the probability of selecting sentences based on the currently selected sentences. we show that this sentence selection process can be modeled as a markov decision process (mdp). we specifically choose to use mdp because there is no supervision for sentence selection from corpus, so re models have to perform iterative learning by trials. subsequently, we train the sentence selection module using reinforcement learning (rl), where the relation prediction score obtained from the selected sentences serves as the reward. through experimental validation, reic outperforms other heuristic methods across various re modules. ## conclusion we present reward-based input construction (reic), a learning-based sentence selection module tailored for cross-document re. we tackle the limitation of document embedding extraction from long documents. reic selects important sentences based on relational evidence, enabling the re module to effectively infer relations. given the unavailability of supervision for evidence sentences, we use rl to train reic. we show the superiority of our method over heuristic methods. this highlights the potential of learning-based approaches to improve the performance of cross-document re.