A sequence-to-sequence approach for document-level relation extraction (2022)

Abstract: Motivated by the fact that many relations cross the sentence boundary, there has been increasing interest in document-level relation extraction (DocRE). DocRE requires integrating information within and across sentences, capturing complex interactions between mentions of entities. Most existing methods are pipeline-based, requiring entities as input. However, jointly learning to extract entities and relations can improve performance and be more efficient due to shared parameters and training steps. In this paper, we develop a sequence-to-sequence approach, seq2rel, that can learn the subtasks of DocRE (entity extraction, coreference resolution and relation extraction) end-to-end, replacing a pipeline of task-specific components. Using a simple strategy we call entity hinting, we compare our approach to existing pipeline-based methods on several popular biomedical datasets, in some cases exceeding their performance. We also report the first end-to-end results on these datasets for future comparison. Finally, we demonstrate that, under our model, an end-to-end approach outperforms a pipeline-based approach. Our code, data and trained models are available at https://github.com/johngiorgi/seq2rel . An online demo is available at https://share.streamlit.io/johngiorgi/seq2rel/main/demo.py .

## introduction pubmed, the largest repository of biomedical literature, contains over 30 million publications and is adding more than two papers per minute. accurate, automated text mining and natural language processing (nlp) methods are needed to maximize discovery and extract structured information from this massive volume of text. an important step in this process is relation extraction (re), the task of identifying groups of entities within some text that participate in a semantic relationship. in the domain of biomedicine, relations of interest include chemical-induced disease, protein-protein interactions, and gene-disease associations. many methods have been proposed for re, ranging from rule-based to machine learning-based @xcite . most of this work has focused on intra-sentence binary re, where pairs of entities within a sentence are classified as belonging to a particular relation (or none). these methods often ignore commonly occurring complexities like nested or discontinuous entities, coreferent mentions (words or phrases in the text that refer to the same entity), inter-sentence and n-ary relations (see figure 1 for examples). the decision not to model these phenomena is a strong assumption. in genia @xcite , a corpus of pubmed articles labelled with around 100,000 biomedical entities, ∼17% of all entities are nested within another entity. discontinuous entities are particularly common in clinical text, where ∼10% of mentions in popular benchmark corpora are discontinuous @xcite . in the cdr corpus @xcite , which comprises 1500 pubmed articles annotated for chemical-induced disease relations, ∼30% of all relations are inter-sentence. some relations, like drug-gene-mutation interactions, are difficult to model with binary re @xcite . in response to some of these shortcomings, there has been a growing interest in document-level re @xcite . docre aims to model inter-sentence rediscontinuous mention of bladder tumours. ## task definition: document-level relation extraction given a source document of s tokens, a model must extract all tuples corresponding to a relation, r, expressed between the entities, e in the document, (e 1 , ..., e n , r) where n is the number of participating entities, or arity, of the relation. each entity e i is represented as the set of its coreferent mentions {e i j } in the document, which are often expressed as aliases, abbreviations or acronyms. all entities appearing in a tuple have at least one mention in the document. the mentions that express a given relation are not necessarily contained within the same sentence. commonly, e is assumed to be known and provided as input to a model. we will refer to these methods as "pipeline-based". in this paper, we are primarily concerned with the situation where e is not given and must be predicted by a model, which we will refer to as "end-to-end". ## conclusion in this paper, we extend generative, seq2seq methods for relation extraction to the document level. we propose a novel linearization schema that can handle complexities overlooked by previous seq2seq approaches, like coreferent mentions and n-ary relations. we compare our approach to existing pipeline-based and end-to-end methods on several benchmark corpora, in some cases exceeding their performance. in future work, we hope to extend our method to full scientific papers and develop strategies to improve performance in the low-data regime and in cases where there are many relations per document.