SAIS : Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction (2022)

Abstract: Stepping from sentence-level to document-level, the research on relation extraction (RE) confronts increasing text length and more complicated entity interactions. Consequently, it is more challenging to encode the key information sources—relevant contexts and entity types. However, existing methods only implicitly learn to model these critical information sources while being trained for RE. As a result, they suffer the problems of ineffective supervision and uninterpretable model predictions. In contrast, we propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for RE. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately so as to enhance interpretability. By assessing model uncertainty, SAIS further boosts the performance via evidence-based data augmentation and ensemble inference while reducing the computational cost. Eventually, SAIS delivers state-of-the-art RE results on three benchmarks (DocRED, CDR, and GDA) and outperforms the runner-up by 5.04% relatively in F1 score in evidence retrieval on DocRED.

## introduction playing a crucial role in the continuing effort of transforming unstructured text into structured knowledge, re @xcite seeks to identify relations between an entity pair based on a given piece of text. earlier studies mostly pay attention to sentence-level re @xcite (i.e., the targeting entity pair co-occur within a sentence) and achieve promising results @xcite . based on an extensive empirical analysis, 1 our code is available at https://github.com/ xiaoyuxin1002/sais . @xcite reveals that textual contexts and entity types are the major information sources that lead to the success of prior approaches. given that more complicated relations are often expressed by multiple sentences, recent focus of re has been largely shifted to the document level @xcite . existing document-level re methods @xcite utilize advanced neural architectures such as heterogeneous graph neural networks @xcite and pre-trained language models @xcite . however, although documents typically include longer contexts and more intricate entity interactions, most prior methods only implicitly learn to encode contexts and entity types while being trained for re. as a result, they deliver inferior and uninterpretable results. on the other hand, it has been a trend that many recent datasets support the training of more powerful language models by providing multi-task annotations such as coreference and evidence @xcite @xcite . therefore, in contrast to existing methods, we advocate for explicitly guiding the model to capture textual contexts and entity type information by supervising and augmenting intermediate steps (sais) for re. more specifically, we argue that, from the input document with annotated entity mentions to the ultimate output of re, there are four intermediate steps involved in the reasoning process. consider the motivating example in figure 1: (1) coreference resolution (cr): although sentence 0 describes the "citizenship" of "carl linnaeus the younger" and sentence 1 discusses the "father" of "linnaeus filius", the two names essentially refer to the same person. hence, given a document, we need to first resolve various contextual roles represented by different mentions of the same entity via cr. (2) entity typing (et): after gathering contextual information from entity mentions, et regularizes entity representations with the corresponding type information (e.g., entity a, "linnaeus filius", is of type "per" (person)). within an entity pair, the type information of the head and tail entities can be used to filter out impossible relations, as the relation "year_of_birth" can never appear between two entities of type "per", for instance. (3) pooled and (4) fine-grained evidence retrieval (per and fer): a unique task for locating the relevant contexts within a document for an entity pair with any valid relation is to retrieve the evidence sentences supporting the relation. nonetheless, some entity pairs may not express valid relations within the given document (e.g., entities d and b in the example). meanwhile some entity pairs possess multiple relations (e.g., entity a is both "ed-ucated_at" and an "employee" of entity d), each with a different evidence set. therefore, we use per to distinguish entity pairs with and without valid supporting sentences and fer to output more interpretable evidence unique to each valid relation of an entity pair. in this way, the four intermediate steps are complementary to re, in the sense that cr, per, and fer capture textual contexts while et preserves entity type information. consequently, by explicitly supervising the model's outputs in these intermediate steps via carefully designed tasks, we extract relations of improved quality. in addition, based on the predicted evidence, we filtrate relevant contexts by augmenting specific intermediate steps with pseudo documents or attention masks. by assessing model confidence, we apply these two kinds of evidence-based data augmentation together with ensemble inference, only when the model is uncertain about its original predictions. eventually, we further boost the performance with negligible computational cost. altogether, our sais method achieves state-ofthe-art re performance on three benchmarks (do-cred @xcite , cdr @xcite , and gda @xcite ) due to more effective supervision and enhances interpretability by improving the evidence retrieval (er) f1 score on docred by 5.04% relatively compared to the runner-up. ## supervising intermediate steps this section describes the tasks that explicitly supervise the model's outputs in the four intermediate steps. together they complement the quality of re. ## augmenting intermediate steps we further improve re after training the pipeline sais o all by augmenting the intermediate steps in sais o all with the retrieved evidence from fer.