TTM - RE : Memory-Augmented Document-Level Relation Extraction (2024)

Abstract: Document-level relation extraction aims to categorize the association between any two entities within a document.We find that previous methods for document-level relation extraction are ineffective in exploiting the full potential of large amounts of training data with varied noise levels. For example, in the ReDocRED benchmark dataset, state-of-the-art methods trained on the large-scale, lower-quality, distantly supervised training data generally do not perform better than those trained solely on the smaller, high-quality, human-annotated training data. To unlock the full potential of large-scale noisy training data for document-level relation extraction, we propose TTM-RE, a novel approach that integrates a trainable memory module, known as the Token Turing Machine, with a noisy-robust loss function that accounts for the positive-unlabeled setting. The trainable memory module enhances knowledge extraction from the large-scale noisy training dataset through an explicit learning of the memory tokens and a soft integration of the learned memory tokens into the input representation, thereby improving the model’s effectiveness for the final relation classification. Extensive experiments on ReDocRED, a benchmark dataset for document-level relation extraction, reveal that TTM-RE achieves state-of-the-art performance (with an absolute F1 score improvement of over 3%). Ablation studies further illustrate the superiority of TTM-RE in other domains (the ChemDisGene dataset in the biomedical domain) and under highly unlabeled settings.

## introduction relation extraction aims to classify the relationships between two specified entities into predefined categories. this task is pivotal in natural language processing, as it involves identifying and categorizing the connections between two entities (for example, "pacific fair" and "queensland" in figure 1 ) into predefined classes (for example, "located in" in figure 1 ). its importance spans across various downstream applications, encompassing question answering @xcite , knowledge graph ## methodology we propose ttm-re, a memory-augmented, document-level relation extraction method. an illustration of the overall framework of ttm-re is shown in figure 3 . ## discussion analysis of memory module the token turing machine (ttm) memory module performs best with a large training set. this leads to an important question: what other applications could provide such extensive training data? while it is true that ttm benefits from a large dataset, potential applications include any field within nlp or computer vision. moreover, these datasets do not necessarily need to be human-annotated. for instance, the docred dataset was distantly supervised using wikipedia and spacy for ner and relation linking via wikidata. similar methods can be employed to create datasets for specific tasks. ttm-re has demonstrated improved performance with these distantly supervised datasets compared to other baselines. ## conclusion in this paper, we investigated ttm-re, integrating ttms relation classification and evaluated our model on redocred and chemdisgene re datasets. to summarize our contributions, no previous work has explored memory in this distantly supervised setting. as such, ttm-re demonstrates a completely new way of increasing performance for the difficult task of document-level relation extraction as opposed to previous work, which mainly improved the loss function @xcite . for this work, we found compelling results by performing ablations that showed that adding (and increasing) the number of memory tokens/layers helped performance over baselines, compared to simply using larger models like deberta v3. we also observed that ttms necessitate either fine-tuning on a large distantly-labeled training dataset or a significantly large human-annotated training dataset (chemdisgene) to optimize memory vector initialization. we believe that this work lends itself to future work in this exciting area, and we hope that our findings will pave the way for future exploration of memory-augmented techniques in large language models for information extraction tasks.