RAPL : A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction (2023)

Abstract: How to identify semantic relations among entities in a document when only a few labeled documents are available? Few-shot document-level relation extraction (FSDLRE) is crucial for addressing the pervasive data scarcity problem in real-world scenarios. Metric-based meta-learning is an effective framework widely adopted for FSDLRE, which constructs class prototypes for classification. However, existing works often struggle to obtain class prototypes with accurate relational semantics: 1) To build prototype for a target relation type, they aggregate the representations of all entity pairs holding that relation, while these entity pairs may also hold other relations, thus disturbing the prototype. 2) They use a set of generic NOTA (none-of-the-above) prototypes across all tasks, neglecting that the NOTA semantics differs in tasks with different target relation types. In this paper, we propose a relation-aware prototype learning method for FSDLRE to strengthen the relational semantics of prototype representations. By judiciously leveraging the relation descriptions and realistic NOTA instances as guidance, our method effectively refines the relation prototypes and generates task-specific NOTA prototypes. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches by average 2.61% F_1 across various settings of two FSDLRE benchmarks.

## introduction document-level relation extraction (docre) aims to identify the relations between each pair of entities within a document, which is crucial for extracting complex cross-sentence relations and implementing large-scale information extraction @xcite @xcite . however, the annotation of docre data is both time-consuming and labor-intensive, thomas miller was born in cardiff in 1935. at the age of 31, he became a lecturer at the london business school, where he met his wife, karen smith, a native londoner. they worked together in london for 30 years. in 1999, miller returned to his hometown of cardiff and passed away six years later. ## problem formulation few-shot document-level relation extraction is defined with an n -doc setting @xcite . in each individual fsdlre task (also called an episode 2 ), there are a set of n support documents {d s,1 , ..., d s,n } and a query document d q , and the entity mentions in each document are pre-annotated. for each support document d s,i , there is also a triple set t s,i containing all valid (e h , r, e t ) triples in the document. here e h and e t are the head and tail entity of a relation instance, and @xmath0, with r episode being the relation type set for which instances are to be extracted. the annotations of support documents are complete, which means any entity pair for which no relation type has been assigned can be considered as nota. given these as inputs, the fsdlre task aims to predict the triple set t q for the query document d q , which contains all valid triples in d q of relation types in r episode . doc encoder rel encoder relation prototype generation support cross token attention class prototype representations support relation instance learning relation-weighted contrastive learning support documents support token embeddings support relation instance embeddings query document relation embeddings query cross token attention query token embeddings query entity pair learning support nota instance learning nota prototype generation support nota instance embeddings learnable base nota prototypes query entity pair embeddings r1 target relation types r2 r3 classification ℒ 𝐵𝐶𝐸 sim(r1, r2) sim(r1, r3) sim(r2, r3) ℒ 𝑅𝐶𝐿 ℒ negative instance pair positive instance pair paradigm. in training phase, we construct a group of training episodes by sampling support and query documents from a training document corpus c train . the set r episode of each training episode is a subset of r train , a relation type set for meta-training. the model aims to learn general knowledge from these training tasks to better generalize to novel tasks. in test phase, the model is evaluated on a group of test episodes sampled from a test document corpus c test , which is disjoint with c train . the set r episode of each test episode is a subset of r test , a relation type set for meta-testing, which is also disjoint with r train . ## methodology the overall architecture of rapl is illustrated in figure 3 . we first introduce the encoding procedure for documents and entities in section 3.1. in section 3.2 and section 3.3, we elaborate on the learning of relation-aware relation prototypes and nota prototypes respectively. the training and inference processes are finally given in section 3.4. ## conclusion in this paper, we propose rapl, a novel relationaware prototype learning method for fsdlre. we reframe the construction of relation prototypes into instance level and further propose a relationweighted contrastive learning method to jointly refine the relation prototypes. moreover, we design a task-specific nota prototype generation strategy to better capture the nota semantics in each task. experiment results and further analyses demonstrate the superiority of our method and effectiveness of each component. for future work, we would like to transfer our method to other few-shot document-level ie tasks such as few-shot document-level event argument extraction, which shares similar task structure with fsdlre.