Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction (2022)

Abstract: Relation extraction (RE) has achieved remarkable progress with the help of pre-trained language models. However, existing RE models are usually incapable of handling two situations: implicit expressions and long-tail relation types, caused by language complexity and data sparsity. In this paper, we introduce a simple enhancement of RE using k nearest neighbors ( k NN-RE). k NN-RE allows the model to consult training relations at test time through a nearest-neighbor search and provides a simple yet effective means to tackle the two issues above. Additionally, we observe that k NN-RE serves as an effective way to leverage distant supervision (DS) data for RE. Experimental results show that the proposed k NN-RE achieves state-of-the-art performances on a variety of supervised RE datasets, i.e., ACE05, SciERC, and Wiki80, along with outperforming the best model to date on the i2b2 and Wiki80 datasets in the setting of allowing using DS. Our code and models are available at: https://github.com/YukinoWan/kNN-RE.

## introduction relation extraction (re) aims to identify the relationship between entities mentioned in a sentence, and is beneficial to a variety of downstream tasks such as question answering and knowledge base population. recent studies @xcite @xcite @xcite in supervised re take advantage of pre-trained language models (plms) and achieve sota performances by fine-tuning plms with a relation classifier. however, we observe that existing re models are usually incapable of handling two re-specific situations : implicit expressions and long-tail relation types. implicit expression refers to the situation where a relation is expressed as the underlying message that is not explicitly stated or shown. for example, for the relation "sibling to", a common expression can be "he has a brother james", while an implicit expression could be "he is the youngest son of liones, comparing with samuel liones and henry liones." in the latter case, the relation "sibling to" between "samuel liones" and "henry liones" is not directly expressed but could be inferred from them both are brothers of the same person. such underlying message can easily confuse the relation classifier. the problem of longtail relation types is caused by data sparsity in training. for example, the widely used supervised re dataset tacred @xcite includes 41 relation types. the most frequent type "per:title" has 3,862 training examples, while over 22 types have less than 300 examples. the majority types can easily dominate model predictions and lead to low performance on long-tail types. inspired by recent studies @xcite @xcite using knn to retrieve diverse expressions for language generation tasks, we introduce a simple but effective knn-re framework to address abovementioned two problems. specifically, we store the training examples as the memory by a vanilla re model and consult the stored memory at test time through a nearest-neighbor search. as shown in figure 1 , for an implicit expression, the expression "son of" may mislead to an incorrect prediction while its retrieved nearest neighbor contains a direct expression "brother of", which is a more explicit expression of the gold label "sibling to". the prediction of long-tail examples, as shown in figure 1 , is usually biased toward the majority class. nearest neighbor retrieval provides direct guidance to the prediction by referring to the labels of its nearest neighbors in the training set, and thus can significantly reduce the imbalanced classification. additionally, we observe that knn-re serves as an efficient way to leverage distant supervision (ds) data for re. ds augments labeled re datasets by matching knowledge base (kb) relation triplets and raw text entity pairs in a weak-supervision fashion @xcite @xcite . recent studies @xcite @xcite , which apply plms to the ds labeled data to improve supervised re, require heavy computation due to the fact that they require pre-training on ds data, whose size is usually dozens of times that of supervised datasets. to address this issue, we propose a lightweight method to leverage ds data to benefit supervised re by extending the construction of stored memory for knn-re to ds labeled data and outperforming the recent best pre-training method with no extra training. in summary, we propose knn-re: a flexible knn framework to solve the re task. we conduct the experiments for knn-re with three diffor the vanilla re model, we follow the recent sota method pure @xcite . to encode an input example to a fixed-length representation by fine-tuning plms such as bert @xcite , pure adds extra marker tokens to highlight the head and tail entities and their types. specifically, given an example x: "he has a brother james. ## experiment settings supervised datasets we evaluate our proposed method on five popular re datasets. test example 1 p re he was the captain of ... team that won the part of: 0.3 ( √ ) participant of: 0.7 1950 world cup after beating ... in the final round match known as the "maracanazo". retrieved nearest neighbors gold label d @xmath0.9 with turkish settlers from turkey and ... it was succeeded as danish representative at participant of 5 → 0.03 2006 contest by sidsel ben semmane with ... test example 2 p re south air command of the indian air force subsidiary: 0.478 ( √ ) has part: 0.496 is headquartered in the city retrieved nearest neighbors gold label d @xmath1.6 of mary, joseph, and the infant jesus, in an enclosed garden, symbolizing mary 's virginity. fidobank (formerly seb bank) is a bank of ukraine subsidiary 3 → 0.3 that until 2012 belonged to the swedish seb group. ## conclusion we propose knn-re: a flexible knn framework with different memory settings for solving implicit expression and long-tail relation issues in re. the results show that our knn-re with training memory outperforms vanilla re model and achieves sota f1 scores on three datasets. in the ds setup, knn-re also outperforms sota ds pre-training methods significantly without extra training.