Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction (2020)

Abstract: Current supervised relational triple extraction approaches require huge amounts of labeled data and thus suffer from poor performance in few-shot settings. However, people can grasp new knowledge by learning a few instances. To this end, we take the first step to study the few-shot relational triple extraction, which has not been well understood. Unlike previous single-task few-shot problems, relational triple extraction is more challenging as the entities and relations have implicit correlations. In this paper, We propose a novel multi-prototype embedding network model to jointly extract the composition of relational triples, namely, entity pairs and corresponding relations. To be specific, we design a hybrid prototypical learning mechanism that bridges text and knowledge concerning both entities and relations. Thus, implicit correlations between entities and relations are injected. Additionally, we propose a prototype-aware regularization to learn more representative prototypes. Experimental results demonstrate that the proposed method can improve the performance of the few-shot triple extraction.

## introduction relational triple extraction is an essential task in information extraction for natural language processing (nlp) and knowledge graph (kg) @xcite , which is aimed at detecting a pair of entities along with their relation from unstructured text. for instance, there is a sentence "paris is known as the romantic capital of france.", and in this example, an ideal relational triple extraction system should extract the relational triple paris, capital of, france , in which capital of is the relation of paris and france. current works in relational triple extraction typically employ traditional supervised learning based on feature engineering @xcite and neural networks @xcite . the main problem with supervised learning models is that they can not perform well on unseen entity types or relation categories (e.g., train a model to extract knowledge triples from the economic text, then run this model to work on scientific articles). as a result, supervised relational triple extraction can not extend to the unseen entity or relation types. a trivial solution is to annotate more data for unseen triple types, then retraining the model with newly annotated data @xcite . however, this method is usually impractical because of the extremely high cost of annotation. intuitively, humans can learn about a new concept with limited supervision, e.g., one can detect and classify new entities with 3-5 examples @xcite . this motivates the setting that we aim at for relational triple extraction: few-shot learning (fsl). in few-shot learning, a trained model rapidly learns a new concept from a few examples while keeping great generalization from observed examples @xcite . hence, if we need to extend relational triple extraction into a new domain, a few examples are needed to activate the system in the new domain without retraining the model. by formulating this fsl relational triple extraction, we can significantly reduce the annotation cost and training cost while maintaining highly accurate results. they continued as members of lds church, under the direction of brigham young. one of their sons, shimon peres, became a catholic priest. hamilton was the fifth bishop of minnesota in the episcopal church. though methods of few-shot learning develop fast in recent yeas, most of these works concentrate on single tasks such as relation extraction and text classification @xcite . however, the effect of joint extraction of entities and relations on low-resource scenarios is still not well understood, which are two subtasks belonging to relational triple extraction. unlike extraction for each single task, joint entity and relation extraction is more challenging, as entity and relations have implicit correlations, which cannot be ignored. to address this issue, we propose a multi-prototype embedding network (mpe) model to extract the few-shot relational triples, inspired by the prototypical network @xcite . to be specific, we utilize two kinds of prototypes regarding both entities and relations. note that entity pairs and relations have explicit knowledge constraints @xcite , such as the born in relation suggests that the type of head entity must be person, and vice versa. based on those observations and motivated by the knowledge graph embedding @xcite , we introduce the hybrid prototypical learning to explicitly inject knowledge constraints. we firstly learn entity and relation prototypes and then leverage translation constraint in hyperspace to regularize prototype embedding. note that such knowledge-aware regularization not only injects prior knowledge from the external knowledge graph, but also leads to a more smooth and representative prototype for few-shot extraction. moreover, we introduce prototype-regularization considering both intramural and mutual similarities between different prototypes. experimental results on the fewrel dataset @xcite demonstrate that our approach outperforms baseline models in the few-shot setting. to summarize, our main contributions include: two main directions have been proposed for relational triple extraction, which has two subtasks: entity extraction and relation extraction, namely, pipeline @xcite @xcite @xcite and joint learning methods @xcite @xcite . the pipeline model can be more flexible because it extracts entity pairs and relations sequentially, but this design will lead to error propagation @xcite . meanwhile, joint relational triple extraction models can solve this problem well by extracting triples end-to-end, and the interaction between entities and relations can be realized within the model, which makes the performance of the two mutually enhanced. however, due to the "data-hungry" attribute of conventional neural networks, these relational triple extraction models need a large amount of data for training. thus, lots of efforts @xcite @xcite have been devoted to few-shot learning, @xcite presents a few-shot relation extraction datasets to promote the research of information extraction in few-shot scenarios and adapt some few-shot learning methods @xcite @xcite for this task. among these models, the prototypical network @xcite achieves comparable results on several few-shot learning benchmarks; meanwhile, it is simple and effective. this model assumes that each class exists a prototype, and it tries to find the prototypes for classes from supporting instances and compares the distance between the query instance under a particular distance metric. in natural language processing, @xcite first proposes a hybrid attention-based prototypical network for few-shot relation extraction. @xcite proposes to utilize the prototypical network to tackle the few-shot named entity recognition. @xcite proposes a collapsed dependency transfer mechanism and a label-enhanced task-adaptive projection network (l-tapnet) for few-shot slot filing. however, all previous few-shot works mainly consider single tasks, while relational triple extraction should take both entity and relation into consideration. to the best of our knowledge, we are the first approach for the few-shot relational triple extraction, which addresses both entities and relations. our work is motivated by knowledge graph embedding @xcite such as transe @xcite from knowledge graph (kg), which is composed of many relational triples like head, relation, tail . transe is first proposed by @xcite to encode triples into a continuous low-dimensional space, which is based on the translation law h + r ≈ t. many follow-up works like transh @xcite , distmult @xcite , and transr @xcite , propose advanced methods of translation by introducing different embedding spaces. in few-shot settings, it is extremely challenging to inject implicit knowledge constrains in vector space. such simple yet effective knowledge constraints provide an intuitive solution. ## conclusion and future work in this paper, we study the few-shot relational triple extraction problem and propose a novel multiprototype embedding network that bridge text representation learning and knowledge constraints. extensive experimental results prove that our model is effective, but remains challenges. those empirical findings shed light on promising future directions, including 1) enhancing entity recognition with effective sequence decoders; 2) studying few-shot relational triple extraction with more triples in a single sentence; 3) injecting logic rules to enable robust extraction; and 4) developing few-shot relational triple extraction benchmarks.