Graph-based Model Generation for Few-Shot Relation Extraction (2022)

Abstract: Few-shot relation extraction (FSRE) has been a challenging problem since it only has a handful of training instances. Existing models follow a ‘one-for-all’ scheme where one general large model performs all individual N-way-K-shot tasks in FSRE, which prevents the model from achieving the optimal point on each task. In view of this, we propose a model generation framework that consists of one general model for all tasks and many tiny task-specific models for each individual task. The general model generates and passes the universal knowledge to the tiny models which will be further fine-tuned when performing specific tasks. In this way, we decouple the complexity of the entire task space from that of all individual tasks while absorbing the universal knowledge.Extensive experimental results on two public datasets demonstrate that our framework reaches a new state-of-the-art performance for FRSE tasks. Our code is available at: https://github.com/NLPWM-WHU/GM_GEN.

## introduction relation extraction (re) aims to detect the implied relations between/among entities mentioned in sentences. it plays a significant role in natural language processing (nlp). massive unstructured text can be transformed into structured factual knowledge using the re technique. however, training re models requires sufficient human-annotated data. the models' performance usually drops dramatically without enough training data. however, in realistic scenarios, the acquisition of highquality annotated data on re is time-consuming and labor-intensive @xcite . to alleviate the labeled data scarcity problem, some researchers employ distant supervision (ds) @xcite or semi-supervised relation extraction (ssre) @xcite 2019 ds and ssre obtain annotation information with the help of knowledge base and model prediction results, respectively. ds assumes that if two entities belong to a relation in kb, all sentences mentioning these entities indicate the relation class in kb. these methods inevitably generate falsepositive and false-negative samples. ssre methods require many unlabeled data and iteratively train a new model using the results on unlabeled data predicted by the trained model and they cannot adapt to the situation with only a few labeled examples either. that is to say, it is still an urgent problem to train a successful model under an extreme data scarcity condition where entire categories are introduced with just one or few examples. a more challenging, yet practical extension, is the few-shot relation extraction (fsre) task which handles the re tasks with a handful of training instances. to better understand fsre, we illustrate the n-way-k-shot scenario in fsre where n and k respectively represent the category number and the samples per category. we provide a 10-way-5-shot example here. given the relation description, 10*5 labeled sentences as a support set, and 10 unlabeled sentences as a query set, the fsre model will predict the relation between two entities mentioned in each sentence of the query set with the help of the support set and possible external knowledge. existing fsre methods often employ metricbased learning techniques to improve fsre performance, which trains a function e ϕ (•) mapping all samples into one same embedding space and then using some distance functions in the embedding space to label query samples. the intuition is that the distance between the query and the instances with the same relation is smaller than those with different relations. more recent fsre methods integrate various information, such as entity description @xcite , relation description @xcite , to enhance the prototypical network. the aforementioned methods try to build a general embedding function learned from all training instances and utilize distance metrics to label instances. however, a general model cannot adapt to the single n-way-k-shot task because there exists a discrepancy between the entire task space and each individual task. the general model can only reach the optimal point in the whole task space instead of each task space. consequently, such a 'one-for-all' learning scheme @xcite , i.e. one model solves all individual nway-k-shot tasks, seriously limits the performance of fsre models. in this paper, we propose a novel model generation framework to overcome the limitation of the 'one-for-all' learning scheme in fsre. the basic idea is to train one general model for all tasks and many tiny models for each individual n-way-kshot task. as shown in figure 1 , we pre-train a general model to generate tiny task-specific models. during the testing phase, the tiny models can be fine-tuned according to the task-specific support samples and labels without affecting the knowledge from the general model. specifically, we believe that the topology information (graph) between the samples and the label representation should also be modeled in addition to the semantic features called attributes of the sample. therefore, we first employ a graph-based model generation module to combine the topology information with the attributes of instances and the relation descriptions. then, the graph-based model generates many tiny classification models which will be fine-tuned and infer on different few-shot tasks. the separation of the general model and task-specific models successfully decouples the complexity of the entire task space from that of all individual tasks yet absorbing the universal knowledge. the contributions of this paper are as follows: ## analysis to get a deep insight into the proposed gm_gen model, we conduct the ablation study, an investigation of hyper-parameter, a complexity analysis, and a visualization experiment. ## conclusions to overcome the limitation of the popular 'onefor-all' learning scheme in current fsre methods, we propose a novel graph-based model generation framework to decouple the complexity of the entire task space from that of each task space. by fine-tuning the generated models on the specific classification task, our method can naturally reach the optimal point in each task. extensive experiments demonstrate the effectiveness of our proposed framework over existing baselines.