Generating Labeled Data for Relation Extraction: A Meta Learning Approach with Joint GPT -2 Training (2023)

Abstract: Relation Extraction (RE) is the task of identifying semantic relation between real-world entities mentioned in text. Despite significant progress in RE research, a remaining challenge for RE concerns the lack of training data for data-hungry deep learning models. Cost of annotation and difficulty of the task are among hindrance to collect a large-scale RE dataset in different domains. To address this limitation, we propose a novel framework to automatically generate labeled data for RE. Our framework presents the pre-trained language model GPT-2 for data generation. In addition, to optimize the generated samples for an RE model, we introduce a meta learning approach to allow the GPT-2 model to be updated during the training process for RE. In particular, to leverage the feedback from the RE model to improve the data generation from GPT-2, we propose a novel reward function to update the GPT-2 model with REINFORCE, seeking to promote the similarity of the RE loss function’s gradients computed for generated data and a meta development set. We conduct extensive experiments on two benchmark datasets to produce state-of-the-art performance for RE.

## introduction one of the fundamental tasks in information extraction (ie) involves relation extraction (re) that aims to identify semantic relations between two entities mentioned in textual data. for instance, in the sentence "after xzy's decision to move to europe, they selected paris as the final location for their headquarters.", the semantic relation part-whole between two entity mentions "europe" and "paris" should be detected. an re system can be employed to populate a knowledge base with relations among entities, provide information for question answering systems, and present facts for text summerization tools. due to the importance of re, in recent years various methods and models have been proposed for this task. these models can be categorized into feature-based @xcite @xcite @xcite @xcite and deep learning @xcite @xcite @xcite @xcite models. the existing models provide solutions for re in various settings including monolingual @xcite , cross-lingual @xcite , cross-domain @xcite , and joint models @xcite . despite those progress, one limitation that hinders on-going research for re is labeled data scarcity. annotating a large-scale re dataset is challenging, due to the expensive nature of annotation task and the high requirement for expertise in specific domains. as such, prior methods have resorted to distantly supervised setting @xcite @xcite or pseudo labeling techniques @xcite that leverage vast amounts of unlabeled data to address the labeled data scarcity issue for re. although these methods are helpful to substantially increase the size of re datasets, they also introduce massive noisy samples which might hurt the training of an re model. consequently, creating cost-efficient large-scale labeled datasets for specific domains remains highly challenging for re. to achieve large-scale labeled datasets, in this work we introduce a novel data augmentation method to automatically generate labeled data for re. in particular, instead of using unlabeled data, we propose to employ the pre-trained language model gpt-2 @xcite to generate synthetic labeled data for re. in our method, the gpt-2 model is first fine-tuned on available manually labeled re datasets. concretely, the language model is trained on the label-augmented sentences in which positive and negative re samples are marked with special tags surrounding two input entity mentions. next, the fine-tuned gpt-2 model is employed to generate new label-augmented indomain sentences that can be map back to produce new labeled data for re. the new labeled data is then combined with the original manually labeled data to train an re model. however, an issue with this approach involves the separation between the fine-tuning process of gpt-2 and the target re model that might cause a mismatch between the generated data from gpt-2 and the data expected by the re model (e.g., the generated data can be noisy or redundant for re). as such, to improve the effectiveness of the generated data for an re model, we propose to further optimize gpt-2 parameters during the training of the re model, thus enabling the interactions between the gpt-2 and re models to generate optimal/customized data for re. in particular, we propose a meta learning framework to treat the parameters of the gpt-2 model as meta-parameters for the re model that will be fine-tuned based on the performance of the re model on a separate meta development set. to leverage the performance on meta development set to optimize gpt-2 parameters, one solution is to employ reinforcement learning where the rewards for the generated sentences can be directly based on some performance metric (e.g., f1 score). however, due to the small size of the available data, this reward can lead to unstable training with high variance. to remedy this issue, in this work we propose a novel reward function that instead relies on gradients of the re model's loss to produce more robust training signals. in particular, our intuition is that a generated sample should have a higher reward if the direction in which the re model should be updated to perform well on the sample and the development data are similar. to fulfil this objective, in the proposed training procedure, after one iteration of training, we first compute the average gradient of the re model's loss function over the meta development set. next, the gradient of the loss of the re model over a generated sample is computed. finally, the reward for the generated sample is obtained via the cosine similarity between the gradients from the development set and the generated sample. while this reward is backed up with intuitive objectives, we also provide mathematical derivation of the reward based on bi-level optimiza-tion to further demonstrate the advantages of our method. finally, we evaluate the effectiveness of the proposed method on two benchmark datasets for re. the experiments show the superiority of the proposed model compared to strong baselines. ## model task definition: we study the problem of sentencelevel relation extraction. in this setting, the objective is to identify semantic relation between two input entity mentions in a sentence. formally, the input to our model involve a sentence @xmath0, w 2 , . . . , w n ] and two indices s and o (1 ≤ s, @xmath1. our goal is to predict label y representing semantic relation between the entity mentions w s and w o from a predefined relation label set r (@xmath2. note that if the two entity mentions are not involved in a relation the special label n one is employed. also, for convenience, let o train be the set of available training data for our re problem (i.e., @xmath3. ## related work relation extraction is one of the fundamental tasks in information extraction. due to its importance, various methods have been proposed for re, ranging from feature-based and kernel-based techniques @xcite @xcite @xcite @xcite to recent advanced deep learning models @xcite @xcite @xcite . the typical neural architectures for re include convolutional neural networks @xcite @xcite , recurrent neural networks @xcite @xcite , and self-attentions in transformer @xcite . to address the key challenge of data scarcity for re, prior work has resorted to distantly supervised methods @xcite @xcite or pseudo labeling techniques @xcite . however, such methods suffer from low quality of obtained training data, thus hindering performance for re. also, we note that data augmentation based on gpt-2 has also been explored for other tasks, such as event extraction @xcite @xcite @xcite . compared to such prior work, our work features a new meta learning framework to jointly train gpt-2 with the downstream re model, leveraging gradient agreement-based reward to improve the quality of generated labeled data. ## conclusion we present a novel data augmentation method for re using the pre-trained language model gpt-2. the language model is fine-tuned over labelaugmented texts to generate in-domain and labeled samples for re. to improve the quality of generated data for re, the gpt-2 model is further optimized along the training process of a re model in a novel meta learning framework (i.e., joint training to promote model interaction). agreement scores between gradients of the re loss function over generated data and a meta development set are proposed as the reward to update the gpt-2 model. we conduct extensive experiments on two benchmark datasets to demonstrate the benefits of the proposed method for re. in the future, we will explore the application of the proposed methods to other related tasks in information extraction.