GDA : Generative Data Augmentation Techniques for Relation Extraction Tasks (2023)

Abstract: Relation extraction (RE) tasks show promising performance in extracting relations from two entities mentioned in sentences, given sufficient annotations available during training. Such annotations would be labor-intensive to obtain in practice. Existing work adopts data augmentation techniques to generate pseudo-annotated sentences beyond limited annotations. These techniques neither preserve the semantic consistency of the original sentences when rule-based augmentations are adopted, nor preserve the syntax structure of sentences when expressing relations using seq2seq models, resulting in less diverse augmentations. In this work, we propose a dedicated augmentation technique for relational texts, named GDA, which uses two complementary modules to preserve both semantic consistency and syntax structures. We adopt a generative formulation and design a multi-tasking solution to achieve synergies. Furthermore, GDA adopts entity hints as the prior knowledge of the generative model to augment diverse sentences. Experimental results in three datasets under a low-resource setting showed that GDA could bring 2.0% F1 improvements compared with no augmentation technique.

## introduction relation extraction (re) aims to extract semantic relations between two entities mentioned in sentences and transform massive corpora into triplets in the form of @xcite . neural relation extraction models show promising performance when high-quality annotated data is available @xcite @xcite . while in practice, human annotations would be labor-intensive and time-consuming to obtain and hard to scale up to a large number of relations @xcite @xcite . this motivates us to solicit data augmentation techniques to generate pseudo annotations. a classical effort devoted to data augmentation in nlp is adopting rule-based techniques, such as synonym replacement @xcite , random deletion @xcite , random swap @xcite and dependency tree morphing @xcite . however, these methods generate synthetic sentences without considering their semantic consistencies with the original sentence, and may twist semantics due to the neglection of syntactic structures. other successful attempts on keeping the semantic consistency of the sentences are modelbased techniques. the popular back translation method @xcite generates synthetic parallel sentences using a translation model to translate monolingual sentences from the target language to the source language. however, it works exclusively on sentence-level tasks like text classification and translation, which is not designed to handle fine-grained semantics in entity-level tasks like relation extraction. @xcite design a specific method for re tasks by fine-tuning gpt-2 to generate sentences for specific relation types. however, it cannot be used in practice because the model generates less diverse sentences -it includes similar entities and identical relational expressions under the same relation. to keep the generated sentences diverse while semantically consistent with original sentences, we propose a relational text augmentation technique named gda. as illustrated in figure 1 , we adopt the multi-task learning framework with one shared encoder and two decoders that are complementary with each other: one decoder aims to predict the original sentence by restructuring words in the syntactic structure, which can maintain the semantics of the original sentence and ensure the model has the ability to generate semantically consistent target sentence. however, restructuring the syntactic ## proposed data augmentation technique the proposed data augmentation technique gda consists of two steps: 1) train a seq2seq generator. 2) generate pseudo sentences. as illustrated in figure 1 , the first step adopts t5 @xcite consisting of encoder and decoder parts as the seq2seq generator (θ). the generator learns to convert two sentences with the same re-lation label. specifically, the encoder part takes a sentence @xmath0, x 2 , ..., x tx ) as input where named entities are recognized and marked in advance, and obtains the contextualized token embeddings @xmath1, h 2 , ..., h tx ). the decoder part takes the h as input and generates target sentence @xmath2, y 2 , ..., y ty ) word by word by maximizing the conditional probability distribution of p(y i |@xmath3, h, θ). the second step randomly selects an annotated sentence as input, and leverages the trained generator to generate pseudo sentence with entity marker and same relation label. now, we introduce the details of each step. ## experiments we conduct extensive experiments on three public datasets and low-resource re setting to show the effectiveness of gda and give a detailed analysis to show its advantages. ## conclusions and future works in this paper, we propose a relational text augmentation technique: gda for re tasks. unlike conventional data augmentation techniques, our technique adopts the multi-task learning framework to generate diverse, coherent, and semantic consistent augmented sentences. we further adopt entity hints as prior knowledge for diverse generation. experiments on three public datasets and low-resource settings could show the effectiveness of gda. for future research directions, we can try to explore more efficient pre-ordering and parsing methods, and apply our data augmentation methods to more nlp applications, such as semantic parsing @xcite , natural language inference @xcite .