Improving Low-resource Named Entity Recognition with Graph Propagated Data Augmentation (2023)

Abstract: Data augmentation is an effective solution to improve model performance and robustness for low-resource named entity recognition (NER). However, synthetic data often suffer from poor diversity, which leads to performance limitations. In this paper, we propose a novel Graph Propagated Data Augmentation (GPDA) framework for Named Entity Recognition (NER), leveraging graph propagation to build relationships between labeled data and unlabeled natural texts. By projecting the annotations from the labeled text to the unlabeled text, the unlabeled texts are partially labeled, which has more diversity rather than synthetic annotated data. To strengthen the propagation precision, a simple search engine built on Wikipedia is utilized to fetch related texts of labeled data and to propagate the entity labels to them in the light of the anchor links. Besides, we construct and perform experiments on a real-world low-resource dataset of the E-commerce domain, which will be publicly available to facilitate the low-resource NER research. Experimental results show that GPDA presents substantial improvements over previous data augmentation methods on multiple low-resource NER datasets.

## introduction data augmentation is an effective solution to improve model performance and robustness, and is especially useful when the labeled data is scarce. in computer vision and speech, simple hand-crafted manipulations @xcite are widely used to generate synthetic data that preserve the original information. however, when applied to natural language processing (nlp), it is challenging to edit a sentence without changing its syntax or semantics. there are two successful attempts of applying data augmentation on sentence-level nlp tasks. one is manipulating a few words in the original sentence, which can be based on synonym replacement @xcite @xcite , random insertion or deletion @xcite , random swap @xcite @xcite . the other is generating the whole sentence with the help of backtranslation @xcite @xcite , sequence to sequence models @xcite or pre-trained language models @xcite . however, when applied to token-level tasks such as ner, these methods suffer heavily from token-label misalignment or erroneous label propagation. to overcome the issue of token-label misalignment, dai and adel (2020) extend the replacement from token-level to entity-level with entities of the same class, which proves to be a simple but strong augmentation method for ner. @xcite adopt a seq2seq model to conditionally generate contexts while leaving entities / aspect terms unchanged. @xcite exploit an auto-regressive language model to annotate entities while treating ner as a text tagging task. @xcite utilize labeled sequence linearization to enable masked entity language model to explicitly condition on label information when predicting masked entity tokens. still, these methods generate synthetic data, which inevitably introduces incoherence, semantic errors and lacking in diversity. in this work, we investigate data augmentation with natural texts instead of synthetic ones. we are inspired by the fact that professional annotators usually understand the semantics of an entity through its rich context. however, in low-resource ner, the semantic information of a specific entity is relatively limited due to fewer annotations. to this end, we propose to improve the ner models by mining richer contexts for the existing labeled entities. more particularly, we propose a graph propagation based data augmentation (gpda) framework for ner, leveraging graph propagation to build relationships between labeled data and unlabeled natural texts. the unlabeled texts are accurately and partially labeled according to their connected labeled data, which has more diversity rather than synthetic hand-crafted annotations. furthermore, not restricted to the existing annotated entities in the training data, we explore external entities from the unlabeled text by leveraging consistency-restricted self-training. the contributions of gpda can be concluded: ## method fig. 1 presents the workflow of our proposed data augmentation framework. first, we build a graph between labeled data nodes and unlabeled text nodes according to their textual similarity. then, the entity annotations are propagated to obtain augmented data. finally, the marginalized likelihood for conditional random field (crf) @xcite is applied during the training phase as the augmented data are partially labeled. moreover, we adopt the consistency-restricted self-training strategy to further improve the model performance. ## discussion retrieving relevant texts from databases has been widely used in nlp tasks. raner @xcite retrieves context using a search system to enhance the token representation for ner tasks. to help entity disambiguation in domain-specific ner, @xcite retrieves the domainspecific database to find the correlated sample. in order to leverage the extensive information about entities in wikipedia and wikidata, @xcite and @xcite construct databases and retrieve context to enhance model performance. in this study, we propose the utilization of retrieval techniques for data augmentation in low-resource settings. furthermore, while they perform retrieval on both the training and testing datasets, we only use the small seed training dataset for retrieval. it's noteworthy that our approach can also be combined with theirs to further enhance the performance of ner in low-resource settings. ## conclusion we present gpda as a data augmentation framework for low-resource ner, which utilizes graph propagation with natural texts for augmentation. to make the most efficient utilization of the explored partially labeled text, we adopt consistencyrestricted self-training. experiment results show