C o A ug: Combining Augmentation of Labels and Labelling Rules (2023)

Abstract: Collecting labeled data for Named Entity Recognition (NER) tasks is challenging due to the high cost of manual annotations. Instead, researchers have proposed few-shot self-training and rule-augmentation techniques to minimize the reliance on large datasets. However, inductive biases and restricted logical language lexicon, respectively, can limit the ability of these models to perform well. In this work, we propose CoAug, a co-augmentation framework that allows us to improve few-shot models and rule-augmentation models by bootstrapping predictions from each model. By leveraging rules and neural model predictions to train our models, we complement the benefits of each and achieve the best of both worlds. In our experiments, we show that our best CoAug model can outperform strong weak-supervision-based NER models at least by 6.5 F1 points.

## introduction named entity recognition (ner) is the task of identifying entity spans of specific types in a given document. while deep learning has led to the development of highly performant supervised ner models @xcite @xcite , their performance is contingent on the availability of high-quality large labeled datasets, which is often expensive to collect. moreover, it is impractical to assume the availability of large datasets for all domains. hence, learning from limited labeled data is a pressing challenge in named entity recognition research. the majority of research in this area can be broadly classified into two distinct paradigms: few-shot learning with pre-trained language models (lms) and weak supervision methods that utilize heuristic rules for entity extraction. in few-shot learning, models are trained to identify novel entities given just a few labeled examples for each entity type. while pretrained lms have been explored for this setting, their susceptibility to overfitting on small datasets results in poor performance. consequently, recent works improve recognition using prototypical networks @xcite , improved representations from self-supervised pre-training of lms @xcite , and self-training @xcite . in the iterative learning process of self-training, many candidate entities are extracted and added into the training set for future iterations. however, premature models from initial iterations also add erroneous entities to the training set, resulting in models whose performance lags behind fully-supervised models that utilize large labeled datasets. on the other hand, rule-based weak supervision methods utilize heuristic rules and manual lexicons @xcite developed by domain experts to supervise entity recognition models. however, experts may find it challenging to enumerate all possible heuristics, which can limit the diversity of identified entities in docu-ments. in recent work, tallor @xcite overcomes this limitation by automatically learning rules given unlabeled data and an initial set of seed rules (tens of rules). nonetheless, while rule-based methods offer high precision, their performance is constrained by the logical language specified by the developer, which limits the set of identifiable entities. moreover, learning rules can fail to identify entities in new linguistic contexts that would otherwise be known. we hypothesize that the two paradigms of fewshot learning and rule-based weak supervision can effectively complement each other, as neural models are skilled at identifying candidates from different linguistic contexts but lack precision, while rulebased methods can identify accurate candidates with precision but lack the flexibility to identify entities in different contexts. therefore, in this work, we propose co-augmentation (coaug), as shown in figure 1 , an iterative bootstrapping framework that effectively combines neural models, rule-based weak supervision methods, and unlabeled data. our proposed framework draws inspiration from co-training @xcite , but it has its own unique approach. like co-training, coaug aims to combine two distinct inductive biases in limited labeled data settings. unlike co-training, instead of improving two models that use different feature sets individually by bootstrapping labels from each other, coaug accomplishes the same goal by using two models that use different forms of supervision to expand the same label set. additionally, in each iteration of coaug, both classifiers are trained with the predictions made by both models, rather than just one. our choice allows the framework to function from really small initial training sets for the individual models. we evaluate our approach on four named entity recognition datasets that span general and science domains. our results indicate that (a) coaug consistently improves performance over self-training ruleaugmentation and few-shot models while being highly precise, (b) utilizing stronger pre-training for the neural models leads to improved performance of models in our framework. in summary, our contributions are as follows: in this work, we consider a setting where we have access to an initial set of seed rules, s, and a large unlabeled corpus, u, to perform the named entity recognition task. applying the rules, s, on u provides the initial set of labeled examples, l, to train models in our framework. our framework, coaug (short for co-augmentation), iteratively improves the performance of two models by leveraging the bootstrapped predictions on unlabeled data by each model. given that prior work in low-resource ner focuses on two parallel tracks of rule-augmentation and few-shot learning methods that do not interact with each other, we instantiate coaug with a rule-augmentation model and a few-shot model to leverage the best of both paradigms. we refer to these components of our framework as rule augmenter and label augmenter (figure 1 ). in the subsections below, we describe the rule augmenter and label augmenter modules. ## conclusion in this work, we introduce coaug, a coaugmentation framework that utilizes unlabeled data to train rule-augmentation and neuralaugmentation models to become better ner taggers. our results on datasets from two domains demonstrate the effectiveness of coaug for lowresource domains. our analysis reveals that coaug is able to perform better than weak-supervision methods like tallor because of an ability to find more positive instances while maintaining high precision. further analysis shows the importance of factors such as the strength of pre-training that can contribute towards the success of models in domain-specific datasets.