Named Entity Recognition without Labelled Data: A Weak Supervision Approach (2020)

Abstract: Named Entity Recognition (NER) performance often degrades rapidly when applied to target domains that differ from the texts observed during training. When in-domain labelled data is available, transfer learning techniques can be used to adapt existing NER models to the target domain. But what should one do when there is no hand-labelled data for the target domain? This paper presents a simple but powerful approach to learn NER models in the absence of labelled data through weak supervision. The approach relies on a broad spectrum of labelling functions to automatically annotate texts from the target domain. These annotations are then merged together using a hidden Markov model which captures the varying accuracies and confusions of the labelling functions. A sequence labelling model can finally be trained on the basis of this unified annotation. We evaluate the approach on two English datasets (CoNLL 2003 and news articles from Reuters and Bloomberg) and demonstrate an improvement of about 7 percentage points in entity-level F1 scores compared to an out-of-domain neural NER model.

## introduction named entity recognition (ner) constitutes a core component in many nlp pipelines and is employed in a broad range of applications such as information extraction @xcite , question answering @xcite , document de-identification @xcite , machine translation @xcite and even conversational models @xcite . given a document, the goal of ner is to identify and classify spans referring to an entity belonging to pre-specified categories such as persons, organisations or geographical locations. ner models often rely on convolutional or recurrent neural architectures, sometimes completed by a crf layer @xcite @xcite . more recently, deep contextualised representations relying on bidirectional lstms @xcite , transformers @xcite or contextual string embeddings @xcite have also been shown to achieve state-of-the-art performance on ner tasks. these neural architectures require large corpora annotated with named entities, such as ontonotes @xcite or conll 2003 @xcite . when only modest amounts of training data are available, transfer learning approaches can transfer the knowledge acquired from related tasks into the target domain, using techniques such as simple transfer @xcite , discriminative fine-tuning @xcite , adversarial transfer @xcite or layer-wise domain adaptation approaches @xcite . however, in many practical settings, we wish to apply ner to domains where we have no labelled data, making such transfer learning methods difficult to apply. this paper presents an alternative approach using weak supervision to bootstrap named entity recognition models without requiring any labelled data from the target domain. the approach relies on labelling functions that automatically annotate documents with named-entity labels. a hidden markov model (hmm) is then trained to unify the noisy labelling functions into a single (probabilistic) annotation, taking into account the accuracy and confusions of each labelling function. finally, a sequence labelling model is trained using a cross-entropy loss on this unified annotation. as in other weak supervision frameworks, the labelling functions allow us to inject expert knowledge into the sequence labelling model, which is often critical when data is scarce or non-existent @xcite . new la-belling functions can be easily inserted to leverage the knowledge sources at our disposal for a given textual domain. furthermore, labelling functions can often be ported across domains, which is not the case for manual annotations that must be reiterated for every target domain. the contributions of this paper are as follows: ## approach the proposed model collects weak supervision from multiple labelling functions. each labelling function takes a text document as input and outputs a series of spans associated with ner labels. these outputs are then aggregated using a hidden markov model (hmm) with multiple emissions (one per labelling function) whose parameters are estimated in an unsupervised manner. finally, the aggregated labels are employed to learn a sequence labelling model. figure 1 illustrates this process. the process is performed on documents from the target domain, e.g. a corpus of financial news. labelling functions are typically specialised to detect only a subset of possible labels. for instance, a gazetteer based on wikipedia will only detect mentions of persons, organisations and geographical locations and ignore entities such as dates or percents. this marks a departure from existing aggregation methods, which are originally designed for crowd-sourced data and where annotators are supposed to make use of the full label set. in addition, unlike previous weak supervision approaches, we allow labelling functions to produce probabilistic predictions instead of deterministic values. the aggregation model described in section 3.2 directly captures these properties in the emission model associated with each labelling function. we first briefly describe the labelling functions integrated into the current system. we review in section 3.2 the aggregation model employed to combine the labelling predictions. the final labelling model is presented in section 3.3. the complete list of 52 labelling functions employed in the experiments is available in appendix a. ## evaluation we evaluate the proposed approach on two englishlanguage datasets, namely the conll 2003 dataset and a collection of sentences from reuters and bloomberg news articles annotated with named entities by crowd-sourcing. we include a second dataset in order to evaluate the approach with a more fine-grained set of ner labels than the ones in conll 2003. as the objective of this paper is to compare approaches to unsupervised domain adaptation, we do not rely on any labelled data from these two target domains. ## conclusion this paper presented a weak supervision model for sequence labelling tasks such as named entity recognition. to leverage all possible knowledge sources available for the task, the approach uses a broad spectrum of labelling functions, including data-driven ner models, gazetteers, heuristic functions, and document-level relations between entities. labelling functions may be specialised to recognise specific labels while ignoring oth- well repairs to lift heidrun product loc oil output -statoil company . oslo gpe 1996-08-22 date cardinal three cardinal plugged water injection wells on the heidrun product loc company oilfield off mid-norway will be reopened over the next month date , operator den norske stats company oljeselskap person as org ( statoil company ) said on thursday date . the plugged wells have accounted for a dip of 30,000 cardinal barrels quantity per day ( bpd ) in heidrun loc output to roughly 220,000 cardinal bpd quantity , according to the company 's status weekly org newsletter . the wells will be reperforated and gravel will be pumped into the reservoir through one cardinal time of the wells to avoid plugging problems in the future , it said . -oslo gpe newsroom neural models: ontonotes-trained ner ;gazetteers: company uncased ; heuristic functions: date detector, snips, and number detector ; document level functions: doc majority uncased ; aggregated predictions: hmm-aggregated model proper nnp compound misc legal company full_name btc sec wiki geo crunchbase product doc_history doc_majority proper nnp compound misc legal company full_name btc sec wiki geo crunchbase product doc_history doc_majority proper nnp compound misc legal company full_name btc sec wiki geo crunchbase product doc_history doc_majority proper nnp compound misc legal company full_name btc sec wiki geo crunchbase product doc_history doc_majority 0.00 0.25 0.50 0.75 1.00 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 ers. furthermore, unlike previous weak supervision approaches, labelling functions may produce probabilistic predictions. the outputs of these labelling functions are then merged together using a hidden markov model whose parameters are estimated with the baum-welch algorithm. a neural sequence labelling model can finally be learned on the basis of these unified predictions. evaluation results on two datasets (conll 2003 and news articles from reuters and bloomberg) show that the method can boost ner performance by about 7 percentage points on entity-level f 1 . in particular, the proposed model outperforms the unsupervised domain adaptation approach through contextualised embeddings of @xcite . of specific linguistic interest is the contribution of document-level labelling functions, which take advantage of the internal coherence and narrative structure of the texts. future work will investigate how to take into account potential correlations between labelling functions in the aggregation model, as done in e.g. @xcite . furthermore, some of the labelling functions can be rather noisy and model selection of the optimal subset of the labelling functions might well improve the performance of our model. model selection approaches that can be adapted are discussed in @xcite ; @xcite . we also wish to evaluate the approach on other types of sequence labelling tasks beyond named entity recognition.