# Distantly Supervised Named Entity Recognition with Self-Adaptive Label Correction
## 1 Introduction  
Named Entity Recognition (NER) aims to detect and classify Named Entities (NEs) mentioned in unstructured text into predefined categories, such as location, person, organization, etc. It is a foundational task in Natural Language Processing (NLP) and can benefit many downstream tasks, e.g., relation extraction, event extraction and question answering systems. The prevalence of deep neural networks has greatly advanced the research in NER [1, 2, 3, 4]. Such success can be largely attributed to the large amounts of data with high-quality manual annotations. In practice, such annotations are expensive or even infeasible. On the contrary, with the help of a knowledge base (e.g., Wikidata [5] and Yago [6]), training labels can be automatically generated [7]. Nevertheless, they inevitably involve noisy labels, as illustrated in Figure 1. Unfortunately, the strong memorization power of deep neural networks makes them susceptible to the presence of noisy labels and overfitted to corrupted labels [8], leading to poor generalization. Even popular regularization techniques, such as data augmentation [9], weight decay [10], dropout [11] and batch normalization [12], fail to narrow the gap in performance between fully supervised and distantly supervised NER models. Attempts have been made in designing noisy label learning algorithms for distantly supervised NER [13, 14, 15]. These methods alleviated the negative effect of noisy labels by discarding or reweighting mislabeled training data, but failed to fully exploit useful information from the mislabeled data. As the noisy label learning study [16] proved, a classifier trained on noisy labels has the ability to identify whether a label has been corrupted. As another proof of this study, they verify that leveraging a simple label-correction algorithm has a guaranteed success rate and recovers the correct labels of mislabeled data with high probability. Hence, this work explores how to better utilize mislabeled data by introducing label correction for distantly supervised NER. In this work, a unified self-adaptive learning framework termed SALO (Self-Adaptive Label cOrrection) is proposed that can dynamically detect corrupted labels and progressively correct them. SALO consists of three key components: implicit noise denoising, a noisy label detector and a pseudolabel estimator. Implicit noise denoising is devised to produce the supervision signals in each training step. A labeled token selection strategy is introduced with denoising classifiers to select consistent and high-confidence labels for hard supervision signals. A noisy label detector dynamically estimates a soft target distribution at each time step via a pretrained NER classifier, which intrinsically splits hard supervision signals into two classes (i.e., high-quality instances and problematic instances). Depending on the maximum probability of the distribution, problematic instances are under corrected label supervision or left intact. By doing so, the built model is prevented from being overconfident and fully mines valuable information from mislabeled data. A pseudolabel estimator is devised to regenerate the type and position of labels, respectively, based on a noisy example detector. Specifically, a label correction mechanism is introduced to smooth out problematic instances and even completely change the training labels if necessary. An iterative training algorithm is further designed to take full advantage of these data-correction processes, which significantly boost performance. The main contributions are as follows:  
* A self-adaptive learning framework termed SALO is proposed to improve the learning of NER models by dynamically incorporating adaptive label correction into training. An iterative training algorithm continuously optimizes the model while correcting noisy labels. * A denoising classifier and a noisy label detector are introduced to identify noisy labels.