Improving the Robustness of Distantly-Supervised Named Entity Recognition via Uncertainty-Aware Teacher Learning and Student-Student Collaborative Learning (2024)

Abstract: Distantly-Supervised Named Entity Recognition (DS-NER) effectively alleviates the burden of annotation, but meanwhile suffers from the label noise. Recent works attempt to adopt the teacher-student framework to gradually refine the training labels and improve the overall robustness. However, we argue that these teacher-student methods achieve limited performance because the poor calibration of the teacher network produces incorrectly pseudo-labeled samples, leading to error propagation. Therefore, we attempt to mitigate this issue by proposing: (1) Uncertainty-Aware Teacher Learning that leverages the prediction uncertainty to reduce the number of incorrect pseudo labels in the self-training stage; (2) Student-Student Collaborative Learning that allows the transfer of reliable labels between two student networks instead of indiscriminately relying on all pseudo labels from its teacher. This approach further enables a full exploration of mislabeled samples rather than simply filtering unreliable pseudo-labeled samples. We evaluate our proposed method on five DS-NER datasets, demonstrating that our method is superior to the state-of-the-art DS-NER denoising methods.

## introduction named entity recognition (ner) aims to locate and classify named entities in text, which plays an important role in many applications such as dialogue systems @xcite @xcite . however, deep learningbased ner methods usually require a substantial quantity of high-quality annotation for training models, which is exceedingly costly. therefore, distantly-supervised named entity recognition (ds-ner) is widely used in real-world scenarios, ## task definition given the training corpus d ds where each sample (x i , y i ), x i represents i-th token, and y i is the label. each entity is a span of the text, associated with an entity type. we use the bio scheme for sequence labeling. the beginning token of an entity is labeled as b-type, and others are i-type. the non-entity tokens are labeled as o. traditional ner is a supervised learning task based on a clean dataset. we focus on the practical scenario where the training labels are noisy due to distant supervision, i.e., the revealed tag y i may not correspond to the underlying correct one. thus, the challenge of ds-ner is to reduce the negative effect of noisy annotations. ## methodology as shown in figure 2 , censor consists of two teacher-student networks to handle the noisy label. to avoid overfitting the incorrect pseudo labels generated by poorly calibrated teacher networks, we introduce uncertainty-aware teacher learning that leverages the prediction uncertainty to guide the label selection. we also propose student-student collaborative learning that allows reliable label transfer between two student networks, further reducing the risk of learning incorrect pseudo labels and making a full use of mislabeled samples. ## related work to alleviate the burden of annotation, previous studies attempted to annotate ner datasets via distant supervision, which suffers from noisy annotation. ## conclusion we introduce censor, a novel teacher-student framework designed for ds-ner. censor firstly incorporates uncertainty-aware teacher learning (utl), utilizing prediction uncertainty to guide the pseudo-label selection. it mitigates the usage of incorrect pseudo labels by avoiding reliance on confidence scores from poorly calibrated teacher networks. we also introduce student-student collaborative learning (scl) to enable a student network not to completely rely on pseudo labels from its teacher network, minimizing the risk of learning incorrect ones. meanwhile, this component also ensures that the training set is comprehensively utilized. our experimental results substantiate that censor achieves superior performance compared to prior methodologies.