Adaptive Self-training for Neural Sequence Labeling with Few Labels

1 INTRODUCTION . Motivation . Deep neural networks typically require large amounts of training data to achieve stateof-the-art performance . Recent advances with pre-trained language models like BERT ( Devlin et al. , 2019 ) , GPT-2 ( Radford et al. , 2019 ) and RoBERTa ( Liu et al. , 2019 ) have reduced this annotation bottleneck . In this paradigm , large neural network models are trained on massive amounts of unlabeled data in a self-supervised manner . However , the success of these large-scale models still relies on fine-tuning them on large amounts of labeled data for downstream tasks . For instance , our experiments show 27 % relative improvement on an average when fine-tuning BERT with the full training set ( 2.5K-705K labels ) vs. fine-tuning with only 10 labels per class . This poses several challenges for many real-world tasks . Not only is acquiring large amounts of labeled data for every task expensive and time consuming , but also not feasible in many cases due to data access and privacy constraints . This issue is exacerbated for sequence labeling tasks that require annotations at token- and slot-level as opposed to instance-level classification tasks . For example , an NER task can have slots like B-PER , I-PER , O-PER marking the beginning , intermediate and out-of-span markers for person names , and similar slots for the names of location and organization . Similarly , language understanding models for dialog systems rely on effective identification of what the user intends to do ( intents ) and the corresponding values as arguments ( slots ) for use by downstream applications . Therefore , fully supervised neural sequence taggers are expensive to train for such tasks , given the requirement of thousands of annotations for hundreds of slots for the many different intents . Semi-supervised learning ( SSL ) ( Chapelle et al. , 2010 ) is one of the promising paradigms to address labeled data scarcity by making effective use of large amounts of unlabeled data in addition to task-specific labeled data . Self-training ( ST , ( III , 1965 ) ) as one of the earliest SSL approaches has recently shown state-of-the-art performance for tasks like image classification ( Li et al. , 2019 ; Xie et al. , 2020 ) performing at par with supervised systems while using very few training labels . In contrast to such instance-level classification tasks , sequence labeling tasks have dependencies between the slots demanding different design choices for slot-level loss optimization for the limited labeled data setting . For instance , prior work ( Ruder & Plank , 2018 ) using classic self-training techniques for sequence labeling did not find much success in the low-data regime with 10 % labeled data for the target domain . Although there has been some success with careful task-specific data selection ( Petrov & McDonald , 2012 ) and more recently for distant supervision ( Liang et al. , 2020 ) using external resources like knowledge bases ( e.g. , Wikipedia ) . In contrast to these prior work , we develop techniques for self-training with limited labels and without any task-specific assumption or external knowledge . For self-training , a base model ( teacher ) is trained on some amount of labeled data and used to pseudo-annotate ( task-specific ) unlabeled data . The original labeled data is augmented with the pseudo-labeled data and used to train a student model . The student-teacher training is repeated until convergence . Traditionally in self-training frameworks , the teacher model pseudo-annotates unlabeled data without any sample selection . This may result in gradual drifts from self-training on noisy pseudo-labeled instances ( Zhang et al. , 2017 ) . In order to deal with noisy labels and training set biases , Ren et al . ( 2018 ) propose a meta-learning technique to automatically re-weight noisy samples by their loss changes on a held-out clean labeled validation set . We adopt a similar principle in our work and leverage meta-learning to re-weight noisy pseudo-labeled examples from the teacher . While prior techniques for learning to re-weight examples have been developed for instance-level classification tasks , we extend them to operate at token-level for discrete sequence labeling tasks . To this end , we address some key challenges on how to construct an informative held-out validation set for token-level re-weighting . Prior works ( Ren et al. , 2018 ; Shu et al. , 2019 ) for instance classification construct this validation set by random sampling . However , sequence labeling tasks involve many slots ( e.g . WikiAnn has 123 slots over 41 languages ) with variable difficulty and distribution in the data . In case of random sampling , the model oversamples from the most populous category and slots . This is particularly detrimental for low-resource languages in the multilingual setting . To this end , we develop an adaptive mechanism to create the validation set on the fly considering the diversity and uncertainty of the model for different slot types . Furthermore , we leverage this validation set for token-level loss estimation and re-weighting pseudo-labeled sequences from the teacher in the meta-learning setup . While prior works ( Li et al. , 2019 ; Sun et al. , 2019 ; Bansal et al. , 2020 ) on meta-learning for image and text classification leverage multi-task learning to improve a target classification task based on several similar tasks , in this work we focus on a single sequence labeling task – making our setup more challenging altogether . Our task and framework overview . We focus on sequence labeling tasks with only a few annotated samples ( e.g. , K = { 5 , 10 , 20 , 100 } ) per slot type for training and large amounts of task-specific unlabeled data . Figure 1 shows an overview of our framework with the following components : ( i ) Self-training : Our self-training framework leverages a pre-trained language model as a teacher and co-trains a student model with iterative knowledge exchange ( ii ) Adaptive labeled data acquisition for validation : Our few-shot learning setup assumes a small number of labeled training samples per slot type . The labeled data from multiple slot types are not equally informative for the student model to learn from . While prior works in meta-learning randomly sample some labeled examples for held-out validation set , we develop an adaptive mechanism to create this set on the fly . To this end , we leverage loss decay as a proxy for model uncertainty to select informative labeled samples for the student model to learn from in conjunction with the re-weighting mechanism in the next step . ( iii ) Meta-learning for sample re-weighting : Since pseudo-labeled samples from the teacher can be noisy , we employ meta-learning to re-weight them to improve the student model performance on the held-out validation set obtained from the previous step . In contrast to prior work ( Ren et al. , 2018 ) on sample re-weighting operating at instance-level , we incorporate the re-weighting mechanism at token-level for sequence labeling tasks . Here the token-level weights are determined by the student model loss on the above validation set . Finally , we learn all of the above steps jointly with end-toend learning in the self-training framework . We refer to our adaptive self-training framework with meta-learning based sample re-weighting mechanism as MetaST . We perform extensive experiments on six benchmark datasets for several tasks including multilingual Named Entity Recognition and slot tagging for user utterances from task-oriented dialog systems to demonstrate the generalizability of our approach across diverse tasks and languages . We adopt BERT and multilingual BERT as encoder and show that its performance can be significantly improved by nearly 10 % for low-resource settings with few training labels ( e.g. , 10 labeled examples per slot type ) and large amounts of unlabeled data . In summary , our work makes the following contributions . ( i ) Develops a self-training framework for neural sequence tagging with few labeled training examples . ( ii ) Leverages an acquisition strategy to adaptively select a validation set from the labeled set for meta-learning of the student model . ( iii ) Develops a meta-learning framework for re-weighting pseudo-labeled samples at token-level to reduce drifts from noisy teacher predictions . ( iv ) Integrates the aforementioned components into an end-to-end learning framework and demonstrates its effectiveness for neural sequence labeling across six benchmark datasets with multiple slots , shots , domains and languages . 2 BACKGROUND . Sequence labeling and slot tagging . This is the task identifying the entity span of several slot types ( e.g. , names of person , organization , location , date , etc . ) in a text sequence . Formally , given a sentence with N tokens X = { x1 , ... , xN } , an entity or slot value is a span of tokens s = [ xi , ... , xj ] ( 0 ≤ i ≤ j ≤ N ) associated with a type . This task assumes a pre-defined tagging policy like BIO ( Tjong et al. , 1999 ) , where B marks the beginning of the slot , I marks an intermediate token in the span , and O marks out-of-span tokens . These span markers are used to extract multi-token values for each of the slot types with phrase-level evaluation for the performance . Self-training . Consider f ( · ; θtea ) and f ( · ; θstu ) to denote the teacher and student models respectively in the self-training framework . The role of the teacher model ( e.g. , a pre-trained language model ) is to assign pseudo-labels to unlabeled data that is used to train a student model . The teacher and student model can exchange knowledge and the training schedules are repeated till convergence . The success of self-training with deep neural networks in recent works ( He et al. , 2019 ; Xie et al. , 2020 ) has been attributed to a number of factors including stochastic regularization with dropouts and data regularization with unlabeled data . Formally , given m-th unlabeled sentence with N tokens Xum = { xu1 , m , ... , xuN , m } and C pre-defined labels , consider the pseudo-labels Ŷ ( t ) m = [ ŷ ( t ) m,1 , ... , ŷ ( t ) m , N ] generated by the teacher model at the t-th iteration where , ŷ ( t ) m , n = argmax c∈C fn , c ( x u m , n ; θ ( t ) tea ) . ( 1 ) The pseudo-labeled data set , denoted as ( Xu , Ŷ ( t ) ) = { ( Xum , Ŷ ( t ) m ) } Mm , is used to train the student model and learn its parameters as : θ̂ ( t ) stu = argmin θ 1 M M∑ m=1 l ( Ŷ ( t ) m , f ( X u m ; θ ( t−1 ) stu ) ) , ( 2 ) where l ( · , · ) can be modeled as the cross-entropy loss . 3 ADAPTIVE SELF TRAINING . Given a pre-trained language model ( e.g. , BERT ( Devlin et al. , 2019 ) ) as the teacher , we first finetune it on the small labeled data to make it aware of the underlying task . The fine-tuned teacher model is now used to pseudo-label the large unlabeled data . We consider the student model as another instantiation of the pre-trained language model that is trained over the pseudo-labeled data . However , our few-shot setting with limited labeled data results in a noisy teacher . A naive transfer of teacher knowledge to the student results in the propagation of noisy labels limiting the performance of the student model . To address this challenge , we develop an adaptive self-training framework to re-weight pseudo-labeled predictions from the teacher with a meta-learning objective that optimizes the token-level loss from the student model on a held-out labeled validation set . This held-out set is adaptively constructed via labeled data acquisition which selects labeled samples with high uncertainty for efficient data exploration .

Summary: This paper proposes an adaptive self-training framework, called MetaST, for tackling few-shot sequence labeling tasks. The framework consists of several components: a teacher model that finetunes with the few-shot training data and generates noisy labels for the unlabeled examples; a student model that learns from re-weighted noisy labels (at the token level), and an iterative process to update the teacher with the trained student. It also uses a meta-learning mechanism to adjust the token-level weights based on a subsampled set of clean data. This subset is sampled based on the student model’s uncertainty to improve learning efficiency.