Original: A DAPTIVE S ELF - TRAINING FOR N EURAL S EQUENCE L ABELING WITH F EW L ABELS
Revised: A DAPTIVE S ELF - TRAINING FOR N EURAL S EQUENCE L ABELING WITH F EW L ABELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Neural sequence labeling is an important technique employed for many Natural
Revised: Neural sequence labeling is an important technique employed for many Natural

Original: Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot tagging for dialog systems and semantic parsing.
Revised: Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot tagging for dialog systems and semantic parsing.

Original: Large-scale pre-trained language models obtain very good performance on these tasks when ﬁne-tuned on large amounts of task-speciﬁc labeled data.
Revised: Large-scale pre-trained language models obtain very good performance on these tasks when ﬁne-tuned on large amounts of task-speciﬁc labeled data.

Original: However, such large-scale labeled datasets are difﬁcult to obtain for several tasks and domains due to the high cost of human annotation as well as privacy and data access constraints for sensitive user applications.
Revised: However, such large-scale labeled datasets are difﬁcult to obtain for several tasks and domains due to the high cost of human annotation as well as privacy and data access constraints for sensitive user applications.

Original: This is exacerbated for sequence labeling tasks requiring such annotations at token-level.
Revised: This is exacerbated for sequence labeling tasks requiring such annotations at token-level.

Original: In this work, we develop techniques to address the label scarcity challenge for neural sequence labeling models.
Revised: In this work, we develop techniques to address the label scarcity challenge for neural sequence labeling models.

Original: Speciﬁcally, we develop self-training and meta-learning techniques forfew-shot trainingof neural sequence taggers, namely MetaST.
Revised: Speciﬁcally, we develop self-training and meta-learning techniques for training neural sequence taggers with few labels.

Original: While self-training serves as an effective mechanism to learn from large amounts of unlabeled data – meta-learning helps in adaptive sample re-weighting to mitigate error propagation from noisy pseudolabels.
Revised: While self-training serves as an effective mechanism to learn from large amounts of unlabeled data – meta-learning helps in adaptive sample re-weighting to mitigate error propagation from noisy pseudo-labels.

Original: Extensive experiments on six benchmark datasets including two  massive multilingual NERdatasets and four slot tagging datasets for task-oriented dialog systems demonstrate the effectiveness of our  method with around 10% improvement over state-of-the-art systems  for the 10-shot setting.
Revised: Extensive experiments on six benchmark datasets including two for massive multilingual NER and four slot tagging datasets for task-oriented dialog systems demonstrate the effectiveness of our method. With only 10 labeled examples for each class for each task, our method obtains 10% improvement over state-of-the-art systems demonstrating its effectiveness for the low-resource setting.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Motivation.
Revised: Motivation.

Original: Deep neural networks typically require large amounts of training data to achieve stateof-the-art performance.
Revised: Deep neural networks typically require large amounts of training data to achieve stateof-the-art performance.

Original: Recent advances with pre-trained language models like BERT (Devlin et al., 2019), GPT-2 (Radford et al., 2019) and RoBERTa (Liu et al., 2019) have reduced this annotation bottleneck.
Revised: Recent advances with pre-trained language models like BERT (Devlin et al., 2019), GPT-2 (Radford et al., 2019) and RoBERTa (Liu et al., 2019) have reduced this annotation bottleneck.

Original: In this paradigm, large neural network models are trained on massive amounts of unlabeled data in a self-supervised manner.
Revised: In this paradigm, large neural network models are trained on massive amounts of unlabeled data in a self-supervised manner.

Original: However, the success of these large-scale models still relies on ﬁne-tuning them on large amounts of labeled data for downstream tasks.
Revised: However, the success of these large-scale models still relies on ﬁne-tuning them on large amounts of labeled data for downstream tasks.

Revised: For instance, our experiments show 27% relative improvement on an average when ﬁne-tuning BERT with the full training set (2.5K-705K labels) vs. ﬁne-tuning with only 10 labels per class.

Original: This poses several challenges for many real-world tasks.
Revised: This poses several challenges for many real-world tasks.

Original: Not only is acquiring large amounts of labeled data for every task expensive and time consuming, but in many cases, it is not feasible  due to data access and privacy constraints.
Revised: Not only is acquiring large amounts of labeled data for every task expensive and time consuming, but also not feasible in many cases due to data access and privacy constraints.

Original: This issue is exacerbated for sequence labeling tasks that require annotations at tokenand slot-level as opposed to instance-level classiﬁcation tasks.
Revised: This issue is exacerbated for sequence labeling tasks that require annotations at token- and slot-level as opposed to instance-level classiﬁcation tasks.

Original: For example, an NER task can have slots like B-PER, I-PER, O-PER marking the beginning, intermediate and out-of-span markers for person names, and similar slots for the names of location and organization.
Revised: For example, an NER task can have slots like B-PER, I-PER, O-PER marking the beginning, intermediate and out-of-span markers for person names, and similar slots for the names of location and organization.

Original: Similarly, language understanding models for dialog systems rely on effective identiﬁcation of what the user intends to do ( intents ) and the corresponding values as arguments ( slots ) for use by downstream applications that perform the desired task.
Revised: Similarly, language understanding models for dialog systems rely on effective identiﬁcation of what the user intends to do ( intents ) and the corresponding values as arguments ( slots ) for use by downstream applications.

Original: Therefore, fully supervised neural sequence taggers arevery expensive to train for such tasks, given the requirement for thousands of annotations for hundreds of slots for the many different intents.
Revised: Therefore, fully supervised neural sequence taggers are expensive to train for such tasks, given the requirement of thousands of annotations for hundreds of slots for the many different intents.

Original: Semi-supervised learning (SSL) (Chapelle et al., 2010) is one of the promising paradigms to address labeled data scarcity by making effective use of large amounts of unlabeled data in addition to task-speciﬁc labeled data.
Revised: Semi-supervised learning (SSL) (Chapelle et al., 2010) is one of the promising paradigms to address labeled data scarcity by making effective use of large amounts of unlabeled data in addition to task-speciﬁc labeled data.

Original: Self-training (ST, (III, 1965)) as one of the earliest SSL approaches has recently shown state-of-the-art performance for  image classiﬁcation (Li et al., 2019; Xie et al., 2020) performing at par with supervised systems while using very few training labels.
Revised: Self-training (ST, (III, 1965)) as one of the earliest SSL approaches has recently shown state-of-the-art performance for tasks like image classiﬁcation (Li et al., 2019; Xie et al., 2020) performing at par with supervised systems while using very few training labels.

Original: Despite their success on instance-level classiﬁcation, these methods may not be directly applicable to sequence labeling or may not yield the best results since sequence labeling tasks have dependenciesbetween the slots demanding different design choices for instance- and slot-level loss optimizations.
Revised: In contrast to such instance-level classiﬁcation tasks, sequence labeling tasks have dependencies

Original: [Figure]
Revised: [Figure]

Revised: between the slots demanding different design choices for slot-level loss optimization for the limited labeled data setting.

Revised: For instance, prior work (Ruder & Plank, 2018) using classic self-training techniques for sequence labeling did not ﬁnd much success in the low-data regime with 10% labeled data for the target domain.

Revised: Although there has been some success with careful task-speciﬁc data selection (Petrov & McDonald, 2012) and more recently for distant supervision (Liang et al., 2020) using external resources like knowledge bases (e.g., Wikipedia).

Revised: In contrast to these prior work, we develop techniques for self-training with limited labels and without any task-speciﬁc assumption or external knowledge.

Original: For self-training, a base model ( teacher ) is trained on some amount of labeled data and used to pseudo-annotate (task-speciﬁc) unlabeled data.
Revised: For self-training, a base model ( teacher ) is trained on some amount of labeled data and used to pseudo-annotate (task-speciﬁc) unlabeled data.

Original: The original labeled data is augmented with the pseudo-labeled data and used to train a student model.
Revised: The original labeled data is augmented with the pseudo-labeled data and used to train a student model.

Original: The student-teacher training is repeated until convergence.
Revised: The student-teacher training is repeated until convergence.

Original: Traditionally in self-training frameworks, the teacher  pseudo-annotates unlabeled data without any sample selection.
Revised: Traditionally in self-training frameworks, the teacher model pseudo-annotates unlabeled data without any sample selection.

Original: This may result in gradual drifts from self-training on noisy pseudo-labeled instances (Zhang et al., 2017).
Revised: This may result in gradual drifts from self-training on noisy pseudo-labeled instances (Zhang et al., 2017).

Original: Sample selection leveraging teacher conﬁdence has been studied in curriculum learning (Bengio et al., 2009), self-paced learning (Kumar et al., 2010) and more recently, for self-training (Li et al., 2019; Liang et al., 2020a).
Revised: In order to deal with noisy labels and training set biases, Ren et al. (2018) propose a meta-learning technique to automatically re-weight noisy samples by their loss changes on a held-out clean labeled validation set.

Original: These works leverage the easiness of the samples to develop a learningschedule like training on easy concepts ﬁrst followed by complex ones. Since it is hard to deﬁne the “easiness” of a sample, these works rely only on the model loss and therefore are prone to training set biases.
Revised: We adopt a similar principle in our work and leverage meta-learning to re-weight noisy pseudo-labeled examples from the teacher. While prior techniques for learning to re-weight examples have been developed for instance-level classiﬁcation tasks, we extend them to operate at token-level for discrete sequence labeling tasks.

Original: To  address such issues stemming from noisy labels and training set biases, learning to re-weight noisy examples (Ren et al., 2018) leverages a meta objective with  the basic assumption that the best weighting strategy should minimize the loss on a held-out clean labeled validation set.
Revised: To this end, we address some key challenges on how to construct an informative held-out validation set for token-level re-weighting. Prior works (Ren et al., 2018; Shu et al., 2019) for instance classiﬁcation construct this validation set by random sampling. However, sequence labeling tasks involve many slots (e.g. WikiAnn has 123 slots over 41 languages) with variable difﬁculty and distribution in the data.

Original: We adopt a similar principle in our work and leverage meta-learning to re-weight pseudo-labeled examples from the teacher.
Revised: In case of random sampling, the model oversamples from the most populous category and slots.

Original: While prior techniques for learning to re-weight examples have been developed for instance-level classiﬁcation tasks, we extend them to operate at token-level for discrete text sequences for our sequence labeling tasks.
Revised: This is particularly detrimental for low-resource languages in the multilingual setting.

Original: Additionally, we develop an adaptive mechanism to create the validation set on the ﬂy that reﬂects the  uncertainty of the modeland subsequently helps us in obtaining token-level weights for  re-weighting pseudo-labeled examples from the teacher.
Revised: To this end, we develop an adaptive mechanism to create the validation set on the ﬂy considering the diversity and uncertainty of the model for different slot types. Furthermore, we leverage this validation set for token-level loss estimation and re-weighting pseudo-labeled sequences from the teacher in the meta-learning setup.

Original: While prior workson few-shot meta-learning for both image and text classiﬁcation (Li et al., 2019; Sun et al., 2019; Bansal et al., 2020)  leverage multi-task learning to improve a target classiﬁcation task based on several similar tasks, in this work we focus on a single sequence labeling task – making our setup more challenging altogether.
Revised: While prior works (Li et al., 2019; Sun et al., 2019; Bansal et al., 2020) on meta-learning for image and text classiﬁcation leverage multi-task learning to improve a target classiﬁcation task based on several similar tasks, in this work we focus on a single sequence labeling task – making our setup more challenging altogether.

Original: Our task and framework overview.
Revised: Our task and framework overview.

Original: We focus on sequence labeling tasks with only a few labeled samples (e.g., K = { 5 , 10 , 20 , 100 } ) per slot type for training and large amounts of task-speciﬁc unlabeled data.
Revised: We focus on sequence labeling tasks with only a few annotated samples (e.g., K = { 5 , 10 , 20 , 100 } ) per slot type for training and large amounts of task-speciﬁc unlabeled data.

Original: Figure 1 shows an overview of our framework with the following components: (i) Self-training: Our self-training framework leverages a pre-trained language model as a teacher and co-trains a student model with iterative knowledge exchange (ii) Labeled data acquisition for validation set: Our few-shot learning setup assumes a small number of labeled training samples per slot type.
Revised: Figure 1 shows an overview of our framework with the following components: (i) Self-training: Our self-training framework leverages a pre-trained language model as a teacher and co-trains a student model with iterative knowledge exchange (ii) Adaptive labeled data acquisition for validation: Our few-shot learning setup assumes a small number of labeled training samples per slot type.

Original: We expose this data to the teacher and to the student model with two different views, namely, for supervised ﬁne-tuning of the teacher model and as a held-out validation set for the student model.

Original: The labeled data from multiple slot types are not equally informative for the student model to learn from at different training iterations.
Revised: The labeled data from multiple slot types are not equally informative for the student model to learn from.

Revised: While prior works in meta-learning randomly sample some labeled examples for held-out validation set, we develop an adaptive mechanism to create this set on the ﬂy.

Original: Therefore, we leverage loss decay as a proxy for model uncertainty toadaptively select informative labeled samples for the student model to learn from in conjunction with the re-weighting mechanism in the next step.
Revised: To this end, we leverage loss decay as a proxy for model uncertainty to select informative labeled samples for the student model to learn from in conjunction with the re-weighting mechanism in the next step.

Original: (iii) Meta-learning for sample re-weighting: Since pseudo-labeled samples from the teacher can be noisy, we employ metalearning to re-weight them to improve the student model performance on the held-out validation set obtained from the previous step.
Revised: (iii) Meta-learning for sample re-weighting: Since pseudo-labeled samples from the teacher can be noisy, we employ meta-learning to re-weight them to improve the student model performance on the held-out validation set obtained from the previous step.

Original: In contrast to prior work (Ren et al., 2018) on sample re-weighting operating at instance-level, we incorporate the re-weighting mechanism at token-level for sequence labeling tasks – where the token weights are determined by the student model loss on the above validation set.
Revised: In contrast to prior work (Ren et al., 2018) on sample re-weighting operating at instance-level, we incorporate the re-weighting mechanism at token-level for sequence labeling tasks. Here the token-level weights are determined by the student model loss on the above validation set.

Original: Finally, we learn all of the above steps jointly with end-to-end learning in the selftraining framework.
Revised: Finally, we learn all of the above steps jointly with end-toend learning in the self-training framework.

Original: We refer  our adaptive self-training framework with meta-learning based sample re-weighting mechanism as MetaST .
Revised: We refer to our adaptive self-training framework with meta-learning based sample re-weighting mechanism as MetaST .

Original: We perform extensive experiments on six benchmark datasets for several tasks including multilingual Named Entity Recognition and slot tagging for user utterances from task oriented dialog systems to demonstrate the generalizability of our approach across diverse tasks and languages
Revised: We perform extensive experiments on six benchmark datasets for several tasks including multilingual Named Entity Recognition and slot tagging for user utterances from task-oriented dialog systems to demonstrate the generalizability of our approach across diverse tasks and languages.

Original: We adopt BERT and multilingual BERT as encoder and show that its performance can be signiﬁcantly improved by nearly 10% for low-resource settings with few training labels (e.g., 10-shot) and large  unlabeled data.
Revised: We adopt BERT and multilingual BERT as encoder and show that its performance can be signiﬁcantly improved by nearly 10% for low-resource settings with few training labels (e.g., 10 labeled examples per slot type) and large amounts of unlabeled data.

Original: In summary our work makes the following contributions.
Revised: In summary, our work makes the following contributions.

Original: (i) Develops a selftraining framework for neural sequence tagging with few labeled training samples.
Revised: (i) Develops a self-training framework for neural sequence tagging with few labeled training examples.

Original: (ii) Leverages an acquisition strategy to adaptively select a validation set from the labeled set for meta-learning for the student model.
Revised: (ii) Leverages an acquisition strategy to adaptively select a validation set from the labeled set for meta-learning of the student model.

Original: (iii) Develops a meta-learning framework for re-weighting pseudo-labeled samples at token-level to reduce drifts from noisy teacher predictions.
Revised: (iii) Develops a meta-learning framework for re-weighting pseudo-labeled samples at token-level to reduce drifts from noisy teacher predictions.

Original: (iv) Integrates the aforementioned components into an end-to-end learning framework and demonstrates its effectiveness for neural sequence labeling across six benchmark datasets including diverse tasks and multiple  languages.
Revised: (iv) Integrates the aforementioned components into an end-to-end learning framework and demonstrates its effectiveness for neural sequence labeling across six benchmark datasets with multiple slots, shots, domains and languages.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: Sequence labeling and slot tagging.
Revised: Sequence labeling and slot tagging.

Original: This is the task  of classifying each token in a sequence of observed values into pre-deﬁned categories (also called slot types), such as names of person, organization, location, date, etc.
Revised: This is the task identifying the entity span of several slot types (e.g., names of person, organization, location, date, etc.) in a text sequence.

Original: Formally, given a sentence with N tokens X = { x 1 , ..., x N } , an entity or slot value is a span of tokens s =
Revised: Formally, given a sentence with N tokens X = { x 1 , ..., x N } , an entity or slot value is a span of tokens s =

Original: [ x i , ..., x j ](0 ≤ i ≤ j ≤ N ) associated with a type.
Revised: [ x i , ..., x j ](0 ≤ i ≤ j ≤ N ) associated with a type.

Original: This task assumes a pre-deﬁned tagging policy like BIO (Tjong et al., 1999), where B marks the beginning of the slot, I marks an intermediate token in the span, and O marks out-of-span tokens.
Revised: This task assumes a pre-deﬁned tagging policy like BIO (Tjong et al., 1999), where B marks the beginning of the slot, I marks an intermediate token in the span, and O marks out-of-span tokens.

Revised: These span markers are used to extract multi-token values for each of the slot types with phrase-level evaluation for the performance.

Original: Self-training.
Revised: Self-training.

Original: Consider f ( · ; θ tea ) and f ( · ; θ stu ) to denote the teacher and student models respectively in the self-training framework.
Revised: Consider f ( · ; θ tea ) and f ( · ; θ stu ) to denote the teacher and student models respectively in the self-training framework.

Original: The role of the teacher model (e.g., a pre-trained language model) is to assign pseudo-labels to unlabeled data that is used to train a student model.
Revised: The role of the teacher model (e.g., a pre-trained language model) is to assign pseudo-labels to unlabeled data that is used to train a student model.

Original: The teacher and student model can exchange knowledge and the training schedules are repeated till convergence.
Revised: The teacher and student model can exchange knowledge and the training schedules are repeated till convergence.

Original: The success of self-training with deep neural networks in recent works (He et al., 2019; Xie et al., 2020) has been attributed to a number of factors including stochastic regularization with dropouts and data regularization with unlabeled data.
Revised: The success of self-training with deep neural networks in recent works (He et al., 2019; Xie et al., 2020) has been attributed to a number of factors including stochastic regularization with dropouts and data regularization with unlabeled data.

Original: Formally, given m -th unlabeled sentence with N tokens X um = { x u 1 ,m , ..., x uN,m } and C pre-deﬁned labels, consider the pseudo-labels ˆ Y ( t ) m = 
Revised: Formally, given m -th unlabeled sentence with N tokens X um = { x u 1 ,m , ..., x uN,m } and C pre-deﬁned labels, consider the pseudo-labels ˆ Y ( t ) m = [ˆ y (

Original: [ˆ y ( t ) m, 1 , ..., ˆ y ( t ) m,N ] generated by the teacher model at the t -th iteration where,
Revised: [Equation] ) m, 1 , ..., ˆ y ( t ) m,N ] generated by the teacher model at the t -th iteration where,

Original: [Equation]
Revised: [Equation]

Original: The pseudo-labeled data set, denoted as ( X u , ˆ Y ( t ) )
Revised: The pseudo-labeled data set, denoted as ( X u , ˆ Y ( t ) )

Original: = { ( X um , ˆ Y ( t ) m ) } Mm , is used to train the student model and learn its parameters as:
Revised: = { ( X um , ˆ Y ( t ) m ) } Mm , is used to train the student model and learn its parameters as:

Original: [Equation]
Revised: [Equation]

Original: where l ( · , · ) can be modeled as the cross-entropy loss.
Revised: where l ( · , · ) can be modeled as the cross-entropy loss.

Original:  A DAPTIVE S ELF T RAINING
Revised:  A DAPTIVE S ELF T RAINING

Original: Given a pre-trained language model (e.g., BERT (Devlin et al., 2019)) as the teacher, we ﬁrst ﬁnetune it on the small labeled data to make it aware of the underlying task.
Revised: Given a pre-trained language model (e.g., BERT (Devlin et al., 2019)) as the teacher, we ﬁrst ﬁnetune it on the small labeled data to make it aware of the underlying task.

Original: The ﬁne-tuned teacher model is now used to pseudo-label the large unlabeled data.
Revised: The ﬁne-tuned teacher model is now used to pseudo-label the large unlabeled data.

Original: We consider the student model as another instantiation of the pre-trained language model that is trained over the pseudo-labeled data.
Revised: We consider the student model as another instantiation of the pre-trained language model that is trained over the pseudo-labeled data.

Original: However, our few-shot setting with limited labeled data results in a noisy teacher.
Revised: However, our few-shot setting with limited labeled data results in a noisy teacher.

Original: A naive transfer of teacher knowledge to the student results in the propagation of noisy label information limiting the performance of the student model.
Revised: A naive transfer of teacher knowledge to the student results in the propagation of noisy labels limiting the performance of the student model.

Original: To address this challenge, we develop an adaptive self-training framework to re-weight pseudo-labeled predictions from the teacher with a meta-learning objective that optimizes the token-level loss from the student model on a held-out labeled validation set.
Revised: To address this challenge, we develop an adaptive self-training framework to re-weight pseudo-labeled predictions from the teacher with a meta-learning objective that optimizes the token-level loss from the student model on a held-out labeled validation set.

Original: In standard meta-learning setup, such loss changes are estimated over a separate set of held-out clean labeled data, which is used as the validation set. However, in order to make effective use of the initial small labeled dataand focus on informative samples for different slots, we introduce an acquisition strategy to adaptively select the validation set consisting of labeled samples with high uncertainty.
Revised: This held-out set is adaptively constructed via labeled data acquisition which selects labeled samples with high uncertainty for efﬁcient data exploration.

Original: 3. A DAPTIVE L ABELED D ATA A CQUISITION
Revised: 3. A DAPTIVE L ABELED D ATA A CQUISITION

Original: Instead of focusing on samples that the model already predicts with high conﬁdence, or the converse where the  model frequently makes a mistake – uncertainty sampling provides an alternative strategy for selecting samples that the model is confused about.
Revised: In standard meta-learning setup for instance-level classiﬁcation tasks, the held-out validation set is usually constructed via random sampling (Ren et al., 2018; Shu et al., 2019). Sequence labeling tasks involve many slot types with variable difﬁculty and distribution in the data. For instance, NER tasks over WikiAnn operate over 123 slot types from 41 languages with additional complexity from variable model performance across different languages. A random sampling leads to oversampling instances with the most populous categories and slot types in the data.

Revised: Therefore, we propose a novel labeled data acquisition strategy to construct the validation set for effective data exploration.

Original: Such strategiesare commonly used for variance reduction in  active learning settings (Settles, 2009; Chang et al., 2017a). 
Revised: We demonstrate its beneﬁt over classic meta-learning approaches from prior works in experiments. In general, data acquisition strategies for prior works in meta-learning and active learning broadly leverage random sampling (Ren et al., 2018; Shu et al., 2019), easy (Kumar et al., 2010) and hard example mining (Shrivastava et al., 2016) or uncertainty-based methods (Chang et al., 2017a). These strategies have been compared in prior works (Chang et al., 2017a; Gal et al., 2017) that show uncertainty-based methods to have better generalizability across diverse settings.

Revised: There are several approaches to uncertainty estimation including error decay (Konyushkova et al., 2017; Chang et al., 2020), Monte Carlo dropouts (Gal et al., 2017) and predictive variance (Chang et al., 2017a).

Original: In this work, we leverageloss decay in the form of stochastic loss of the modelin successive iterations as a proxy to obtain the uncertainty in its predictions. This enables thestudent model  to focus more on samples on which it can learn better in contrast to outliers or very hard examples.
Revised: We follow a similar principle of error decay to ﬁnd samples that the model is uncertain about and can correspondingly beneﬁt from knowing their labels (similar to active learning settings). To this end, we leverage stochastic loss decay from the model as a proxy for the model uncertainty to generate validation set on the ﬂy. This is used for estimating token-level weights and re-weighting pseudo labeled data in Section 3.2.

Original: Consider the loss of the student model with parameters θ ( t ) stu on the labeled data ( X lm , Y m ) in the t -th iteration as l ( Y m , f ( X lm ; θ ( t ) stu )) .
Revised: Consider the loss of the student model with parameters θ ( t ) stu on the labeled data ( X lm , Y m ) in the t -th iteration as l ( Y m , f ( X lm ; θ ( t ) stu )) .

Original: To measure the loss decay value, we need to calculate the difference between the current and previous loss values.
Revised: To measure the loss decay value at any iteration, we use the difference between the current and previous loss values.

Original: Consideringthat these values may ﬂuctuate across iterations, we adopt the moving average of the loss values for ( X lm , Y m ) in the latest R iterations as a baseline l mb for loss decay estimation.
Revised: Considering these values may ﬂuctuate across iterations, we adopt the moving average of the loss values for ( X lm , Y m ) in the latest R iterations as a baseline l mb for loss decay estimation.

Original: The baseline l mb is calculated as follows:
Revised: Baseline measure l mb is calculated as follows:

Original: [Equation]
Revised: [Equation]

Original: Since the loss decay values are estimated on the ﬂy, we want to balance exploration and exploitation.
Revised: Since the loss decay values are estimated on the ﬂy, we want to balance exploration and exploitation.

Original: To this end, we add a smoothness constant δ to prevent the low loss decay samples  from never being selected again.
Revised: To this end, we add a smoothness factor δ to prevent the low loss decay samples (i.e. samples with low uncertainty) from never being selected again.

Original: Considering all of the above factors, we obtain the sampling weight of labeled data ( X lm , Y lm ) as follows:
Revised: Considering all of the above factors, we obtain the sampling weight of labeled data ( X lm , Y lm ) as follows:

Original: [Equation]
Revised: [Equation]

Original: The smoothness constant δ needs to be adaptive with dynamic training loss scale.
Revised: The smoothness factor δ needs to be adaptive since the training loss is dynamic.

Original: In general, we can set the value of δ as the average or maximum of the loss decay values among all the labeled samples in ( X l , Y ) . However, to make our sampling mechanism more robust to extreme values, we use the maximum of the loss decay value asδ in this paper and limit the maximum weight ( W max ) to two times minimum ( W min ).
Revised: Therefore, We adopt the maximum of the loss decay value as the smoothness factor δ to encourage exploration.

Original: The aforementioned acquisition function is re-estimated after a ﬁxed number of steps to capture model changes.
Revised: The aforementioned acquisition function is re-estimated after a ﬁxed number of steps to adapt to model changes.

Original: With labeled data acquisition, wecan rely on informative uncertain samples to improve learning efﬁciency.
Revised: With labeled data acquisition, we rely on informative uncertain samples to improve learning efﬁciency.

Original: The sampled mini-batches of labeled data {B ls } are used as a validation set for the student model in the next step for re-weighting pseudo-labeled data from the teacher model.
Revised: The sampled mini-batches of labeled data {B ls } are used as a validation set for the student model in the next step for re-weighting pseudo-labeled data from the teacher model.

Revised: We demonstrate its impact via ablation study in experiments.

Original: Also, note that the labeled data is only used to compute the acquisition function and not used forthe explicit training of the student model in this phase.
Revised: Note that the labeled data is only used to compute the acquisition function and not used for explicit training of the student model in this step.

Original: 3. R E - WEIGHTING P SEUDO -L ABELED D ATA
Revised: 3. R E - WEIGHTING P SEUDO -L ABELED D ATA

Original: To mitigate error propagation from noisy pseudo-labeled instances from the teacher, we leverage meta-learning to adaptively re-weight samples based on the student model loss on a held-out validation set following (Ren et al., 2018).
Revised: To mitigate error propagation from noisy pseudo-labeled sequences from the teacher, we leverage meta-learning to adaptively re-weight them based on the student model loss on the held-out validation set obtained via labeled data acquisition from the previous section.

Original: However, in contrast to prior work focusing on instance-level tasks like image classiﬁcation – sequence labeling operates on discrete text sequences as input and assigns labels to each token in the sequence.
Revised: In contrast to prior work focusing on instance-level tasks like image classiﬁcation – sequence labeling operates on discrete text sequences as input and assigns labels to each token in the sequence.

Original: Since teacher predictions vary for different slot labels and types, we adapt the meta-learning framework to re-weight samples at a token-level resolution.
Revised: Since teacher predictions vary for different slot labels and types, we adapt the meta-learning framework to re-weight samples at a token-level resolution.

Original: Token Re-weighting.
Revised: Token Re-weighting.

Original: Consider the pseudo-labels { ˆ Y ( t ) m =
Revised: Consider the pseudo-labels { ˆ Y ( t ) m =

Original: [ˆ y ( t ) m, 1 , ..., ˆ y ( t ) m,N ] } Mm =1 from the teacher in the t -th iteration with m and n indexing the instance and a token in the instance, respectively.
Revised: [ˆ y ( t ) m, 1 , ..., ˆ y ( t ) m,N ] } Mm =1 from the teacher in the t -th iteration with m and n indexing the instance and a token in the instance, respectively.

Original: In classic self-training, we update the student parameters leveraging pseudo-labels as follows:
Revised: In classic self-training, we update the student parameters leveraging pseudo-labels as follows:

Original: [Equation]
Revised: [Equation]

Original: Now, to downplay noisy token-level labels, we leverage meta-learning to re-weight the pseudolabeled data.
Revised: Now, to downplay noisy token-level labels, we leverage meta-learning to re-weight the pseudolabeled data.

Original: To this end, we follow a similar analysis  (Koh & Liang, 2017; Ren et al., 2018) to perturb the weight for each token in the mini-batch by (cid:15) as follows.
Revised: To this end, we follow a similar analysis from (Koh & Liang, 2017) and (Ren et al., 2018) to perturb the weight for each token in the mini-batch by (cid:15) .

Revised: Weight perturbation is used to discover data points that are most important to improve the model performance on a held-out validation set (Koh & Liang, 2017) where the sample importance is given by the magnitude of the the negative gradients.

Revised: We extend prior techniques to obtain token-level perturbations as:

Original: [Equation]
Revised: [Equation]

Original: The token weights are obtained by minimizing the student model loss on a held-out validation set of clean labeled samples.
Revised: The token weights are obtained by minimizing the student model loss on the held-out validation set.

Original: Here, we employ the labeled data acquisition strategy from Eq.
Revised: Here, we employ the labeled data acquisition strategy from Eq.

Original: 4 to sample informative mini-batches of labeled data B ls locally at step t .
Revised: 4 to sample informative mini-batches of labeled data B ls locally at step t .

Original: To obtain a cheap estimate of the meta-weight at step t , we take a single gradient descent step for the sampled labeled mini-batch B ls :
Revised: To obtain a cheap estimate of the meta-weight at step t , we take a single gradient descent step for the sampled labeled mini-batch B ls :

Original: [Equation]
Revised: [Equation]

Original: Considering the diversity of slot types and their interactions in the  sequence labeling task, we sample
Revised: We set the token weights to be proportional to the negative gradients to reﬂect the importance of pseudo-labeled tokens in the sequence. Since sequence labeling tasks have dependencies between the slot types and tokens, it is difﬁcult to obtain a good estimation of the weights based on a single mini-batch of examples.

Original:  S mini-batches of labeled data {B l 1 , ..., B lS }  to get a stable gradient estimate. 
Revised: Therefore, we sample S mini-batches of labeled data {B l 1 , ..., B lS } with the adaptive acquisition strategy and calculate the mean of the gradients to obtain a robust gradient estimate. Note that S is a constant number that is the same for each token and the proportional sign in Eq. 8. Since a negative weight indicates a pseudo-label of poor quality that would potentially degrade the model performance, we set such weights to 0 to ﬁlter them out.

Original: The impact of S is investigated in the experiments (refer to Appendix A.1).
Revised: The impact of S is investigated in the experiments (refer to Appendix A.1).

Original: The overall meta-weight of pseudo-labeled token ( x um,n , ˆ y m,n ) is obtained as:
Revised: The overall meta-weight of pseudo-labeled token ( x um,n , ˆ y m,n ) is obtained as:

Original: [Equation]
Revised: [Equation]

Original: To further ensure the stability of the loss function in each mini-batch, we normalise the weight w m,n .
Revised: To further ensure the stability of the loss function in each mini-batch, we normalise the weight w m,n .

Original: Finally, we update the student model parameters while accounting for token-level re-weighting as:
Revised: Finally, we update the student model parameters while accounting for token-level re-weighting as:

Original: [Equation]
Revised: [Equation]

Original:  U PDATES
Revised: We demonstrate the impact of our re-weighting mechanism with an ablation study in experiments. 3. T EACHER M ODEL I TERATIVE U PDATES

Original: At the end of every self-training iteration, we assign the student model as a new teacher model (i.e., θ tea = θ ( T ) stu ) .
Revised: At the end of every self-training iteration, we assign the student model as a new teacher model (i.e., θ tea = θ ( T ) stu ) .

Original: Since the student model uses the labeled data only as a held-out validation set for meta-learning, we further utilize the labeled data ( X l , Y ) to ﬁne-tune the new teacher model f ( · , θ ( t ) tea ) with standard supervisedcross-entropy loss minimization.
Revised: Since the student model uses the labeled data only as a held-out validation set for meta-learning, we further utilize the labeled data ( X l , Y ) to ﬁne-tune the new teacher model f ( · , θ ( t ) tea ) with standard supervised loss minimization.

Original: We explore the effectiveness of thisﬁne-tuning step via an ablation study in our experimental section 4.
Revised: We explore the effectiveness of this step with an ablation study in experiments.

Original: We further re-initialize the student model at the beginning of the self-training iteration.

Original: The overall training procedure is summarized in Algorithm 1. 
Revised: The overall training procedure is summarized in Algorithm 1. Algorithm 1: MetaST Algorithm.

Original: [Table]
Revised: [Table]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Encoder.
Revised: Encoder.

Original: Pre-trained language models like BERT (Devlin et al., 2019),
Revised: Pre-trained language models like BERT (Devlin et al., 2019),

Original: GPT-2 (Radford et al., 2019) and RoBERTa (Liu et al., 2019) have shown state-of-the-art performance for various natural language processing tasks.
Revised: GPT-2 (Radford et al., 2019) and RoBERTa (Liu et al., 2019) have shown state-of-the-art performance for various natural language processing tasks.

Original: In this work we adopt one of them as a base encoder by initializing the teacher with pre-trained BERT-base model and a randomly initialized token classiﬁcation layer. 
Revised: In this work we adopt one of them as a base encoder by initializing the teacher with pre-trained BERT-base model and a randomly initialized token classiﬁcation layer. [Table] Email [Table]

Revised: 5K 1k

Original:  Datasets.
Revised: [Table] 13K [Table] Datasets.

Original: We perform large-scale experiments with six different datasets including user utterances for task-oriented dialog systems and  Named Entity Recognition tasks as summarized in Table 1.
Revised: We perform large-scale experiments with six different datasets including user utterances for task-oriented dialog systems and multilingual Named Entity Recognition tasks as summarized in Table 1.

Original: (a)
Revised: (a)

Original: Email .
Revised: Email .

Original: This consists of natural language user utterances for email-oriented user actions like sending, receiving or searching emails with attributes like date, time, topic, people, etc.
Revised: This consists of natural language user utterances for email-oriented user actions like sending, receiving or searching emails with attributes like date, time, topics, people, etc.

Original: (b) SNIPS is a public benchmark dataset (Coucke et al., 2018) of user queries from multiple domains including music, media, and weather.
Revised: (b) SNIPS is a public benchmark dataset (Coucke et al., 2018) of user queries from multiple domains including music, media, and weather.

Original: (c) MIT Movie and Restaurant corpus (Liu et al., 2013) consist of similar user utterances for movie and restaurant domains.
Revised: (c) MIT Movie and Restaurant corpus (Liu et al., 2013) consist of similar user utterances for movie and restaurant domains.

Original: (d) CoNLL

Revised: (d) CoNLL

Original: (Sang & Meulder, 2003) and Wikiann (Pan et al., 2017) are public benchmark datasets for multilingual Named Entity Recognition.
Revised: (Sang & Meulder, 2003) and Wikiann (Pan et al., 2017) are public benchmark datasets for multilingual Named Entity Recognition.

Original: CoNLL03 is a collection of news wire articles from the Reuters Corpus from 4 languages with manual annotations, whereas Wikiann comprises of extractions from Wikipedia articles from 41 languages with automatic annotation leveraging meta-data for different entity types like ORG, PER, LOC etc.
Revised: CoNLL03 is a collection of news wire articles from the Reuters Corpus from 4 languages with manual annotations, whereas Wikiann comprises of extractions from Wikipedia articles from 41 languages with automatic annotation leveraging meta-data for different entity types like ORG, PER, LOC etc.

Original: For every dataset, we sample K ∈ { 5 , 10 , 20 , 100 } labeled instances from the training data for each slot type, and add the remaining to the unlabeled dataset.
Revised: For every dataset, we sample K ∈ { 5 , 10 , 20 , 100 } labeled sequences for each slot type from the Train data, and add the remaining to the unlabeled set while ignoring their labels – following standard setups for semi-supervised learning.

Original: We repeatedly sample K labeled instances three times for multiple runs to report average performance.. [Table]
Revised: We repeatedly sample K labeled instances three times for multiple runs to report average performance with standard deviation across the runs.

Original: Baselines.
Revised: Baselines.

Original: The ﬁrst baseline we consider is the fully supervised BERT model trained on all available training data.
Revised: The ﬁrst baseline we consider is the fully supervised BERT model trained on all available training data which provides the ceiling performance for every task.

Original: Each of the other models are trained on K training labels per slot.
Revised: Each of the other models are trained on K training labels per slot type.

Original: We adopt several state-of-theart semi-supervised methods as baselines: (1) CVT1 (Clark et al., 2018) is a semi-supervised sequence labeling method based on crossview training; (2) SeqVAT (Chen et al., 2020) incorporates adversarial training with conditional random ﬁeld layer for semi-supervised sequence labeling; (3) Mean Teacher (MT) (Tarvainen & Valpola, 2017) averages model weights to form an aggregated teacher; (4) VAT (Miyato et al., 2018) adopts virtual adversarial training to make the model robust to noise; (5) classic ST (III, 1965) is simple self-training method with hard pseudo-labels; (6) BOND (Liang et al., 2020b) employs simple self-training for sequence labeling.
Revised: We adopt several state-of-the-art semi-supervised methods as baselines: (1) CVT (Clark et al., 2018) is a semi-supervised sequence labeling method based on cross-view training; (2) SeqVAT (Chen et al., 2020) incorporates adversarial training with conditional random ﬁeld layer for semi-supervised sequence labeling; (3) Mean Teacher (MT) (Tarvainen & Valpola, 2017) averages model weights to obtain an aggregated teacher; (4) VAT (Miyato et al., 2018) adopts virtual adversarial training to make the model robust to noise; (5) classic ST (III, 1965) is simple self-training method with hard pseudo-labels; (6) BOND (Liang et al., 2020) is the most recent work on self-training for sequence labeling with conﬁdence-based sample selection and forms a strong baseline for our work.

Original: We implement our framework in Pytorch and use Tesla V100 gpus for experimentation.
Revised: We implement our framework in Pytorch and use Tesla V gpus for experiments.

Original: Hyper-parameter conﬁgurations withdetailed model settings presented in Appendix.
Revised: Hyper-parameter conﬁgurations with model settings presented in Appendix.

Original: Neural sequence labeling performance with few training labels.
Revised: Neural sequence labeling performance with few training labels.

Original: Table 2 shows the performance comparison among different models for the K= 10 shot setting.
Revised: Table 2 shows the performance comparison among different models with K= 10 labeled examples per slot type.

Original: The fully supervised BERT trained on thousands of labeled examples provides the ceiling performance for the few-shot setting.
Revised: The fully supervised BERT trained on thousands of labeled examples provides the ceiling performance for the few-shot setting.

Original: We observe our method MetaST to signiﬁcantly outperform all methods across all datasets including the models that also use the same BERT encoder as ours like MT, VAT, Classic ST and BOND with corresponding average performance improvements as 14 .
Revised: We observe our method MetaST to signiﬁcantly outperform all methods across all datasets including the models that also use the same BERT encoder as ours like MT, VAT, Classic ST and BOND with corresponding average performance improvements as 14 .

Original: 46% and 8 .
Revised: 46% and 8 .

Original: Non BERT models like CVT and SeqVAT are consistently worse than other baselines.
Revised: Non BERT models like CVT and SeqVAT are consistently worse than other baselines.

Original: [Table]
Revised: [Table]

Original: All models (except CVT and SeqVAT) use the same BERT-Base encoder.

Original: The F1 score of our framework for each task is followed by the standard deviation in parentheses and percentage improvement ( ↑ ) over the BERT model with few-shot supervision.

Original: We also observe variable performance of the models across differenttasks.
Revised: We also observe variable performance of the models across different

Original: More speciﬁcally the  gap between the best few-shot model and the fully supervised model varies signiﬁcantly.
Revised: Speciﬁcally, the per formance gap between the best few-shot model and the fully supervised model varies signiﬁcantly.

Original: As such,

Original: MetaST can achieve close performance to the fully-supervised model in some datasets (e.g. SNIPS and Email) but has bigger room for improvement with others (e.g. CoNLL03 (EN) and Wikiann (EN)).
Revised: MetaST achieves close performance to the fully-supervised model in some datasets (e.g. SNIPS and Email) but has bigger room for improvement in others (e.g. CoNLL03 (EN) and Wikiann (EN)).

Original: This can be attributed to the following factors.
Revised: This can be attributed to the following factors.

Original: (i)
Revised: (i)

Original: Labeled training samples and slots .
Revised: Labeled training examples and slots .

Original: The total number of labeled training instances for our K-shot setting is given by K × # Slots .
Revised: The total number of labeled training instances for our K-shot setting is given by K × # Slots .

Original: Therefore, for tasks witha higher number of slots and consequently more training labels, most of the models perform better including MetaST.
Revised: Therefore, for tasks with higher number of slots and consequently more training labels, most of the models perform better including MetaST.

Original: Additionally, for sequence labeling tasks with  inherent dependency between the slot structure, task-oriented systems with more slots and richer interactions beneﬁt more than NER tasks.
Revised: Task-oriented dialog systems with more slots and inherent dependency between the slot types beneﬁt more than NER tasks.

Original: (ii) Task difﬁculty: User utterances from task-oriented dialog systems for some of the domains like weather, music and emails contain predictive query patterns and limited diversity.
Revised: (ii) Task difﬁculty: User utterances from task-oriented dialog systems for some of the domains like weather, music and emails contain predictive query patterns and limited diversity.

Original: In contrast, Named Entity Recognition datasets are comparatively diverse and require more training labels to generalize well.
Revised: In contrast, Named Entity Recognition datasets are comparatively diverse and require more training labels to generalize well.

Original: Similar observations are also depicted in Table 3 for multilingual NER tasks with more slots and consequently more training labels from multiple languages as well as richer interactions across the slots from different languages.
Revised: Similar observations are also depicted in Table 3 for multilingual NER tasks with more slots and consequently more training labels from multiple languages as well as richer interactions across the slots from different languages.

Original: [Table]
Revised: [Table]

Revised: deviation

Revised: ↑

Original: Controlling for the total amount of labeled data.
Revised: Controlling for the total amount of labeled data.

Original: In order to control for the variable amount of training labels across different datasets, we perform another experiment where we vary the number of shots for different slot types while keeping the total number of labeled instances for each dataset similar (ca. 200 ).
Revised: In order to control for the variable amount of training labels across different datasets, we perform another experiment where we vary the number of labels for different slot types while keeping the total number of labeled instances for each dataset similar (ca. 200 ).

Original: Results are shown in Table 4.
Revised: Results are shown in Table 4.

Original: To better illustrate the effect of the number of training labels, we choose tasks with lower performance in Table 2 for this experiment.
Revised: To better illustrate the effect of the number of training labels, we choose tasks with lower performance in Table 2 for this experiment.

Original: Comparing the results in Tables 2 and 4, we observe the performance of MetaSTon all the tasks to improve with more training labels. [Table]
Revised: Comparing the results in Tables 2 and 4, we observe the performance of MetaST to improve with more training labels for all the tasks .

Original: Effect of varying the number of few-shots K .
Revised: Effect of varying the number of labels K per slot.

Original: Table 5 shows the improvement in the performance of MetaST when increasing the number of labels for each slot type in the SNIPS dataset. 
Revised: Table 5 shows the improvement in the performance of MetaST when increasing the number of labels for each slot type in the SNIPS dataset. [Table]

Original: Similar trends can be found on other datasets (results in Appendix).
Revised: Similar trends can be found on other datasets (results in Appendix).

Original: We observe that the performance of MetaST with only 100 labels per slot can match the fully supervised BERT performance trained on 13 K labeled instances. As we increase the amount of labeled training instances, the performance of BERT also  improves 
Revised: As we increase the amount of labeled training instances, the performance of BERT also improves, and correspondingly the margin between MetaST and these baselines decreases although MetaST still improves over all of them. In the self-training framework

Original: [Table]
Revised: [Table]

Original: and correspondingly the margin between MetaST and these baselines decreases although MetaST still improves over all ofthem. For example, while MetaST improves over BERT by 15% for the 5 -shot setting
Revised: given the ceiling performance for every task and the improved performance of the teacher with more training labels

Original: [Table]
Revised: [Table]

Original: the corresponding improvement reduces to 2%  for  the 100 -shot setting.
Revised: there is less room for (relative) improvement of the student over the teacher model. Consider SNIPS for example. Our model obtains 12% and 2% improvement over the few-shot BERT model for the 10-shot and 100-shot setting with F1-scores as 88.22% and 95.39%, respectively. The ceiling performance for this task is 95.8% on training BERT on the entire dataset with 13K labeled examples.

Original: This phenomenon indicates that MetaST is most impactful forthe low-resource settings 
Revised: This demonstrates that MetaST is most impactful for low-resource settings with few training labels for a given task.

Original: [Table]
Revised: [Table]

Original: Ablation analysis.
Revised: Ablation analysis.

Original: Table 6 demonstrates the impact of different MetaST components with ablation studies.
Revised: Table 6 demonstrates the impact of different MetaST components with ablation analysis.

Original: We observe thatusing soft pseudo-labelsmay hurt the model performance compared to hard pseudo-labels, as also shown in recent work (Kumar et al., 2020).
Revised: We observe that soft pseudo-labels hurt the model performance compared to hard pseudolabels, as also shown in recent work (Kumar et al., 2020).

Original: Such a performance drop may be attributed to soft labels being less informative compared to sharpened labels.
Revised: Such a performance drop may be attributed to soft labels being less informative compared to sharpened ones.

Original: Removing the iterative teacher ﬁne-tuning step (Section 3.1) also hurts the overall performance.
Revised: Removing the iterative teacher ﬁne-tuning step (Section 3.1) also hurts the overall performance [Table]

Original: Re-weighting strategies .
Revised: Continued pre-training v.s. self-training. To contrast continued pre-training with self-training, we further pre-train BERT on in-domain unlabeled data and then ﬁne-tune it with few labeled examples denoted as “BERT (Continued Pre-training + Few-shot Supervision)”. The pre-training step improves the BERT performance over the baseline on SNIPS but degrades the performance on CoNLL03. This indicates that continued pre-training can improve the performance of few-shot supervised BERT on specialized tasks (e.g., SNIPS) with different data distribution than the original pre-training data (e.g., Wikipedia), but may not help for general domain ones like CoNLL03 with overlapping data from Wikipedia.

Original: To explore the role of re-weighting mechanism for the pseudo-labeled data discussed in Section 3.2, we perform an ablation study where we replace our meta-learning
Revised: In contrast to the above baseline, MetaST brings signiﬁcant improvements on both datasets. This demonstrates the generality and ﬂexibility of self-training over pre-training as also observed in contemporary work (Zoph et al., 2020) on image classiﬁcation. Adaptive labeled data acquisition . We perform an ablation study by removing adaptive labeled data acquisition from MetaST (denoted as “MetaST w/o Labeled Data Acq.”).

Original: [Figure]
Revised: Removing this component leads to around 2% performance drop on an average demonstrating the impact of labeled data acquisition. Moreover, the performance drop on SNIPS (39 slots) is larger than that on CoNLL03 ( slots). This demonstrates that adaptive acquisition is more helpful for tasks with more slot types – where diversity and data distribution necessitate a better exploration strategy in contrast to random sampling employed in prior meta-learning works. Re-weighting strategies .

Original:  component with different sample selection strategies based on the model conﬁdence for different tokens.
Revised: To explore the role of token-level re-weighting for pseudo-labeled sequences (discussed in Section 3.2), we replace our meta-learning component with different sample selection strategies based on the model conﬁdence for different tokens.

Original: One sampling strategy would choose samples uniformly without any re-weighting (referred to as MetaST w/o Re-weighting).
Revised: One sampling strategy chooses samples uniformly without any re-weighting (referred to as “MetaST w/o Re-weighting”).

Original: A sampling strategy with weights proportional to the model conﬁdence favors easy samples (referred to as MetaST-Easy), whereas the converse favors difﬁcult ones (referred to as MetaST-Difﬁcult).We observe the meta-learning based re-weighting strategy to perform the best.
Revised: The sampling strategy with weights proportional to the model conﬁdence favors easy samples (referred to as “MetaST-Easy”), whereas the converse favors difﬁcult ones (referred to as “MetaSTDifﬁcult”).We observe the meta-learning based re-weighting strategy to perform the best.

Original: Interestingly, MetaST-Easy outperforms MetaST-Difﬁcult signiﬁcantly on CoNLL (EN) but achieves slightly lower performance on SNIPS.
Revised: Interestingly, MetaST-Easy outperforms MetaST-Difﬁcult signiﬁcantly on CoNLL03 (EN) but achieves slightly lower performance on SNIPS.

Original: This may demonstrate that difﬁcult samples are more helpful when the quality of pseudo-labeled data is relatively high, whereas the sample selection strategy focusing on difﬁcult samples introduces noisy samples with lower pseudo-label quality.
Revised: This demonstrates that difﬁcult samples are more helpful when the quality of pseudo-labeled data is relatively high. On the converse, the sample selection strategy focusing on difﬁcult samples introduces noisy examples with lower pseudo-label quality.

Original: Therefore, sampling strategies may need to vary for different datasets, thereby, demonstrating the necessity of adaptive data re-weighting as in our framework MetaST
Revised: Therefore, sampling strategies may need to vary for different datasets, thereby, demonstrating the necessity of adaptive data re-weighting as in our framework MetaST.

Revised: Moreover, MetaST signiﬁcantly outperforms classic self-training strategies with hard and soft pseudo-labels demonstrating the effectiveness of our design.

Original: Analysis of pseudo-labeled data re-weighting.
Revised: Analysis of pseudo-labeled data re-weighting.

Original: To visually explore the adaptive re-weighting mechanism, we illustrate token re-weighting of MetaST on CoNLL03 (EN) dataset with K=10 shot at step 100 in Fig.
Revised: To visually explore the adaptive re-weighting mechanism, we illustrate token-level re-weighting of MetaST on CoNLL03 (EN) dataset with K= shot at step 100 in Fig.

Original: We include the re-weighting visualisation on SNIPS in Appendix A.1.
Revised: We include the re-weighting visualisation on SNIPS in Appendix A.1.

Original: We observe that the selection mechanism ﬁlters out most of the noisy pseudo-labels (colored in blue) even  with high teacher conﬁdence as shown in Fig.
Revised: We observe that the selection mechanism ﬁlters out most of the noisy pseudo-labels (colored in blue) even those with high teacher conﬁdence as shown in Fig.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Semi-supervised learning has been widely used for consistency training (Bachman et al., 2014;
Revised: Semi-supervised learning has been widely used for consistency training (Bachman et al., 2014;

Original: Rasmus et al., 2015; Laine & Aila, 2017; Tarvainen & Valpola, 2017; Miyato et al., 2018), latent variable models (Kingma et al., 2014) for sentence compression (Miao & Blunsom, 2016) and code generation (Yin et al., 2018).
Revised: Rasmus et al., 2015; Laine & Aila, 2017; Tarvainen & Valpola, 2017; Miyato et al., 2018), latent variable models (Kingma et al., 2014) for sentence compression (Miao & Blunsom, 2016) and code generation (Yin et al., 2018).

Original: More recently, methods like UDA (Xie et al., 2019) leverage consistency training for few-shot learning of instance-classiﬁcation tasks leveraging auxiliary resources like paraphrasing and back-translation (BT) (Sennrich et al., 2016).
Revised: More recently, methods like UDA (Xie et al., 2019) leverage consistency training for few-shot learning of instance-classiﬁcation tasks leveraging auxiliary resources like paraphrasing and back-translation (BT) (Sennrich et al., 2016).

Original: Sample selection.
Revised: Sample selection.

Original: Curriculum learning (Bengio et al., 2009) techniques are based on the idea of learning easier aspects of the task ﬁrst followed by the more complex ones.
Revised: Curriculum learning (Bengio et al., 2009) techniques are based on the idea of learning easier aspects of the task ﬁrst followed by the more complex ones.

Original: Prior work leveraging self-paced learning (Kumar et al., 2010) and more recently self-paced co-training (Ma et al., 2017) leverage teacher conﬁdence to select easy samples during training.
Revised: Prior work leveraging self-paced learning (Kumar et al., 2010) and more recently self-paced co-training (Ma et al., 2017) leverage teacher conﬁdence to select easy samples during training.

Original: Sample selection for image classiﬁcation tasks have been explored in recent works with meta-learning (Ren et al., 2018;
Revised: Sample selection for image classiﬁcation tasks have been explored in recent works with meta-learning (Ren et al., 2018;

Original: Li et al., 2019) and active learning (Panagiota Mastoropoulou, 2019; Chang et al., 2017b).
Revised: Li et al., 2019) and active learning (Panagiota Mastoropoulou, 2019; Chang et al., 2017b).

Original: However, all of these techniques rely on only the model outputs applied to instance-level classiﬁcation tasks.
Revised: However, all of these techniques rely on only the model outputs applied to instance-level classiﬁcation tasks.

Original: Semi-supervised sequence labeling.
Revised: Semi-supervised sequence labeling.

Original: Miller et al.
Revised: Miller et al.

Original: ; Peters et al.
Revised: ; Peters et al.

Original: (2017) leverage large amounts of unlabeled data to improve token representation for sequence labeling tasks.
Revised: (2017) leverage large amounts of unlabeled data to improve token representation for sequence labeling tasks.

Original: Another line of research introduces latent variable modeling (Chen et al., 2019;
Revised: Another line of research introduces latent variable modeling (Chen et al., 2019;

Original: Zhou & Neubig, 2017) to learn interpretable and structured latent representations.
Revised: Zhou & Neubig, 2017) to learn interpretable and structured latent representations.

Original: Recently, adversarial training based model
Revised: Recently, adversarial training based model

Original: SeqVAT (Chen et al., 2020) and cross-view training method CVT (Clark et al., 2018) have shown promising results for sequence labeling tasks.
Revised: SeqVAT (Chen et al., 2020) and cross-view training method CVT (Clark et al., 2018) have shown promising results for sequence labeling tasks.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: In this work, we develop an adaptive self-training framework MetaST that leverages self-training and meta-learning for few-shot training of neural sequence taggers.
Revised: In this work, we develop an adaptive self-training framework MetaST that leverages self-training and meta-learning for few-shot training of neural sequence taggers.

Original: We address the issue of error propagation from noisy pseudo-labels from the teacher in the self-training framework by adaptive sample selection and re-weighting with meta-learning.
Revised: We address the issue of error propagation from noisy pseudo-labels from the teacher in the self-training framework by adaptive sample selection and re-weighting with meta-learning.

Original: Extensive experiments on six benchmark datasets and different tasks including multilingual NER and slot tagging for task-oriented dialog systems demonstrate the effectiveness of the proposed method particularly for low-resource settings.
Revised: Extensive experiments on six benchmark datasets and different tasks including multilingual NER and slot tagging for task-oriented dialog systems demonstrate the effectiveness of the proposed method particularly for low-resource settings.

Original: A A PPENDIX A. E XPLORATIONS ON UNLABELED DATA AND MINI - BATCH S
Revised: A A PPENDIX A. E XPLORATIONS ON UNLABELED DATA AND MINI - BATCH S

Original: Variation in model performance with unlabeled data.
Revised: Variation in model performance with unlabeled data.

Original: Table 12 shows the improvement in model performance as we inject more unlabeled data with diminishing returns after a certain point.
Revised: Table 12 shows the improvement in model performance as we inject more unlabeled data with diminishing returns after a certain point.

Original: Variation in model performance with mini-batch S. We set the value of S in Eq.
Revised: Variation in model performance with mini-batch S. We set the value of S in Eq.

Original: 8 to { 1 , 3 , 5 } respectively to explore its impact on the re-weighting mechanism.
Revised: 8 to { 1 , 3 , 5 } respectively to explore its impact on the re-weighting mechanism.

Original: From Figure 3 we observe that the model is not super sensitive to hyper-parameter S but can achieve a better estimate of the weights of the pseudo-labeled data with increasing mini-batch values.
Revised: From Figure 3 we observe that the model is not super sensitive to hyper-parameter S but can achieve a better estimate of the weights of the pseudo-labeled data with increasing mini-batch values.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: A. A NALYSIS OF R E - WEIGHTING ON SNIPS AND C O N LL
Revised: A. A NALYSIS OF R E - WEIGHTING ON SNIPS AND C O N LL

Original: Analysis of pseudo-labeled data re-weighting.
Revised: Analysis of pseudo-labeled data re-weighting.

Original: To visually explore the adaptive re-weighting mechanism, we illustrate token re-weighting of MetaST on CoNLL03 and SNIPS datasets with
Revised: To visually explore the adaptive re-weighting mechanism, we illustrate token re-weighting of MetaST on CoNLL03 and SNIPS datasets with

Original: K=10 shot at step 100 in Fig. 4.
Revised: K=10 shot at step 100 in Fig. 4.

Original: Besides the observation in the experimental section, we observe that many difﬁcult and correct pseudo-labeled samples (low teacher conﬁdence) are selected according to Fig. 4a.
Revised: Besides the observation in the experimental section, we observe that many difﬁcult and correct pseudo-labeled samples (low teacher conﬁdence) are selected according to Fig. 4a.

Original: [Figure]
Revised: [Figure]

Original: A. K- SHOTS
Revised: A. K- SHOTS

Original: Effect of varying the number of few-shots K. We show the performance changes with respect to varying number of few-shots K { 5 , 10 , 20 , 100 } on Wikiann (en), MIT movie, MIT Restaurant,
Revised: Effect of varying the number of few-shots K. We show the performance changes with respect to varying number of few-shots K { 5 , 10 , 20 , 100 } on Wikiann (en), MIT movie, MIT Restaurant,

Original: CoNLL2003 (En), Multilingual CoNLL and Multilingual Wikiann in Table 9-13.
Revised: CoNLL2003 (En), Multilingual CoNLL and Multilingual Wikiann in Table 9-13.

Original: Since the number of labeled examples for some slots in Email dataset is around 10, we only show 5 and 10 shots for
Revised: Since the number of labeled examples for some slots in Email dataset is around 10, we only show 5 and 10 shots for

Original: Email dataset in Table 8.
Revised: Email dataset in Table 8.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A. I MPLEMENTATIONS AND H YPER - PARAMETER
Revised: A. I MPLEMENTATIONS AND H YPER - PARAMETER

Original: We do not perform any hyper-parameter tuning for different datasets.
Revised: We do not perform any hyper-parameter tuning for different datasets.

Original: The batch size and maximum sequence length varies due to data characteristics and are as shown in Tbale 14.
Revised: The batch size and maximum sequence length varies due to data characteristics and are as shown in Tbale 14.

Original: The hyperparameters are as shown in Table 14.
Revised: The hyperparameters are as shown in Table 14.

Original: Also, we retain parameters from original BERT implementation from https://github.com/ huggingface/transformers .
Revised: Also, we retain parameters from original BERT implementation from https://github.com/ huggingface/transformers .

Original: We implement SeqVAT based on https://github.com/jiesutd/NCRFpp  .[Table] [Table] [Figure]
Revised: We implement SeqVAT based on https://github.com/jiesutd/NCRFpp and implement CVT following https://github.com/tensorflow/models/tree/master/ research/cvt_text .

Original: [Table]
Revised: [Table]

Original: [Figure] [Table] [Table] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]