Self-Training with Weak Supervision (2021)

Abstract: State-of-the-art deep neural networks require large-scale labeled training data that is often expensive to obtain or not available for many tasks. Weak supervision in the form of domain-specific rules has been shown to be useful in such settings to automatically generate weakly labeled training data. However, learning with weak rules is challenging due to their inherent heuristic and noisy nature. An additional challenge is rule coverage and overlap, where prior work on weak supervision only considers instances that are covered by weak rules, thus leaving valuable unlabeled data behind. In this work, we develop a weak supervision framework (ASTRA) that leverages all the available data for a given task. To this end, we leverage task-specific unlabeled data through self-training with a model (student) that considers contextualized representations and predicts pseudo-labels for instances that may not be covered by weak rules. We further develop a rule attention network (teacher) that learns how to aggregate student pseudo-labels with weak rule labels, conditioned on their fidelity and the underlying context of an instance. Finally, we construct a semi-supervised learning objective for end-to-end training with unlabeled data, domain-specific rules, and a small amount of labeled data. Extensive experiments on six benchmark datasets for text classification demonstrate the effectiveness of our approach with significant improvements over state-of-the-art baselines.

## introduction the success of state-of-the-art neural networks crucially hinges on the availability of large amounts of annotated training data. while recent advances on language model pre-training @xcite ; ## self-training with weak supervision we now present our approach, astra, that leverages a small amount of labeled data, a large amount of unlabeled data, and domain-specific heuristic rules. our architecture has two main components: the base student model (section 2.1) and the rule attention teacher network (section 2.2), which are iteratively co-trained in a self-training framework. formally, let x denote the instance space and @xmath2, . . . , k} denote the label space for a kclass classification task. we consider a small set of manually-labeled examples d @xmath3, y l )}, where x @xmath4. we also consider a set of pre-defined heuristic rules @xmath5, where each rule r j has the general form of a labeling function that considers as input an instance x @xmath6, and ran (teacher) (3) semi-supervised ran training few labeled data (1) self-training (2) rule attention network rule embeddings rule attention rule weights instance x i q j i α j i rules r j rule labels q i ran prediction unlabeled data student pseudo-labeled data few labeled data either assigns a weak label q j @xmath7, 1} k (one-hot encoding) or does not apply, i.e., does not assign a label for x i . our goal is to leverage d l , d u , and r to train a classifier that, given an unseen test instance @xmath8, predicts a label @xmath9. in the rest of this section, we present our astra framework for addressing this problem. ## experiments datasets. we evaluate our framework on the following six benchmark datasets for weak supervision from @xcite and @xcite . ( 1 on average across all the datasets, 66% of the instances are covered by fewer than 2 rules, whereas 40% are not covered by any rule at all -demonstrating the sparsity in our setting. we also report the accuracy of the rules in terms of majority voting method learning to weight unlabeled rules instances (no rules) majority ---snorkel @xcite --posteriorreg @xcite --l2r @xcite --implyloss @xcite -self-train --astra on the task-specific unlabeled datasets. additional details on the dataset and examples of rules are presented in the appendix. evaluation. we train astra five times for five different random splits of the labeled training data and evaluate on held-out test data. we report the average performance as well as the standard deviation across multiple runs. we report the same evaluation metrics as used in prior works @xcite for a fair comparison. model configuration. our student model consists of embeddings from pre-trained language models like elmo @xcite or bert @xcite for generating contextualized representations for an instance, followed by a softmax classification layer. the ran teacher model considers a rule embedding layer and a multilayer perceptron for mapping the contextualized representation for an instance to the rule embedding space. refer to the appendix for more details. baselines. we compare our method with the following methods: (a) majority predicts the majority vote of the rules with ties resolved by predicting a random class. (b) labeledonly trains classifiers using only labeled data (fully supervised baseline). (c) self-train @xcite leverages both labeled and unlabeled data for iterative self-training on pseudolabeled predictions over task-specific unlabeled data. this baseline ignores domain-specific rules. (e) snorkel+labeled @xcite trains classifiers using weakly-labeled data with a generative model. the model is trained on unlabeled data for computing rule weights in an unsupervised fashion, and learns a single weight per rule across all instances. it is further fine-tuned on labeled data. (f) l2r @xcite learns to re-weight noisy or weak labels from domain-specific rules via meta-learning. it learns instance-specific but not rule-specific weights. (g) posteriorreg @xcite trains classifiers using rules as soft constraints via posterior regularization @xcite . (h) implyloss @xcite leverages exemplar-based supervision as additional knowledge for learning instance-specific and rulespecific weights by minimizing an implication loss over unlabeled data. this requires maintaining a record of all instances used to create the weak rules in the first place. ## related work in this section, we discuss related work on selftraining and learning with noisy labels or rules. refer to @xcite for a thorough survey of approaches addressing low-resource scenarios. self-training. self-training @xcite @xcite as one of the earliest semi-supervised learning approaches @xcite trains a base model (student) on a small amount of labeled data; applies it to pseudo-label (task-specific) unlabeled data; uses pseudo-labels to augment the labeled data; and re-trains the student in an iterative manner. self-training has recently been shown to obtain state-of-the-art performance for tasks like image classification @xcite @xcite , few-shot text classification (mukherjee and awadallah, 2020; @xcite , and neural machine translation @xcite and has shown complementary advantages to unsupervised pre-training @xcite . a typical issue in self-training is error propagation from noisy pseudo-labels. this is addressed in astra via rule attention network that computes the fidelity of pseudo-labels instead of directly using them to re-train the student. learning with noisy labels. classification under label noise from a single source has been an active research topic @xcite . a major line of research focuses on correcting noisy labels by learning label corruption matrices @xcite @xcite . more related to our work are the instance reweighting approaches @xcite , which learn to up-weight and down-weight instances with cleaner and noisy labels respectively. however, these operate only at instance-level and do not consider rule-specific importance. our approach learns both instance-and rule-specific fidelity weights and substantially outperforms ren text what was president lyndon johnson 's reform program called ? clean label enty astra teacher enty weak source label weight feature / regular expression pattern student enty @xmath10.0 hi (contextualized instance embedding) rule 8 hum @xmath11.0 ( |^)(who|what|what)[^\w] * (\w+ ){0,1}(person| man|woman|human|president|president)[^\w] * ( |@xmath0) rule 42 desc @xmath12.0 ( |^)(explain|describe|what|what)[^\w] * ( |@xmath1) ## conclusions and future work we developed a weak supervision framework, astra, that efficiently trains classifiers by integrating task-specific unlabeled data, few labeled data, and domain-specific knowledge expressed as rules. our framework improves data coverage by employing self-training with a student model. this considers contextualized representations of instances and predicts pseudo-labels for all instances, including those that are not covered by heuristic rules. additionally, we developed a rule attention network, ran, to aggregate various weak sources of supervision (heuristic rules and student pseudo-labels) with instance-specific weights, and employed a semi-supervised objective for training ran without strong assumptions about the nature or structure of the weak sources. extensive experiments on several benchmark datasets demonstrate our effectiveness, particularly at high levels of rule sparsity. in future work, we plan to extend our framework to support a broader range of natural language understanding tasks and explore alternative techniques for rule embedding.