Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning (2022)

Abstract: Weakly-supervised learning (WSL) has shown promising results in addressing label scarcity on many NLP tasks, but manually designing a comprehensive, high-quality labeling rule set is tedious and difficult. We study interactive weakly-supervised learning—the problem of iteratively and automatically discovering novel labeling rules from data to improve the WSL model. Our proposed model, named PRBoost, achieves this goal via iterative prompt-based rule discovery and model boosting. It uses boosting to identify large-error instances and discovers candidate rules from them by prompting pre-trained LMs with rule templates. The candidate rules are judged by human experts, and the accepted rules are used to generate complementary weak labels and strengthen the current model. Experiments on four tasks show PRBoost outperforms state-of-the-art WSL baselines up to 7.1%, and bridges the gaps with fully supervised models.

## introduction weakly-supervised learning (wsl) has recently attracted increasing attention to mitigate the label scarcity issue in many nlp tasks. in wsl, the training data are generated by weak labeling rules obtained from sources such as knowledge bases, frequent patterns, or human experts. the weak labeling rules can be matched with unlabeled data to create large-scale weak labels, allowing for training nlp models with much lower annotation cost. wsl has recently achieved promising results in many tasks including text classification @xcite @xcite , relation extraction @xcite , and sequence tagging @xcite @xcite . despite its success, wsl is limited by two major factors: 1) the labeling rules, and 2) the static learning process. first, it is challenging to provide a comprehensive and high-quality set of labeling rules a priori. labeling rules are often humanwritten @xcite , but the process of writing labeling rules is tedious and time-consuming even for experts. a few works attempt to automatically discover labeling rules by mining labeled data @xcite , or enumerating predefined types. however, the preextracted rules are restricted to frequent patterns or predefined types, which are inadequate for training an accurate model. second, most existing wsl methods are static and can suffer from the noise in the initial weak supervision @xcite @xcite @xcite . as the labeling rule set remains fixed during model training, the initial errors can be amplified, resulting in an overfitted end model. interactive rule discovery has been explored in two recent works @xcite , which solicits human feedback on candidate rules to refine the rule set. unfortunately, their rule forms are limited to simple repetitive structures such as n-grams @xcite , and the huge rule search space makes an enumerating-pruning pipeline not scalable for large datasets @xcite . due to the above reasons, state-of-the-art wsl methods still underperform fully-supervised methods by significant gaps on many nlp tasks. as shown in a recent study @xcite , the best wsl methods fall behind the best fullysupervised methods in 15 out of 18 nlp benchmarks; and the average performance gap is 18.84% in terms of accuracy or f1 score. to bridge the gap between weakly-supervised and fully-supervised approaches, we propose an iterative rule discovery and boosting framework, namely prboost for interactive wsl. compared to existing works on wsl and active learning, pr-boost features three key designs: first, we design a rule discovery module that uses rule templates for prompting pre-trained language models (plms). by feeding difficult instances and rule templates into plms, the module distills knowledge from plms via prompting and generates candidate rules that capture key semantics of the input instances. compared to prior works based on n-grams @xcite , our prompt-based rule discovery is more expressive and applicable to any tasks that support prompting. second, we design a boosting-style ensemble strategy to iteratively target difficult instances and adaptively propose new rules. in each iteration, we reweigh data by the boosting error to enforce the rule discovery module to focus on larger-error instances. this avoids enumerating all the possible rules and implementing post-filtering for novel rules, but directly targets rule discovery on largeerror instances to provide complementary information to the current model. third, we strategically solicit human feedback to evaluate the candidate rules. humans are asked to judge whether a candidate rule should be accepted or abstained. the accepted high-quality rules are then used to generate new weak labels that are fed into boosted model training. as the promptgenerated rules are highly interpretable, the rule evaluation is simply a binary choice task for human experts and thus effortless. unlike traditional active learning methods that annotate individual instances, such a rule-level annotation is more labelefficient because the annotated rules can match large amounts of instances. we compare our method with supervised, weakly-supervised and interactive learning baselines on four tasks: relation extraction, ontology classification, topic classification, and chemicalprotein interaction prediction. the results show: 1) our method outperforms state-of-the-art weaklysupervised baselines by up to 7.1%; 2) the rulelevel annotation helps the model achieve higher model performance compared to the instance-level annotation under the same budget; 3) the machinediscovered and human-evaluated rules are of high quality, which consistently refine the weak labels and the model in each iteration. our key contributions are: (1) a prompt-based rule discovery framework for interactive wsl, which provides flexible rule representation while capturing subtle semantics in rule generation; (2) an iterative boosting strategy for discovering novel rules from hard instances and strengthening the model by an ensemble of complementary weak models; (3) an interpretable and easy-to-annotate interactive process for rule annotation; (4) comprehensive experiments demonstrating the effectiveness of our framework. ## related work weakly-supervised learning wsl has recently attracted much attention in various nlp tasks. despite their promising performance on various tasks, manually designing the rules can be timeconsuming. moreover, the noise and incompleteness of the initial rules could be propagated in model training @xcite . a few works attempt to reduce human efforts in manually designing labeling rules by discovering rules from data. for example, snuba @xcite generates heuristics based on a small labeled dataset with pre-defined rule types; tallor @xcite and glara @xcite study rule expansion for ner problem based on lexical information and then select rules based on a hand-tuned threshold. however, these methods discover rules in a static way and are constrained to task-specific rule types. in contrast, our framework discovers rules iteratively from the entire unlabeled dataset, which can refine the rule set and enlarge its diversity on-the-fly. interactive learning our work is related to active learning (al) as both involve human annotators in the learning process. however, the key difference is that al labels instances based on various query policies @xcite @xcite @xcite , while our method does not annotate individual instances, but uses annotated rules to match unlabeled data. this makes our method more label-efficient in leveraging human feedback for creating large-scale labeled data. to the best of our knowledge, only a few works have studied interactive wsl @xcite @xcite as in our problem. however, they either use simple n-gram based rules @xcite that fail to capture sentence-level semantics, or suffer from a huge searching space for context-free grammar rules @xcite . unlike these works, our method uses flexible rule representations based on prompts, and also uses boosting for targeted rule discovery to avoid enumerating all possible rules and performing post-filtering for novel rules. language model prompting our work is also related to prompt-based learning for plms, which converts the original task to a cloze-style task and leverages plms to fill the missing information @xcite . prompting has been explored in various tasks, including text classification @xcite @xcite , information extraction @xcite ) and text generation @xcite . recent works focus on generating better prompt templates or learning implicit prompt embeddings @xcite . however, none of these works studied prompting for generating weak labels. our work is orthogonal to them since we do not aim to optimize prompts for the original task, but uses prompts and plms as a knowledge source for rule discovery. ## preliminaries problem formulation weakly-supervised learning (wsl) creates weak labels for model training by applying labeling rules over unlabeled instances d u . given an unlabeled instance @xmath0, a labeling rule r(•) maps x into an extended label space: r(x) → @xmath1. here y is the original label set for the task, and 0 is a special label indicating x is unmatchable by r. given a set r of labeling rules, we can apply each rule in r on unlabeled instances to create a weakly labeled dataset d l . however, the initial weak labels d l can be highly noisy and incomplete, which hinder the performance of wsl. we thus study the problem of interactive wsl: how can we automatically discover more high-quality labeling rules to enhance the performance of wsl? besides d u and d l , we also assume access to a small set of clean labels for example, keyword-based rules are widely used to map certain keywords to their highly correlated labels @xcite @xcite . regular expression is another common rule format, which matches instances with pre-defined surface patterns @xcite @xcite . logical rules @xcite perform logical operations (such as conjunction ∧ and negation ¬) over atomic rules and can thus capture higher-order compositional patterns. we adopt a prompt-based rule representation (section 4.1), which is flexible to encompass any existing rule representations. our prompt-based rule relies on a rule template τ (•) for the target task, which contains a [mask] token to be filled by a plm m along with an unlabeled instance x. from the rule template τ , each candidate rule can be automatically derived by @xmath2, τ, x). such a prompt-based rule representation is highly flexible and can be applied to any nlp tasks that support prompting (see examples in ## methodology overview prboost is an iterative method for interactive wsl. in each iteration, it proposes candidate rules from large-error instances, solicits human feedback on candidate rules, generates weak labels, and trains new weak models for ensembling. figure 1 shows the process in one iteration of pr-boost, which relies on three key components: large-error instance 𝒙 𝒆 𝒊 microsoft is an american technology corporation founded by bill gates. founded started called    rule1: person [founded] organization prompt template 𝒙 𝒑 𝒊 [input] the person bill gates [mask] the organization microsoft. rule2: person [started] organization clean data 𝒟 𝑙 data weights 𝒘 𝒊 unmatched data 𝒟 𝑢 rule-matched data 𝒟 𝑟 model 𝑚 𝑡-1 model 𝑚 𝑡 self-training model ensemble 𝒑 @xmath3. candidate rules generation 2. interactive rule evaluation 3. weakly supervised model training & ensemble human annotators 𝒟 𝑟 ∪ 𝒟 𝑡-1 figure 1: overall framework for prboost. in each iteration, prboost (1) identifies large-error instances from the limited clean data and converts each large-error instance to a prompt template for prompting-based rule discovery; (2) presents candidate rules to human experts for annotation and uses accepted rules to generate new weak labels; (3) trains a new weak model with self-training and ensembles it with the previous models. ## conclusion we proposed prboost to iteratively discover prompt-based rules for interactive weaklysupervised learning. through a boosting-style ensemble strategy, it iteratively evaluates model weakness to identify large-error instances for new rule proposal. from such large-error instances, its prompt-based rule discovery module leads to expressive rules that can largely improve rule coverage while being easy to annotate. the discovered rules complement the current rule set and refine the wsl model continuously. our experiments on four benchmarks demonstrate that prboost can largely improve wsl and narrow the gaps between wsl models and fully-supervised models.