Fixing Model Bugs with Natural Language Patches (2022)

Abstract: Current approaches for fixing systematic problems in NLP models (e.g., regex patches, finetuning on more data) are either brittle, or labor-intensive and liable to shortcuts. In contrast, humans often provide corrections to each other through natural language. Taking inspiration from this, we explore natural language patches—declarative statements that allow developers to provide corrective feedback at the right level of abstraction, either overriding the model (“if a review gives 2 stars, the sentiment is negative”) or providing additional information the model may lack (“if something is described as the bomb, then it is good”). We model the task of determining if a patch applies separately from the task of integrating patch information, and show that with a small amount of synthetic data, we can teach models to effectively use real patches on real data—1 to 7 patches improve accuracy by ~1–4 accuracy points on different slices of a sentiment analysis dataset, and F1 by 7 points on a relation extraction dataset. Finally, we show that finetuning on as many as 100 labeled examples may be needed to match the performance of a small set of language patches.

## introduction natural language enables humans to communicate a lot at once with shared abstractions. for example, in teaching someone about the colloquial use of the term "bomb", we might say describing food as 'bomb' means it is very good, while saying someone bombed means it was disappointing. this simple sentence uses various abstractions (e.g., "food") to provide context-dependent information, making it easy for humans to generalize and understand sentences such as "the tacos were bomb" or "the chef bombed" without ever having seen such examples. ## training patchable models assuming f has a text encoder and a classification head, we have two finetuning stages. in the task finetuning stage, we train f on a labeled dataset {x i , y i } (standard supervised learning). in the patch finetuning stage, we use the learnt encoder and learn g (initialized randomly) and i (initialized with the classification head). for the patch finetuning stage, we write a small set of patch templates covering the kinds of patches users may write for their own application (see if food is described as wug, then food is good y: positive x: service was alright, but food was wug patch: if food is described as wug, then food is bad y: negative x: service was alright, but food was yummy y: positive x: service was alright, but food was greasy patch: if food is described as greasy, then food is bad y: negative x: food was wug, but service was horrible patch: if food is described as wug, then food is good y: negative x: service was alright, but food was wug patch: if food is described as wug, then food is bad y: negative x: service was amazing, even though food was wug patch: if food is described as wug, then food is bad y: positive if food is described as wug, then food is good y: positive patch: if food is described as yummy, then food is good @xcite x: service was alright, but food was wug patch: if food is described as wug, then food is good y: positive x: service was alright, but food was wug patch: if food is described as wug, then food is bad y: negative x: service was alright, but food was yummy y: positive x: service was alright, but food was greasy patch: if food is described as greasy, then food is bad y: negative if food is described as wug, then food is good y: negative x: service was alright, but food was wug patch: if food is described as wug, then food is bad y: negative x: service was amazing, even though food was wug patch: if food is described as wug, then food is bad y: positive if food is described as wug, then food is good y: positive patch: if food is described as yummy, then food is good @xcite figure 2 : a model can learn from just the labels that "yummy" and "greasy" are positive and negative words respectively, and learn to perfectly fit training data without ever using patch features (a, top). this behavior can be explicitly prevented via eits (a, bottom). a model may also fit the data without using the input features by always predicting 1 / 0 for "food is good" / "food is bad" (a, top/bottom). thus, we additionally ensure that the label cannot be inferred from the patch alone (b). entropy increasing transformations. patch finetuning will fail if the synthetic data can be fit by a model that ignores the input or the patch (fig. 2a ). thus, to ensure our model cannot fit the synthetic data without combining patch features with inputs, we perturb the inputs with entropy increasing transformations (eits). we identify words from the input template for which the patch supplies additional information e.g., aspect adjectives, relationship between entities, and transform these into a small set of nonce words. crucially, the meanings of these nonce words vary from example to example, and can only be inferred from the patch (fig. 2a bottom; more examples in appendix a.2). intuitively, the transformations inject an additional source of randomness which can only be recovered via the patch features. such transformations are also used in @xcite in the context of meta-learning. eits alone do not fix the failure mode where the model can fit the data without using input features at all. for example, in fig. 2a bottom, the model might learn a shortcut so that it always predicts 1/0 for "food is good" / "food is bad", regardless of the input. thus, in addition to eits, to ensure that the model uses input features, we ensure that a given patch consequence q and the target label are independent (fig. 2b ). ## experimental setup applications. we apply our method to binary sentiment analysis and relation extraction. for sentiment analysis, our task finetuning data comes from sst2 @xcite . for relation extraction, we use the spouse dataset @xcite for task finetuning, where the objective is to determine whether two entities are married or not given a textual context about them. model. we use t5-large @xcite as implemented in the transformers library @xcite for all experiments. both the gating and interpreter heads are separate decoders learnt on top of a shared encoder and each of these components are initialized with the corresponding t5 pre-trained weights. to prevent catastrophic forgetting on the original task during patch finetuning, we also multi-task learn the patch finetuning loss along with the original task loss. templates for generating patches for patch finetuning are in baselines. we report performance of the original model with only task finetuning (orig) and the model obtained after patch finetuning (orig+pf) without using any patches, to isolate the gains of language patches from those induced by training on additional synthetic data. we also report results obtained from prompting orig with our patches (prompt), i.e., inserting the patch text before the input text to see how well finetuned t5 follows instructions. to use multiple patches for this baseline, we prompt the model with each individual patch and ensemble results with majority voting. finally, we experiment with regex-based patches (regex) where patch conditions are converted to regex rules and consequents are converted into functions rule q (x). for override patches, this function simply outputs the specified label. for sentiment analysis, where feature based patches supply contextual meanings, rule q (x) replaces words with specified meanings e.g., replacing "bomb" with "good" in "the food was bomb". for feature based patches on relation extraction, rule q (x) appends the patch consequent to the input text. ## controlled experiments we test the behavior of language patches (and baselines) under different controlled conditions with checklist @xcite . patches and example inputs are presented in fig. 3 . we test cases where patches apply and are relevant for predictions, and corresponding cases where they either do not apply or are not relevant. thus, models that rely on shortcuts such as copying the label word from the patch or merely performing token matching perform poorly on the checklist. for sentiment analysis, we test override patches with abstract conditions (e.g., "if food is described as weird, then label is negative" ) on various concrete instantiations such as "the pizza at the restaurant was weird". we also construct invariance tests (o-inv), where adding such patches should not change predictions on inputs where the condition is false (e.g., "the waiter was weird", "the tacos were not weird"). we also construct tests for feature-based patches (feat) where patches provide meaning for nonce adjectives, with analogous invariance tests (feat-inv). finally, we construct analogous tests for relation extraction, where patches fill in reasoning gaps in the model such as "if entity1 gave entity2 a ring, then entity1 and entity2 are engaged". we present the results in ## patching models on real benchmarks 6.1 sentiment analysis unless noted otherwise, all datasets in this subsection are derived from yelp review @xcite . to fix errors on low-accuracy slices, we write patches by inspecting a random subset of 10-20 errors made by orig+pf. controlling the model. in order to check if patches can control model behavior with abstract conditions "in the wild", we manually annotate a random subset of 500 reviews with food and service specific sentiment ("the food was good, service not so much" is labeled as service: 0, food: 1). we then construct override patches of the form "if food / service is good / bad, then label is positive / negative", and evaluate models as to how often (on average) the prediction is as expected when the patch applies and how often it is unchanged when the patch does not apply. we present results in patching low-accuracy slices. we identify slices where our base model has (comparatively) low accuracy, and check whether patches can improve performance. yelp-stars consists of all examples in yelp review with the word 'star' present. for this subset, we use two overrides patch: "if review gives 1 or 2 stars, then label is negative", "if review gives 0 stars, then label is negative". yelp-colloquial in ## analysis the goal of entropy increasing transformations (eits; section 3) is to prevent the interpreter head from learning shortcuts that either ignore patch features or rely exclusively on them. we perform an ablation, comparing our model to a model trained without eits on the checklists in ## conclusion when faced with the task of fixing bugs in trained models, developers often resort to brittle regex rules or finetuning, which requires curation and labeling of data, is computationally intensive, and susceptible to shortcuts. this work proposes natural language patches which are declarative statements of the form "if c, then q" that enable developers to control the model or supply additional information with conditions at the right level of abstraction. we proposed an approach to patching that models the task of determining if a patch applies (gating) separately from the task of integrating the information (interpreting), and showed that this approach results in significant improvements on two tasks, even with very few patches. moreover, we show that patches are efficient (1-7 patches are equivalent or better than as many as 100 finetuning examples), and more robust to potential shortcuts. our system is a first step in letting users correct models through a single step "dialogue". avenues for future work include extending our approach to a back-and-forth dialogue between developers and models, modeling pragmatics, interpreting several patches at once, and automating patch finetuning.