Robustness of Demonstration-based Learning Under Limited Data Scenario (2022)

Abstract: Demonstration-based learning has shown great potential in stimulating pretrained language models’ ability under limited data scenario. Simply augmenting the input with some demonstrations can significantly improve performance on few-shot NER. However, why such demonstrations are beneficial for the learning process remains unclear since there is no explicit alignment between the demonstrations and the predictions. In this paper, we design pathological demonstrations by gradually removing intuitively useful information from the standard ones to take a deep dive of the robustness of demonstration-based sequence labeling and show that (1) demonstrations composed of random tokens still make the model a better few-shot learner; (2) the length of random demonstrations and the relevance of random tokens are the main factors affecting the performance; (3) demonstrations increase the confidence of model predictions on captured superficial patterns. We have publicly released our code at https://github.com/SALT-NLP/RobustDemo.

## introduction current large pretrained language models (plms) struggle to learn nlp tasks under limited data scenarios @xcite @xcite @xcite . in contrast, humans can solve natural language tasks with only a few illustrative examples @xcite . motivated by this, demonstration-based learning has been introduced to augment the input with a few examples and labels. for instance, @xcite simply picked up to 32 randomly sampled instances and directly concatenated them with the input to perform in-context learning with the model frozen and significantly boosted the performance. @xcite concatenated the input with task demonstrations to create augmented input and fed them into plms to obtain improved token representations to do sequence labeling in a building on these prior works, we take a deeper dive into the robustness of demonstration-based learning @xcite , especially for structured prediction tasks like named entity recognition (ner). demonstrations might not be robust for more structured prediction settings since these limited amounts of examples might not include much inductive bias. also, using classifier-based fine-tuning demonstrations could be even more unreliable since there is no alignment between the demonstrations and the prediction space. concretely, we investigate the robustness of demonstration-based sequence labeling by design-ing pathological demonstrations: gradually ruling out the helpful information from the demonstrations. we surprisingly find that a working demonstration does not need to contain correct examples to observe improved performance. furthermore, randomly replacing every token in the demonstration can still make a better few-shot learner even it is no longer a meaningful sentence and does not make any sense (figure 1 ). this observation conflicts with some existing hypotheses @xcite that models are learning meaningful knowledge from these demonstrations. we also find that the length of the pathological demonstration and the relevance of its random tokens drastically affect the performance. empirical results on name regularity bias (nrb) diagnose dataset @xcite shows that the demonstrations rarely help the performance when there is no easy patterns. additionally, we show the pathological demonstrations can obtain similar or better performance on nlu tasks such as classification and natural language inference. in summary, our empirical results encourage the rethinking on how the demonstration helps the model obtain better few-shot capability and provides some insights. ## problem definition this section focuses on demonstration-based learning under limited data scenario, by introducing concepts of limited data sequence labeling tasks in section 3.1, as well as describing traditional token classification methods in section 3.2 and demonstration-based learning in section 3.3. ## pathological demonstrations we refer to the demonstration constructed in section 3.3 as standard (st) demonstration. suppose the model leverages the demonstrations in a humananalogous way and understands the meaning of them, there will be no more performance gains if we no longer provide correct example-label pairs or actual examples. to this end, we design three pathological demonstrations by gradually removing such intuitively helpful information from st demonstrations: we show templates and examples for these pathological demonstrations modified from standard demonstration for ner in ## understanding the demonstrations we take a closer look at the surprising performance of (pathological) demonstrations to examine whether such strong performance has any connections with spurious patterns or dataset bias, which the deep learning models are constantly being accused of leveraging @xcite . as a case study, we use a carefully designed testbed nrb @xcite to diagnose name regularity bias in the ner models learned with demonstrations (section 6.1). we also conduct experiments on the more popular way of utilizing demonstrations with prompts in section 6.2 to show the effectiveness of pathological demonstrations. ## discussion and conclusion in this paper, we study the robustness of demonstration-based learning by designing pathological demonstrations. we found that, replacing demonstrations with random tokens still makes the model a better few-shot learner; the length of random token strings and the sampling space for random tokens are the main factors affecting the performance; and demonstrations increase the confidence of model predictions on captured superficial patterns such as the name regularity bias. below we discuss the broader impacts of our findings. our findings imply that natural language is not sufficiently understood by the plms when being utilized together with demonstrations, since random token strings also lead to similar strong performances. similar to our work, recent studies (ri and tsuruoka, 2022; chiang and lee, 2022) also found models pretrained on random token strings with a nesting dependency structure still provide transferable knowledge to downstream fine-tuning, suggesting the insufficient utilization of natural language during pretraining. we urge future work on demonstration based learning to think twice about their model performance gains by designing more robustness tests and ablation studies. our work also calls for a better design of demonstrations that are free of spurious patterns, as existing demonstrations are less effective while there is no spurious patterns to leverage (see section 6.1). future work should not only optimize the performances of demonstration based methods on the validation set, but also pay attention to whether these models suffer from spurious patterns and how to increase their generalization abilities.