Robustness of Learning from Task Instructions
Year: 2023

Abstract:
Traditional supervised learning mostly works on individual tasks and requires training on a large set of task-specific examples. This paradigm seriously hinders the development of task generalization since preparing a taskspecific example set is costly. To build a system that can quickly and easily generalize to new tasks, textual instructions have been adopted as an emerging trend of supervision recently. These instructions give the model the definition of the task and allow the model to output the appropriate answer based on the instructions and inputs. However, task instructions are often expressed in different forms, which can be interpreted from two threads (Lou et al., 2023): firstly, some instructions are short sentences and are pretrained language model (PLM) oriented, such as prompts, while other instructions are paragraphs and are human-oriented, such as those annotation guidelines in Amazon MTurk; secondly, different end-users very likely explain the same task with instructions of different textual expressions. A robust system for task generalization should be able to handle any new tasks regardless of the variability of instructions. However, the system robustness in dealing with a wide range of instructions is still unexplored. This work investigates the system robustness when the instructions of new tasks are (i) manipulated, (ii) paraphrased, or (iii) from different levels of conciseness. To our knowledge, this is the first work that systematically studies how robust a PLM is when it is supervised by instructions with different factors of variability.1

heading: 1 Introduction
text: One goal of AI is to achieve cross-task generalization, i.e., the model can handle new tasks after being trained on existing tasks. To solve target tasks, the system needs task-specific supervision.
1All code and data: https://github.com/jiashenggu/ Tk-Instruct/tree/main
In the conventional machine learning paradigm, task-specific supervision means a large set of labeled examples particularly for this task. Despite the progress of conventional machine learning, collecting task-specific examples of large volumes is costly and even infeasible if we want the system to cope with new tasks immediately. Therefore, another form of task-specific supervision—task instructions—is attracting increasing attention in the community. Task instructions, commonly a piece of text, describe the task semantics and tell the model how to approach the expected outputs for the inputs. A couple of recent studies (Mishra et al., 2022a; Yin et al., 2022; Wang et al., 2022; Sanh et al., 2022) have shown promising progress in cross-task generalization with only instructions as supervision.
In the real world, task instructions are often presented in different forms, which can be interpreted from two dimensions: on the one hand, as Lou et al. (2023) defined, instructions have different degrees of conciseness and were created for different audiences—some instructions are short sentences and are PLM-oriented, such as prompts like “translate from English to French”, while other instructions are paragraphs and are human-oriented, e.g., those annotation guidelines in Amazon MTurk; on the other hand, the same task is likely to be explained by different end-users with instructions of varying textual expressions. For example, “translate from English to French” can also be expressed as “given an English sentence, translate it into French”. Therefore, a robust system for task generalization is expected to handle any new tasks regardless of the variability of instructions.
However, despite the limited earlier work studying the PLM robustness in terms of prompts (Webson and Pavlick, 2022; Khashabi et al., 2022), it is still unclear how robust a pretrained instruction learning system can handle broader types of instructions, including human-oriented as well as
13935
PLM-oriented ones, and their cross-type generalization. This work focuses on human-oriented instructions and explores the system robustness when the instructions of new tasks are (i) manipulated, e.g., words removal, words replacement, sentence shuffle, etc., (ii) paraphrased with dramatic surface change, or (iii) from different levels of conciseness, e.g., training tasks have human-oriented instructions while testing tasks are equipped with PLM-oriented prompts.
To get the answers for the above three threads of instruction variability, we finetune a T5 (Raffel et al., 2020) model on the train of SUPERNATURALINSTRU (Wang et al., 2022), a dataset with the largest number of NLP tasks and each task is described by an Amazon-Mturk-style instruction, and test it on (i) the test of SUPERNATURALINSTRU, where we modify the original instructions with word-level, sentence-level and instruction-level perturbations, (ii) a newly crowdsourced dataset, Para-Instructions, with 23 unseen tasks and each has 4 distinct human-oriented instructions in the same format as the test of SUPER-NATURALINSTRU, and (iii) a new dataset DIVERSEPROMPT (Honovich et al., 2022), in which each task has prompt-style instructions.
To our knowledge, this is the first work that systematically studies how robust a PLM is when it is supervised by instructions with different factors of variability. Our experiments show that:
• The presence of demonstrations minimizes the impact of instruction perturbations and paraphrasing, as they typically have an unpredictable effect on performance. Demonstrations play a crucial role in ensuring system robustness, even when the instruction provided is a random text fragment, i.e., “random instruction + positive examples” mostly outperforms “correct instruction only”;
• When demonstrations are absent, applying a model that learns from training tasks with human-oriented instructions to unseen tasks with PLM-oriented prompts works poorly;
• So far, we still lack a method to effectively learn instruction to solve tasks in the absence of demonstration.

heading: 2 Related work
text: Our work is related to prompts (a special case of instruction learning), cross-task generalization, and
the study of robustness under adversarial attack.
Prompt as instruction. Brown et al. (2020a) attempted to unleash the power of PLMs, such as GPT-3 (Brown et al., 2020b), through prompts. Since they discovered prompts, many studies have followed that direction in various ways, with robustness being an aspect that cannot be ignored. Liu et al. (2021) introduced the effectiveness of prompt learning and the wide application prospects. Min et al. (2022) explored how the model learns prompts and what aspects of the prompt contribute to the final task performance. Webson and Pavlick (2022) tried to figure out whether promptbased models truly understand the meaning of their prompts. Zhao et al. (2021) investigated ways to improve the stability of prompt-based models. Bach et al. (2022) created a platform for the production, distribution, and use of natural language prompts. Despite the large number of papers on prompts, little attention has been paid to studying the robustness of systems under prompt learning. Our work has a broader vision that considers the robustness in terms of PLM-oriented prompts as well as more natural human-oriented instructions.
Instruction-driven task generalization. Recent work has worked on using the information in task instructions to allow PLMs to respond to inputs that match the task instructions after they have understood task instructions. Efrat and Levy (2020) tested GPT-2 (Radford et al., 2019) to understand real-world MTurk instructions to annotate some popular datasets, and concluded that GPT-2 works poorly. Mishra et al. (2022b) collected more than 60 distinct NLP tasks with MTurk instructions consisting of items like title, definition, things to avoid, etc., and claimed that BART (Lewis et al., 2020) and GPT-3 benefit from instructions to generalize across tasks. Wang et al. (2022) further extended this dataset to cover over 1.6k crosslingual tasks, each with a piece of natural language instruction. More work in this thread can be found in (Lou et al., 2023), which provides a comprehensive survey for various instruction learning. The robustness studied in this work is also based on instruction-driven task generalization; however, our emphasis lies on analyzing the variability in system performance as opposed to actively enhancing overall system performance.
Robustness under adversarial attack. There is a lot of work on adversarial attacks in NLP systems,
and adversarial attacks are often used as a way to enhance robustness (Zhang et al., 2020; Goel et al., 2021; Wang et al., 2020; Morris et al., 2020; Raman et al., 2023). Adversarial perturbations are considered to be worst-case scenarios that do not often occur in the real world and represent a very specific kind of noise. To generate valid adversarial examples, most attack methods require access to the structure, internal weights, and hyperparameters of the NLP model. In addition, adversarial perturbations should not be perceived by humans. This is a serious challenge because even small changes in the text may be easily recognized by the user but ignored by the machine. In contrast, some minor changes may cause the machine to react drastically but are not important to humans. Our investigation encompasses not only deliberate perturbations, but also more natural discrepancies that arise between
the training instructions and test instructions.

heading: 3 Experiment
text: This work studies the robustness of learning from task instructions from three scenarios: (i) the system robustness when dealing with instructions that are deliberately perturbed (Section 3.1); (ii) the system robustness when a task confronts diverse instructions that are written by different annotators (Section 3.2); (iii) the system robustness when it tests on tasks that have instructions with distinct abstractiveness (Section 3.3). The following subsections present our experiments about the three dimensions of robustness study, respectively.
The pretrained instruction learning model. To test the robustness of a pretrained model that learned NLP tasks from instructions, we choose the T5 model (Raffel et al., 2020) with 3B and 11B parameters finetuned on the training set of SUPERNATURALINSTRU (Wang et al., 2022).
SUPER-NATURALINSTRU consists of 1,616 NLP tasks in total and each task is explained by four items: Definition (a short paragraph describing the task semantics), Positive Examples (a couple of examples with inputs, correct outputs, and explanations), Negative Examples (a couple of examples with inputs, incorrect outputs, and explanations), and Instances (a large set of instances with inputs and correct outputs). One sample is shown in Table 1. As far as we know, SUPERNATURALINSTRU is the largest dataset containing human-oriented task instructions, and T5 denotes one of the state-of-the-art PLM models (we do not
consider GPT-3 due to the hardware constraints). As the data for training the instruction learning model, the train of SUPER-NATURALINSTRU has 1,462 tasks.

heading: 3.1 Robustness when instructions are perturbed
text: Dataset. This experiment uses the test of SUPER-NATURALINSTRU, which has 154 tasks (119 English tasks and 35 tasks in other languages). The robustness of the pretrained T5 model on train is tested on this test where instructions are perturbed. All test tasks are clustered into 12 categories with the official evaluation metrics either EXACTMATCH or ROUGE-L (Lin, 2004). Table 2 lists all the 12 categories and their respect evaluation metrics.
Note that Wang et al. (2022) concatenated Task name, Definition, Positive Examples and Negative Examples in various combinations as a long instruction, referred to as “Instruction with Examples” (INSTRU-W-EXAM). This paper uses the definition and two positive examples as INSTRU-W-EXAM configuration. In order to study the robustness when those examples are available or not, we report INSTRU-W-EXAM as well as INSTRU-WO-EXAM which uses only the Definition as instructions.
Instruction perturbation. To imitate how instructions are perturbed in the real world, we design various perturbation methods that try to change the
original instruction by its surface form or even semantics. Table 3 summarizes all the perturbation methods this work uses, including word-level perturbations, such as i) delete stop words; ii) randomly delete five to ten words; iii) insert masks at random positions and replace with words predicted by pretrained BERT (Devlin et al., 2019); iv) randomly replace words by the predictions of pretrained BERT, and v) shuffle the words in the instruction, and sentence-level perturbations, such as i) repeating a random sentence, and ii) shuffle the sentences in the instruction, and instruction-level perturbation, where we try to replace the original task definition with the definition of a randomly chosen task.
Spacy (Schmitt et al., 2019) is used to remove stop words from instructions and to split instructions into separate sentences, and BERT is used to insert and replace words in instructions.
Results. Each metric was run three times, and the mean and standard deviation of the three runs were reported. Tables 4&10 presents the performance on English and non-English subsets of SUPERNATURALINSTRU’s test, respectively. Each table summarizes the perturbation results by word-level, sentence-level, and instruction-level for both the INSTRU-WO-EXAM and INSTRU-W-EXAM setups. Our goal is to study i) the effects of different levels of perturbations; ii) the effects of including positive examples in the instructions; iii) whether there are any distinct phenomena between English tasks and
non-English tasks or not. From the two tables, we observe that:
• Effects of different levels of perturbations in INSTRU-WO-EXAM: i) word-level perturbations degrade the performance slightly, among which the “shuffle words” has the worst effect; ii) generally, sentence-level perturbations have less influence than the word-level perturbations: we only notice tiny drop compared with “original instructions”
and no clear difference between “repeat sentences” and “shuffle sentences” is observed; iii) “shuffle instructions” doesn’t work; this is within expectation since a random instruction provides no useful supervision for the target task.
• When the instructions contain examples (i.e., INSTRU-W-EXAM): i) the system is pretty robust among all perturbations; ii) “shuffle instructions” (note that the examples are still specific to
the target task) only decreases the performance to a small extent, which indicates that “correct examples+incorrect instruction” is better than “correct instruction without examples”. In other words, providing a couple of examples in the instruction rather than merely a short piece of task description is still the key to instruction-driven learning.
• Does larger models lead to higher robustness? We report INSTRU-W-EXAM results of T511B in Table 5. We only observe occasional performance drops by word-level perturbations in a few task categories, such as CR, DAR, and OR. In general, only “shuffle instruction” can lead to clear deterioration while other perturbations do not have a noticeable effect.
• Difference between English and non-
English: To save space, we move the results of non-English test tasks to the Table 10 in Appendix B. By comparing Table 4 and Table 10, we do not notice any different phenomena in non-English tasks.
How to interpret the phenomena caused by instruction perturbations? Based on the aforementioned observations, we conclude that perturbations, retaining as much task semantic as the original instructions, can yield less influence on the system’s robustness. In other words, two instructions that are more likely to be paraphrased tend to have similar performance. We further validate this argument in the subsequent Section 3.2.
Since INSTRU-W-EXAM generally performs bet-
ter than INSTRU-WO-EXAM, hereafter, we report INSTRU-W-EXAM for the remaining experiments.

heading: 3.2 Robustness when instructions are rephrased
text: In addition to the instruction perturbation experiments which may lead to grammatically incorrect instructions, this subsection studies a scenario where multiple valid instructions are available for the same task. This is a real-world challenge since the end users are free to write any open-form instructions to guide the system.
Data construction. Unfortunately, there are no existing datasets that contain tasks with diverse human-oriented instructions. To build a dataset, we randomly choose 2 English2 tasks from each of the 12 categories in the test of SUPERNATURALINSTRU; this results in 23 tasks3 that are uniformly distributed over those categories. We hire three graduate students in the NLP area to construct a dataset of diverse instructions with the following two steps:
• Writing step: for each task, each annotator is provided with 10 labeled instances, the task name, and the original task Definition. Then each annotator is required to write a new definition based on her/his understanding of the task. After this step, each task will have 3 new versions of raw instructions.
• Validation step: For each raw instruction, we forward it to two checkers who judge if it is valid for the task and if it is different enough from the original Definition. If any issues are found, the
2We considered English only since it is hard to find annotators who are familiar with many non-English languages.
3The “grammar error correction” has only one task.
original author will be asked to modify the instruction. This process is repeated until both checkers are satisfied with the new instruction. After this step, each task will have 4 versions of high-quality instructions (one original, three new).
Table 6 lists an example task with 4 instructions in our constructed data. Compared with the perturbed instructions demonstrated in Table 3, it is clear that the tasks in our new dataset are equipped with more valid instructions and they are more like paraphrases of each other while conveyed with more distinct textual expressions. Our constructed dataset, “Para-Instructions”, as the first one that contains multiple human-oriented instructions for each task, is released to the community.
Results. For all three new instructions, we report their averages and standard deviations. Table 7 compares the performance of the original instruction and the three new instructions by T53B and T5-11B models. We notice that the system is considerably robust among all four versions of instructions. It suggests that providing a paraphrasing instruction can generally keep the performance once demonstrations are available.

heading: 3.3 Robustness when instructions have different abstractiveness
text: Another challenge in instruction-driven task generalization is to encourage the same system to work well on datasets that contain instructions constructed from totally different templates or abstractiveness. This phenomenon often exists between two different datasets. For example, SUPERNATURALINSTRU has instructions that are mostly a paragraph while DIVERSEPROMPT (Honovich et al., 2022) contains prompts only—mostly short
and abstractive sentences.
Dataset. To check the robustness of the model finetuned on SUPER-NATURALINSTRU, we test it on the DIVERSEPROMPT dataset. DIVERSEPROMPT has 24 tasks and each task has an average of 7.75 prompts. DIVERSEPROMPT was the only dataset that provides multiple prompts for a task before we wrote this paper, and the prompts are much more concise (average word size 10.75) than the training instructions of SUPER-NATURALINSTRU.
Results. Table 8 presents the results on DIVERSEPROMPT by the model finetuned on SUPERNATURALINSTRU. For this experiment, we observe a totally different story:
• First, the pretrained model on SUPERNATURALINSTRU does not work well on DIVERSEPROMPT. Regardless of the task types (shown by the evaluation metrics: EXACTMATCH or ROUGE-L), most tasks show unsatisfactory performance. This can result from two
reasons: on the one hand, the instruction format (i.e., prompts in DIVERSEPROMPT) is significantly different from that in SUPER-NATURALINSTRU— DIVERSEPROMPT has prompts that mostly contain a few words while instructions in SUPERNATURALINSTRU are more specific, informative and are human-oriented; on the other hand, the tasks in DIVERSEPROMPT are very distinct from those in SUPER-NATURALINSTRU, such as “sum”, “diff”, “first word letter”, “common concept”, etc.
• Second, even for some tasks that co-exist in SUPER-NATURALINSTRU and DIVERSEPROMPT, e.g., “word in context”, “translation”, “sentiment”, etc., their performances fall short of expectations. For example, “Text Generation” (TG) in Table 7 gets consistent performance around 24.6 by ROUGE-L, whereas the translation tasks in DIVERSEPROMPT ranging from 8 to 13.
The preceding two items hint at the fragility of the model when dealing with unseen tasks and unfamiliar instruction formats, despite its training on more than 1,000 instruction-laden tasks. So far, we still lack a method to effectively learn instruction to solve tasks in the absence of demonstration. The majority of instruction-tuning datasets commonly furnish only a single instruction for each input-output pair, aiming to enhance the system’s capacity for instruction following through an increased number of training tasks. However, this approach may prove suboptimal in supervising a system that can effectively generalize to unfamiliar tasks with unfamiliar formats of instructions.
• Third, it is worth mentioning that this DIVERSEPROMPT dataset does not provide demonstrations. In this case, we notice mostly greater standard deviations in Table 8 than that in Table 7. This indicates that even paraphrasing prompts can not guarantee consistent performance if no demonstrations exist. Then we dive into the performance per prompt for tasks that have larger standard deviations of ROUGE-L, such as “negation”, “cause and effect”, “taxonomy animal” and “active to passive”, in Table 9. We found the following factors that affect prompt performance in text generation tasks: (i) it is better to use a verb that clearly has a single sense. For example, “write” and “extract” are generally better than “output”, “list”, which can be verbs or nouns; (ii) explicitly presenting the action is better than that implicitly. For example, “· · ·, write the · · ·” is better than a question “which of the · · · is · · ·?”.
This story shows that there is still a long way to go for instruction-driven task generalization, especially when the system deals with less familiar tasks with abstractiveness-distinct instructions.

heading: 4 Conclusion
text: This paper proposed a new issue, the robustness of instruction-tuned models. Experiments show that the instruction-tuned model may not learn the
content of the instructions itself in some tasks, but rather the pattern of positive examples. The robustness of the instruction-tuned model is very important, which determines the stability and reliability of instruction fine-tuning models in real-world applications. Using only instructions to tell models what they should do is a long-term goal of AI, and we hope that future work will be done to improve the robustness of the instruction-tuned model.

heading: Acknowledgments
text: The authors would like to thank all the anonymous reviewers for their insightful comments and suggestions.
Limitations
We summarize the limitations of this work as follows:
• First, it is still challenging to figure out why a particular instruction results in better or worse performance. We tried to explain a part of the observations, but some of the phenomena are still mysterious.
• Technically we can create a much larger Para-Instruction dataset than 23 that provides diverse instructions for NLP tasks. Due to the budget limitation, we only collected 23 tasks.
• Some prior work showed that larger PLMs often bring better performance in dealing with instruction learning. Due to hardware constraints, we are unable to try PLMs that are larger than T5-11B.
Ethics Statement
We do not anticipate any ethical issues particularly to the topics of this work. Nevertheless, some work presented here extensively uses large-scale pretrained models with self-attention, which may lead to substantial financial and environmental costs.

heading: A Finetuning Details
text: In this work, we finetune T5 using the HuggingFace library. We use a scheduler with a constant learning rate of 5e-05. All models are optimized by AdamW (Loshchilov and Hutter, 2019) with a batch size of 8. T5-3B is fine-tuned on one NVIDIA RTX A5000 GPU and T5-11B on four NVIDIA A100 GPUs. DeepSpeed4 is used to parallelize the models, and bfloat16 precision is enabled to save GPU memory.
4https://github.com/microsoft/DeepSpeed

heading: B T5-3B Results on Non-English Tasks
text: Table 10 presents the INSTRU-WO-EXAM (in black) and INSTRU-W-EXAM (in gray) results of the T5-3B model on the 35 non-English test tasks of SUPER-NATURALINSTRU by different levels of perturbations.

heading: C T5-11B Results on Non-English Tasks
text: Table 11 presents the INSTRU-W-EXAM results of the T5-11B model on the 35 non-English test tasks of SUPER-NATURALINSTRU by different levels of perturbations.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
5
3 A2. Did you discuss any potential risks of your work? 1
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? 3
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? 3
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? No response.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? No response.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? No response.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? No response.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? 3
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Using only instructions to tell models what they should do is a long-term goal of AI, and we hope that future work will be done to improve the robustness of the instruction-tuned model