HINT : Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation (2023)

Abstract: Recent NLP models have shown the remarkable ability to effectively generalise ‘zero-shot’ to new tasks using only natural language instructions as guidance. However, many of these approaches suffer from high computational costs due to their reliance on concatenating lengthy instructions with every input example, resulting in costly reprocessing of the instruction. To avoid this, we introduce Hypernetworks for INstruction Tuning (HINT), which convert task instructions and examples into parameter-efficient modules inserted into an underlying model using a pretrained text encoder, eliminating the need to include instructions in the model input. The hypernetwork in HINT also produces an encoded instruction, which we concatenate with encoded inputs during decoding to further improve performance. HINT models outperform strong state-of-the-art baselines by over 10% when controlling for compute (measured in FLOPs). By converting instructions into modules, HINT models can effectively disregard the length of instructions and few-shot example inputs in terms of compute usage. As a result, HINT can enhance its performance by up to 25% by incorporating additional few-shot data, while utilizing only up to 5% more compute. This combines the strengths of parameter-efficient fine-tuning and in-context learning.

## introduction large pretrained language models have demonstrated a striking ability to perform new tasks through the use of in-context examples or instructions alone @xcite , or after training on input instances augmented with instructions @xcite @xcite @xcite . this ability allows a single model to 1 our code is available at: https://github.com/allenai/hyper-task-descriptions . ## hint model here, we introduce the main elements of our hint model. the model has two core parts: the hypernetwork, which takes in text instructions and outputs parameter-efficient modules, and the underlying model, into which we insert the generated parameter weights. the underlying model is simply an encoder-decoder foot_2 transformer model @xcite with additional parameter-efficient adaptations inserted in, while the hypernetwork has a more complex architecture which we describe below. figure 1 provides a visual overview. ## experimental details we evaluate our approach on two popular instruction-based datasets: super-natural instructions (sni) @xcite and the t0 split of p3 @xcite . we use t5x and seqio @xcite to handle data preprocessing and model training. we use t5 v1.1 + lm adaptation @xcite as our base models, using the 3b size unless otherwise stated. unless otherwise stated, the hypernetwork generates prefixes of length 30 and adapters with a bottleneck size of 512, matching the sizes recommended by @xcite . we use the adafactor optimizer @xcite with a constant learning rate of 0.001. unless otherwise stated, we report results from single runs. pretraining we pretrain all models for 10,000 steps (7,000 for 11b size models) using c4 @xcite with a batch size of 1,024 samples and sequences of length 512. super-natural instructions (sni) for sni, we examine two settings: providing the hypernetwork with the task definition and the underlying network with the instance input only ('def'), and providing the hypernetwork with the task definition and two few-shot task examples ('def + 2 pos.'). to train, we finetune our pretrained hint models for 1,000 steps with a batch size of 1,024, with a maximum sequence length of 1,024 for both the underlying model and the hypernetwork input. we then evaluate the final checkpoint on the test split of sni, which is a set of 119 unseen tasks. we use v2.6 of super-natural instructions. p3 for p3, we explore two settings: (a) 'joint', where we give the hypernetwork a templated form of the prompt with instance information removed and give the underlying model the full prompted input. (b) 'split', where we give the hypernetwork the templated prompt without instance information and give the underlying model only the instance information without the prompt. in both cases, we fully finetune our model for 10,000 steps with a batch size of 2,048. we use a maximum input length of 1,024 for the underlying model and 512 for the hypernetwork. we train and evaluate on the same tasks and splits as t0 @xcite . baselines we primarily compare against t0 and tk-instruct, models fully-finetuned on p3 and sni respectively with all task information concatenated with the input. we replicate these models, matching the finetuning settings used for hint models, and find that our replications significantly outperform previously reported results, making these baselines extremely strong. we note where results are our replications or reported from prior work. we additionally compare against 'x + peft', the prior models with adapters and prefixes added in before finetuning, hypertune @xcite , a concurrent work that primarily makes use of a pretrained hypernetwork but without instruction fusion, hypter (ye and ren, 2021), a prior hypernetwork-based model that does not use pretraining, gpt-2 @xcite , a strong decoder-only model, which we fully finetune, and opt @xcite , another strong decoderonly model, which we also fully finetune. ## analysis 6.1 pretraining we compare using no pretraining, our pretraining scheme, and the pretraining scheme proposed by @xcite with and without using instruction fusion ('hint' and 'hint -no instr. fus.', respectively). we find that: (a) using pretraining gives a large boost in performance for hypernetwork-only models, showing that pretraining is essential to good hypernetwork performance, and (b) using our pretraining scheme works best overall. we hypothesise this reflects the fact that our scheme is closer to the super-natural instructions format than caclm. unlike @xcite , we found that further pretraining did not aid performance. this is likely due to the fact that we tie the underlying model encoder and hypernetwork encoder weights together, meaning that the model weights must balance between acting as the hypernetwork and underlying model encoder. ## conclusion we introduce hypernetworks for instruction tuning (hint) models and show that they consistently outperform strong full-input baselines when controlling for inference compute. this is primarily due to the fact that hint models process their task instructions once per task, while current state-ofthe-art models re-encode instructions with every task input. we show that the success of hint models relies on a pretrained hypernetwork, which converts task instructions into parameter-efficient modules and an encoded instruction, both of which we insert into the underlying model. future work could investigate how hint aids in few-shot settings, further building on hint's strong few-shot efficiency and taking advantage of the improved initialisation provided by hypernetworks @xcite . overall, hint models combine the benefits of parameter-efficient learning with the benefits of instruction-based learning, allowing one to easily turn pretrained language models into efficient, task-customised models.