Few-Shot Parameter-Efﬁcient Fine-Tuning is Better and Cheaper than In-Context Learning
Anonymous Author(s)
Afﬁliation
Address email
Abstract
Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding asmall number of training examples as part of the input.
ICL incurs substantialcomputational, memory, and storage costs because it involves processing all of thetraining examples every time a prediction is made.
Parameter-efﬁcient ﬁne-tuning(PEFT)
(e.g. adapter modules, prompt tuning, sparse update methods, etc.)
offersan alternative paradigm where a small set of parameters are trained to enable amodel to perform the new task.
In this paper, we rigorously compare few-shot
ICL and PEFT and demonstrate that the latter offers better accuracy as well asdramatically lower computational costs.
Along the way, we introduce a new PEFTmethod called (IA) 3 that scales activations by learned vectors, attaining strongerperformance while only introducing a relatively tiny amount of new parameters.
We also propose a simple recipe based on the T0 model [1] called T-Few thatcan be applied to new tasks without task-speciﬁc tuning or modiﬁcations.
Wevalidate the effectiveness of T-Few on completely unseen tasks by applying it tothe RAFT benchmark [2], attaining super-human performance for the ﬁrst timeand outperforming the state-of-the-art by 6% absolute.
All of the code used in ourexperiments will be publicly available. All of the code used in our experimentswill be made publicly available.
Introduction
Pre-trained language models have become a cornerstone of natural language processing, thanksto the fact that they can dramatically improve data efﬁciency on tasks of interest – i.e., using apre-trained language model for initialization often produces better results with less labeled data.
Ahistorically common approach has been to use the pre-trained model’s parameters for initializationbefore performing gradient-based ﬁne-tuning on a downstream task of interest.
While ﬁne-tuninghas produced many state-of-the-art results [1], it results in a model that is specialized for a singletask with an entirely new set of parameter values, which can become impractical when ﬁne-tuning amodel on many downstream tasks.
An alternative approach popularized by [3, 4] is in-context learning (ICL), which induces a modelto perform a downstream task by inputting prompted examples.
Few-shot prompting converts asmall collection of input-target pairs into (typically) human-understandable instructions and examples[3, 4], along with a single unlabeled example for which a prediction is desired.
Notably, ICL requiresno gradient-based training and therefore allows a single model to immediately perform a wide varietyof tasks.
Performing ICL therefore solely relies on the capabilities that a model learned duringpre-training.
These characteristics have led to a great deal of recent interest in ICL methods [5–10]. [Table]
[Figure]
Despite the practical beneﬁts of ICL, it has several major drawbacks.
First, processing all promptedinput-target pairs every time the model makes a prediction incurs signiﬁcant compute costs.
Second,
ICL typically produces inferior performance compared to ﬁne-tuning [4].
Finally, the exact formattingof the prompt (including the wording [11] and ordering of examples [12]) can have signiﬁcant andunpredictable impact on the model’s performance, far beyond inter-run variation of ﬁne-tuning.
Recent work has also demonstrated that ICL can perform well even when provided with incorrectlabels, raising questions as to how much learning is taking place at all [9].
An additional paradigm for enabling a model to perform a new task with minimal updates is parameterefﬁcient ﬁne-tuning (PEFT), where a pre-trained model is ﬁne-tuned by only updating a small numberof added or selected parameters.
Recent methods have matched the performance of ﬁne-tuning thefull model while only updating or adding a small fraction (e.g. 0.01%) of the full model’s parameters[13, 14].
Furthermore, certain PEFT methods allow mixed-task batches where different examples ina batch are processed differently [14], making both PEFT and ICL viable for multitask models.
While the beneﬁts of PEFT address some shortcomings of ﬁne-tuning (when compared to ICL), therehas been relatively little focus on whether PEFT methods work well when very little labeled datais available.
Our primary goal in this paper is to close this gap by proposing a recipe – i.e., a model, a
PEFT method, and a ﬁxed set of hyperparameters – that attains strong performance on novel, unseentasks while only updating a tiny fraction of the model’s parameters.
Speciﬁcally, we base our approachon the T0 model [1], a variant of T5 [15] ﬁne-tuned on a multitask mixture of prompted datasets.
To improve performance on classiﬁcation and multiple-choice tasks, we add unlikelihood [16, 17]and length normalization-based [4] loss terms.
In addition, we develop (IA) 3 , a PEFT methodthat multiplies intermediate activations by learned vectors.
(IA) 3 attains stronger performance thanfull-model ﬁne-tuning while updating up to 10,000 ⇥ fewer parameters.
Finally, we demonstratethe beneﬁts of pre-training the (IA) 3 parameters before ﬁne-tuning [18, 19].
Our overall recipe,which we dub “ T-Few ”, performs signiﬁcantly better than ICL (even against 16 ⇥ larger models)and outperforms humans for the ﬁrst time on the real-world few-shot learning benchmark RAFT [2]while requiring dramatically less compute and allowing for mixed-task batches during inference.
Tofacilitate the use of T-Few on new problems and future research on PEFT, we release our code.
After providing background on ICL and PEFT in the following section, we discuss the design of
T-Few in section 3.
In section 4, we present experiments comparing T-Few to strong ICL baselines.
Finally, we discuss related work in appendix B and conclude in section 5.
Background
In this section, we provide am verview of ICL and PEFT with a focus on characterizing the computation, memory, and on-disk storage costs of making a prediction.
Real-world costs depend onimplementation and hardware, so we report costs in terms of FLOPs for computation and bytes formemory and storage, respectively.
Additional related work is discussed in appendix B.
2.1 Few-shot in-context learning (ICL)
ICL [3, 4] aims to induce a model to perform a task by feeding in concatenated and promptedinput-target examples (called “shots”) along with an unlabeled query example.
Taking the cycled letter task from Brown et al.
[4] as an example, a 4-shot input or context would be “ Pleaseunscramble the letters into a word, and write that word: asinoc = casino,yfrogg = froggy, plesim = simple, iggestb = biggest, astedro = ”, for which thedesired output would be “ roasted ”.
ICL induces an autoregressive language model to performthis task by feeding in the context and sampling from the model.
For classiﬁcation tasks, eachlabel is associated with a string (e.g. “ positive ” and “ negative ” for sentiment analysis) anda label is assigned by choosing the label string that the model assigns the highest probability to.
For multiple-choice tasks (e.g. choosing between N possible answers to a question), the model’sprediction is similarly determined by determining which choice is assigned the highest probability.
The primary advantage of ICL is that it enables a single model to perform many tasks immediatelywithout ﬁne-tuning.
This also enables mixed-task batches , where different examples in a batch of datacorrespond to different tasks by using different contexts in the input.
ICL is also typically performedwith only a limited number of labeled examples – called few-shot learning – making it data-efﬁcient.
Despite these advantages, ICL comes with signiﬁcant practical drawbacks: First, making a predictionis dramatically more expensive because the model needs to process all of the in-context labeledexamples.
Speciﬁcally, ignoring the quadratic complexity of self-attention operations in Transformerlanguage models (which are typically small compared to the costs of the rest of the model [20]),processing the k training examples for k -shot ICL increases the computational cost by approximatelyk + 1 times compared to processing the unlabeled example alone.
Memory costs similarly scaleapproximately linearly with k , though during inference the memory costs are typically dominated bystoring the model’s parameters.
Separately, there is a small amount of on-disk storage required forstoring the in-context examples for a given task.
For example, storing 32 examples for a task wherethe prompted input and target for each example is 512 tokens long would require about 66 kilobytesof storage on disk ( 32 examples ⇥ 512 tokens ⇥ 32 bits).
Beyond the aforementioned costs, ICL also exhibits unintuitive behavior.
Zhao et al.
[12] showedthat the ordering of examples in the context heavily inﬂuences the model’s predictions.
Min et al.[9] showed that ICL can still perform well even if the labels of the in-context examples are swapped(i.e.
made incorrect), which raises questions about whether ICL is really “learning” from the labeledexamples.
Various approaches have been proposed to mitigate these issues.
One way to decrease computationalcosts is to cache the key and value vectors for in-context examples.
This is possible because decoderonly Transformer language models have a causal masking pattern, so the model’s activations for thecontext do not do not depend on the unlabeled example.
In an extreme case, 32 -shot ICL with 512tokens per in-context example would result in over 144 gigabytes of cached key and value vectors forthe GPT-3 model ( 32 examples ⇥ 512 tokens ⇥ 96 layers ⇥ 12288 d model ⇥ 32 bits each for the key 109 and value vectors).
Separately, Min et al.
[21] proposed ensemble ICL , where instead of using theoutput probability from concatenating the k training examples, the output probabilities of the modelon each training example (i.e. 1 -shot ICL for each of the k examples) are multiplied together.
Thislowers the non-parameter memory cost by a factor of k/ 2 but increases the computational cost bya factor of 2 .
In terms of task performance, Min et al.
[21] ﬁnd that ensemble ICL outperforms thestandard concatenative variant.
2.2 Parameter-efﬁcient ﬁne-tuning
While standard ﬁne-tuning updates all parameters of the pre-trained model, it has been demonstratedthat it is possible to instead update or add a relatively small number of parameters.
Early methodsproposed adding adapters [22–24], which are small trainable feed-forward networks inserted betweenthe layers in the ﬁxed pre-trained model.
Since then, various sophisticated PEFT methods have beenproposed, including methods that choose a sparse subset of parameters to train [25, 26], producelow-rank updates [13], perform optimization in a lower-dimensional subspace [27], add low-rankadapters using hypercomplex multiplication [28], and more.
Relatedly, prompt tuning [14] and preﬁxtuning [29] concatenate learned continuous embeddings to the model’s input or activations to induceit to perform a task; this can be seen as a PEFT method [30].
State-of-the-art PEFT methods canmatch the performance of ﬁne-tuning all of the model’s parameters while updating only a tiny fraction(e.g.
0.01%) of the model’s parameters.
PEFT drastically reduces the memory and storage requirements for training and saving the model.
Inaddition, certain PEFT methods straightforwardly allow mixed-task batches – for example, prompt tuning enables a single model to perform many tasks simply by concatenating different promptembeddings to each example in the batch [14].
On the other hand, PEFT methods that re-parameterizethe model (e.g. [27, 13]) are costly or onerous for mixed-task batches.
Separately, different PEFTmethods increase the computation and memory required to perform inference by different amounts.
For example, adapters effectively add additional (small) layers to the model, resulting in small butnon-negligible increases in computational costs and memory.
An additional cost incurred by PEFTis the cost of ﬁne-tuning itself, which must be performed once and is then amortized as the modelis used for inference.
However, we will show that PEFT can be dramatically more computationallyefﬁcient when considering both ﬁne-tuning and inference while achieving better accuracy than ICL.
Designing the T-Few Recipe
Given that PEFT allows a model to be adapted to a new task with relatively small storage requirementsand computational cost, we argue that PEFT presents a promising alternative to ICL.
Our goalis therefore to develop a recipe that allows a model to attain high accuracy on new tasks withlimited labeled examples while allowing mixed-task batches during inference and incurring minimalcomputational and storage costs.
By recipe , we mean a speciﬁc model and hyperparameter settingthat provides strong performance on any new task without manual tuning or per-task adjustments.
In this way, we can ensure that our approach is a realistic option in few-shot settings where limitedlabeled data is available for evaluation [31, 32].
3.1 Model and Datasets
As a ﬁrst step, we must choose a pre-trained model.
Ideally, the model should attain high performanceon new tasks after ﬁne-tuning on a limited number of labeled examples.
In preliminary experimentsapplying PEFT methods to different pre-trained models, we attained the best performance with T0[1].
T0 is based on T5 [15], an encoder-decoder Transformer model [33] that was pre-trained via amasked language modeling objective [34] on a large corpus of unlabeled text data.
T0 was created byﬁne-tuning T5 on a multitask mixture of datasets in order to enable zero-shot generalization, i.e. theability to perform tasks without any additional gradient-based training.
Examples in the datasets usedto train T0 were prompted by applying the prompt templates from the Public Pool of Prompts (P3[35]), which convert each example in each dataset to a prompted text-to-text format where each labelcorresponds to a different string.
For brevity, we omit a detailed description of T0 and T5; interestedreaders can refer to Sanh et al.
[1] and Raffel et al.
T0 was released in three billion and elevenbillion parameter variants, referred to as “T0-3B” and simply “T0” respectively.
In this section (whereour goal is to design the T-Few recipe through extensive experimentation), we use T0-3B to reducecomputational costs.
For all models and experiments, we use Hugging Face Transformers [36].
While T0 was designed for zero-shot generalization, we will demonstrate that it also attains strongperformance after ﬁne-tuning with only a few labeled examples.
To test T0’s generalization, Sanh et al.[1] chose a set of tasks (and corresponding datasets) to hold out from the multitask training mixture– speciﬁcally, sentence completion (COPA [37], H-SWAG [38], and Story Cloze [39] datasets),natural language inference (ANLI [40], CB [41], and RTE [42]), coreference resolution (WSC [43]and Winogrande [44]), and word sense disambiguation (WiC [45]).
Evaluation of generalizationcapabilities can then be straightforwardly done by measuring performance on these held-out datasets.
We also will later test T-Few ’s abilities in the RAFT benchmark [2] in section 4.3, a collection ofunseen “real-world” few-shot tasks with no validation set and a held-out test set.
ANLI, WiC, WSC islicensed under a Creative Commons License.
Winogrande is licnsed under an Apache license.
COPAis under a BSD-2 Clause license.
We could not ﬁnd the license of RTE and CB but they are part of
SuperGLUE which mentions the datasets are allowed for use in research context.
To ease comparison, we use the same number of few-shot training examples for each dataset as Brownet al.
[4], which varies from 20 to 70.
Unfortunately, the few-shot dataset subsets used by Brownet al.
[4] have not been publicly disclosed.
To allow for a more robust comparison, we thereforeconstructed ﬁve few-shot datasets by sampling subsets with different seeds and report the medianand interquartile range.
We prompt examples from each dataset using the prompt templates from P
Bach et al.
[35], using a randomly-sampled prompt template for each example at each step.
Unlessotherwise stated, we train our model for 1K steps with a batch size of 8 and report performance at theend of training.
For evaluation, we use “rank classiﬁcation”, where the model’s log-probabilities for all possible labelstrings are ranked and the model’s prediction is considered correct if the highest-ranked choice is the correct answer.
Rank classiﬁcation evaluation is compatible with both classiﬁcation and multiplechoice tasks.
Since model performance can vary signiﬁcantly depending on the prompt template used,we report the median accuracy across all prompt templates from P3 and across few-shot data subsetsfor each dataset.
For all datasets, we report the accuracy on the test set or validation set when the testlabels are not public (e.g. SuperGLUE datasets).
In the main text, we report median accuracy acrossthe nine datasets mentioned above.
Detailed results on each dataset are provided in the appendices.
3.2 Unlikelihood Training and Length Normalization
Before investigating PEFT methods, we ﬁrst explore two additional loss terms to improve theperformance of few-shot ﬁne-tuning of language models.
Language models are normally trainedwith cross-entropy loss L LM =  1 T P t log p ( y t | x , y <t ) where the model is trained to increase theprobability of the correct target sequence
y =
( y 1 , y 2 , . . .
, y T )
given the input sequence x .
For evaluation, we use rank classiﬁcation (described in section 3.1) which depends on both theprobability that the model assigns to the correct choice as well as the probabilities assigned by themodel to the incorrect choices.
To account for this during training, we consider adding an unlikelihoodloss [16, 17]:
[Equation]
which discourages the model from predicting tokens from incorrect target sequences, where ˆy ( n ) =(
ˆ y 1 , ˆ y 2 , . .
, ˆ y T ( n ) ) is the n -th of N incorrect target sequences.
We hypothesize that adding L UL willimprove results on rank classiﬁcation because the model will be trained to assign lower probabilitiesto incorrect choices, thereby improving the chance that the correct choice is ranked highest.
The possible target sequences for a given training example can have signiﬁcantly different lengths,especially in multiple-choice tasks.
Ranking each choice based on probability can therefore “favor”shorter choices because the model’s assigned probability to each token is  1 .
To rectify this,we consider using length normalization when performing rank classiﬁcation, which divides themodel’s score on each possible answer choice by the number of tokens in the choice (as used in
GPT-3 [4]).
When using length normalization during evaluation, we introduce an additional lossterm during training that more closely reﬂects length-normalized evaluation.
First, we compute thelength-normalized log probability of a given output sequence  ( x , y ) =
[Equation] T
[Equation]
Then, we maximize the length-normalized log probability of the correct answer choice by minimizingthe softmax cross-entropy loss:
[Equation]
When training a model with L LM , L UL , and L LN , we simply sum them.
This avoids introducing anyhyperparameters that would be problematic to tune in the few-shot setting (where realistically-sizedvalidation sets are tiny by necessity [31, 32]).
We report the results of ﬁne-tuning all of T0-3B’s parameters with and without length normalizationon all datasets in appendix C.
We ﬁnd that adding L LN improves the accuracy from 60.7% to 62.71%and including both L UL and L LN provides a further improvement to 63.3%.
Since these loss termsimprove performance without introducing any additional hyperparameters, we include them in ourrecipe and use them in all following experiments.
3.3 Parameter-efﬁcient ﬁne-tuning with (IA) 3
In order to compare favorably to few-shot ICL, we need a PEFT method that has the followingproperties: First, it must add or update as few parameters as possible to avoid incurring storageand memory costs.
Second, it should achieve strong accuracy after few-shot training on new tasks.
Finally, it must allow for mixed-task batches, since that is a capability of ICL.
In order to easilyenable mixed-task batches, a PEFT method should ideally not modify the model itself.
Otherwise,each example in a batch would effectively need to be processed by a different model or computationalgraph.
A more convenient alternative is provided by methods that directly modify the activations ofthe model since this can be done independently and cheaply to each example in the batch accordingto which task the example corresponds to.
Prompt tuning and preﬁx tuning methods [14, 29] work byconcatenating learned vectors to activation or embedding sequences and are therefore examples ofactivation-modifying PEFT methods that allow for mixed-task batches.
However, as we will discuss later, we were unable to attain reasonable accuracy with prompt tuning and found that the moreperformant PEFT methods did not allow for mixed-task batches.
We therefore developed a new PEFTmethod that meets our desiderata.
As an alternative, we explored element-wise multiplication (i.e. rescaling) of the model’s activationsagainst a learned vector.
Speciﬁcally, we consider adaptation of the form l  x where l 2 R d is alearned task-speciﬁc vector,  represents element-wise multiplication, and x 2 R T ⇥ d is a length- Tsequence of activations.
We use “broadcasting notation” [46] so that the ( i, j ) th entry of l  x is l j x i,j .
In preliminary experiments, we found it was not necessary to introduce a learned rescaling vectorfor each set of activations in the Transformer model.
Instead, we found it was sufﬁcient to introducerescaling vectors on the keys and values in self-attention and encoder-decoder attention mechanismsand on the intermediate activation of the position-wise feed-forward networks.
Speciﬁcally, usingthe notation from Vaswani et al.
[33], we introduce three learned vectors l k 2 R d k , l v 2 R d v , andl ↵ 2 R d ff , which are introduced into the attention mechanisms as:
[Equation]
and in the position-wise feed-forward networks as ( l ↵   ( W 1 x )) W 2 , where  is the feed-forwardnetwork nonlinearity.
We introduce a separate set of l k , l v , and l ↵ vectors in each Transformer layerblock.
This adds a total of L ( d k + d v + d ↵ ) new parameters for a L -layer-block Transformer encoderand L (2 d k + 2 d v + d ↵ ) (with factors of 2 accounting for the presence of both self-attention andencoder-decoder attention) for a L -layer-block decoder.
l k , l v , and l ↵ are all initialized with ones sothat the overall function computed by the model does not change when they are added.
We call ourmethod (IA) 3 , which stands for “Infused Adapter by Inhibiting and Amplifying Inner Activations”.(IA) 3 makes mixed-task batches possible because each sequence of activations in the batch can beseparately and cheaply multiplied by its associated learned task vector.
We also note that, in theevent that a model will only be used on a single task, the modiﬁcations introduced by (IA) 3 canalso be applied to weight matrices permanently so that no elementwise multiplication is required andthe model’s architecture remains unchanged.
This possible because element-wise multiplicationsperformed in (IA) 3 always co-occur with a matrix multiplication, and l
Wx =
( l  W ) x .
In thiscase, our method incurs no additional computational cost compared to the original model.
To validate (IA) 3 , we compare it to a large variety of existing adaptation methods in our setting ofﬁne-tuning T0-3B on few-shot datasets from held-out tasks.
Speciﬁcally, we compare with 9 strong
PEFT methods: BitFit [47] which updates only the bias parameters; Adapters [23] which introducetask-speciﬁc layers after the self-attention and position-wise feed-forward networks; Compacter and
Compacter++ [28] which improve upon adapters by using low-rank matrices and hypercomplex multiplication; prompt tuning [14] which learns task-speciﬁc prompt embeddings that are concatenated tothe model’s input; FISH Mask [26] which chooses a subset of parameters to update based on their approximate Fisher information; Intrinsic SAID [27] which performs optimization in a low-dimensionalsubspace; preﬁx-tuning [29] which learns task-speciﬁc vectors that are concatenated to the model’sactivations; and LoRA [13] which assigns low-rank updates to parameter matrices.
Additionally, weinclude the baselines of full-model ﬁne-tuning and updating only the layer normalization parameters.
For certain methods that allow changing the parameter efﬁciency, we report results for differentbudgets: 0.2% and 0.02% sparsity for FISH Mask, 10 and 100 learned prompt vectors for prompttuning, and 20,000- or 500,000-dimensional subspaces for Intrinsic SAID.
The results are shown in ﬁg. 2, with detailed per-dataset results in appendix D. We ﬁnd that (IA) 3is the only method that attains higher accuracy than the full-model-ﬁne-tuning baseline.
Whileother PEFT methods (e.g. Intrinsic SAID and prompt tuning) update or introduce fewer parameters,(IA) 3 performs considerably better.
Our results and setting differ with some past work on the
PEFT methods we compare against.
Mahabadi et al.
[28] report that Compacter and Compacter++outperform full-model ﬁne-tuning, including in the few-shot setting.
Lester et al.
[14] found thatprompt tuning could match full-model ﬁne-tuning, and in subsequent work Wei et al.
[48] found thatprompt tuning performed well when applied to a multitask ﬁne-tuned model in the few-shot setting.
In both cases, we experimented with various hyperparameter choices to try to match past results.
We hypothesize the disagreement comes from us using a different model and different datasets.
Forprompt tuning speciﬁcally, we noticed that the validation set performance could ﬂuctuate wildly overthe course of training, hinting at possible optimization issues.
3.4 Pre-training (IA) 3
In recent work, Gu et al.
[18], Vu et al.
[19] showed that pre-training the prompt embeddings inprompt tuning can improve performance when ﬁne-tuning on downstream few-shot tasks.
For pretraining, Gu et al.
[18] use a suite of self-supervised tasks applied to unlabeled text data, and Vuet al.
[19] consider using embeddings from a separate task or multitask mixture.
We follow Vu et al.[19] and simply pre-train the new parameters introduced by (IA) 3 on the same multitask mixtureused to train T0.
We pre-train for 100,000 steps with a batch size of 16 before ﬁne-tuning the (IA) 3parameters on each individual downstream dataset.
A full comparison of accuracy with and withoutpre-training (IA) 3 is detailed in appendix E. We ﬁnd that pre-training improves ﬁne-tuned accuracyfrom 64.6 to 65.8 and therefore add it to our recipe.
3.5 Combining the ingredients
In summary, the T-Few recipe is deﬁned as follows: We use the T0 model as a backbone.
We add(IA) 3 for downstream task adaptation and use parameters initialized from pre-training (IA) 3 on thesame multitask mixture for T0.
As an objective, we use the sum of a standard language modelingloss L LM , an unlikelihood loss L UL for incorrect choices, and a length-normalized loss L LN .
Wetrain for 1,000 steps with a batch size of 8 sequences using the Adafactor optimizer [49] with alearning rate of 3 e  3 and a linear decay schedule with a 60-step warmup.
We apply prompt templatesto downstream datasets during training and inference to convert each example into an instructivetext-to-text format.
Importantly, we apply this recipe to every downstream dataset in exactly the sameway without per-dataset hyperparameter tuning or modiﬁcations.
This makes the recipe a realisticoption for few-shot learning settings where validation sets are tiny by deﬁnition [31, 32].
Outperforming ICL with T-Few
Having designed and established the T-Few recipe on T0-3B, we now apply it to T0 (with 11 billionparameters) and compare performance to strong few-shot ICL baselines.
From this point onwards,we use exactly the same recipe and hyperparameters across all tasks.
4.1 Performance on T0 tasks
First, we evaluate T-Few on the datasets that were held out from T0’s training mixture.
We compareagainst zero-shot learning with T0 [1] (since we found few-shot ICL to performed worse than zero- shot for T0, see appendix F); few-shot ICL with T5+LM [14] (the next-step-prediction languagemodel upon which T0 is based); and few-shot ICL with the 6.7, 13, and 175 billion parameter variantsof GPT-3.
See appendix F for more details on these baselines.
The accuracy on the held-out T0datasets (described in section 3.1) is shown in table 1 and ﬁg. 3, with per-dataset results reportedin
appendix F. We ﬁnd that T-Few outperforms all other methods by a substantial margin.
Notably, T-Few achieves a 6% higher accuracy than few-shot ICL with GPT-3 175B despite being about 16 ⇥smaller and outperforms the smaller GPT-3 variants by an even larger margin.
T-Few also attainssigniﬁcantly higher accuracy than both zero-shot learning with T0 and few-shot ICL with T5+LM.
[Table]
[Table]
4.2 Comparing computational costs
Having established that T-Few signiﬁcantly outperforms ICL-based models, we now compare therelative costs of each few-shot learning approach.
For simplicity, we use the FLOPs-per-tokenestimates for Transformer-based language models introduced by Kaplan et al.
Speciﬁcally, weestimate that a decoder-only Transformer (e.g. the GPT series) with N parameters uses 2 N FLOPsper token for inference and 6 N FLOPs per token for training.
Encoder-decoder models like T0 and
T5 (where the encoder and decoder have the same number of layers and layer sizes) only processeach token with either the encoder or decoder (each having roughly half the parameters of the fullmodel), so the FLOPs per token estimates are halved to N and 3 N FLOPs per token for inference andtraining.
We note that FLOPs are not a direct measurement of real-world computational cost becauselatency, power usage, and other costs can vary signiﬁcantly depending on hardware and other factors[52].
However, we focus on FLOPs because it is a hardware-independent metric that closely withreal-world costs the hardware setup used for running the different methods we consider would likelyvary signiﬁcantly across methods.
We summarize the costs in table 1 and discuss them below.
For allestimates, we use the median number of shots (41) across the datasets we consider.
Rank evaluationand our unlikelihood loss both require processing every possible output choice to attain a predictionfor an unlabeled example.
The median combined tokenized sequence length for the input and allpossible targets is 103 for the datasets we consider.
For in-context examples processed for few-shot
ICL, only the correct target is required, producing a median sequence length of 98.
Assuming thatkey and value vectors are cached, processing a single example with ICL therefore involves processing41 ⇥ 98 + 103 tokens.
A summary of our cost estimates is provided in table 1.
Inference cost.
Beyond improved accuracy, the primary advantage of avoiding few-shot ICL isdramatically lower inference costs.
Processing a single input and all target choices with T-Fewrequires 11 e 9 ⇥ 103 = 1 . 1 e 12 FLOPs, whereas few-shot ICL with GPT-3 175B requires 2 ⇥ 175 e 9 ⇥(41 ⇥ 98 + 103) = 1 .
4 e 15 FLOPs – more than 3 orders of magnitude more.
Inference costs with ICLusing the smaller GPT-3 variants are also dramatically higher than the inference cost of T-Few .
Asdiscussed in section 2.1, caching the key and value vectors when the same set of in-context examplesis to be reused can reduce the computational cost of ICL.
However, this would only result in anapproximately 41 ⇥ reduction, which is not nearly enough to make any of the GPT-3 ICL costs as lowas T-Few .
Training cost.
Since T-Few is the only method that involves updating parameters, it is the onlymethod that incurs a training cost.
Training an eleven billion parameter encoder-decoder model for1,000 steps with a batch size of 8 length-103 sequences requires approximately 3 ⇥ 11 e 9 ⇥ 1 , 000 ⇥ 8 ⇥ 103 = 2 .
7 e 16 FLOPs.
While not insigniﬁcant, this is only about 20 times larger than the FLOPsrequired to process a single example with few-shot ICL using GPT-3 175B. In other words, training
T-Few costs as much as using GPT-3 175B to process 20 examples with few-shot ICL.
We alsofound that ﬁne-tuning T0 with T-Few on a single dataset only takes about a half an hour on a single
NVIDIA A100 GPU.
As of writing, this would cost about $2 USD using Microsoft Azure.
Storage cost.
T-Few also incurs the largest storage cost.
When stored as single-precision ﬂoats, theparameters added by (IA) 3 take up 4.2 MB of space on disk.
In contrast, ICL methods only requirestoring the tokenized in-context examples (typically stored as 32-bit integers), resulting in a smaller41 ⇥ 98 ⇥ 32 bits = 16 kB disk space requirement.
However, we note that 4.2 MB is dwarfed by theon-disk size of the model checkpoints themselves – storing the (IA) 3 adaptation vectors for 10,000tasks would take about as much space as the T0 checkpoint (41.5 GB).
Memory usage.
During inference, the primary memory cost is incurred by the model’s parameters.
The only model smaller than T0 (used by T-Few ) is GPT-3 6.7B; otherwise, T-Few will incur a lowermemory cost during inference.
Additional memory costs are incurred when training T-Few due to theneed to cache intermediate activations for backpropagation and for the gradient accumulator variablesin Adafactor.
However, as mentioned above, it is possible to use the T-Few recipe on a single 80GB
A100 GPU.
4.3 Performance on Real-world Few-shot Tasks (RAFT)
So far, we have evaluated performance on a collection of datasets that were not explicitly designedfor benchmarking few-shot learning.
To better evaluate T-Few ’s performance in the real world, weevaluated our approach on the RAFT benchmark [2].
RAFT consists of 11 “economically valuable”tasks that aim to mirror real-world applications.
Importantly, each RAFT datasets has only 50 trainingexamples with no validation set and a (larger) test set with no public labels, so it is impossible to“cheat” by tuning on an unrealistically-large validation set or by peeking at the test set [32, 31].
Weapply T-Few to RAFT by using the standard prompts released alongside the dataset.
The accuracy ofthe current top-5 methods is shown in table 2, with further details provided in appendix H. T-Fewattains a state-of-the-art accuracy of 75.8% and outperforms the human baseline (73.5% accuracy)for the ﬁrst time.
The next-best model (from Schick and Schütze [50]) achieves 6% lower accuracyand GPT-
175B attains only 62.7%.
These results validate that T-Few can be readily applied as-is tonovel real-world tasks to attain strong performance.
4.4 Ablation experiments
Given that our T-Few design experiments were on T0-3B, we perform an ablation of some of theingredients of T-Few on T0.
Detailed results are shown in appendix G. While the gains from addingeach ingredient does not always signiﬁcant increase the accuracy on each individual datasets, eachingredient consistently improves the average performance across datasets: Removing pre-trainingdecreases accuracy by 1.6%, removing unlikelihood training and length normalization decreasesaccuracy by 4.1%, and removing both pre-training and our additional loss terms reduces accuracy by2.5%.
Conclusion
We introduced T-Few , a parameter-efﬁcient few-shot learning recipe that attains higher accuracy thanfew-shot ICL at a lower computational cost.
T-Few uses (IA) 3 , a new PEFT method that rescalesinner activations with learned vectors.
Using (IA) 3 produces better performance than ﬁne-tuningthe full model while only introducing a tiny amount of additional parameters.
T-Few also uses twoadditional loss terms that encourage the model to output lower probabilities for incorrect choicesand account for the length of different answer choices.
When applying T-Few as-is (with no taskspeciﬁc hyperparameter tuning or other changes) to the RAFT benchmark, we attained super-humanperformance for the ﬁrst time and outperformed prior submissions by a large margin.
Throughdetailed characterization of computational costs, we found that T-Few uses over 1,000 ⇥ fewer FLOPsduring inference than few-shot ICL with GPT-3 and only requires 30 minutes to train on a single
NVIDIA A100 GPU.
Since all of our experiments were on classiﬁcation tasks, we are interested inapplying T-Few to generative tasks like as summarization and question answering in future work.
We hope our results provide a new perspective on how best to perform few-shot learning with largelanguage models. en
azure virtual ndm v4-series
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes] We compare different PEFT methods in Figure 2 anddemonstate PEFT outperforms ICL in section 4.
b ) Did you describe the limitations of your work?
[Yes] Section {+
+}
(c) Did you discuss any potential negative societal impacts of
your
work?
[No]
(d) Have you read the ethics review guidelines and ensured that your
paper
conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[Yes] Included code with README
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Section 3.1, Section 3.5, Appendix D , H
(c) Did you report error bars (e g., with respect to the random seed after running experi
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Appendix A
[Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes] Section 3.1, Section4.3(b)
Did you mention the license of the assets?
[Yes] Section 3.
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
[Table] (d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[No]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[No]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A] [Table]