Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Anonymous Author(s)
Affiliation
Address email
Abstract
Few-shot in-context learning (ICL) enables pre-trained language models to perform a previously-unseen task without any gradient-based training by feeding asmall number of training examples as part of the input.
ICL incurs substantialcomputational, memory, and storage costs because it involves processing all of thetraining examples every time a prediction is made.
Parameter-efficient fine-tuning
(PEFT) (e.g. adapter modules, prompt tuning, sparse update methods, etc.)
offersan alternative paradigm where a small set of parameters are trained to enable amodel to perform the new task.
In this paper, we rigorously compare few-shot
ICL and PEFT and demonstrate that the latter offers better accuracy as well asdramatically lower computational costs.
Along the way, we introduce a new PEFT method called (IA) 3 that scales activations by learned vectors, attaining strongerperformance while only introducing a relatively tiny amount of new parameters.
We also propose a simple recipe based on the T0 model [1] called T-Few that can be applied to new tasks without task-specific tuning or modifications.
Wevalidate the effectiveness of T-Few on completely unseen tasks by applying it tothe RAFT benchmark [2], attaining super-human performance for the first timeand outperforming the state-of-the-art by 6% absolute.
All of the code used in ourexperiments will be publicly available. All of the code used in our experimentswill be made publicly available.
Introduction
Pre-trained language models have become a cornerstone of natural language processing, thanksto the fact that they can dramatically improve data efficiency on tasks of interest – i e., using apre-trained language model for initialization often produces better results with less labeled data.
Ahistorically common approach has been to use the pre-trained model’s parameters for initializationbefore performing gradient-based fine-tuning on a downstream task of interest.
While fine-tuninghas produced many state-of-the-art results [1], it results in a model that is specialized for a singletask with an entirely new set of parameter values, which can become impractical when fine-tuning amodel on many downstream tasks.
An alternative approach popularized by [3, 4] is in-context learning (ICL), which induces a modelto perform a downstream task by inputting prompted examples.
Few-shot prompting converts asmall collection of input-target pairs into (typically) human-understandable instructions and examples[3, 4], along with a single unlabeled example for which a prediction is desired.
Notably, ICL requiresno gradient-based training and therefore allows a single model to immediately perform a wide varietyof tasks.
Performing ICL therefore solely relies on the capabilities that a model learned duringpre-training.
These characteristics have led to a great deal of recent interest in ICL methods [5–10].
[Figure]
Despite the practical benefits of ICL, it has several major drawbacks.
First, processing all promptedinput-target pairs every time the model makes a prediction incurs significant compute costs.
Second,
ICL typically produces inferior performance compared to fine-tuning [4].
Finally, the exact formattingof the prompt (including the wording [11] and ordering of examples [12]) can have significant andunpredictable impact on the model’s performance, far beyond inter-run variation of fine-tuning.
Recent work has also demonstrated that ICL can perform well even when provided with incorrectlabels, raising questions as to how much learning is taking place at all [9].
An additional paradigm for enabling a model to perform a new task with minimal updates is parameterefficient fine-tuning (PEFT), where a pre-trained model is fine-tuned by only updating a small numberof added or selected parameters.
Recent methods have matched the performance of fine-tuning thefull model while only updating or adding a small fraction (e.g. 0.01%) of the full model’s parameters[13, 14].
Furthermore, certain PEFT methods allow mixed-task batches where different examples ina batch are processed differently [14], making both PEFT and ICL viable for multitask models.
While the benefits of PEFT address some shortcomings of fine-tuning (when compared to ICL), therehas been relatively little focus on whether PEFT methods work well when very little labeled datais available.
Our primary goal in this paper is to close this gap by proposing a recipe – i.e., a model, a
PEFT method, and a fixed set of hyperparameters – that attains strong performance on novel, unseentasks while only updating a tiny fraction of the model’s parameters.
Specifically, we base our approachon the T0 model [1], a variant of T5 [15] fine-tuned on a multitask mixture of prompted datasets.
To improve performance on classification and multiple-choice tasks, we add unlikelihood [16, 17]and length normalization-based [4] loss terms.
In addition, we develop (IA) 3 , a PEFT methodthat multiplies intermediate activations by learned vectors.
(IA) 3 attains stronger performance thanfull-model fine-tuning while updating up to 10,000 × fewer parameters.
Finally, we demonstratethe benefits of pre-training the (IA) 3 parameters before fine-tuning [18, 19].
Our overall recipe,which we dub “ T-Few ”, performs significantly better than ICL (even against 16 × larger models)and outperforms humans for the first time on the real-world few-shot learning benchmark RAFT [2]while requiring dramatically less compute and allowing for mixed-task batches during inference.
Tofacilitate the use of T-Few on new problems and future research on PEFT, we release our code.
After providing background on ICL and PEFT in the following section, we discuss the design of
T-Few in section 3.
In section 4, we present experiments comparing T-Few to strong ICL baselines.
Finally, we discuss related work in appendix B and conclude in section 5.
Background
In this section, we provide am verview of ICL and PEFT with a focus on characterizing the computation, memory, and on-disk storage costs of making a prediction.
Real-world costs depend onimplementation and hardware, so we report costs in terms of FLOPs for computation and bytes formemory and storage, respectively.
Additional related work is discussed in appendix B.
2.1 Few-shot in-context learning (ICL)
ICL [3, 4] aims to induce a model to perform a task by feeding in concatenated and promptedinput-target examples (called “shots”) along with an unlabeled query example.
Taking the cycled letter task from Brown et al.
[4] as an example, a 4-shot input or context would be “ Pleaseunscramble the letters into a word, and write that word: asinoc = casino,yfrogg = froggy, plesim = simple, iggestb = biggest, astedro = ”, for which thedesired output would be “ roasted ”.
ICL induces an autoregressive language model to performthis task by feeding in the context and sampling from the model.
For classification tasks, eachlabel is associated with a string (e.g. “ positive ” and “ negative ” for sentiment analysis) anda label is assigned by choosing the label string that the model assigns the highest probability to.
For multiple-choice tasks (e.g. choosing between N possible answers to a question), the model’sprediction is similarly determined by determining which choice is assigned the highest probability.
The primary advantage of ICL is that it enables a single model to perform many tasks immediatelywithout fine-tuning.
This also enables mixed-task batches , where different examples in a batch of datacorrespond to different tasks by using different contexts in the input.
ICL is also typically performedwith only a limited number of labeled examples – called few-shot learning – making it data-efficient.
Despite these advantages, ICL comes with significant practical drawbacks: First, making a predictionis dramatically more expensive because the model needs to process all of the in-context labeledexamples.
Specifically, ignoring the quadratic complexity of self-attention operations in Transformerlanguage models (which are typically small compared to the costs of the rest of the model [20]),processing the k training examples for k -shot ICL increases the computational cost by approximatelyk + 1 times compared to processing the unlabeled example alone.
Memory costs similarly scaleapproximately linearly with k , though during inference the memory costs are typically dominated bystoring the model’s parameters.
Separately, there is a small amount of on-disk storage required forstoring the in-context examples for a given task.
For example, storing 32 examples for a task wherethe prompted input and target for each example is 512 tokens long would require about 66 kilobytesof storage on disk ( 32 examples × 512 tokens × 32 bits).
Beyond the aforementioned costs, ICL also exhibits unintuitive behavior.
Zhao et al.
[12] showedthat the ordering of examples in the context heavily influences the model’s predictions.
Min et al.[9] showed that ICL can still perform well even if the labels of the in-context examples are swapped(i.e.
made incorrect), which raises questions about whether ICL is really “learning” from the labeledexamples.
Various approaches have been proposed to mitigate these issues.
One way to decrease computationalcosts is to cache the key and value vectors for in-context examples.
This is possible because decoderonly Transformer language models have a causal masking pattern, so the model’s activations for thecontext do not do not depend on the unlabeled example.
In an extreme case, 32 -shot ICL with 512tokens per in-context example would result in over 144 gigabytes of cached key and value vectors forthe GPT-3 model ( 32 examples × 512 tokens × 96 layers × 12288 d model × 32 bits each for the keyand value vectors).
Separately, Min et al.
[21] proposed ensemble ICL , where instead of using theoutput probability from concatenating the k training examples, the output probabilities of the modelon each training example (i.e. 1 -shot ICL for each of the k examples) are multiplied together.
Thislowers the non-parameter memory cost by a factor of k/ 2 but increases the computational cost bya factor of 2 .
In terms of task performance, Min et al.
[21] find that ensemble ICL outperforms thestandard concatenative variant.
2.2 Parameter-efficient fine-tuning
While standard fine-tuning updates all parameters of the pre-trained model, it has been demonstratedthat it is possible to instead update or add a relatively small number of parameters.
Early methodsproposed adding adapters [22–24], which are small trainable feed-forward networks inserted betweenthe layers in the fixed pre-trained model.
Since then, various sophisticated PEFT methods have beenproposed, including methods that choose a sparse subset of parameters to train [25, 26], producelow-rank updates [13], perform optimization in a lower-dimensional subspace [27], add low-rankadapters using hypercomplex multiplication [28], and more.
Relatedly, prompt tuning [14] and prefixtuning [29] concatenate learned continuous embeddings to the model’s input or activations to induceit to perform a task; this can be seen as a PEFT method [30].
State-of-the-art PEFT methods canmatch the performance of fine-tuning all of the model’s parameters while updating only a tiny fraction(e.g.
0.01%) of the model’s parameters.
PEFT drastically reduces the memory and storage requirements for training and saving the model.
Inaddition, certain PEFT methods straightforwardly allow mixed-task batches – for example, prompt tuning enables a single model to perform many tasks simply by concatenating different promptembeddings to each example in the batch [14].
On the other hand, PEFT methods that re-parameterizethe model (e.g. [27, 13]) are costly or onerous for mixed-task batches.
Separately, different PEFTmethods increase the computation and memory required to perform inference by different amounts.
For example, adapters effectively add additional (small) layers to the model, resulting in small butnon-negligible increases in computational costs and memory.
An additional cost incurred by PEFTis the cost of fine-tuning itself, which must be performed once and is then amortized as the modelis used for inference.
However, we will show that PEFT can be dramatically more computationallyefficient when considering both fine-tuning and inference while achieving better accuracy than ICL.
Designing the T-Few Recipe
Given that PEFT allows a model to be adapted to a new task with relatively small storage requirementsand computational cost, we argue that PEFT presents a promising alternative to ICL.
Our goalis therefore to develop a recipe that allows a model to attain high accuracy on new tasks withlimited labeled examples while allowing mixed-task batches during inference and incurring minimalcomputational and storage costs.
By recipe , we mean a specific model and hyperparameter settingthat provides strong performance on any new task without manual tuning or per-task adjustments.
In this way, we can ensure that our approach is a realistic option in few-shot settings where limitedlabeled data is available for evaluation [31, 32].
3.1 Model and Datasets
As a first step, we must choose a pre-trained model.
Ideally, the model should attain high performanceon new tasks after fine-tuning on a limited number of labeled examples.
In preliminary experimentsapplying PEFT methods to different pre-trained models, we attained the best performance with T0[1].
T0 is based on T5 [15], an encoder-decoder Transformer model [33] that was pre-trained via amasked language modeling objective [34] on a large corpus of unlabeled text data.
T0 was created byfine-tuning T5 on a multitask mixture of datasets in order to enable zero-shot generalization, i.e. theability to perform tasks without any additional gradient-based training.
Examples in the datasets usedto train T0 were prompted by applying the prompt templates from the Public Pool of Prompts (P3[35]), which convert each example in each dataset to a prompted text-to-text format where each labelcorresponds to a different string.
For brevity, we omit a detailed description of T0 and T5; interestedreaders can refer to Sanh et al.
[1] and Raffel et al.
T0 was released in three billion and elevenbillion parameter variants, referred to as “T0-3B” and simply “T0” respectively.
In this section (whereour goal is to design the T-Few recipe through extensive experimentation), we use T0-3B to reducecomputational costs.
For all models and experiments, we use Hugging Face Transformers [36].
While T0 was designed for zero-shot generalization, we will demonstrate that it also attains strongperformance after fine-tuning with only a few labeled examples.
To test T0’s generalization, Sanh et al.[1] chose a set of tasks (and corresponding datasets) to hold out from the multitask training mixture– specifically, sentence completion (COPA [37], H-SWAG [38], and Story Cloze [39] datasets),natural language inference (ANLI [40], CB [41], and RTE [42]), coreference resolution (WSC [43]and Winogrande [44]), and word sense disambiguation (WiC [45]).
Evaluation of generalizationcapabilities can then be straightforwardly done by measuring performance on these held-out datasets.
We also will later test T-Few ’s abilities in the RAFT benchmark [2] in section 4.3, a collection ofunseen “real-world” few-shot tasks with no validation set and a held-out test set.
ANLI, WiC, WSC islicensed under a Creative Commons License.
Winogrande is licnsed under an Apache license.
COPAis under a BSD-2 Clause license.
We could not find the license of RTE and CB but they are part of
SuperGLUE which mentions the datasets are allowed for use in research context.
To ease comparison, we use the same number of few-shot training examples for each dataset as Brownet al.
[4], which varies from 20 to 70.
Unfortunately, the few-shot dataset subsets used by Brownet al.
[4] have not been publicly disclosed.
To allow for a more robust comparison, we thereforeconstructed five few-shot datasets by sampling subsets with different seeds and report the medianand interquartile range.
We prompt examples from each dataset using the prompt templates from P
Bach et al.
[35], using a randomly-sampled prompt template for each example at each step.
Unlessotherwise stated, we train our model for 1K steps with a batch size of 8 and report performance at theend of training.
For evaluation, we use “rank classification”, where the model’s log-probabilities for all possible labelstrings are ranked and the model’s prediction is considered correct if the highest-ranked choice is the correct answer.
Rank classification evaluation is compatible with both classification and multiplechoice tasks.
Since model performance can vary significantly depending on the prompt template used,we report the median accuracy across all prompt templates from P3 and across few-shot data subsetsfor each dataset.
For all datasets, we report the accuracy on the test set or validation set when the testlabels are not public (e.g. SuperGLUE datasets).
In the main text, we report median accuracy acrossthe nine datasets mentioned above.
Detailed results on each dataset are provided in the appendices.
3.2 Unlikelihood Training and Length Normalization
Before investigating PEFT methods, we first explore two additional loss terms to improve theperformance of few-shot fine-tuning of language models.
Language models are normally trainedwith cross-entropy loss L LM = − 1 T (cid:80) t log p ( y t | x , y <t ) where the model is trained to increase theprobability of the correct target sequence
y =
( y 1 , y 2 , . . .
, y T )
given the input sequence x .
For evaluation, we use rank classification (described in section 3.1) which depends on both theprobability that the model assigns to the correct choice as well as the probabilities assigned by themodel to the incorrect choices.
To account for this during training, we consider adding an unlikelihoodloss [16, 17]:
[Equation]
which discourages the model from predicting tokens from incorrect target sequences, where ˆy ( n ) =(
ˆ y 1 , ˆ y 2 , . .
, ˆ y T ( n ) ) is the n -th of N incorrect target sequences.
We hypothesize that adding L UL willimprove results on rank classification because the model will be trained to assign lower probabilitiesto incorrect choices, thereby improving the chance that the correct choice is ranked highest.
The possible target sequences for a given training example can have significantly different lengths,especially in multiple-choice tasks.
Ranking each choice based on probability can therefore “favor”shorter choices because the model’s assigned probability to each token is ≤ 1 .
To rectify this,we consider using length normalization when performing rank classification, which divides themodel’s score on each possible answer choice by the number of tokens in the choice (as used in
GPT-3 [4]).
When using length normalization during evaluation, we introduce an additional lossterm during training that more closely reflects length-normalized evaluation.
First, we compute thelength-normalized log probability of a given output sequence β ( x , y )
= 1 T (cid:80)
Tt =1 log p ( y t | x , y <t ) .
Then, we maximize the length-normalized log probability of the correct answer choice via a standardsoftmax cross-entropy loss:
[Equation]
When training a model with L LM , L UL , and L LN , we simply sum them.
This avoids introducing anyhyperparameters that would be problematic to tune in the few-shot setting (where realistically-sizedvalidation sets are tiny by necessity [31, 32]).
We report the results of fine-tuning all of T0-3B’s parameters with and without length normalizationon all datasets in appendix C.
We find that adding L LN improves the accuracy from 60.7% to 62.71%and including both L UL and L LN provides a further improvement to 63.3%.
Since these loss termsimprove performance without introducing any additional hyperparameters, we include them in ourrecipe and use them in all following experiments.
3.3 Parameter-efficient fine-tuning with (IA) 3
In order to compare favorably to few-shot ICL, we need a PEFT method that has the followingproperties: First, it must add or update as few parameters as possible to avoid incurring storageand memory costs.
Second, it should achieve strong accuracy after few-shot training on new tasks.
Finally, it must allow for mixed-task batches, since that is a capability of ICL.
In order to easilyenable mixed-task batches, a PEFT method should ideally not modify the model itself.
Otherwise,each example in a batch would effectively need to be processed by a different model or computationalgraph.
A more convenient alternative is provided by methods that directly modify the activations ofthe model since this can be done independently and cheaply to each example in the batch accordingto which task the example corresponds to.
Prompt tuning and prefix tuning methods [14, 29] work byconcatenating learned vectors to activation or embedding sequences and are therefore examples ofactivation-modifying PEFT methods that allow for mixed-task batches.
However, as we will discuss later, we were unable to attain reasonable accuracy with prompt tuning and found that the moreperformant PEFT methods did not allow for mixed-task batches.
We therefore developed a new PEFTmethod that meets our desiderata.
As an alternative, we explored element-wise multiplication (i.e. rescaling) of the model’s activationsagainst a learned vector.
Specifically, we consider adaptation of the form l ⊙ x where l ∈ R d is alearned task-specific vector, ⊙ represents element-wise multiplication, and x ∈ R T × d is a length- Tsequence of activations.
We use “broadcasting notation” [46] so that the ( i, j ) th entry of l ⊙ x is l j x i,j .
In preliminary experiments, we found it was not necessary to introduce a learned rescaling vectorfor each set of activations in the Transformer model.
Instead, we found it was sufficient to introducerescaling vectors on the keys and values in self-attention and encoder-decoder attention mechanismsand on the intermediate activation of the position-wise feed-forward networks.
Specifically, usingthe notation from Vaswani et al.
[33], we introduce three learned vectors l k ∈ R d k , l v ∈ R d v , andl ff ∈ R d ff , which are introduced into the attention mechanisms as:
[Equation]
and in the position-wise feed-forward networks as ( l ff ⊙ γ ( W 1 x )) W 2 , where γ is the feed-forwardnetwork nonlinearity.
We introduce a separate set of l k , l v , and l ff vectors in each Transformer layerblock.
This adds a total of L ( d k + d v + d ff ) new parameters for a L -layer-block Transformer encoderand L (2 d k + 2 d v + d ff ) (with factors of 2 accounting for the presence of both self-attention andencoder-decoder attention) for a L -layer-block decoder.
l k , l v , and l ff are all initialized with ones sothat the overall function computed by the model does not change when they are added.
We call ourmethod (IA) 3 , which stands for “Infused Adapter by Inhibiting and Amplifying Inner Activations”.(IA) 3 makes mixed-task batches possible because each sequence of activations in the batch can beseparately and cheaply multiplied by its associated learned task vector.
We also note that, in theevent that a model will only be used on a single task, the modifications introduced by (IA) 3 canalso be applied to weight matrices permanently so that no elementwise multiplication is required andthe model’s architecture remains unchanged.
This possible because element-wise multiplicationsperformed in (IA) 3 always co-occur with a matrix multiplication, and l ⊙
Wx =
( l ⊙ W ) x .
In thiscase, our method incurs no additional computational cost compared to the original model.
To validate (IA) 3 , we compare it to a large variety of existing adaptation methods in our setting offine-tuning T0-3B on few-shot datasets from held-out tasks.
Specifically, we compare with 8 strong
PEFT methods: BitFit [47] which updates only the bias parameters; Adapters [23] which introducetask-specific layers after the self-attention and position-wise feed-forward networks; Compacter and
Compacter++ [28] which improve upon adapters by using low-rank matrices and hypercomplex multiplication; prompt tuning [14] which learns task-specific prompt embeddings that are concatenated tothe model’s input; FISH Mask [26] which chooses a subset of parameters to update based on their approximate Fisher information; Intrinsic SAID [27] which performs optimization in a low-dimensionalsubspace; and LoRA [13] which assigns low-rank updates to parameter matrices.
Additionally, weinclude the baselines of full-model fine-tuning and updating only the layer normalization parameters.
For certain methods that allow changing the parameter efficiency, we report results for differentbudgets: 0.2% and 0.02% sparsity for FISH Mask, 10 and 100 learned prompt vectors for prompttuning, and 20,000- or 500,000-dimensional subspaces for Intrinsic SAID.
The results are shown in fig. 2, with detailed per-dataset results in appendix D. We find that (IA) 3is the only method that attains higher accuracy than the full-model-fine-tuning baseline.
Whileother PEFT methods (e.g. Intrinsic SAID and prompt tuning) update or introduce fewer parameters,(IA) 3 performs considerably better.
Our results and setting differ with some past work on the
PEFT methods we compare against.
Mahabadi et al.
[28] report that Compacter and Compacter++outperform full-model fine-tuning, including in the few-shot setting.
Lester et al.
[14] found thatprompt tuning could match full-model fine-tuning, and in subsequent work Wei et al.
[48] found thatprompt tuning performed well when applied to a multitask fine-tuned model in the few-shot setting.
In both cases, we experimented with various hyperparameter choices to try to match past results.
We hypothesize the disagreement comes from us using a different model and different datasets.
Forprompt tuning specifically, we noticed that the validation set performance could fluctuate wildly overthe course of training, hinting at possible optimization issues. 0.001% 0.01% 0.1% % of parameters updated [Figure] cc
[Figure]
y
All parameters (IA)³ LoRA BitFit Layer Norm
Compacter
Compacter++
Prompt Tuning
Prefix Tuning
Adapter
FISH Mask
Intrinsic SAID
FLOPs per example
A cc u r a c y
T-Few
T
T5+LM
GPT-3 6.7B
GPT-3 13B
GPT-3 175B
3.4 Pre-training (IA) 3
In recent work, Gu et al.
[18], Vu et al.
[19] showed that pre-training the prompt embeddings inprompt tuning can improve performance when fine-tuning on downstream few-shot tasks.
For pretraining, Gu et al.
[18] use a suite of self-supervised tasks applied to unlabeled text data, and Vuet al.
[19] consider using embeddings from a separate task or multitask mixture.
We follow Vu et al.[19] and simply pre-train the new parameters introduced by (IA) 3 on the same multitask mixtureused to train T0.
We pre-train for 100,000 steps with a batch size of 16 before fine-tuning the (IA) 3parameters on each individual downstream dataset.
A full comparison of accuracy with and withoutpre-training (IA) 3 is detailed in appendix E. We find that pre-training improves fine-tuned accuracyfrom 64.6 to 65.8 and therefore add it to our recipe.
3.5 Combining the ingredients
In summary, the T-Few recipe is defined as follows: We use the T0 model as a backbone.
We add(IA) 3 for downstream task adaptation and use parameters initialized from pre-training (IA) 3 on thesame multitask mixture for T0.
As an objective, we use the sum of a standard language modelingloss L LM , an unlikelihood loss L UL for incorrect choices, and a length-normalized loss L LN .
Wetrain for 1,000 steps with a batch size of 8 sequences using the Adafactor optimizer [49] with alearning rate of 3 e − 3 and a linear decay schedule with a 60-step warmup.
We apply prompt templatesto downstream datasets during training and inference to convert each example into an instructivetext-to-text format.
Importantly, we apply this recipe to every downstream dataset in exactly the sameway without per-dataset hyperparameter tuning or modifications.
This makes the recipe a realisticoption for few-shot learning settings where validation sets are tiny by definition [31, 32].
Outperforming ICL with T-Few
Having designed and established the T-Few recipe on the T0-3B model, we now apply it to T0 (witheleven billion parameters) and compare performance to strong few-shot ICL methods.
We do notmake any modifications to the T-Few recipe to account for the change in model size.
4.1 Performance on T0 tasks
First, we evaluate T-Few on the datasets that were held out from T0’s training mixture.
We compareagainst zero-shot learning with T0 [1] (since we found few-shot ICL to performed worse than zeroshot for T0, see appendix F); few-shot ICL with T5+LM, the next-step-prediction language model upon which T0 is based; and few-shot ICL with the 6.7, 13, and 175 billion parameter variants of GPT-3.
See appendix F for more details on these baselines.
The accuracy on the held-out T0 datasets (described in section 3.1) is shown in table 1 and fig. 3,with per-dataset results reported in appendix
F. We find that T-Few outperforms all other methods bya substantial margin.
Notably, T-Few achieves a 6% higher accuracy than few-shot ICL with GPT-3175B despite being about 16 × smaller and outperforms the smaller GPT-3 variants by an even largermargin.
T-Few also attains significantly higher accuracy than its sibling models, outperforming bothzero-shot learning with T0 and few-shot ICL with T5+LM.
[Table]
[Table]
[Table]
4.2 Comparing computational costs
Having established that T-Few significantly outperforms ICL-based models, we now compare therelative costs of each few-shot learning approach.
For simplicity, we use the FLOPs-per-tokenestimates for Transformer-based language models introduced by Kaplan et al.
Specifically, weestimate that a decoder-only Transformer (e.g. the GPT series) with N parameters uses 2 N FLOPsper token for inference and 6 N FLOPs per token for training.
Encoder-decoder models like T0 and
T5 (where the encoder and decoder have the same number of layers and layer sizes) only processeach token with either the encoder or decoder (each having roughly half the parameters of the fullmodel), so the FLOPs per token estimates are halved to N and 3 N FLOPs per token for inference andtraining.
We note that FLOPs are not a direct measurement of real-world computational cost becauselatency, power usage, and other costs can vary significantly depending on hardware and other factors[52].
However, we focus on FLOPs because it is a hardware-independent metric that closely withreal-world costs the hardware setup used for running the different methods we consider would likelyvary significantly across methods.
We summarize the costs in table 1 and discuss them below.
For allestimates, we use the median number of shots (41) across the datasets we consider.
Rank evaluationand our unlikelihood loss both require processing every possible output choice to attain a predictionfor an unlabeled example.
The median combined tokenized sequence length for the input and allpossible targets is 103 for the datasets we consider.
For in-context examples processed for few-shot
ICL, only the correct target is required, producing a median sequence length of 98.
Assuming thatkey and value vectors are cached, processing a single example with ICL therefore involves processing41 × 98 + 103 tokens.
A summary of our cost estimates is provided in table 1.
Inference cost.
Beyond improved accuracy, the primary advantage of avoiding few-shot ICL isdramatically lower inference costs.
Processing a single input and all target choices with T-Fewrequires 11 e 9 × 103 = 1 . 1 e 12 FLOPs, whereas few-shot ICL with GPT-3 175B requires 2 × 175 e 9 ×(41 × 98 + 103) = 1 .
4 e 15 FLOPs – more than 3 orders of magnitude more.
Inference costs with ICLusing the smaller GPT-3 variants are also dramatically higher than the inference cost of T-Few .
Asdiscussed in section 2.1, caching the key and value vectors when the same set of in-context examplesis to be reused can reduce the computational cost of ICL.
However, this would only result in anapproximately 41 × reduction, which is not nearly enough to make any of the GPT-3 ICL costs as lowas T-Few .
Training cost.
Since T-Few is the only method that involves updating parameters, it is the onlymethod that incurs a training cost.
Training an eleven billion parameter encoder-decoder model for 1,000 steps with a batch size of 8 length-103 sequences requires approximately 3 × 11 e 9 × 1 , 000 ×8 × 103 = 2 .
7 e 16 FLOPs.
While not insignificant, this is only about 20 times larger than the FLOPsrequired to process a single example with few-shot ICL using GPT-3 175B. In other words, training
T-Few costs as much as using GPT-3 175B to process 20 examples with few-shot ICL.
We alsofound that fine-tuning T0 with T-Few on a single dataset only takes about a half an hour on a single
NVIDIA A100 GPU.
As of writing, this would cost about $17 USD using Microsoft Azure.
Storage cost.
T-Few also incurs the largest storage cost.
When stored as single-precision floats, theparameters added by (IA) 3 take up 4.2 MB of space on disk.
In contrast, ICL methods only requirestoring the tokenized in-context examples (typically stored as 32-bit integers), resulting in a smaller41 × 98 × 32 bits = 16 kB disk space requirement.
However, we note that 4.2 MB is dwarfed by theon-disk size of the model checkpoints themselves – storing the (IA) 3 adaptation vectors for 10,000tasks would take about as much space as the T0 checkpoint (41.5 GB).
Memory usage.
During inference, the primary memory cost is incurred by the model’s parameters.
The only model smaller than T0 (used by T-Few ) is GPT-3 6.7B; otherwise, T-Few will incur a lowermemory cost during inference.
Additional memory costs are incurred when training T-Few due to theneed to cache intermediate activations for backpropagation and for the gradient accumulator variablesin Adafactor.
However, as mentioned above, it is possible to use the T-Few recipe on a single 80GB
A100 GPU.
4.3 Performance on Real-world Few-shot Tasks (RAFT)
So far, we have evaluated performance on a collection of datasets that were not explicitly designedfor benchmarking few-shot learning.
To better evaluate T-Few ’s performance in the real world, weevaluated our approach on the RAFT benchmark [2].
RAFT consists of 11 “economically valuable”tasks that aim to mirror real-world applications.
Importantly, each RAFT datasets has only 50 trainingexamples with no validation set and a (larger) test set with no public labels, so it is impossible to“cheat” by tuning on an unrealistically-large validation set or by peeking at the test set [32, 31].
Weapply T-Few to RAFT by using the standard prompts released alongside the dataset.
The accuracy ofthe current top-5 methods is shown in table 2, with further details provided in appendix H. T-Fewattains a state-of-the-art accuracy of 75.8% and outperforms the human baseline (73.5% accuracy)for the first time.
The next-best model (from Schick and Schütze [50]) achieves 6% lower accuracyand GPT-
175B attains only 62.7%.
These results validate that T-Few can be readily applied as-is tonovel real-world tasks to attain strong performance.
4.4 Ablation experiments
Given that our T-Few design experiments were on T0-3B, we perform an ablation of some ofthe ingredients of T-Few on T0.
We experiment with omitting the step of pre-training (IA) 3 andremoving unlikelihood training and length normalization Detailed results are shown in appendix G. We confirm that each of the ingredients provides a boost in accuracy: Removing pre-training decreasesaccuracy by 1.6%, and removing both pre-training and our additional loss terms reduces accuracy byan additional 2.5%.
Conclusion
We introduced T-Few , a parameter-efficient few-shot learning recipe that attains higher accuracy thanfew-shot ICL at a lower computational cost.
T-Few uses (IA) 3 , a new PEFT method that rescalesinner activations with learned vectors.
Using (IA) 3 produces better performance than fine-tuningthe full model while only introducing a tiny amount of additional parameters.
T-Few also uses twoadditional loss terms that encourage the model to output lower probabilities for incorrect choicesand account for the length of different answer choices.
When applying T-Few as-is (with no taskspecific hyperparameter tuning or other changes) to the RAFT benchmark, we attained super-humanperformance for the first time and outperformed prior submissions by a large margin.
Throughdetailed characterization of computational costs, we found that T-Few uses over 1,000 × fewer FLOPsduring inference than few-shot ICL with GPT-3 and only requires 30 minutes to train on a single
NVIDIA A100 GPU.
Since all of our experiments were on classification tasks, we are interested inapplying T-Few to generative tasks like as summarization and question answering in future work.
We hope our results provide a new perspective on how best to perform few-shot learning with largelanguage models.
https://docs.microsoft.com/en-us/azure/virtual-machines/ndm-a100-v4-series
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] We compare different PEFT methods in Figure 2 anddemonstate PEFT outperforms ICL in section 4.
(b) Did you describe the limitations of your work?
[Yes] Section[-
-]
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Included codewith README
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] Section 3.1, Section 3.5, Appendix D, H
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Appendix C, D, E, F, G(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Appendix A
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] Section 3.1, Section4.
(b) Did you mention the license of the assets?
[Yes] Section 3.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re [Table] using/curating?
[No]
[Table]
Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[No]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
Did you include the full text of instructions given to participants and screenshots, [Table] applicable
N A]
[Table]
you describe any participant risks, with links to Institutional Review
[Table]
approvals,
[N
[Table]
Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation
[N/A] [Table]