FANTASTICALLY ORDERED PROMPTS AND WHERE TO FIND
THEM: OVERCOMING FEW-SHOT PROMPT ORDERSENSITIVITY
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, Pontus Stenetorp

Presented by: Sarkar Snigdha Sarathi Das

IN-CONTEXT LEARNING (ICL)•Two samples:•

Review: the greatest musicians. Sentiment: positive.

•Review: redundant concept. Sentiment: negative•

Test Sample: Review: Amazing movie. Sentiment:______•

Priming by concatenation•

Review: the greatest musicians. Sentiment: positive. Review: redundant concept. Sentiment: negative.
Review: Amazing movie. Sentiment: _____

IN-CONTEXT LEARNING (ICL)•

Review: the greatest musicians. Sentiment: positive. Review: redundant concept. Sentiment: negative.
Review: Amazing movie. Sentiment: _____•

Popularized by GPT-3•

No finetuning/gradient updates required!

•Input length during inference increases•

Highly dependent on the template/prompt

IN-CONTEXT LEARNING (ICL) – ORDER SENSITIVITY•

Review: the greatest musicians.
Sentiment: positive. Review:
redundant concept. Sentiment:
negative. Review: Amazing movie.
Sentiment: _____•

Review: redundant concept.
Sentiment: negative. Review: thegreatest musicians. Sentiment:
positive. Review: Amazing movie.
Sentiment: _____•

Unfortunately, this ordering cancause significant variance inperformance.

IMPACT OF TRAINING SAMPLE AND MODEL SIZE•

Large models can get great
performance, however cannot
guarantee low variance inpermutations•

Adding more training samples
does not reduce variancesignificantly

ARE PROMPTS TRANSFERRABLE ACROSS MODELS?
•A specific permutation’s
performance may drop from88.7% to 51.6% by changing theunderlying model from GPT2-XL(1.5B) to GPT2-Large (0.8B)•Taken 4 samples – all 24
permutations of prompts andthen on prediction -> calculate
pairwise Spearman’s rankcorrelation coefficient.

•No correlation at all

LABEL ORDERINGS ALSO DON’T MATTER ACROSS MODELS•Measured across six label
patterns: NNPP, NPNP, NPPN,PNNP, PNPN, PPNN

•Seems like random behavior

DEGENERATE BEHAVIOUR OF BAD PROMPTS

 Most of the failing prompts suffer from highly unbalancedpredicted label distribution
 A way can be using output distribution calibration (Zhao et al.

2021)1
 Basically, using N/A as prompt and see output distribution,then add one affine transformation to counter the imbalance While that improves performance, variance seems still highacross different orders of prompts

1 Calibrate Before Use: Improving Few-Shot Performance of Language Models

HOW TO DETERMINE THE BEST ORDERING?

 Simple way: use a dev set.
 Violates True Few shot setting where we don’t use any dev set.
 Can we generate a probing set by querying the language model instead? Basically, we are using probing

set as a substitute for dev set

PROBING SET GENERATION

There is no guarantee of the validity of the labels of these artificial samples!

PROBING METRICS

 Two different methods are used to select best prompt orderings using the generated/probed samples(minus the labels)

Global Entropy

Local Entropy

PROBING METRICS: GLOBAL ENTROPYPROBING METRICS: GLOBAL ENTROPY

Thus, we calculate the global Entropy for a context
permutation m(allows us to combat extremely unbalanced prediction)

PROBING METRICS: LOCAL ENTROPY

Motivation: if a prompt is overly confident, it is likely that it is not behaving as desired/poorly calibrated

SUMMARY: FROM PROBING TO USING ENTROPY••••

Find probe samples for each of the orderings (artificial x’ s)
Using those x’ s calculate either Global or Local entropy for each of the orderings
Select top k permutations having highest entropy -> Performant prompts
Performant prompts are used to evaluate performance in different datasets

MAIN RESULTS

MAIN RESULTS - ROBUSTNESSCOMPARISON AGAINST SUBSET OF TRAINING DATA FOR TUNING 4-shot data is split in half : dev and test set
 LocalE and GlobalE still outperforms this technique

COMPARISON AGAINST SUBSET OF TRAINING DATA FOR TUNING 4-shot data is split in half : dev and test set
 LocalE and GlobalE still outperforms this technique

CONCLUSION
 This paper effectively shows how few-shot prompts suffer form order sensitivity
 A thorough analysis has shown that order sensitivity is present across tasks, model sizes, prompttemplates, samples, number of training samples
 Without using any external dev set, the local and global entropy calculation using probed samples caneffectively detect any prompt that may cause imbalance Through thorough investigation, on average 13% improvement can be gotten across 11 text classification

tasks

CRITIQUE The limited context window already severely limits how many samples can be used as context. (Liu et al.

20212). As a result, in practice often few shot examples are filtered out, those can be used as dev set.
 While no finetuning is required, ICL itself is computationally heavy during inference due to huge context

window needed (less sparsity)
 Performance still not exactly same as full finetuning even with GPT-3, sometimes quite a bit far away.
 Performance with few shot finetuning could be compared, at least in smaller models which could serve

as a good baseline

2What makes good in-context examples for GPT-3?

FUTURE WORK Apart from ICL, sometimes in finetuning we have trouble finding dev sets in few shot cases since we canuse all the few shot samples there. This approach can be adapted and modified to be used in calibrating
those models as well, for model selection
 Is it possible to generate **ONE** artificial sample that gives the same effect as all the in-context few-shot examples we have? This can both reduce computational load in in-context learning and effectively
use all the few-shot samples to get better performance.