How Many Demonstrations Do You Need for In-context Learning?
Authors: Jiuhai Chen
Lichang Chen
Chen Zhu
Tianyi Zhou
Venue: EMNLP 2023 Findings (2023)
Keywords: In-context learning: Large language model

Abstract: Large language models (LLMs) are capable to perform complex reasoning by in-context learning (ICL) when provided with a few input-output demonstrations (demos) and more powerful when intermediate reasoning steps (chain of thoughts (CoT)) of the demos are given. Is it necessary to use multi-demo in ICL? In this paper, we study ICL using fewer demos for each test query on the tasks in~\cite{wei2022chain}. Surprisingly, we do not observe significant degradation when using only one randomly chosen demo. To study this phenomenon, for each test query, we categorize demos into ''positive demos'' leading to the correct answer, and ``negative demos'' resulting in wrong answers. Our analysis reveals an inherent bias in those widely studied datasets and the redundancy of demos: most demos are positive for a majority of test queries, which explains the good performance of ICL with one random demo. Moreover, ICL (with and w/o CoT) using only one positive demo significantly outperforms multi-demo ICL adopted by most previous works, indicating the weakness of LLMs in finding positive demo(s) for input queries, which is difficult to evaluate on the biased datasets. Furthermore, we observe a counterintuitive behavior of ICL using multi-demo, i.e., its accuracy degrades(improves) when given more positive(negative) demos. This implies that ICL can be easily misguided by interference among demos and their spurious correlations. Our analyses highlight several fundamental challenges that need to be addressed in LLMs training, ICL, and benchmark design.

How Many Demonstrations Do You Need for In-context Learning?
Jiuhai Chen jchen I 69@ umd.edu
Lichang Chen bobchen @umd.edu
Chen Zhu
Tianyi Zhou tianyi @ umd.edu
chenzhu @umd.edu
University of Maryland
Abstract
is in-context learning (ICL) (Brown al., 2020), which provides LLMs with human-written instruc- tion and a few exemplars or demonstrations (de- mos), along with the input queries. However, con- ventional few-shot prompting performs poorly on complex reasoning tasks (Wei et al., 2022). Re- cently, an effective ICL strategy for complex rea- soning tasks, including arithmetic reasoning, com- monsense reasoning, and symbolic reasoning, elaborate the intermediate reasoning step each demo (Wei et al., 2022), namely Chain-of-thoughts (CoT) prompting.
Large language models (LLMs) are capable perform complex reasoning by in-context learning (ICL) when provided with few input- output demonstrations (demos) and more pow- erful when intermediate reasoning steps ('chain of thoughts (CoT)") of the demos are given. it necessary to use multi-demo in ICL? In this paper, we study ICL using fewer demos for each test query on the tasks in (Wei et al., 2022). Surprisingly, we do not observe significant degradation when using only one randomly chosen demo. To study this phenomenon, for each test query, we categorize demos into "pos- itive demos" leading to the correct answer, and "negative demos" resulting in wrong answers. Our analysis reveals inherent bias in those widely studied datasets and the redundancy of demos: most demos are positive for a majority of test queries, which explains the good perfor- mance of ICL with one random demo. More- over, ICL (with and w/o CoT) using only one positive demo significantly outperforms multi- demo ICI, adopted by most previous works, indicating the weakness of LLMs in finding positive demo(s) for input queries, which is difficult to evaluate on the biased datasets. Fur- thermore, observe a counterintuitive be- havior of ICL using multi-demo, i.e., its accu- racy degrades(improves) when given more posi- tive(negative) demos. This implies that ICL can be casily misguided by interference among de- mos and their spurious correlations. Our anal- yses highlight several fundamental challenges that need to be addressed in LLMs training, ICL, and benchmark design.
ICL relies on human engineering and expertise in designing demo questions, intermediate reasoning steps, and final answers so the LLM can generalize them to variety of unseen queries. However, it is inefficient and impractical to design demos for different queries but fixed set of demos might not cover all the possible queries. In addition, adding demos (especially in CoT prompting) significantly increases input tokens, which are costly and may exceed the maximum input length of LLMs. To provide better demos for efficient ICL and save human efforts, a very recent line of work studies automatic prompting (Zhang et al., 2022; Wang et al., 2023; Arora et al., 2022), which leverage LLMs to select demo questions and construct their answers and CoTs for ICL. They save human labor on creating demos but do not address several fun- damental questions in ICL, c.g., How many demos are necessary for ICL? Can LLMs in figure out which demo(s) is more useful to each test query? Does ICL leverage all the demos or mainly rely on few of them resolve each test query? Can LLMs in ICL combine the strengths of multiple demos improve the answers?
1 Introduction
The recent race of large Language models (LLMs) (Brown et al., 2020; Chowdhery et al., 2022; Thop- pilan et al., 2022; Rae al., 2021) shows that the capability of reasoning can be significantly im- proved with the scaling of model size. One of the most remarkable behaviors observed in LLMs
In this paper, we take the first step toward bet- ter understanding the effect of multiple demos in ICL. through series empirical studies on the de- mos and benchmarks widely used in CoT prompt- ing (Wei et al., 2022), which covers a diverse set

---

reasoning tasks. In particular, we investigate how the ICL (with and w/o CoT) performance changes when varying the number of demos and the im- pact of each demo on different test queries. We start with an extreme case of ICL using only one demo randomly chosen for each query. Surpris- ingly, compared to the default 8 (or 7)-demo ICL in previous work, we do not observe a significant drop in the test accuracy. But does this imply that multiple demos are unnecessary to ICL? To study this phenomenon, we take closer look the pro- portion of positive demos (i.e., the demos leading to correct answers in one demo ICL) for each test query. Statistics on all the datasets reveal a widely existing bias of easy queries, i.e., most demos are positive for majority of queries, for which one (random) demo is all they need.
test query (Rubin et al., 2022), estimating the LLM's bias for better learning calibration parame- ters (Zhao et al., 2021), etc.
Chain-of-Thoughts (CoT) and its variants. CoT has been recently introduced to elicit the rea- soning abilities of LLMs (Wei et al., 2022) by aug- menting each demo with chain of rational steps, Many follow-ups ups works further improve the perfor- mance of CoT, e.g., self-consistency (Wang et al., 2022b) draws an ensemble of outputs for major- ity voting to replace the greedy decoding. How- ever, CoT still heavily relies on human expertise to annotate the reasoning chains. A handful of re- cent works have explored the idea of automatic prompting (Zhang et al.. 2022: Huang et al.. 2022; Wang et al.. 2023). For instance, Auto-CoT (Zhang et al., 2022) proposes to select queries of the de- mos via clustering all test queries and sampling demo queries with diversity. (Huang et al., 2022) fine-tunes an LLM with high-confidence rationale- augmented answers for unlabeled questions. Wang et al. (2023) views the LLM as a topic model and proposes an algorithm selecting the optimal demo from a set of annotated data. While beneficial, most automatic prompting methods focus on by- passing human engineering and building better de- mos from set of questions. But they do not inves- tigate whether demos are used in the correct way by LLMs in ICL. In contrast, we find that the origi- nal demos provided by (Wei et al., 2022) include adequate information (c.g., one positive demo per query) for the LLMs to produce correct answers.
That being said, how does ICL perform on the test queries with fewer positive demos? Unfortu- nately, though provided with some positive demos, ICL fails to produce correct answers for many of them. We verify this by evaluating ICL with one positive demo, which significantly outperforms the widely used multi-demo ICL. This exposes a weak- ness of LLMs, i.e., they are not good at identifying the positive demo(s) and ignoring the negative ones for each query in multi-demo ICL, even when more details such as CoT are given. Our further analy- sis reveals another deeper reason for this. Specif- ically, we start from ICL with one positive (neg- ative) demo but adding more positive (negative) demos results in counterintuitive degradation (im- provement) of ICL accuracy, indicating a negative impact of the interference or spurious correlation among demos on the LLMs. Therefore, multiple demos might provide more information than a sin- gle demo but the current LLMs and ICL methods cannot fully exploit them and filter out misleading interference.
The role of demos in ICL. Several works have explored the mechanism behind the success of CoT prompting. Min et al. (2022) observes that label correctness is not the critical reason for the suc- cess of few-shot ICI/prompting. Madaan and Yaz- danbakhsh (2022) also finds that the label correct- ness is immaterial to the task on GSM8K. Instead, Madaan and Yazdanbakhsh (2022) constructs three key components in rational and identifies which component plays a vital role in CoT. Saparov and He (2022) concludes that LLMs are capable of mak- ing correct individual deduction steps but have diffi- culty systematically exploring the different options. Wang al. (2022a) shows that CoT reasoning is possible even with invalid demos. These works try to understand what makes CoT prompting effective. However, few works focus on varying the number of demos and inherent dataset bias in sho- ICL
2 Related Work
In-Context Learning (ICL). In-context learn- ing (ICL) provides an efficient strategy to per- form downstream task adaptations on pretrained LLMs (Brown et al., 2020). By prepending task- specific instructions and some demos each test query, the LLM is able accomplish highly spec- ified tasks. Recent work in ICL focuses on au- tomatically determining the prompts, e.g., train- ing a dense retriever to allocate semantically simi- lar training examples (Liu et al., 2022) for each

---

CoT prompting.
performance? How many demos are needed for complex reasoning tasks? To answer these ques- tions, we start by investigating the simplest case, i.e., one-demo ICL. Surprisingly, reducing the num- ber of demos to one does not bring critical degra- dation even when the demo is randomly selected. When we can filter out negative demos (defined in Section 4.2) for each test query, one demo ICL sig- nificantly outperforms the widely used multi-demo ICL. We provide an in-depth analysis of the rea- sons behind these phenomena and they reveal some fundamental issues of LLMs and benchmarks.
3 Background and Experimental Setup
3.1 Tasks and Datasets
We conduct series of experiments on various reasoning benchmarks: arithmetic reasoning: GSM8K (Cobbe et al., 2021), MultiArith (Roy and Roth, 2016), AddSub (Hosseini et al., 2014), SVAMP (Patel et al., 2021), AQuA (Ling et al.. 2017) and SingleOp (Wei et al., 2022). common- sense reasoning: CSQA (Talmor et al., 2019). symbolic reasoning: Coin-flip (Wei al., 2022). The overall statisties are listed in table
4.1 Prompting with One Random Demo
We compare ICL with one random demo with ICL with all demos when each demo associated with/without CoT.
TASK
# Demo
# Query
GSM8K
Arithmetic
1319
MultiArith
Arithmetic
8
600
AddSub
Arithmetic
8
395
SVAMP
Arithmetic
1000
One Random Demo is randomly selected from few demos crafted by (Wei et al., 2022). We prepend this single demo to the test sample and query the language model once.
AQuA
Arithmetic
SingleOp
Arithmetic
8
508
CSQA
Commonsense
1221
Coin-flip
Symbolic
500
All Demos is the baseline reported by (Wei al., 2022), prepending all demos (e.g., 8 demos for arithmetic reasoning tasks) to the test sample and query the model once.
Table Statistics of datasets. Demo is the number of CoT exemplars provided by Wei et al. (2022).
3.2 Language Model and In-Context Learning
Results for few-shot (without CoT) and CoT prompting on variety of datasets are reported in Fig. and Fig. 2, respectively, For both ICL meth- ods, reducing seven eight demos (green bar) to one random demo (blue bar) causes only slight degradation (0-7%) on the test accuracy, while sig- nificantly reducing the input length and compu- tational cost. These savings are attractive since most API LLMs are billed based on the number of input tokens (e.g., $ 0.02 per 1000 tokens for GPT-3). On most evaluated tasks, one random demo suffices to achieve the most phenomenal improvement by ICL but using more than demo only brings marginal improvement. It indicates an inefficient usage of demos in ICL, despite their presumed high quality and diversity (as they are carefully created by humans).
To efficiently conduct the experiments, we focus on code-davinci-002 (Chen et al., 2021: Chowdhery al., 2022) from the GPT-3 model family. Because when we started the experiments, code-davinci-002 is a highly efficient programming generation en- gine that offers superior performance at an afford- able price, especially for reasoning tasks (Wang et al., 2022b; Zhang et al., 2022). We explore two prompting settings for in-context learning:
Few-shot prompting. Standard few-shot prompt- ing (Brown et al., 2020) in which demos are for- matted as Question + Answer pairs appended to each test query.
CoT prompting. We also conduct experiments on CoT prompting where each demo is augmented by a chain of thoughts (Wei et al., 2022) in the form of Question + rationale + Answer.
But what are the reasons behind this inefficient usage of multiple demos? Is it due to weakness of current LLMs or ICI. on exploiting demos or an inherent redundancy of the handcrafted demos for these benchmark tasks? Given the above obser- vations, it is plausible that different demos might provide redundant information to each test query so any randomly chosen one should do the same job. But does this hold for all test queries? Does
4 One-Demo Prompting
It is common use multiple demos in ICL, e.g.. manual-Cofi (Wei et al., 2022) relies on humans to create a few demos for different tasks as shown in Table 1. But do multiple demos really improve ICL

---

One Random demo
One Postibve Damo
demos
main purpose of this section is analyze the unex- pected behaviors of single-demo ICL and compare its best and worst performance. Our observations lead to novel insights that can potentially be used to design demo selection methods but we do not aim to develop such methods in this paper.
100
80
40
A demo be positive for query but negative for another query. For example, Fig. 4 shows that the eight demos designed for GSM8K are all posi- tive for an easy query but all negative and lead to incorrect answers for another hard query. Hence, it is interesting to study the proportion of easy and hard queries in the widely used benchmark datasets, Given that we have eight demos in total, it is rea- sonable to define the Easy Sample to be the queries with 6 demos to be positive and Hard Sample to be the queries with merely S 1 positive demo. Therefore, the probability of choosing a positive demo for easy samples in the one random demo prompting is 2 75% (at least 6 positive demos from 8 demos) while the probability for hard sam- ples is S 12.5% (at most positive demos from 8 demos). To explain the high accuracy of prompting with one random demo, natural problem to study IS: what is the percentage of easy/hard samples in each dataset?
20-
0StraK k. AddBub
SWAMP
AQUA
Single0p
CSQA
Figure 1: ICL without CoT: Prompting with one ran- dom demo has slightly lower accuracy than few-shot prompting (8 or demos). Prompting with one positive demo significantly outperforms few-shot prompting.
100-
80-
Accurity
4.-
20-
COMIKKoRAAttaaas... SWAMP
SingieOp
CRGA
Letter
We report the statistics of easy and hard sam- ples according to the number of positive demos for each sample in two commonly used ICL datasets, CSQA and GSM8K, where the former is for arith- metic reasoning and the latter is for commonsense reasoning. In particular, Fig. 5 (a) and Fig. 6 (a) give statisties of all test queries in terms of the num- ber of positive demos in GSM8K and CSQA. On both datasets, we observe that easy samples are the majority while hard samples take up a very small fraction. Notably, as shown in Fig. 6 (a), ~ 58% of test queries in CSQA dataset are easy (having 2 6 positive demos out of 8 so the success probability of one random demo prompting for these samples is N 75%.
ICL with CoT: Prompting with one random demo has a slightly lower accuracy than CoT prompting (8 or 7 demos). Prompting with one positive demo significantly outperforms CoT prompting.
Figure 2:
there exist the best demo for each query? When the LLMs are API models, their weights cannot be farther finetuned, is it still possible to improve their ICL performance through the demos?
4.2 Positive/Negative Demos and Hard/Easy Samples in Datasets
For an in-depth study of these questions, we cate- gorize all the demos into positive/negative demos for each input query in the one-demo prompting setting, i.e., "positive Demos" enabling the LLMs to produce a correct answer while "negative De- mos" results in wrong answers. One example of "negative/positivee Demo" under CoT prompting is shown in Fig. 3. We then study the proportions of positive demos for test queries in each benchmark dataset, which reflect the probability of randomly sampling a positive demo that can be used to ex- plain previous observations. We point out that the
In contrast, only 8 % of CSQA dataset are hard samples, for which randomly selecting a demo out of the eight results in < 12.5% ICL accuracy. Hence, easy samples dominate CSQA, for which the 8 or 7 demos are highly redundant. Morcover, observe similar statistics on other datasets such as GSM8K and almost all the datasets used in CoT prompting papers (see Table 2 in the appendix). This explains the marginal improvement of multi- demo ICL over ICL with only one random demo

---

Model Input
Negative Demo
Model Input
Positive Demo
Q: If there are cars the perking lot and arrive, how many are the parking lot?
au Shown has five toys. For Christmas, he got two tays each from now?
mom dad. How many toys
A: There originally 3 cars, more cars arrive. 3 2 The howwor S
A: Shawn started with 5 toys. If he got toys each from his mom toys.
and
The school cafeteria ordered red and green apples for students lunches, But, only 5 students wanted fruit, hew many did cafeteria end with?
Q: The school cafeteria ordered 6 red apples and 15 apples students lunches. But, only students wanted fruit, how many extra did the calleteria end with?
Model Output
Model Output
A: are originally apples green apples, more students wanted fruit. 5+ 15 =5 26. The answer 26.
A: There are 21 apples in total. students take 5 apples. The cafeteria has 21 = apples left.
Figure 3: Negative/Positive Demo, In one demo ICL for a test query, a negative demo leads to an incorrect answer
while a positive demo results in the correct answer.
(shown in Fig. and Fig. 2): most queries in these datasets are easy samples that only require one random demo produce the correct answers increasing the demos does not bring significant improvement.
(with high probability) for easy samples when most demos are positive but can be easily con- fused/misguided by few negative demos, even the majority are still positive demos. This also ex- plains the marginal difference between all-demo ICL and one random demo ICL on different data groups as shown in Fig. 7-8, in which ICL with one positive demo instead can achieve 100% accu- racy for data groups with > 1 positive demos. In other words, LLMs cannot precisely distinguish positive and negative demos for a query. Unfortu- nately, this weakness of LLMSs cannot be reflected by evaluations on most existing ICL benchmarks because of the aforementioned dataset bias.
Given the statistics of positíve demos per sam- ple in dataset, can estimate the accuracy of prompting with one random demo by the expected probability of a randomly chosen demo being pos- itive for queries from each dataset. Specifically, let NN be the number of available demos (N for GSM8K and N = 7 for CSQA) and p.. be the percentage of samples with n positive demos, then the estimated accuracy of one random demo ICI. is EN Pn N. For instance, given the statistics in Fig. 5 (a) and Fig. 6 (a), the estimated accuracy is 52% for GSM8K and 71% for CSQA, which matches the empirical accuracy for one random demo ICL. reported in Fig. 2.
4.3 Prompting with One positive Demo
Multi-demo ICL is inefficient in the usage of input tokens and can easily be misguided by a few neg- ative demos. On the other hand, by definition, a positive demo results in a correct answer, and most samples in those datasets have at least one positive demo, according to the statistics such as Fig. 5 (a) and Fig. 6 (a).
This reveals widely existing dataset bias, i.e., easy samples dominate these benchmark datasets and the difficulty follows a long tail dis- tribution. Though this could be claimed as an ad- vantage of the desannde demos, it implics redundancy and inefficient usage of these demos: one positive demo suffices to produce the correct answers for most queries (as they are easy), while multi-demo ICL multiplies the cost but only brings marginal improvement
Hence, it is intuitive compare the widely used multi-demo ICL with ICL including only one sin- gle positive demo in the prompt Surprisingly, shown Fig. and 2, one positive demo ICL significantly outperforms the multi-demo ICL. even the latter spends 8x (or 7x) cost of the for- mer and includes the demo used in the former. For example, there are 83% samples in GSM8K with > 1 positive demos so the one positive demo ICL enjoys accuracy of ~ 83%, which is much bet- ter than the 60% accuracy of ICL. using all the eight demos. We consistently observe similar per-
few queries. Since the maximum input length for LLMs is strictly limited, this also indicates a bottleneck of multi-demo ICL when the targeted tasks become practically more complicated and requires more diverse demos for different types of queries.
The inefficient usage of multiple demos also exposes a weakness of LLMs when applied for ICL, i.e., they can only produce correct answers
We randomly chonse one demo for samples without any positive demo

---

One demo ICL
Output
Demo
Hard Sample
Negative Dema
512
Mark plants bronstolk
trees grove.
bolow hiss secund
the grove today, After they are done, there will be 21 trees, flow many trees the grove worksrs plant todiey?
256
Negetive
windowe, whech is fownt off
2048
Megative Denea
doubiees The bre
2f there 3 cars the parking and more arrivw, how
Magitive Donee
beonszall
2048
Megative Dems
taller
Leah chocolates
had
how
1024
Megatie
many talipaps. gave Denny lalipaps,
1024
Megatha
has lippps.. How many tollipops lason give Danny?
Output
Damo Type
Easy Sample
Shawn has Thve toys. For Christmas, he two toys each from dad.
Poctfee
now?
Poulthv Demoo
Alisa bikod 12 miles per hour for 4.5 hours, Stanley per for 2.5 hours. Hew many miles did Alisa and Stanley tocal?
There were nine computers in the server room. more
computers were installedd each day, from Monday to Thursday, How many computers are now the server room?
Positee Dunno
Poulthue
had 58 Wednesday, Wirdneday??
he golt balls. many gol
Postuve Demo
Positrcc Damo
8. Olivis has 23. She bought five bagels for 3 each. How much money does she have lef??
79
Posthec
Figure 4: Easy/Hard Samples from GSM8K: for hard query (Mark plants a beanstalk ...), all 8 demos are negative and result in wrong answers in one-demo ICL; for the easy query (Alisa biked 12 miles ...). all the 8 demos are positive and lead to the correct answer. The 8 demos for arithmetic problems are from (Wei et al., 2022).
(a). GSM8K
GSMBK-Hard
(b). CSQA-Hard
17%
8%
11M
Toulive Demoss
(c). GSMBK-Easy
(c). CSQA-Easy Easy
41%
bwg Pos Demos
5.%
3%
12%
PAR
7%
Figure 6: Pie chart on the number of positive demos (ICL with CoT) per sample/query (0 inside the pic chart) for queries in (a) the whole CSQA dataset (b) CSQA-Hard; (c): CSQA-Easy.
Figure 5: Pie chart on the number positive demos (ICL with CoT) per sample/query (0 6 inside the pie chart) for queries in (a) the whole GSM8K dataset (b) GSM-Hard; (c): GSM-Easy.
demo ICL. In contrast, all-demo ICL, even with most (> 6) demos are positive, is prone to cross- demo interference and misguidance, leading to poorer accuracy than one random demo ICL. But which is the essential reason for this gap? Can we improve the ICL performance by introducing more positive-only demos? Is it possible for LLMs stitch the relevant/correct pieces of negative demos to build correct answer?
formance gaps on all evaluated datasets, in both ICL without CoT (few-shot prompting) and ICL with CoT prompting. Therefore, these comparisons suggest that selecting one positive demo can be both more efficient and more effective than us- ing multiple demos in ICL.
Moreover, considering that the positive demo for each query is already included in the multiple demos, the poorer performance of multi-demo indi- cates misguidance from negative demos or harmful interference between multiple demos. As shown in Fig. 7-8, ICL with one random demo, despite ignoring useful information in the rest demos, also removes the misguidance and interference and thus achieves similar or even better accuracy than all-
Adding More Demos to Prompt: Does improve or confuse ICL?
In the previous section, mainly focus on the per- formance of one demo ICL. and the reasons behind it. counterintuitive observation is that ICL with multiple demos performs even worse than ICL with

---

only one positive demo. Why does multi-demo ICL perform worse and when can it bring additional im- provement? To address these questions, we study the following two problems.
and AilAee
Dre sedendens C.
C1
0
Problem I: Starting from a prompt of one positive demo, will the accuracy be further boasted if adding more positive demos?
0
Problem II: Starting from a prompt of one negative demo, what will happen if adding more negative demos?
Peckin Domoi
Figure 8: Accuracy of ICL. using seven demos and one random demo (with CoT) fine-grained data groups different in the number of positive demos (Fig, (a)) for CSQA dataset. The size of each dot pro- portional to the data percentage. The observations and conclusions on CSQA are similar to those for Fig. 7.
These two scenarios mainly focus all-positive all-negative demo cases. This is because we al- ready observed in the previous section that a mix of negative and positive demos can misguide ICL since the evaluated LLMs are not good at distin- guishing negative and positive demos. On the other hand, it is still unclear whether the LLMs exploit the correlations between multiple demos and how they affect the ICL process. For example, when all the demos are positive (negative), will the LLMs treat all demos to be independent and thus keep the answer correct (wrong), or will the answer be changed due to the cross demo correlations?
have < 1 positive demo. Hence, we are able increase the number of positive (negative) demos from to in the two studied problems, producing a full spectrum of varying accuracy over different numbers of demos.
In Fig. 9-10 and Fig. 11-12 (appendix). we re- port the results for Problem I & II when apply- ing CoT prompting and few-shot prompting to CSQA-Fasy/Hard and GSM8K-Fasy/Hard, Sur- prisingly, on all datasets and ICL strategies, we consistently observe that increasing positive de- mos in the prompt results in lower accuracy on the easy samples, while increasing the negative demos improves the accuracy on the hard sam- ples. This indicates that LLMS in ICL, when given multiple demos, do take the correlations among de- mos into account rather than simply treating them independently. However, the correlations do not al always bring improvement to ICL for multiple pos- itive demos, for example, ICL with one positive demo achieves nearly 99% accuracy on GSM8K- Easy but adding an additional positive demo leads to significant degradation. In this case, the inter- ference and spurious correlations among multiple demos concatenated in the prompt are harmful to ICL and tend to misguide the LLMs toward finding the correct answer. On the other hand, ICL with multiple negative demos is able to extract the rele- vant information for the test query from multiple demos and combine them by LLMs achieve an improved answer. Though the improvement is not highly phenomenal, we consistently observe in all the plots, indicating a non- non-trivial composition of clues from multiple demos commonly happening
will
dema
Pheciiie Eeroos
Figure 7: Accuracy of ICL using all eight demos and one random demo (with CoT) on ane-igr data groups different in the number of positive demos (Fig. 5 (a)) for GSMSK datasct. The size of each dot is proportional to the data percentage. All-demo ICL brings improvement only to hard samples with fewer positive demos, while one random demo performs similarly or even better than all eight demos, indicating inefficient usage of multiple demos when easy samples dominate the dataset.
To ensure enough number of positive (negative) demos are added for cach sample, we use CSQA- Easy and GSMyK-Easy as evaluation sets for Prob- lem and CSQA-Hard and GSM8K-Hard as eval- uation sets for Problem II. As illustrated in Fig. and Fig. 6, the two easy sets of samples have 2 6 positive demos while the two hard sets of samples

---

Adding Positve Demo (RS04-Easy)
age beneficial cross-demo attention and restrain harmful interference.
Adding Negative Demo IGSM Hard)
0.00
0.97
Discussion
0.96
200
0.08
0.05
In-context learning (ICL) plays an important role in the ecosystem of LLMs. Recent LLMs are ca- pable of directly generating customized outputs by following the demos appended to input. However, it is not clear how many demos suffice to produce high-quality answers. In this paper, for the first time, we study the performance of ICL (with or without CoT prompting) under different numbers of demos and provide an in-depth investigation of the observations across several widely used bench- mark datasets.
00.05
0.94
0.04
Damo
Increasing demos in CoT Prompting GSM8K: for each query in (SMMFasy(GSM-Harr)) we start from a positive(negative) demo, add more posi- tive(negative) demos to the prompt, but observe an curacy degradatioon(mprovveennn))
Figure 9:
Adidnng Positive Demo (CSOA Easyl Aiding Negative Dumo (CSQ)-Har))
In particular, we found that randomly selecting one single demo barely hurts the performance while increasing the demos merely brings marginal im- provement. We then study how many demos can lead to correct answers in the one-demo ICL for each sample and analyze its statistics over all sam- ples in widely used benchmark datasets. The statis- tics reveal widely existing dataset bias that easy samples with many positive demos dominate the datasets, which explains the high accuracy of ICL with one random demo. also exposes a weak- ness of LLMs in distinguishing negative/positive demos in ICL. Moreover, we found that only one positive demo is sufficient to significantly outper- form multi-demo ICL., while saving great amount of cost. Furthermore, we study the contribution and interference of cross-demo correlations to ICL by investigating how the accuracy changes as we add more positive (negative) demos to the prompt. Surprisingly, adding positive demos reduces the accuracy while adding negative demos brings im- provement, indicating a problematic interpretation and exploitation of the cross-demo correlations by LLMs in ICL.
1.00
0.99
0.90
0.08
30300
0.97
0.00
0.99
0.02
0.05
Increasing demos in CoT Prompting CSQA: for each query in CSQA-FasyCCSQAAHartu we start from a positive(negative) demo, add more posi- tive(negative) demos to the prompt, but observe an ac- curacy deggradation(improvemnnn)
Figure 10:
during ICL and resulting in better answers even for hard samples.
Hence, increasing the number of positive (nega- tive) demos does not intuitively improve (weaken) the ICL performance and the main reason lies in the extraction and exploitation of cross demo cor- relations in ICL. Since multiple demos in ICL are concatenated together and then appended to the query as the whole input, a pretrained LLM might lack the capability to completely separate all demos and choose the positive one to follow during the ICL inference process, especially when the LLM's pretraining does not cover such tasks. Therefore, study cxposes a weakness of the current LLMs in modeling cross-demo correlations, which can be one of the main reasons for the marginal improve- ment brought by multi-demo ICI. To mitigate this problem, one may modify the pretraining recipe with additional training tasks/objectives to encour-
Our analyses highlight several fundamental chal- lenges that need to be addressed in the future, e.g.. how to design less biased benchmarks and more diverse demos that can be used to better evaluate LLMs' capability of distinguishing positive demos from the negative ones; how to improve the effi- ciency and effectiveness of multi-demo usage in ICL; how avoid the harmful interference caused by cross-demo correlations and meanwhile lever- age them to improve the ICL performance on hard samples with fewer positive demos; how to select positive demos for a given query, etc.

---

References
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. CoRR, abs/21 10.14168.
Simran Arora, Avanika Narayan, Mayee F. Chen, Lau- rel J. Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, and Christopher Ré. 2022. Ask me anything: A simple strategy for prompting language models. CoRR, abs/2210.00 10.0241
Mohammad Javad Hosseini, Hannanch Hajishirzi, Oren Etziomi, and Nate Kushman. 2014, Learning to solve arithmetic word problems with verb categorization. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, Special Interest Group of the ACL. pages 523-533, ACL.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. 7iegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Fiii Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
Jiaxin Huang. Shixiang Shane Gu. Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-immpove. CoRR. 1../2210011111
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blun- som. 2017. Program induction by rationale genera- tion: Learning to solve and explain algebraic word problems. pages 158-167.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sas- uy, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cum- mings, Matthias Plappert, Fotios Chantzis, Eliza- beth Barnes, Ariel Herbert-Voss William Hehgen Guss, Alex Nichol. Alex Paino, Nikolas Tezak, Jie Tang. Igor Babuschkin. Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr. Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluat- ing large language models trained on code. CoRR, abs/2107.0333 107744344.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for gpt-3? pages 100-
Aman Madaan and Amir Yazdanbakhsh. 2022. Text and patterns: For effective chain of thought, it takes two tango. CoRR, abs/2209.0766
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? pages 11048- 11064.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP models really able solve simple math word problems? pages 2080-2094.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes. Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang. Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstem, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- eling with pathways. CoRR, 0bs/222440021111
Jack W. Rac, Sebastian Borgeaud. Trevor Cai, Katie Millican, Jordan Hoffmann, H. Francis Song. John Aslanides, Sarah Henderson, Roman Ring, Susan- nah Young, Eliza Rutherford, Tom Hennigan, Ja- cob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Mari- beth Rauh, Po-Sen Huang, Amelia Glaese, Jo- hannes Welbl, Sumanth Dathathri, Saffron Huang. Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Sid- dhant M. Jayakumar, Elena Buchatskaya, David Bud- den, Esme Sutherland, Karen Simonyan, Michela Pa- ganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou. Arthur Mensch, Jcan-Baptist Lespiau, Maria Tsim- poukelli. Nikolai Grigorev, Doug Fritz, Thibault Sot- tiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong. Daniel Toyama, Cyprien de Masson d' Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy. Chris Jones, James Bradbury, Matthew J. Johnson,

---

Blake Hechtman, Laura Weidinger, lason Gabriel, William S. Isaac, Edward Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hass- abis. Koray Kavukcuoglu, and Geoffrey Irving. 2021. Scaling language models: Methods, analysis & in- sights from training gopher. CoRR, abs/21 12. 1446.
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. CoRR, abs/22 10.03493.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improv- ing few-shot performance of language models. In Proceedings the 38th International Conference Machine Learning, 1CML 2021, 18-24 July 2021, Vir- tual Event, volume 139 of Proceedings of Machine Learning Research, pages 2669-12776. PMLR.
Subhro Roy and Dan Roth. 2016. Solving general arith- metic word problems. CoRR, abs/1608.0141.
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. Learning retrieve prompis for in-context learning. pages 2655-2671.
Abulbair Saparov and He He. 2022. Language models are greedy reasoners: systematic formal analysis of chain-of-thought. CoRR, abs/2210.01240.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019, Commonsenseqa: A question answering challenge targeting commonsense knowl- edge. pages 4159-4158.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin. James Qin. Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Mare Pickett, Kathleen S. Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny So- raker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Ale- jandra Molina, Erin HoffmannJohnn Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Co- hen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera- Arcas, Claire Cui, Marian Croak, Ed H. Chi, and Quoc Le. 2022. Lamda: Language models for dialog applications. CoRR, abs/2201.055 .08239.
Boshi Wang. Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2022a. Towards understanding thain-of-thowe prompt- ing: An empirical study of what matters. CoRR, 11s22212.1001.
Xinyi Wang, Wanrong Zhu, and William Yang Wang. 2023. Large language models are implicitly topic models: Explaining and finding good demonstrations for in-context learning. CoRR, abs/2301. 1916.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H. Chi, and Denny Zhou. 2022b. Self- consistency improves chain of thought reasoning in language models. CoRR, abs/2203. 1171.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoe Le, and Denny Zhou. 2022. Chain thought prompting elicits reasoning large language models. CoRR, abs/2201. 1903..

---

A Easy/Hard Sample Ration
Easy Sample
Hard Sample
GSM8K
40.0%
28.3%
MultiArith
94.3%
0.0%
AddSub
82.5%
6.1%
SVAMP
62.6%
14.3%
AQuA 28.3%
55.1% 1%
SingleEq 89.0%
5.3%
CSQA
58.0%
13.0%
Table 2: The percentage of Easy/Hard samples (ICL with CoT) in each benchmark dataset. Easy samples dominate in most datasets while hard samples only take up small fraction.
B More results for Few-shot Prompting
Adding Posiive Demo Easyy Adding Negetive Demo (CSM-ttard)
1.00
-0.15
0.32
5.005
0.88
# Dano
Figure 11: Increasing demos in Few-shot Prompting on GSM8K: for each query in GSM S.MM-EayyGGSMHale start from a (nseaive(negative) demo, add more posi- tive(negative) demos to the prompt, but observe an accu- racy (dgradationn(mppovemmmt))
Adding Positive Demo (CSOA Easy!
Adding Negative Demo (ESQA-Htand)
0.12
8.9955
0,990
2.00
0.907%
0,0550
Duma
Figure 12: Increasing demos in Few-shot Prompting on CSQA: for each query in CSQA ASQQA-asyCCSQAAEEE -ard), we start from a positive(negative) demo, add more posi- tive(negative) demos the prompt, but observe an accu- racy degradation(mmporveeene))


Reviews:
[{"review_id": "V3eBZEUXV0", "reviewer": "Reviewer_XyZd", "summary": "", "questions": "Question A: As also stated in the paper, Min et al (2022) and Madaan & Yazdanbakhsh (2022) observe that label correctness is not as important. How does that finding align with \"The accuracy of multi-demo ICL degrades (improves) when given more positive (negative) demos\"? Does label correctness matter in this particular observation?\n\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your detailed comments, we address each one as follow: \n\n **Question**: More discussions on the related work\n\n **Answer**: Thank you for the suggestion! We will add more detailed discussion in the related work. \n \n **Question**: Min et al (2022) and Madaan & Yazdanbakhsh (2022) observe that label correctness is not as important. How does that finding align with \"The accuracy of multi-demo ICL degrades (improves) when given more positive (negative) demos\"? Does label correctness matter in this particular observation?\n\n\n**Answer**: Our observation is orthogonal to the conclusion of Min et al (2023) and Madaan & UYAU (2022). Label correctness means that the answer within the demo does not necessarily have to be accurate. A positive (or negative) demo is one that guides towards a correct (or incorrect) final answer. Label accuracy does not inluence the observation in our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Clearly written and easy to read\n* Interesting findings reported in the paper (as listed above)\n", "reasons_to_reject": "* The paper is focused on demos for ICL. More discussions on the related work (esp. in the subsection of The role of demos in ICL) and how they are related to the findings of this paper should be included in the paper.", "questions_to_authors": "Question A: As also stated in the paper, Min et al (2022) and Madaan & Yazdanbakhsh (2022) observe that label correctness is not as important. How does that finding align with \"The accuracy of multi-demo ICL degrades (improves) when given more positive (negative) demos\"? Does label correctness matter in this particular observation?\n\n", "paper_topic_and_main_contributions": "This paper investigates the effect of multiple demos in ICL in LLMs. Several findings are reported in the paper.\n- A single, randomly selected demo barely affect the performance.\n- Increasing the demos merely results in marginal improvement.\n- A single positive demo can outperform multi-demo ICL.\n- The accuracy of multi-demo ICL degrades (improves) when given more positive (negative) demos. \n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UH34P2ME0x", "reviewer": "Reviewer_byCC", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your detailed comments, we address each one as follow:\n\nAs suggested by the reviewer, we run the main experiments on GSM8K using the newer GPT-3.5-turbo. The new results and conclusions are consistent with the ones reported in our paper:\n\n1. One **random demo** suffices to achieve the most phenomenal improvement by ICL (in-context learning) but using more than one demo (**8 demos**) only brings marginal improvement. It indicates an inefficient usage of demos in ICL.\n\n    |ICL on GPT-3.5-turbo       \t| Using all 8 demos \t| One random demo  \t| \n    |:-------:|:-------------------:|:------------------:|\n    | GSM8K \t| 73.60($\\pm$ 0.80)         \t| 72.59 ($\\pm$ 0.70)| \n\n2. Existing dataset bias: as reported below, easy samples still dominate benchmark datasets, leading to a long tail distribution of sample difficulty. For most queries (around 67.6% of the whole dataset), randomly selecting one demo suffices to produce the correct answers, while hard samples (no demo leads to the correct answer under single-demo ICL) take up a very small fraction.\n\n    |Number of positive demos| Percentage(%)\t| \n    |:-------:|:-------------------:|\n    | <=1 (**hard samples**)\t| 16.7         \t| \n    | 2\t| 4.0      \t| \n    | 3\t| 3.2       \t| \n    | 4\t| 4.2        \t| \n    | 5\t| 4.2         \t| \n    | >=6 (**easy samples**)\t| 67.6      | \n    \n    \n3. For **easy samples** (around 67.6% of the whole dataset), adding more positive demos to the prompt results in lower accuracy, indicating that the interference and spurious correlations among multiple demos can be harmful to ICL and tend to misguide the LLMs toward finding the correct answer.\n\n    |Num. of positive demo   | 1\t                 | 3 \t             | 8                |\n    |:-------:|:--------------------:|:-------------------:|:------------------:|\n    | Acc. \t| 96.63 ($\\pm$ 0.00) | 95.46 ($\\pm$ 0.20)|94.95 ($\\pm$ 0.01)| \n\n\n    On the other hand, increasing the number of negative demos improves the accuracy on the hard samples (only 16.7% of the whole dataset). Hence, multiple negative demos (each alone leading to an incorrect answer), when combined by LLMs in ICL, can provide useful information regarding the test query and improve the answer to it.\n\n\n    |Num. of positive demo   | 1\t                 | 3 \t             | 8                |\n    |:-------:|:--------------------:|:-------------------:|:------------------:|\n    | Acc. \t| 4.98 ($\\pm$ 0.01) | 10.63 ($\\pm$ 0.20)|14.02  ($\\pm$ 0.01)| \n\n\n **Question**: Evaluation on a single LLM which may not be representative \n\n **Answer**: We conduct all experiments with GPT-3.5-turbo and the findings align consistently with our submitted paper. Moreover, the use of GPT-3.5-turbo strengthens our conclusions because: 1. Multiple demos appear to be excessively redundant, with the difference between one demo and all 8 demos being smaller. 2. There is a higher proportion of easy samples, while hard samples are less common. This implies a more significant bias in the benchmarks that demands more investigation. \n \n **Question**: No statistical significance test was done.\n\n **Answer**: We conduct statistical significance tests on GPT-3.5-turbo, repeated all experiments 5 times, and reported the variance in the above tables. The results, taking into account the variance, are in line with our discoveries. Moreover, most datasets have around 1000 samples (for instance, GSM8K has 1318, and CSQA has 1221), which further strengthens the reliability of our conclusions.\n \n \n **Question**:  It is unclear how well this generalizes to datasets with a larger set of annotated demonstrations.\n\n **Answer**: The reviewer raised a crucial issue that requires future attention. Our paper uncovers a widely existing dataset bias that easy samples with many positive demos dominate the datasets. The challenge will be considered in the future work to create less biased benchmarks with more diverse demos to evaluate LLMs’ capability more effectively.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well written and easy to follow.\n2. The paper deals with a widely-used setup and hence, its observations can be beneficial to the field of in-context learning.", "reasons_to_reject": "1. Evaluation on a single LLM which may not be representative - The analyses in this paper were conducted using one model solely (code-davinci-002), which means that the empirical results obtained may not be applicable to other models. As a result, the conclusions drawn from this study are specific to the mentioned model and cannot be generalized to encompass other LLMs.\n2. No statistical significance test was done - The authors did not conduct statistical significance tests on their results, which is a crucial aspect, particularly considering the relatively small datasets they utilized. Such tests are essential for validating the significance and reliability of the observed findings, especially when working with limited data.\n3. Bias in datasets - The authors asserted that the existing benchmarks suffer from bias, primarily because a significant portion of the test samples can be optimally solved by treating each annotated demonstration as a one-shot prompt. However, they only briefly mentioned that this situation might arise from the fact that these handcrafted demonstrations were specifically designed to tackle the majority of the dataset's samples. It is unclear how well this generalizes to datasets with a larger set of annotated demonstrations.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper provides a comprehensive analysis of in-context learning (ICL)setup by comparing the performance of large language models (LLMs) under different demonstration quantities. Their main objective is to determine whether increasing the number of examples in the prompt improves the LLM's output for different types of samples (classified as hard or easy based on their framework). Furthermore, the authors evaluate a few well-known benchmarks, demonstrating that even with just one random demonstration included in the prompt, a significant number of test samples can still be optimally solved. They also discuss the trade-off associated with using more demonstrations, which leads to longer prompts and by that, more costly execution at the gain of marginal improvements in the results.\n\nMoreover, the study highlights a noteworthy finding - while each individual demonstration may produce correct answers independently, including more demonstrations in the prompt can adversely affect the LLM's output. Finally, the authors discuss the susceptibility of ICL to be influenced by interference among demonstrations and spurious correlations, supporting their claims with empirical evidence.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "* L24-26 - the sentence is unclear\n* L201 - is → was\n* Figure 7-8 - consider to enlarge the font"}, "extra_text": {}}, {"review_id": "p471JuYyaw", "reviewer": "Reviewer_k4LP", "summary": "", "questions": "Please explain the point in the second reason to reject", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your detailed comments, we address each one as follow:\n\nAs suggested by the reviewer, we run the main experiments on GSM8K using the newer GPT-3.5-turbo. The new results and conclusions are consistent with the ones reported in our paper:\n\n1. One **random demo** suffices to achieve the most phenomenal improvement by ICL (in-context learning) but using more than one demo (**8 demos**) only brings marginal improvement. It indicates an inefficient usage of demos in ICL.\n\n    |ICL on GPT-3.5-turbo       \t| Using all 8 demos \t| One random demo  \t| \n    |:-------:|:-------------------:|:------------------:|\n    | GSM8K \t| 73.60($\\pm$ 0.80)         \t| 72.59 ($\\pm$ 0.70)| \n\n2. Existing dataset bias: as reported below, easy samples still dominate benchmark datasets, leading to a long tail distribution of sample difficulty. For most queries (around 67.6% of the whole dataset), randomly selecting one demo suffices to produce the correct answers, while hard samples (no demo leads to the correct answer under single-demo ICL) take up a very small fraction.\n\n    |Number of positive demos| Percentage(%)\t| \n    |:-------:|:-------------------:|\n    | <=1 (**hard samples**)\t| 16.7         \t| \n    | 2\t| 4.0      \t| \n    | 3\t| 3.2       \t| \n    | 4\t| 4.2        \t| \n    | 5\t| 4.2         \t| \n    | >=6 (**easy samples**)\t| 67.6      | \n    \n    \n3. For **easy samples** (around 67.6% of the whole dataset), adding more positive demos to the prompt results in lower accuracy, indicating that the interference and spurious correlations among multiple demos can be harmful to ICL and tend to misguide the LLMs toward finding the correct answer.\n\n    |Num. of positive demo   | 1\t                 | 3 \t             | 8                |\n    |:-------:|:--------------------:|:-------------------:|:------------------:|\n    | Acc. \t| 96.63 ($\\pm$ 0.00) | 95.46 ($\\pm$ 0.20)|94.95 ($\\pm$ 0.01)| \n\n\n    On the other hand, increasing the number of negative demos improves the accuracy on the hard samples (only 16.7% of the whole dataset). Hence, multiple negative demos (each alone leading to an incorrect answer), when combined by LLMs in ICL, can provide useful information regarding the test query and improve the answer to it.\n\n\n    |Num. of positive demo   | 1\t                 | 3 \t             | 8                |\n    |:-------:|:--------------------:|:-------------------:|:------------------:|\n    | Acc. \t| 4.98 ($\\pm$ 0.01) | 10.63 ($\\pm$ 0.20)|14.02  ($\\pm$ 0.01)| \n\n\n **Question**: the generalizability of the findings to other language models or architectures.\n\n **Answer**: We conduct all experiments with GPT-3.5-turbo and the findings align consistently with our original paper. Moreover, the use of GPT-3.5-turbo strengthens our conclusions because: (1) Multiple demos appear to be excessively redundant, with the difference between ICL with one demo and ICL with all 8 demos being smaller; (2) There is a higher proportion of easy samples, while hard samples are less common. This implies a more significant bias in the benchmarks that demands more investigation.\n \n **Question**:  the way to find the positive demo is to see whether the output is correct, which is not available during inference.\n\n **Answer**: In-context learning (ICL) plays an important role in the modern LLMs. However, it is not clear how many demos suffice to produce high-quality answers. In this study, we primarily explore the mechanism behind the success of ICL, focusing on the variation in demo numbers and the intrinsic bias in existing benchmark datasets. We offer a comprehensive analysis of our findings across multiple popular benchmark datasets, which could steer future approaches in selecting positive demos.\n \n \n **Question**: Conclusion and Limitation section is missing. \n\n **Answer**: Thanks for your suggestion, we will add more detailed discussion in the conclusion and limitation section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper uncovers an inherent bias in widely studied datasets used for ICL tasks. It reveals that the majority of demos are positive (leading to the correct answer) for most test queries, which explains the good performance of ICL with only one random demo. \n2. The study identifies a weakness in LLMs, specifically their inability to consistently find positive demos for input queries. \n3. The analyses presented in the paper have practical implications for  ICL design.", "reasons_to_reject": "1. The paper's experimental evaluation is exclusively conducted on a single large language model, code-davinci-002. This limitation raises concerns about the generalizability of the findings to other language models or architectures.\n2. This paper's conclusion may be useless because it points out that positive demo is the key to effective ICL. However, the way to find the positive demo is to see whether the output is correct, which is not available during inference.\n3. Conclusion and Limitation section is missing.\n ", "questions_to_authors": "Please explain the point in the second reason to reject", "paper_topic_and_main_contributions": "The paper addresses the topic of in-context learning (ICL) in large language models (LLMs) using fewer input-output demonstrations (demos) for each test query. The study highlights an interesting phenomenon where ICL performance remains surprisingly strong even with just one randomly chosen demo. This finding challenges the conventional belief that multi-demo setups are necessary for ICL tasks and provides valuable insights into the effectiveness and redundancy of demos.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper provides an empirical study of the impact of multiple demonstrations for in-context learning with large language models. Reviewers all agree that the empirical findings are interesting and would be of benefit to the community to understand, especially since they contrast with conventional wisdom that more demonstrations are better. The primary concern from R2 and R3 are with the experimental setup: they perform their analysis with only one model, so the results might not generalize to other LLMs. R2 points out that existing bias in the used datasets may have impacted their results towards confirming their initial hypotheses. R3 notes that certain findings, namely that positive demonstrations are key for in context learning, might have limited utility since they require knowledge of whether a demonstration is positive a priori, which is not available at inference (though the main purpose of the paper, as the authors state, is to gain insight into what makes ICL work). The authors responded by providing further experiments on GPT-3.5-turbo which they claim to be in line with the findings in their paper, as well as some additional dataset analysis. The paper explores and gives insight into a timely research question, and would likely spur on good discussions at the conference.
Decision: Accept-Findings