In-Context Learning Learns Label Relationships but Is Not Conventional Learning
Authors: Jannik Kossen
Yarin Gal
Tom Rainforth
Venue: ICLR 2024 poster (2024)
Keywords: in-context learning
large language models
Primary area: generative models

Abstract: The predictions of Large Language Models (LLMs) on downstream tasks often improve significantly when including examples of the input–label relationship in the context. However, there is currently no consensus about how this in-context learning (ICL) ability of LLMs works. For example, while Xie et al. (2022) liken ICL to a general-purpose learning algorithm, Min et al. (2022b) argue ICL does not even learn label relationships from in-context examples. In this paper, we provide novel insights into how ICL leverages label information, revealing both capabilities and limitations. To ensure we obtain a comprehensive picture of ICL behavior, we study probabilistic aspects of ICL predictions and thoroughly examine the dynamics of ICL as more examples are provided. Our experiments show that ICL predictions almost always depend on in-context labels and that ICL can learn truly novel tasks in-context. However, we also find that ICL struggles to fully overcome prediction preferences acquired from pre-training data and, further, that ICL does not consider all in-context information equally.

Published as a conference paper at ICLR 2024
IN IN-CONTEXT LEARNING LEARNS LABEL RELATION- SHIPS BUT Is NOT CONVENTIONAL LEARNING
Jannik Kossen
Yarin Gal'
Tom Rainforth 2A
1 OATML, Department of Computer Science, University of Oxford Department of Statisties, University of Oxford
ABSTRACT
The predictions of Large Language Models (LLMs) on downstream tasks often improve significantly when including examples of the input-label relationship in the context. However, there is currently no consensus about how this in-context learning (ICL) ability of LLMs works. For example, while Xie et al. (2022) liken ICI. general-pus purpose learning algorithm, Min et al. (2022b) argue ICI. does not even learn label relationships from in-context examples. In this paper, we provide novel insights into how ICL leverages label information, revealing both capabilities and limitations. To ensure we obtain comprehensive picture of ICL behavior, we study probabilistic aspects of ICL predictions and thoroughly exam- ine the dynamics of ICL as more examples are provided. Our experiments show that ICL predictions almost always depend on in-context labels and that ICL can learn truly novel tasks in-context. However, we also find that ICL struggles to fully overcome prediction preferences acquired from pre-training data and, further, that ICL does not consider all in-context information equally.
INTRODUCTION
Brown et al. (2020) have shown that Large Language Models (LLMs) (Radford et al., 2019; Chowd- hery et al., 2022; Hoffmann et al., 2022; Zhang et al., 2022a) can perform so-called in-context learning (ICL) of supervised tasks. contrast to standard in-weights learning, e.g. gradient-based finetuning of model parameters, ICL requires no parameter updates. Instead, examples of the input- label relationship
the downstream task are simply prepended to the query for which the LLM predicts. This is sometimes also referred to few-shot ICL to differentiate from other ICL variants that do not use example demonstrations (Liu et al., 2023). Few-shot ICL is widely used, e.g. in all LLM publications cited above, improve predictions across variety of established NLP tasks, such as sentiment or document classification, question answering, or natural language inference.
However, there is currently senting large variety of often contradictory perspectives. For example, Brown et al. (2020) high- light similarities between the behavior of ICL and finetuning of LLMs, such as improvements with model size and number of examples. Since then, some have argued that ICL works because it im- plements purpose learning algorithms such as Bayesian inference or gradient descent (Xie et al., 2022: Huszár, 2023; Hahn & Goyal, 2023; Jiang, 2023; Zhang et al., 2023; Von Oswald et al., 2023; Akyürek et al., 2023; Han et al., 2023). In contrast, others have highlighted practical short- comings of ICI, suggesting ICI. does not really 'learn' from examples in the way one expects (Liu et al., 2022: Lu et al., 2022; Zhao et al., 2021; Chen et al., 2022; Agrawal et al., 2023; Chang & Jia, 2022; Razeghi et al., 2022; Li & Qiu, 2023; Wei et al., 2023). In particular, Min et al. (2022b) claim their 'findings suggest that [LLMs]] do not learn new tasks at test time' in the sense of 'capturing the input-label correspondence' Clearly, these claims, if true, are not compatible with the behavior would expect from a general-purpose learning algorithm.
consensus why ICL improves predictions, with prior work pre-
In this paper, we address the pressing need for improved understanding of how information given in-context affects ICL predictions. Concretely, we formulate a set of questions that encode our beliefs about how an idealized conventional learning algorithm should incorporate label information
Correspondence kunnik.kossenée .c..ux..c.uk Equal advising.

---

Published as a conference paper at ICLR 2024
SS1-2
Subj
F. Phraseb.
Hate Speech
AG News
Log Likelibood f
Defauk
Randomized
Guessing
-1-
100
50
0
50
50
25
No Observed In-Con Ii-Context Examples
Figure I: ICL predictions generally depend on the conditional label distribution of in-context exam- ples: when in-context labels are randomized, average log likelihoods of label predictions decrease compared to ICL with default labels for LLaMa-2-708B across a variety tasks. Results averaged over 500 in-context datasets and thin lines 99 % confidence intervals. See $5 for details.
and then study ICL behavior in relation to this concept of a conventional learner. (1) Does ICL take the input-label relationship the in-context examples into account when predicting for test queries? (2) Is ICL powerful enough to overcome prediction preferences originating from pre-training? (3) Does ICL, treat all information provided in-context equally? To study these questions rigorously, rephrase them null hypotheses that we study empirically. Our results yield improved understanding of the similarities and differences between ICL and idcalized conventional learners.
Unlike prior work, we study in detail how ICL predictions evolve an increasing number of exam- ples are provided, from no examples at all up to the maximum possible, across a range of LLMs and tasks. We further show that using probabilistic metrics better highlights the resulting ICL dynam- ics, often revealing large changes in the confidence of ICL predictions, even when accuracy metrics barely change all. These measures ensure we obtain a comprehensive picture of ICL behavior.
In our experiments, we first examine if ICL predictions depend on the labels of in-context examples by studying how probabilistic metries react randomized in-context labels ($5, Fig. 1). Further, we study ICL on truly novel task the LLM cannol know from pre-training ($6). Both experiments show that ICL typically considers in in-context label relations. We then investigate if ICL is powerful enough to overcome prediction preferences learned from pre-training data ($7). In our experiments, we find this typically not the case as ICL performance plateaus if label relations oppose pre- training preference. Further, while additional prompting can improve ICL. here, we ultimately do not find prompts that lead to the desired behavior. Finally, we study if ICL. treats all information provided in-context equally ($8). This is important when the context contains multiple different label relationships. By modifying label relations during ICL, we find it does not treat all in-context information equally, and, instead, ICL preferentially makes use of information closer to the query.
In summary, our results suggest a new middle ground regarding the capabilities and limitations of ICL. While ICL can learn from label information, it does differently than idealized learner. Our findings thus contribute to a better understanding of information processing in ICL, which, in turn, is crucial to ability deploy LLMs safely and effectively. For example, Bai et al. (2022b) suggest to use ICL for alignment, which relies on ICL being able to sufficiently adjust LLM behavior.
2 BACKGROUND
A large and growing body of prior work studies ICL. We here highlight those studies that are most relevant to our motivation, and discuss other related work later in $10.
Since its introduction by Brown et al. (2020), few-shot ICL. has become an integral part of LLM eval- uations: e.g. many recent publications rely on few-shot ICI. tasks, such as the popular HELM bench- mark (Liang et al., 2022), evaluate their LLMs (Chowdhery al., 2022; Hoffmmann et al., 2022).
In the wake of ICL's success, follow up work has speculated ICL implements a general purpose learning algorithm such as gradient descent (Von Oswald et al., 2023) or Bayesian inference (Xie et al., 2022). This line of work implies that ICL captures not just how much LLMs have learned during pre- traiirinn but, rather, how much LLMs have learned how learn novel supervised tasks in-context. However, so far arguments have been largely theoretical, lacking solid experimental evidence in actual I. L.MSs (Zhang al., 2023; Huszár, 2023; Wies et al., 2023; Jiang, 2023).
2

---

Published as a conference paper at ICLR 2024
Conversely, variety of studies have highlighted unexpected shortcomings of ICL, For example, ICL can be sensitive to the formatting (Min et al., 2022b) or order (Lu et al., 2022) of the in-context learning examples. Further, LLMs prefer to predict labels that are common in the pre-training data (Zhao et al., 2021). they can predict drastically different for similar prompts (Chen et al., 2022), and they rely on task formulations similar to those observed in the pre-training data (Wu al., 2023).
In particular, Min et al. (2022b) claim that ICL does not learn label relationships from in-context examples and that ICL 'performance drops only marginally when labels in the demonstrations are replaced by random labels'. Further, they suggest that, instead of learning input-label relationships, ICL only works because the model learns about the general label space, the formatting of the exam- ples, and their input distribution. They assert that ICL does 'not learn new tasks at test time' and that "ground truth demonstrations are in fact not required' in many common scenarios. In the first part of our evaluation, we will revisit and ultimately disagree with these claims.
3 NULL HYPOTHESES ON HOw ICL INCORPORATES LABEL INFORMATION
We wish to obtain better understanding of how ICL uses information about the input-label relation- ship provided in-context. We therefore study to what extent ICL behavior matches our expectations of how a machine learning algorithm should behave in an idealized world. To this end, we introduce the concept of a 'conventional learning algorithm' as an algorithm that conforms to our intuitions of how an idealized learner will make predictions given data. In particular, we focus on the following intuitions of an idealized learner: (1) makes use of the labels for learning. (2) it allows the true la- bel relationship to be learned when provided with sufficient data, and (3) it considers all information in the data equally. Note here that some existing approaches may not always conform to some or all of these intuitions, e.g. due to imperfections in training schemes, but concept of the conventional learning algorithm reflects how we would expect them to behave our training worked perfectly.
To allow these institutions to be tested for ICL, now introduce series of null hypotheses that we will subsequently try to falsify. Our first hypothesis follows from the notion that conventional learners make use of the conditional distribution of labels given inputs.
Null Hypothesis 1 (NHI): ICL. predictios are independent of the conditional label distribution of the examples given in-context.
We will investigate NHI in multiple ways. In $5, we revisit and refine the randomized in-context label experiment of Min al. (2022b): in addition revising their results for point predictions. we propose the use of probabilistic metrics to study if label randomization really does not affect ICL predictive beliefs. Then, in $6, we study ICL on a novel task that we create and which ICL can only solve by learning a novel label relationship that it cannot know from pre-training.
Next, we study how ICL incorporates label information. The pre-trained model already contains in- formation towards many NLP tasks: even without any ICL., predictions are significantly better than random guessing. This raises the question of how information given in-context interacts with this pre-training preference. In typical applications of conventional learners, we would expect that pre- dictions eventually follow the label relation of the training examples when provided with sufficient data. A learner that does not conform to this is inherently limited in what it can learn. With NH2, we study if ICL conforms to these intuitions and can overcome the initial pre- pre-training preference.
Null Hypothesis 2 (NH2): ICL can overcome the zero-shot prediction preferences of the pre-trained model.
In $7, modify in-context label relationships and study how this changes ICL predictions. If NH2 is true, ICL should eventually predict according to any label relation given in-context.
Our last hypothesis relies on the following typical property of conventional learning algorithms: they will consider all information in the examples equally. If a dataset contains multiple sources of infor- mation about label relationship, e.g. the dataset itself is union of multiple datasets, then all infor- mation is considered equally by the learner. NH3 investigates if this is holds true for ICL as well.
Null Hypothesis 3 (NH3): ICL considers all information given in-context equally.
In $8, we study non-stationary input distributions for which label relations change during ICL. If NH3 is true, ICL predictions should not depend on the order in which we present label relations.
3

---

Published as a conference paper at ICLR 2024
Accuracy t
Log Likelihood f
Entropy
0.2-
0.6
Falcon
Falcon 7B
0.8-
Falcon 40B
-0.4-
Falcon 40B Instr.
0.4-
Guessing
0.6
0.6
0.2
0
50
50
50
No Observed In-Context Examples
Figure 2: Few-shot ICL training dynamics default label scenario on SST-2, Accuracy (t) and log likelihood (1) improve with in-context dataset size, and entropies decrease appropriately. Averages over 500 random subsets, thick lines with moving average (window size 5) for clarity.
EXPERIMENTAL SETUP& & ICL TRAINING DYNAMICS
We here detail our experimental setup for the subsequent evaluation of few-shot ICL behavior.
Models & Tasks. We employ LLMs from the LLaMa-2 (Touvron et al., 2023b), LLaMa (Touvron et al., 2023a), and Falcon (TII, 2023) families due to their strong performance and open source nature. We evaluate SST-2, Subjective (Subj.), Financial Phrasebank (FP). Hate Speech (HS), AG News (AGN), MQP, MRPC, RTE, and WNLL We provide citations for all tasks in $D.
Context Size. We always report few-shot ICL performance across all possible numbers of in-context demonstrations, i.e. from zero-shot performance up to the maximum number of examples within the LLMs' input token limit. This is in contrast to prior work, which often evaluates few-shot ICL at only a few context set sizes, and allows us to obtain comprehensive picture of ICL 'training dynamics'.
Computationally Cheap ICL Evaluations. We propose novel evaluation strategy that obtains ICL predictions at all possible numbers of in-context demonstrations without incurring any addi- tional cost. Concretely, we exploit the fact that each forward pass through the model gives not just a prediction for the next token, but rather, the predicted probabilities for each input token (given all preceding tokens). By extracting those token predictions that correspond to labels of in-context examples, we obtain few-shot ICL predictions at all in-context dataset sizes with each forward pass. We refer to $B for a formalization of few-shot ICL and further description of our evaluation strategy.
Evaluation Metrics. We evaluate few-shot ICL performance in terms of accuracy (f) and (average) log likelihood (f) of label predictions. We also report entropy, which, while not a performance met- ric, is useful for understanding how much predicted probabilities are spread over classes. We average metrics over sets of in-context examples drawn randomly and without replacement from the training set, and we compare a guessing baseline that predicts with probabilities equal to class frequencies.
Default Training Dynamics. Before modifying label relationships in the following sections, Fig. shows standard few-shot ICL training dynamics for Falcon models on SST-2. We observe reasonable behavior for all models: as more in-context examples are observed, accuracies and log likelihoods increase, while entropies decrease. Notably, log likelihoods and entropies show in-context learning more clearly: predicted probabilities continue to improve at larger context sizes, whereas accuracies saturate quickly. Differences between models are more noticeable for probabilistic metrics, too: en- tropies reveal that larger or instruction-turre Falcon models predict with higher certainty on SST-2. Similar findings also hold for LLaMa and LLaMA-2 models, for which we provide results in Fig. F.1.
5 Do ICL PREDICTIONS DEPEND ON IN-CONTEXT LABELS?
We now study the null hypotheses (NH) formulated in $3, starting with NH1, which states that ICL predictions are independent of the conditional label distribution of the in-context examples. To this end, first revisit the experiments of Min et al. (2022b), replacing all labels of in-context examples with labels drawn randomly from the training set of the task. If NHI is true, then accuracy, log-likelihood, and entropy should be identical for the randomized and standard label scenario. We note that, while we believe the results of this experiment are already sufficient to reject NH1, the experiments in $6-$8 will provide additional strong evidence for this conclusion.
Observations & Discussion. We evaluate NHI across all our models, tasks, and metrics, computing full ICL training curves introduced in $4. Figure shows log likelihoods for LLaMa-2-70BB for
4

---

Published as a conference paper at ICLR 2024
Accuracy t
Log Likelihood f
Entropy
Falcon 7B
0.2
0.6
Falcon
0.8
Falcon 40B
00.4
0.4-
Guessing
Falcon 40B Instr.
Sal Al
0.6
-0.6-
0.2
0
50
50
No Observed In-Context Examples
Figure 3: Few-shot ICL with randomized labels for SST-2: Compared to default ICL. behavior (dashed lines), log likelihoods and entropies of the Falcon models degrade when in-context labels are randomized. Accuracies show differences less clearly than probabilistic log likelihood and entropy. Averages over 500 repetitions, thick lines with moving average (window size 5) for clarity.
Table I: Average differences between ICL log likelihoods for default and randomized labels. Bold entries indicate differences statistically significant. We can disregard lightgray entries: for them, default ICL performance not significantly better than a random guessing baseline. Whenever de- fault ICL outperforms the baseline, ICL almost always performs significantly worse (positive differ- ences) for random labels. Averages over 500 runs at max. context size, standard errors in Table F.1.
À Log Likelihood
SST-2
Subj
FP
HS
AGN
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
0.42
0.39
0.57
0.18
0.53
0.03
0.02
0.03
0.02
LLaMa-2 13B
0.41
0.62
0.49
0.24
0.81
0.04
0.01
0.06
0.02
LLaMa-2 70B
0.51
0.53
0.57
0.34
0.80
0.29
0.04
0.22
0.18
Falcon 7B
0.20
0.19
0.25
0.06
0.31
0.01
0.01
-0.01
0.01
Falcon 7B Instr.
0.13
0.08
0.11
0.03
0.15
0.03
0.02
-0.00
0.00
Falcon 40B
0.34
0.35
0.31
0.18
0.90
0.06
0.01
0.01
0.02
Falcon 40B Instr.
0.25
0.37
0.27
0.02
0.77
0.06
0.02
0.02
0.04
a selection of tasks, and Fig. 3 shows all metrics for all Falcon models on SST-2. We observe significant differences ICL behavior for the default and randomized label scenario. As the context size grows, likelihoods eventually degrade significantly for randomized labels. In Fig. 3, we can further see entropies increase when randomizing labels. This is reasonable from probabilistic learning perspective: as noisy labels are observed, estimates of uncertainty will typically increase. While differences are large for probabilistic log likelihood and entropy, they can be harder to spot for accuracy. In Fig. 2, only Falcon-40B experiences sizeable accuracy decrease. (For LLaMa(-2) models, accuracy decreases more frequently and can approach guessing level, cf. Figs. F.3 to F.8.)
We provide results for label randomization across all our models, tasks, and metries: we invite the reader to view the full set of training curves SF and provide a summary of the results in Table 1. It shows the average difference in log likelihoods between the default and randomized labels at the maximum number of demonstrations for each task and model. We gray out entries where ICI. default labels does outperform the guessing baseline studying label randomization when ICL works in the first place. When default label performance is better than random (black entries), differences are almost always significantly positive (bold entries), indicating ICL performs worse for randomized labels. Based on these results, we reject NH1 that ICL predictions do not depend on the conditional label distribution of in-context examples.
are only interested in
Notably, Table shows that LLaMa-2-700 2-BB, our largest and most capable model, always performs worse under label randomization. This suggests the importance of labels in ICL will increase as models become more powerful in the future. However, performance often degrades significantly even for smaller models, although they struggle to reach better than random performance on the en- tailment tasks MQP, MRPC, RTE, and WNLL Occasionally, likelihoods improve despite random la- bels, e.g. for the small Falcon models in Fig. 3. Following Pan et al. (2023); Min al. (2022b), we at- tribute this to ICL 'reognizzigg' rather than learning, the task from the random label demonstrations. However, we note that, even here, there are significant performance gaps to the default scenario. We conclude that label randomization adversely affecting ICL predictions is the rule not the exception.
Discussion of Min et al. (2022b). Lastly, we discuss possible reasons for why Min et al. (2022b) arrive at the conclusion that label randomization only 'barely hurts' ICL performance: (1) They do
5

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood f
Entropy
LLaMa-2 7B
0.8
0.5
LLaMa
LLaMa-2 70B
-1.0-
Guessing
0.6
0.4-
-1.5
0.2
0.4
0
50
100
100
50
100
No Observed In-Contest Examples
Figure 4: Few-shot ICL achieves accuracies significantly better than random guessing on our novel author identification task. Thus, LLMs can learn novel label relationships entirely in-context. Averages over 500 runs, thick lines with additional moving average (window size 5) for clarity.
not study probabilistic metrics, which dataset size of 16, but effects of random labels increase with growing contexts. (3) Only one model they study has more than 20B parameters (GPT-3), but we observe that larger models react more to randomization. (Pan et al. (2023) also observe this, cf. $ $10.) (4) On some tasks, performance for Min al. (2022b) could be close to random guessing, where label randomization has less of an effect.
sensitive randomization. (2) They use fixed ICL
CAN ICL LEARN TRULY NOVEL LABEL RELATIONSHIPS?
The results of $5 show that ICL predictions do depend on the label relationship of in-context exam- ples. Here, we explore the extent which ICL can extract label information from the context. Con- cretely. we study if LLMs can learn truly novel label relationships in-contex: To do this, we create task that is guaranteed to not appear in the pre-training data. The task needs to distinet from estab- lished NLP tasks, for which the pre-training data could be contaminated and for which, often, strong zero-shot performance shows the model has learned the task, perhaps implicitly, during pre-training.
Specifically, we create an authorship identification (Stamatatos, 2009) dataset from private messages between two authors of this paper. The task is to identify the author corresponding to given message. As messages stem from private communication, they are guaranteed to not be part of the pre-training corpus. For ICL to succeed here, it needs to learn the novel input-label relationship provided in-context: while the LLM could have some general notion of authorship identification tasks, the specific input-label relationship is definitely novel, as the authors' private writing styles cannot be known the LLM. We give further details on the task in $C.
Observations & Discussion. Figure 4 shows that ICL with LLaMa-2 succeeds at learning the author identification task. Accuracies and log likelihoods increase, agreeing with expectations about conventional learning. Performance improves with model size, but all models perform better than random. We show results for more LLMs in Fig. F.13: all models, except Falcon-77 Instruutt achieve better than random performance. We conclude that LLMs can learn truly novel tasks in- context, correctly inferring the label relation from examples. These results also strongly support our previous rejection of NHI as, clearly, ICL predictions must depend on labels to learn the novel task.
7 CAN ICL OVERCOME TRE-TRAINING PREFERENCE?
With NH2, we explore how in-context label information trades off against the LLM's pre-training preference, i.e. its zero-shot predictions based on label relationships inferred from pre- pre-training and stored in the model parameters. Often, pre-training preference and in-context label relationships agree: e.g. in Fig. 3, performance is high zero-shot and then improves with ICI. To test NH2 if ICL. can overcome pre-training preference, we create scenarios where pre-training preference and in-context observations are not aligned. We then study if ICL behavior is compatible with fully overcoming pre-training preference as we would expect from a conventional learner.
Concretely, we use replacement label relationships when constructing the in-context examples. (1) We flip the default labels, e.g. (negative, positive) get mapped to (positive, negative) for SST-2. (2) We study arbitrary labels, e.g. (negative, positive) become (A, B) or (B, A)-we deliberately choose arbitrary labels here such that the LLM should not have a significant preference for assign- ing them to positive negative. We then evaluate ICI. performance for predicting the replacement

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Falcon 40B
0.6
Label Relationship
SST-2
0.6
0.5
Defauls
Flipped
0.3-
-1.0
0.3
Arbitrary
Achitrary Plipped
25
50
25
50
0
25
50
Hate Speech
LLaMa 65B
0.8-
0.5
1.0
0.5
1..5
0.4-
15
30
30
15
30
LLaMa-2 70B
0.8-
0.5
MQP
0.6
-1.0
0.5
1.5
0.4-
20
40
20
40
0
20
40
No Observed In-Context Examples
Figure 5: Few-shot ICL with replacement labels for Falcon-40B on SST-2, LLaMa-65B on Hate Speech, and LLaMa-2-70B on MQP. Table 2 and $F contain results for all other models and tasks. ICL achieves better than guessing performance for all label relations and models. However, predic- tions for flipped labels (dashed blue) plateau at a higher entropies and lower likelihoods than those for the default label relation (solid blue). For arbitrary labels (pink), the model performs similarly for both label directions. Averages over 100 runs and thick lines with moving average (window size 5).
Table 2: Average differences between ICL entropies for default and flipped labels. Bold entries in- dicate differences are statistically significant. Again, we disregard entries for which default ICL per- formance is not significantly better than the guessing baseline (lightgray entries). When default ICL outperforms the baseline, ICI. entropies are almost always significantly different between scenarios. We average 100 runs, report results maximum context size, and show standard errors in Table F.2.
A Entropy
SST-2
Subj
FP
HS
AGN
MQP
MRPC
RTE
WNLI
L.aMMa22 7B
-0.52
0.01
-0.40
0..00
-0.75
-0.00
0.05
-0.03
-0.01
LLaMa-2 13B
-0.48
0.06
00.47
0.19
-0.95
0.03
-0.07
-0.11
0..7
LLaMa-2 70B
-0.17
-0.10
-0.40
-0.26
-1.00
-0224
-0.15
-0.26
-0.21
Falcon 7B
-0.28
-0.12
-0.02
-0.06
-0.52
0.00
0.00
-0.00
-0.01
Falcon 7B Instr.
-0.37
-0.07
-0.33
-0.05
-0.22
-0.02
0.13
0.01
0.00
Falcon 40B
-0.39
-0.23
-0.42
0.19
-0.90
-0.00
-0.10
-0.02
-0.00
Falcon 40B Instr.
-0.48
-0.16
-0.43
0.31
-0.92
0.10
-0.02
00.06
00.01
label relationship, e.g. the flipped labels in (1). Note that, we rely the flipped label experiments to evaluate NH2; results on arbitrary labels serve to complete the picture, and we discuss them later.
Observations & Discussion. We evaluate NH2 across all our models, tasks, and metrics. Figure 5 shows results for a selection of large models and datasets. Evidently, the LLMs can, to some extent, learn to predict the flipped label relationships against pre-training preference. Accuracies on flipped labels reach levels significantly better than random guessing. However, in particular for entropies, there is consistent gap between the default and flipped label scenarios: ICL predictions on flipped labels are much less certain. Importantly, given the plateauing behavior, this gap is unlikely to disap- pear with additional in-context observations. (Practically speaking, we cannot actually add any addi- tional examples as imput size is maximal already and will deteriorate when exceeding the LLMs' in- put token limit; this is itself a limitation of ICL compared conventional learning.) It seems that la- bel relationships inferred from pre-training have permanent effect that cannot be overcome through in-context observations. This does not agree with conventional learning: predictions on flipped la- bels should continue to improve observations continue to contradict pre-training preference.
Crucially, we observe this behavior across models and tasks. We encourage the reader to view the full set of training curves in $F. Table 2 provides a summary of the results, showing differences in entropy between default and flipped label scenarios at maximum context size, highlighting statistical significance in bold and graying out entries where ICL fails on default labels. Across the board, again observe that predictions on flipped labels plateau: a significant gap between predictions on default and flipped labels remains, even maximum input size. For the models study, reject

---

Published as a conference paper at ICLR 2024
NH2 that ICL can overcome prediction preferences from pre-training. Again, the results here strongly support our previous rejection of NHI, as clearly, predictions change for replacement labels.
Figure also shows that for replacement labels (A, B) and (B, A) both directions similarly easy for ICL to learn. This suggests the LLM has not learned a preference for them from pre- pre-training. agreeing with intuition. Further, learning arbitrary labels is slower than learning default labels but faster than learning flipped labels, which agrees with intuitions about inductive biases.
Can Prompting Help? In $A, we further study if specific prompts, i.e. instructions that inform the LLMs of the flipped labels, can improve ICL predictions. We find that, while prompts initially can help the model predict flipped labels, eventually, prompts longer improve predictions.
8 HOW DOES ICL AGGREGATE IN-CONTETT INFORMATION?
With NH3, we study if ICL considers all in-context information equally. We have just seen that ICL does not treat pre-training preference equivalently to in-context label information. However, it is similarly important to understand how ICL treats different sources purely in-context information.
To test NH3, change the label relationship during in-context learning in three different scenarios. (D - F): after N observations the default label relation we flip the label relation for all following observations, e.g. from (negative, positive) to (positive, negative) for SST-2. (F - D): with N flipped label observations and then expose the model to default labels. (Alternate F ++ D): alternate between the default and flipped labels after each observation. For all three setups, after 2N observations, the model has observed the same number of the flipped and default label examples. If NH3 is true, ICL should treat all observed label relations equally, no matter their position in the input. This means, predictions should be the same for all three scenarios after 2.N observations.
start
Observations & Discussion. Figure shows accuracies for selection of models and tasks, and we report full probabilistic metrics for all tasks and model combinations for which ICL was able to learn the flipped relationships well in Figs. F:32 to F.36. We observe that, across almost all tasks and model combinations, predictions are significantly different between the three setups after observing the same number of examples of both label relationships (after 2.N total observations, red dashed line in the figure). We thus reject NH3 that ICL treats all information provided in-context equivalently.
After the changepoint N, predictions immediately begin to adjust to the new label relationship. In particular, after 2NV observations, the (F + D) setup has a bias for predicting according to the default label relationship, while the (D + F) setup has a bias for predicting the flipped label relationship.
SST-2
Subj
AG News
1.0
1.0
LLaMa-2 70B
D F
0.8
Alternate F Changepoint
0.5-
0.5
Changepoint
0.6
Gueezing
0.0
0.8
Falcon 40B
0.8
0.75-
0.6
0.6-
0.4-
0.50
0.4
25
50
0
20
0
10 20
No Observed In-Context Examples
Figure 6: Few-shot ICL accuracies when the label relationship changes throughout ICL. For (D + F), we start with default labels and change to flipped labels at the changepoint, for (F + D) we change from flipped to the default labels at the changepoint, and for (Alternating F <+ D) we alternate berween the two label relationships after every observation. For setups, at '2 x Change- point', the LLMs have observed the same number of examples for both label relations. If, according to NH3, ICL treats all in-context information equally, predictions should be equal at that point- they not. Bootstrapped 99 % confidence intervals, moving averages (size 3), and 500 repetitions.
8

---

Published as a conference paper at ICLR 2024
In other words, LLMs prefer to use information that is closer to the query, instead of considering all available information equally. Our finding is distinct from Zhao et al. (2021), who observe that ICL preferentially predicts labels that appear frequently near the query for a single fixed label relation. Lastly, we note once more that the results here also strongly support our previous rejection of NH1.
DISCUSSION & LIMITATIONS
Alignment. For alignment of LLMs, it is crucial to understand how pre-training preference and in- puts trade-off, as well as how different parts of the input, such context string and user input, in- teract and influence predictions. Our results suggest prompt-base alignment (Bai et al., 2022b) may struggle to overwrite pre-training preference and could itself easily be overcome by future user input.
Do Labels Always Matter? It is plausible that labels matter less for other NLP tasks such as question answering, where in-context examples may provide limited information towards the answer of the query question. However, for the randomized label experiment, capable LLMs might still identify that the provided in-context answers are random and imitate this in their predictions.
Limitations. We focus on few-shot ICL tasks where evaluation is based on logits and not free-form generation. We do this mostly avoid complications around evaluating free-form generation tasks and believe our results should transfer to this setting. Further, our experiments do not cover RLHF-finetuned LLMs (Christiano et al., 2017; Ziegler et al., 2019; Ouyang et al., 2022).
10 RELATED WORK
Some recent work has studied the effect of labels in ICI. Yoo et al. (2022) also revisit label ran- domization and find significant variance across tasks and models. Pan et al. (2023) further separate ICL into aabel-iddependent and -dependent learning, which they study by replacing labels with bitrary tokens. Wei et al. (2023) find that smaller or instruction-tuned models are less capable when performing ICL with replacement labels. Similar to Min et al. (2022b), the above studies do not con- sider probabilistic metrics or full ICL training curves, and thus can underestimate changes in ICL predictions for modified labels. For example, Pan al. (2023) find that the gap between random and default labels is 'insignificant" for small models, which our results, in particular for probabilistic metries, contradict, cf. $5. Further, Wei et al. (2023) claim that 'large models can override prior knowledge from pretraining [...] in-context' and 'small models do not change their predictions when seeing flipped labels', which is not supported by our results in $7. Lastly, Gao et al. (2021) observe that replacement labels can also degrade performance when finetuning language models. More generally, ICL has been the subject of many recent studies. For example, Min et al. (2022a) fine-tune language models to improve ICL, Si et al. (2023) measure the inductive bias of ICL pre- dictions, Chan et al. (2022b); Dasgupta et al. (2022) study differences between in-weights and in- context generalization, and Chang & Jia (2022); Liu al. (2022); Zhang et al. (2022b) observe that the selection of examples affect ICL predictions significantly. In this paper, we emphasize a prob- abilistic treatment of ICL predictions. Uncertainty Kadavath et al. (2022); Lin et al. (2023); Bai et al. (2022a); Gonen et al. (2022). On non-language tasks, Kirsch et al. (2022); Chan et al. (2022a) study properties that lead to the emergence of ICL. Also related are Garnelo et al. (2018); Kossen et al. (2021); Gordon et al. (2019), who propose deep non non- parametrri models on non-language tasks. Unlike ICL, they can guarantee invariance to example-order or closely approximate Bayesian predictive distributions (Müller et al., 2022).
LLMs has previously been studied, e.g. by
11 CONCLUSIONS
In this paper, we have investigated how the conditional label distribution of in-context examples af- fects ICL predictions. To ensure our conclusions represent ICL behavior well, we have studied ICL across all possible in-context dataset sizes and considered probabilistic aspects of ICL predictions. In some sense, we have shown that ICL both better and worse than expected. On the one hand, our results demonstrate that, against expectations set by prior work, ICL does incorporate in-context label information and can even learn truly novel tasks in-context. On the other hand, we have shown that analogies between ICL and conventional learning algorithms fall short in variety of ways. In particular, label relationships inferred from pre-training have a lasting effect that cannot be sur- mounted by in-context observations. Additional prompting can improve but likely not overcome this deficiency. Further, ICL does not treat all information provided in co-tentext equally and preferentially makes use of label information that appears closer to the query.
9

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
We discuss the details of our experimental evaluation in $E. We provide the code to reproduce our results at the following repository: htttbb.com/jkkofnncooee.....c.
ACKNOWLEDGMENTS
Tom Rainforth is supported by the UK EPSRC grant HP/Y037200/1. Jannik Kossen acknowledges funding from the New College Ycotown Scholarship.
REFERENCES
Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. In- context examples selection for machine translation. ACL, 2023. URL https://arxiv: org/abs/ 211.222222222222222 02437.
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. In ICLR, 2023. URL https: 15661.
/
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv:2204.05862, 2022a. URL
https:/ /aras///ariiie aaraiiiivv.////./ org/ abss 2204.222222 05862.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv:22/2.08073, 2022b. URL https:://axxi.
abs/2212..88 08073..
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D) Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurIPS, 2020. URL
Stephanic Chan, Adam Santoro, Andrew Lampinen, Jane Wang, Aaditya Singh, Pierre Richemond, James McClelland, and Felix Hill. Data distributional properties drive emergent in-context learn- ing in transformers. NeurlPS, 2022a. URL https: /ttpss///riivvi org/ abs 2205. 05055.
Stephanie CY Chan, Ishita Dasgupta, Junkyung Kim, Dharshan Kumaran, Andrew Lampinen, and Felix Hill. Transformers generalize differently from information stored in context vs in weights. arXiv:2210.05675, 2022b. URL https: /arxiv.vvvvv org/ abs 2210. 675.
Ting- Yun Chang and Robin Jia. Careful data curation stabilizes in-context learning. In EMNLP, 2022. URL https:///// aawii.iiivv. www.i.ww......w.t.........o....e att.aa////a... 2212.222222222 10378.
Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, and He He. On the relation between sen- sitivity and accuracy in in-context learning. arXiv:2209.07661, 2022. URL https: //appii///rxv....
org/abs/ 22092 07661.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. Palm: Scaling language modeling with pathways. arXiv:2204.02311, 2022. URL httht://aaxxi org/ brrg/bs// abs/ 2204. 02311.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. NeurlPS, 2017. URL https://arxix.
org/abs/Di /1066. 03741.
Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment chal- lenge. In Machine learning challenges workshop, 2005. URL https: ///nnk/.////////////////////ktk/ springer.
10

---

Published as a conference paper at ICLR 2024
Ishita Dasgupta, Erin Grant, and Tom Griffiths. Distinguishing rule and seemplarbbssed generaliza- tion in learning systems. In JCML, 2022. URL https:: /tttppp://axxvvooo org/ abs 2210 04328.
Ona de Gibert, Naiara Perez, Aitor García-Pablos, and Montse Cuadros. Hate Speech Dataset from White Supremacy Forum. In ACL Workshop Abusive Language Online (ALW2), 2018. URL
https: : / www. acl web. org/ wanthologyy/o/aaaaooaooooglgolat
/wtgg////ww/ /w18 1
Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases.
In Third International Workshop on Paraphrasing (IWP2005), 2005. URL https://
aclanthology 0ww.:0o0-00r000000200... 5002/.
Edwin Fong and Chris C Holmes. On the marginal likelihood and cross-validation. Biometrika, 2020. URL https::/// http///aivvvvv//. org/ abs 905.905000 08737.
Tianyu Gao, Adam Fisch, and Dangi Chen. Making pre-trained language models better few-shot 2012.15723.
learners. In ACI., 2021. URL https: http://axvvvrrrg...o...
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes, arXiv:1807.01622, 2018, URL httpp:://aaiii..rrtee/r// abs/1807 622.
Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer. Demystifying prompts in language models via perplexity estimation. arXiv:2212.04037. 2022. URL https:
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Tumne. Meta- learning probabilistic inference for prediction. ICLR, 2019. URL https: /arxiv.org// abs/ /105. 09921.
Michael Hahn and Navin Goyal. A theory of emergent in-context learning as implicit structure induction. arXiv:2303.07977, 2023. URL hitps: : /arxiaiivooo/// org/ abs / 2303. 3. 07971.
Chi Han, Ziqi Wang, Han Zhao, and Heng Ji. In-context learning of large language models explained as kernel regression. arXiv:2305.12766, 2023. URL https: arxiv. org/ abs/2305. 12766.
Jordan Hoffmaan, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Train- ing compute-optimal large language models. In NeurIPS, 2022. URL pttcttss///aiiieeee// abs/ 2203 15556.
Ferenc Huszár. Implicit bayesian inference large language models, 2023. URL httttt:///aat /www. inference: , vc/ implieiiii ieii line; accessed 10-July-2023).
[On-
Hui Jiang. A latent space theory for emergent abilities large language models. arXiv:2304.09960, 2023. URL https::/arxiv..ot wwt./bs////aaa//e// ab///304532000.. 020./.9944000000.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language mod- els (mostly) know what they know. arXiv:2207.05221, 2022. URL https://arxiv..or
2oss/220..2m 05221.
Louis Kirsch, James Harrison, Jascha Sohl-Dickstein, and Luke Metz. General-purpose in-context learning by meta-learning transformers. arX1v:22/2.044458 2022. URL https://arxixs
org/abs/ 221222222222 04458.
Jannik Kossen, Neil Band, Clare Lyle, Aidan N Gomez, Tom Rainforth, and Yarin Gal. Self- attention between datapoints: Going beyond individual input-output pairs in deep learning. In NeurIPS, 2021. URL https: / aarxivi org/ abs/abasaaa /2106. 02584.
Hector Levesque, Emnest Davis. and Leora Morgenstern. The winograd schema challenge. In Thir- teenth international conference on the principles of knowledge representation and reasoning, 2012. URL htwps:t/t///ecctc cdn. aaal. oorg ocs /492 /42//4492222 218022221111111 111.443-111..--0

---

Published as a conference paper at ICLR 2024
Xiaonan Li and Xipeng Qiu. Finding supporting examples for in-context learning. arXiv:2302.13559 2023. URL atttp:://axiiv org/ abs 2302.13599
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXh::2211.091 2022. URL https: //tpxvaa.vvvv//aa//////aa..aaavl org/ wwgg/bs//////oo/oeo/g//////eebe/ abs/22111 09110.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. TMLR, 2023. URL https: http://axxvvvvv. org/ abs//25s.22222 14334.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3? ACL, 2022. URL htttps://azii...../// orgg /
2101. 06804.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. ACM Computing Surveys, 2023. URL https: //txaaaiii./..////////a/////aiva www.abs/////eooooooeett...ta..ae / 2207 0000077118880011 13586.
Peter Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. Generating wikipedia by summarizing long sequences. In ICLR, 2018. URL // //arxiv.org//a/aai...
https:
1801. 10198.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In ACL, 2022.
URL https:/p//// aarxiiivooo.. org/ abs/2104 086.6.
Malo, A. Sinha, P. Korhonen, J. Wallenius, and Takala. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 2014. URL https::///// //tp///axvvvv///////////.a///aaa org/ abs 53366.3.33..
Clara H. McCreery, Namit Katariya, Anitha Kannan, Manish Chablani, and Xavier Amatriain. Ef- fective transfer learning for identifying similar questions: Matching user questions to covid-19 faqs. arXiv:2008. /3546, 2020. https:/
orgg abtrg/bb//// attg/bbs/200 2008 13546.
Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannanch Hajishirzi. Metaicl: Learning to learn in context. In NAACL, 2022a. URL https:/ pttttpss//aaxi..oo/
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemyyr. Rethinking the role of demonstrations: What makes in-context learning work? In ACL, 2022b. URL https::////
Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Trans- formers can do bayesian inference. ICLR, 2022. URL
org.
10510.
Kevin P Murphy, Probabilistic machine learning: an introduction. MIT press, 2022. URL https: / //probml github. 1o/.mlllmmmmmmmmmm
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. NewrIPS, 2022. URL https:////// /ttpx//axivvv///////////.////wv/ org/ abs/2203. 02155.
Jane Pan, Tianyu Gao, Howard Chen, and Danqi Chen. What in-context learning "learns" in-context: Disentangling task recognition and task leamning. In ACL, 2023. URL cuttps://ae //ta/aaxvvvorro..oeeoo abbs/2055.. 09731.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary De Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NewrIPS, 2019. URL https: / httppps://yyttttt /ptttrcc..cccccc org/.
12

---

Published as a conference paper at ICLR 2024
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language derstanding with unsupervised learning. Technical report, OpenAl, 2018. URL https:
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever,
et al. Language models are unsupervised multitask learners. OpenAl blog, 2019. URL
Yasaman Razeghi, Robert Logan IV, Matt Gardner, and Sameer Singh. Impact of pretraining term frequencies few-shot reasoning. In EMNLP, 2022. URL https: org/ abs/ 2202. 67206.
Chenglei Si, Dan Friedman, Nitish Joshi, Shi Feng, Danqi Chen, and He He. Measuring inductive biases of in-context learning with underspecified demonstrations. In ACL, 2023. URL https: //arxiv. org/ habbs/2305 13299.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In EMNLP, 2013. URL https: /taettss:/accaatttee http///aaaaaanlnaaaaaaaaa or1p//3-1//0000000000000 1170//
Efstathios Stamatatos. A survey of modern authorship attribution methods. American Society for information Science and Technology. 2009. WRRLhtttss///olieeeeee www.bbrrrr..l.lly...........eeee com/
doi/10. 00000.01000//00 was.0/ii/ii..... aas........... 21001.
Technology Innovation Institute TII. Falcon 1lm, 2023. URL URLttt:::/falcotnll tii ae/.
[Online; accessed 10-July-2233..
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. Llama: Open and efficient foundation language models. arXiv:2302.1397 13971, 2023a. URL https:
//arxiv. org/ abs 2307.13971.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foun- dation and fine-tuned chat models. arXiv:2337.009288 2023b. URL https:/ http:///axvvv///////oo abs/2307 09289.
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordy- intsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In ICML, 2023. URL httpt:tt////em //arxiv//iivvv orroorrg///... /21..222222222222 07677.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE; multi-task benchmark and analysis platform for natural language understanding, In ICLR, 2019, URL patpps:// // arxiv. org abs/ 18044 07461.
Sida Wang and Christopher D Manning. Baselines and bigrams: Simple, good sentiment and topic classification. In ACL, 2012. URL https: //ll. acm. wwt//oo/////oooooooooooooooooooo 555523356555555555. 2390688.
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, al. Larger language models do in-context learning differently.
NXXi:23303.08460 2023. URL https::///// //apssi///x/..../ wttpaaiwv.v.g....aaa/ooo..oooolt
www./ogg//////ooooooooooeeeeeo abs/2303 03846.
Noam Wies, Yoav Levine, and Amnon Shashua. The learnability of in-context learning.
arXiv:2303.07895, 2023, URL httphttt//// htttps:/aaxxv..rrg wttp:///ww..rr//////3/33/3 /2/.//2333........
Ronald J Williams and David Zipser. A learning algorithm for continually rumning fully recurrent neural networks. Neural computation, 1989. URL sttps://ieeexpae ieee org/
(bcument//795226
13

---

Published as a conference paper at ICLR 2024
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's 's transformers: State-of-thee art natural language processing. In EMNLP: System Demonstrations, 2020. URL
https: / rxiv. orgg alss 1910 09910.01777....
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks. arXiv:2307.02477, 2023. URL https://
a arxi org/ a abs / 2307 02477.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning implicit bayesian inference. In ICLR, 2022. URL https://arxiv.c abs/ 2111. 02080.
Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang- goo Lee, and Taeuk Kim. Ground-truth labels matter: A deeper look into input-label demonstra- tions. In EMNLP, 2022. URL https: /aalss////xiiii /arxivv.vvvv.///////
org/ abs / 2205 12685.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Victoria Lin, al. Opt: Open pre-trained transformer lan- guage models. arXiv:2205.0100 2022a. URL http:: httaraivvoorrr/ee/eeo www.rr////aa..eeo /t.b///22055 01 01068.
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NeurIPS, 2015. URL
Yiming Zhang, Shi Feng, and Chenhao Tan. Active example selection for in-context learning. In EMNLP, 2022b. URL https::/aaxiissiee:ee
abs
Yufeng Zhang, Fengzhuo Zhang, Zhuoran Yang, and Zhaoran Wang. What and how does
in-context learning learn? bayesian model averaging, parameterization, and generalization.
arXiv:2305.19420, 2023. URL https:://aaxiiiiie http:////wwioo/o///////oooooooot
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving
few-shot performance of language models. In ICMI., 2021. URL https://arxiv..rrtt abs /2002. 09690.
Daniel M Ziegler, Nisan Stiemnon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano. and Geoffrey Irving. Fine-tuning language models from human preferences. 08593.
arXiv: 199.:2999, 2019, www.hhtts////iiiiieeee
14

---

Published as a conference paper at ICLR 2024
A CAN PROMPTS HELP ICL LEARN FLIPPED LABEL RELATIONSHIPS?
In this section, we explore prompts can be used to overeome the plateauing ICL performance when default labels are flipped. Before the in-context examples, we insert prompt strings that inform the LLM of the flipped label relationship some fashion, and should thus help the L.LM adjust to during ICL. These prompts could fundamentally change few-shot ICL behavior. In fact, can think of the in-context learner as the union of LLM and prompt, where so far the prompt was simply left empty. There could exist prompts that help ICL learn the flipped label relationship better than without them, or as well as in the default scenario. Note that, regardless of the outcome here, NH2 remains rejected as ICL should not need to rely prompt to correctly consider in-context observations. Nevertheless, for the 'prompted few-shot ICL' setup, NH2 should be reconsidered.
We explore the following three prompts: 'In the following ...,, (Instruct Prompt) "...negative means positive and positive means negative' (here for SST-2 and adapted to other tasks), (Ignore Prompt) ...ignore all prior knowledge', and (Invert Prompt) ...ffip the meaning for all answers'.
Observations & Discussions. Figure A. gives results for the prompted few-shot ICL setup for LLaMa-65B and Falcon-40B on SST-2. Figures F.38 to F.46 give results for our largest models across all tasks. However, prompting is most successful for the scenarios in Fig. A. making this the most interesting result to study NH2. In particular, prompting has surprisingly weak effect for L.L.M----70B, In Fig. A.1 we observe that prompts, in particular the instruct and invert prompts, can help improve ICL performance. However, it seems that the positive impact from prompting restricted to an initial boost at small in-context datasets sizes. We then sometimes observe 'dip' in performance, which could indicate ICL forgetting about the prompt. At large context sizes, none of prompts have any advantage, and flipped label performance again plateaus short of performance for the default label setup. Therefore, we reject NH2 for the prompted ICL variations that we study.
It is possible that there exist prompts that have not found- found-fornnn which cannot reject NH2. However, we are sceptical these prompts exist for the models we study, as their behavior at large context sizes is strikingly similar across all prompts investigate. For more capable LLMs, we pect may be possible to obtain a zero-shot performance on the flipped scenario that is equal to the zero-shot of the default scenario, i.e. the prompt leads the LLM perfectly flipping all its zero-shot predictions. However, we are unsure if, in addition to flipping zero-shot predictions, such prompts would also improve ICL on flipped label observations be as good in the default scenario.
Accuracy
Log Likelihood
Entropy
Plipped w/ Prompt None
LLaMa 65B
0.75
0.5
0.6
Instruct
Ignore
0.50-
1.00
0.4
Default Guessing
0.25
1.5
Falcon 40B
0.75-
-0.5
0.6
0.4-
-1.0
0.25
0
25
50
25
50
0
26
60
No Observed In-Context Examples
Figure A.1: Prompted few-shot ICL with flipped labels on SST-2, Some prompts are able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). However, improvements have no lasting effect: performance at larger context sizes does not improve and still plateaus short of the default scenario (solid grey line). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
15

---

Published as a conference paper at ICLR 2024
B EVALUATION APPROACH FOR CHEAP IN-CONTEXT LEARNING DYNAMICS
In this section, we suggest a-to the best of our knowledge- novel way of evaluating ICL that gives performance metries at all in-context dataset sizes in a single forward pass without incurring additional cost. We start by introducing the notation necessary to formalize few-shot ICL in LLMs.
Dataset to Input String. The few-shot task defined by dataset D {(S,, ()) where S E Tue, are input sentences from which predict the associated labels Y; T and T are text strings of length v. A verbalizer ((S. Y) takes a sentence-label pair and maps it to an example, e.g. the sequence 'I am happy' and label 'positive' are verbalized as 'Sentence: am happy's n Label: positive\n' n'". We also define a query verbalizer (S) that maps test query S to a query example, e.g. 'I am sad' is mapped to 'Sentence: am sad n Label:', such that the next-token prediction of an LLM will be encouraged to predict the label for the query. We apply the verbalizer to the entire dataset set and concatenate its output to obtain the context = e VS,,, Y,). Finally, we concatenate context C and verbalized query (S), where S is sentence drawn from separate test set, to obtain the input to the language model, 7 = C ( (S)) E Td'
Input String Tokens. The input I is tokenized before can be processed by the language model. The tokenizer, T(1) = (Xi,..., X Xar), maps an input sequence / to sequence of integers, or tokens, X; E (1...., D), where D is the vocabulary size, i.e. the number of unique tokens. We keep track of which token positions correspond to labels, L = (1,.... IN), e.g. the indices of the tokens immediately following the string 'Label:' in the above example.
Tokens to Predictions. In the following, we use capital letters to denote random variables and lower-case letters for their realizations. Here, we describe the behavior of decoder-only language models (Liu et al., 2018; Radford et al., 2018), a popular architecture choice for LLMs. Given the observed sequence of input tokens (X1 = æ1..... XM = TMI), a single forward pass through the language model gives an estimate of the joint probability,
p(X, = =1) p(X2 = 22 X, =1) ....... p(X = JM = =I......... X = IM-1)- (1)
We highglight that Eq. (1) gives the joint probability at the observed outcome:: we obtain M "one- step ahead' predictions, each conditioned only on observed outcomes and not on model predictions. Equation (1) is a common objective in LLM training, where 'the joint probability the model assigns to the observations' is sometimes referred to as teacher forcing (Williams & Zipser, 1989).
At test time, LLMs are usually iteratively conditioned on their own predictions, generating novel outputs via multiple forward passes, i.e. one first samples a p(Xasl...) .). and then ăM+1 ~ p(XM+1 I...,Xa,
d.),, and so on. We here use "..." to stand in for any additional tokens also conditioned on, e.g. (Ii,,.... -M One usually ignores all other terms of the joint here- the predictions for (XI.... XM-1) that are generated in each forward pass only the last term p(XM ....) )) is needed to sample the next token, i.c. the label in standard few-shot ICL applications.
Single-Forward Pass ICL Training Dynamics. We now explain our approach for efficient eval- uation of ICL training dynamics. Given input tokens (XI,..,,Xay for the few-shot ICL setup described above, we first select those terms from Eq. (1) that correspond to label token predictions,
II N p(X,, = 21, Xi = 2i, Xmet, = Imel))
For each term, the model predicts a distribution over the entire token vocabulary, i.c p(X,, I...) is categorical distribution, p (pi....,PP) which is then evaluated at the observed tokens in Eq. (2). We can transform this into a prediction over only the few-shot task label Y by selecting the indices of the categorical distribution which correspond to the tokenized labels and then renormalizing, p(Y) = (ti....,te) ,t)). With this, we rewrite Eq. (2) as the joint probability the model assigns to the sequence of labels given input sentences
....., ). where C' is the number of unique labels which are encoded to tokens
p(V = 31 S s1) p(y2 1/2 S = $1. Yi 31, S2 52) ....
p(NN = YN S = Y1 V/1 ....... SN-1 = -1 YN-1 = UN = sx). (3)
Note how, because the joint is evaluated at the observations, its individual terms are always condi- tioned on the true labels and not previous predictions. This allow to cheaply compute the training
16

---

dynamics of ICL as function of increasing in-context dataset size. With each forward pass, obtain the individual terms of Eq. (3), which are the few-shot ICL predictions at all possible context dataset sizes, i = ,I....,.43 ,.). In contrast, in standard few-shot ICL evaluations, each forward pass only yields predictions for a single test query. neglecting the information the joint contains about the first N - lahel predictions. There may be interesting applications of Eq. (3) to model selection, (Fong & Published as a conference paper at ICLR 2024
as the quantity has links to both Bayesian evidence (Murphy, 2022) and Holmes, 2020), although we do not explore this any further in this paper.
Multi-Token Labels. So far, we have assumed that each label string is encoded as a single token. However, our approach can also be applied if some or all labels are encoded as multiple tokens. In essence, we continue to measure only the probability the model assigns to the first token of each label, making the (fairly harmless) assumption that the first (or only) token that each label encoded to is unique among labels. We believe this is justified, as, given the first token for a label, the model should near-deterministically predict the remaining tokens, i.e. all the predictive information is contained in the first token the model predicts for a label. For example, for the Subjectivity dataset, the label 'objective' is encoded by the LLaMa tokenizer as a single token but the label 'subjective' is encoded two tokens, [subject, ive]. We only use the probability assigned to [subject] to assign probabilities to 'subjective', and ignore any predictions for [ive]. However, if the model successfully accommodates the pattern of the in-context example labels, we would expect probabilities for [ive] following [subject] to be close to always.
For LLaMa-7B on Subjectivity, we have investigated the above assumption empirically. After the first observation of the 'sujjectivity" label in-context, the probability of predicting [ive] after ob- serving [subject] are 0.9998 + 0.0003 for the following 12 instances of the "subjectivity" label, with probabilities normalized over all tokens of the vocabulary here. In other words, we can safely eval- uate the performance of the LLaMa model from its predictions of only the [subject] token, even though the full label is split over two tokens [subject, ive].
C AUTHORSHIP IDENTIFICATION TASK
We here give details on our novel authorship identification task.
Data Collection & Processing. We extract the last 151 messages sent between two authors of this paper on the Slack messaging platform. If multiple messages are sent in a row by one author, these count as multiple inputs. We filter out 42 messages that were of low quality: URLs, article summaries, missed call notifications, and meeting notes. This leaves us with 58 and 51 messages per author. We set the maximum message length to be 200 and truncate any messages longer than that. Before truncation, the longest message is 68 before and after truncation, mean and standard deviation shrink from 100 = 98 to 88 + 65. For use in ICI., treat this dataset as we would any other and present messages in random order.
579 characters long. The median message length
For now, we have decided to not make this dataset public for two reasons: (1) It contains genuinely private communication, and (2) releasing the data would mean that future LLMs might be trained on it, so we could no longer use it to test their ability to learn truly novel label relationships in-context. However, below, we give random examples from the dataset:
Data Release.
Author 1
Would 10.30am on Tuesday work?
Author 1 Sounds good. When are you back again?
Author 1 Yeah, we might have to find somewhere else depending on whether my office mates are in, but its out of term so should be plenty of free meeting rooms if needed
Author 2 No problem!
Author 2 I'll be in [redacted] next week, so do you think we can meet in person?
Author 2 The vacation is 16 days!
D DATASET CITATIONS
We evaluate on SST-2 (Socher et al., 2013), Subjective (Wang & Manning, 2012), Financial Phrase- bank (Malo et al., 2014), Hate Speech (de Gibert et al., 2018), AG News (Zhang et al., 2015), Med- ical Questions Pairs (MQP) (McCreery et al., 2020), as well Microsoft Research Paraphrase Cor- pus (MRPC) (Dolan& & Brockett, 2005), Recognizing Textual Entailment (RTE) (Dagan et al., 2005), and Winograd Schema Challenge (WNLI) (Levesque et al., 2012) from GLUE (Wang et al., 2019).
17

---

Published as a conference paper at ICLR 2024
E EXPERIMENT DETAILS
Below give additional details on our experimental evaluation.
Guessing Baseline. In our experiments, we frequently display a "guessing based on class frequen- cies' baseline grey-dashed line. This baseline presents an informed guess that relies only on knowing the class frequencies of the task and makes the exact same prediction for each input dat- apoint. We here explain how we compute this baseline for accuracy, entropy, and log likelihood. We are given a classification task with C' classes which appear with frequencies p = .Pi,....3 Pcl in the training set. The baseline always predicts p = (pt..... Pc]. i.e. it predicts the class frequencies. For accuracy, it thus always predicts the majority class c" = arg max;; Pk. which leads to accuracy Per. Further, the baseline prediction leads to a log likelihood of Σσ Ph log Pk and an entropy of >E Pk Po Pı under the training data distribution.
Class Flipping. While most of our tasks are hinary classification, Financial Phrasebank and AG News are not. For these datasets, when 'flipping" labels in $7, $A, and $8, we actually rotate labels instead, i.e. we reassign labels as + (1+1) mod C, where the number of classes. For AG News, [ ['world', 'sports", "business', 'science and technology"] get mapped to ['sports', 'business', 'science and technology', 'world']. For Financial Phrasebank, l'negative', 'neutral', 'positive'] get mapped to ['neutral', 'positive', 'negative']. Note that, for Financial Phrasebank, rotating the labels is harder than naively inverting the label order, as rotating does not leave the meaning of the "neutral" label unchanged.
In-Context Example Formatting. We use the following simple templates to format the in-context examples. For SST-2, Subjectivity, Financial Phrasebank, Hate Speech, and our author identification lask, we use the following line of Python code to format each input example: [Label)(nnrr
e"Sentence: ' "(sentence)" InAnswer:
For MRPC, WNLI, and RTE, we format instances with
f"Sentence 1 (sentencel } \nSentence 2: (seentence2)' \nAnswer:
(1abe1)(nk"". For MQP, we use
f"Question 1: '(sentencel) AnQuestion 2: '(sentence2) (acpaanccee"/aaaaaaaaaaaaa \pAnswer:
[label]aaa""
Implementation. We rely the Hugging Face Python library (Wolf et al., 2020) and PyTorch (Paszke et al., 2019) to implement the experiments of this paper. We use half- salf-precision floating-poin point numbers for LLaMa-65B and LLaMa-2-70BB and we use 8 bit-quantization for all other models, which we have found to not affect performance notably. In Fig. 1, we illustrate this by showing the difference between 8 bit quantization and full 32 bit precision for default ICI. and ICL with label randomization for LLaMa-2-7B on the Subjectivity dataset: there is no significant loss of precision or change in behavior from 8 bit quantization.
Datasets. We use Hugging Face Spaces to access all tasks considered in this paper. For Hate Speech, we select the first 1000 examples with labels and 1, skipping datapoints with labels 2 and We do not use custom processing for any other dataset.
Whitespace Tokenization. To evaluate few-shot ICL performance as introduced in $4, we need to identify the tokens that individual task labels are encoded to. We here detail how to achieve this at the example of the SST-2 label 'positive'. In particular, we highlight the, perhaps unexpected, effects of whitespaces on input tokenization. These details are important and, if not considered correctly, can degrade performance significantly.
For Falcon models, the tokenizer encodes 'Answer:' [20309, 37]. 'Answer:-' as [20309, 37, 204]. and 'Answer: -pos: ivv as [20309, 37, 3508]. We here use dashes ._' instead of whitespaces for improved legibility. Clearly, the relevant token for the label "positive" is (350]]. J. Note how the token [204] for the trailing whitespace disappears again af- ter appending the label. Further, just encoding ipositive without preceding whitespace gives [28265]. However, this token does not appear in the input, where the label is preceded by a whitespace we should thus use the token [3508] measure ICL performance.
The LLaMa and LLaMa-2 tokenizer encodes 'Answer:' as [673, 29901], 'Answer as [673, 29901, 29871] 1, and 'Answer: -posittvve as 1673, 29901, 6374]. Thus,
18

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelihood f
Entropy
Bit random
0.8
0.6
32 Bit Default
32 Bit Random
0.6
Guessing
0.6
0.5
50
0
50
50
No Observed In-Context Examples
Figure E.1: Few-shot ICL at bit and 32 bit precision with default and randomized labels for L- on Subjectivity. There is no significant performance degradation from bit quantiza- tion. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals, and we apply moving averages (window size 5) for clarity.
Table E.1: Maximum number of in-context examples we consider for each model-task combination. Below, we shorten AG News as AGN, Hate Speech as HS, and Financial Phrasebank as FP.
SST-2
Subj
FP
HS
AGN
MQP
MRPC
RTE
WNLI
LLaMa-2
140
79
73
76
45
47
40
28
57
LLaMa
66
37
33
26
21
21
20
13
27
Falcon
67
39
37
28
25
23
21
15
30
the relevant token for the label 'positive' is [6374]. In contrast to the Falcon tokenizer, just encod- ing 'positive' without preceding whitespace also gives [6374].
Lastly, similar caveats apply to the classic evaluation procedure for few-shot ICL, where we only evaluate the prediction for a single test query at the end of the input. Here, it is crucially important that we do not end inputs with a trailing whitespace. As we have seen above, for both LLaMa and Falcon tokenizers, the trailing whitespace leads to the generation of extra token that not present when encoding complete in-context examples, as the whitespace would usually be included in the label prediction itself. This change in tokenization between in-context examples and test query can adversely affect ICL performance.
Statistical Significance. In Tables 1, F.I, and F.2 we bold differences if they are statistically significant at a 95 % level. Concretely, we compute if the absolute average differences are larger than 1.96 times the standard error. Similarly, when deciding if default label performance significantly better than random guessing performance, we check if mean performance plus 1.645 times the standard error is larger than the guessing baseline across accuracy and log likelihood.
Maximum Context Dataset Size. For each task, we create in-context datasets by sub-sampling from the training set of the task. Falcon and LLaMa support input sizes up to 2048 tokens, and LLaMa-2 supports up to 4096 input tokens. For all models, performance will degrade if the input size exceeds this limit. This caps the number of in-context examples we can include for each task. For tasks where the individual input sentences are longer, will be able to include fewer examples in-context. Across all in-context datasets that we sample for a task, we compute the minimum num- ber of in-context examples needed to exceed the token limit of 2048 4096. This is the maximum in-context dataset size up to which we report results for that task. We list these numbers in Table E.
Calibration. We do not calibrate predicted probabilities by first dividing them by a 'prior' probabil- ity and then renormalizing as suggested by Zhao et al. (2021). We have found this rarely improves, and sometimes degrades predictions, ef. Fig, E.2. We observe this happening in particular for tasks where labels are encoded as multiple tokens.
19

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
Accuracy f
0.8
Log Likelihood f
-3-
1.25
Defoult Defou't Oal
Random
Random: Cal.
Entropy
Guetzing
0.75
0.50
0.25
0
50
20
20
0
20
10
20
10
20
5
10
15
10
0
20
Observed In-CCotextt Examples
Figure E.2: Few-shot ICL with randomized labels for LLaMa-65B. We additionally display performance for random and default labels when 'calibrating' the predicted probabilities as suggested by Zhao et al. (2021). We not find calibration helpful to improve ICL performance. We average over 500 random in-context datasets and thin lines are bootstrapped 99 % confidence intervals and we apply moving average (window size 3) for clarity.

---

Published as a conference paper at ICLR 2024
F EXTENDED RESULTS
Section 4 Training Dynamics: Figure F.1 shows few-shot ICL training dynamics on SST-2 for selection of models different parameter counts.
Section 5 - Label Randomization: Figure F.2 gives results for randomized labels for all models on SST2. Figures F.3 to F.12 give results for all tasks and models comparing ICL with randomized labels to the default label setup. Table F.1 gives full summary statistics across all models, tasks, and metries for the label randomization experiment.
Section 6 Author ID Task: Figure F.13 gives few-shot ICL results for all models on our novel authorship identification task.
Section Flipped Labels: Figures F.14 to F.31 give results for all tasks and models for the modified label relationship experiments. We also report performance for additional replacement labels across task and models. Table F.2 gives full summary statistics across all models, tasks, and metries for the difference between default and flipped label performance.
Section 8 Dynamic Label Flipping: In Figs. F332 to E36, we give results for the experiments investigating NH3 for all large models on tasks where label flipping gave strong performance in $7. For LLaMa-2-70B Hate Speech in Fig. F.35, metrics appear very similar initially. However, when exploring additional changepoints in Fig. F.37, we find significant differences in predictions.
Appendix A Prompting with Flipped Labels: Figures F.38 to F.46 give results for all tasks and models for the prompted seo-shot ICL setup.
21

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
LLaMa-2
0.2
0.6
1.aMa-2 13R
0.8
LLaMa-2 70B
Gueeeing
0.4
0.4-
0.6
0.6
0.2-
100
100
100
8
LLaMa 7B
0.2
0.6
LLaMa 13B
0.8
65B
Gueeeing
0.4-
0.6
0.6
0.2-
25
50
25
50
50
Falcon 7B
0
0.2
0.6
Falcon 7B Inats.
0.8
Falcon 40B
Falcon 40B Instr.
0.4
0.4
Guessing
0.6
0.6
0.2-
No Observed In-Context Examples
Figure F.1: Few-shot ICL training dynamics standard scenario on SST-2. Accuracy (1) and log likelihood (f) improve with in-context dataset size, and entropies decrease appropriately. Aver- ages over 500 random subsets of the SST-2 training set (thin lines), additionally applying a moving average with window size for clarity (thick lines).
22

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
0.2
0.6
L.aMa-2 13R
0.8
LLaMa-1 70B
0.4
Gueeeing
0.4-
0.6
-0.6
0.2
100
100
100
LLaMa 7B
0.6
LLaMa 13B
0.8
65B
0.4-
Gueeeing
0.4-
0.6
-0.6-
0.2-
25
50
25
50
50
0.2
Falcon 7B
8
0.6-
Falcon 7B Inats.
0.8
Falcon 40B
0.4
Falcon 40B Instr.
0.4-
Guessing
0.6
0.2
N Observed In-Context Examples
Figure F.2: Few-shot ICL with randomized labels for SST-2: Compared to default ICL behavior (dashed lines, cf. Fig. F.1), log likelihoods and entropies of the models degrade when in-context example labels are randomized. Thin lines are averages over 500 repetitions, thick lines with moving average (window size 5), guessing baseline based class frequencies.
23

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
1.0
Detaul: Randomized Guonsinng
Accurary
0.4
0.0
Log Likeelhood f
0.5
-10
-1.
1.0
Entropy
0.5
0.0
100
0
50
0
50
50
0
25
0
25
0
25
0
20
50
No Observed In-Context Examples
Figure F.3: Few-shot ICL with randomized labels for LLaMa-2-7B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can be noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
1.0
Detaul: Randomized Guoesing
0.8
Accurary
0.6
0.0
Log Likeelhood f
0.5
-10
-1.
1.0
Entropy
0.0
100
0
50
0
50
50
0
25
0
25
0
25
0
20
50
No Observed In-Context Examples
Figure F.:: Few-shot ICL with randomized labels for LLaMa-2-13B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can be noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPO
RTE
WNLI
1.0
Accuracy f
0.6
Randomland
0.0
Log Likellhood f
0.5
-1.0
-1.5
Entropy
0.0
100
0
50
0
50
50
0
25
25
0
25
20
0
50
Observed In-Context Examples
Figure F.5: Few-shot ICL with randomized labels for LLaMa-2-70B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can be noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
Detsslt Randomized Goossing
0.8
Accurary
0.6
0.4
0.0
Log Likelihood f
0.5
-100
-1.5
1.00
0.75
Entropy
0.50
0.25
0.00
0
50
20
0
20
20
0
20 0
20
0
10
0
10
20
No Observed In-Context Examples
Figure F.6: Few-shot ICL with randomized labels for LLaMa-7B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
Defsslt Randomized Gooscing
0.8
Accurary
0.6
0.4
0.0
Log Likelihood f
0.5
-100
-1.5
1.00
0.75
Entropy
0.50
0.25
0.00
0
50
20
0
20
20
0
20 0
20
0
10
0
10
20
No Observed In-Context Examples
Figure F.7: Few-shot ICL with randomized labels for LLaMa-13B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPO
RTE
WNLI
1.0
Accuracy f
0.8
0.6
Randomised
0.0
Log Likellhood f
0.5
-1.0
-1.5
Entropy
0.0
50
0
20
0
20
20
0
20
20
0
10
10
0
20
Observed In-Context Examples
Figure F.8: Few-shot ICL with randomized labels for LLaMa-65B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
Detaul: Randomized
0.8
Guossingg
Accurary
0.6
0.4
0.0
Log Likeelihodd f
0.5
-10
-1.5
1.00
0.75
Entropy
0.50
0.25
0.00
0
50
0
25
0
20
20
0
20
0
20
0
20
0
10
20
No Observed In-Context Examples
Figure F.9: Few-shot ICL with randomized labels for Falcon-7B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
Detaul: Razdomized Guonsinng
0.8
Accurary
0.6
0.4
0.0
Log Likeelhood f
0.5
-10
1155
0.8
Entropy
0.6
0.4
0.2
0.0
0
50
25
0
20
20
0
20
0
20
0
20
0
10
20
No Observed In-Context Examples
Figure F.10: Few-shot ICL with randomized labels for Falcon-7B-Instruet. accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can be noisy, differences are clearly visible for probabilistic entropy and log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
1.0
Detaul: Razdomized
0.8
Gunssingg
Accurary
0.6
0.4
0.2
0.0
Log Likeelhood f
0.5
-10
-1.
1.0
Entropy
0.5
0.0
0
50
0
25
0
20
20
0
20
0
20
0
20
0
10
20
No Observed In-Context Examples
Figure F.11: Few-shot ICL with randomized labels for Falcon-40B: accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can noisy, differences are clearly visible for probabilistic entropy log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
1.0
Detaul: Randomized Guoncinng
0.8
Accurary
0.4
0.0
Log Likeelhood f
0.5
-10
1.5
1.0
Entropy
0.5
0.0
0
50
0
25
0
20
20
0
20
0
20
0
20
0
10
20
No Observed In-Context Examples
Figure F.12: Few-shot ICL with randomized labels for Falcon-40B-Instruce accuracies, entropies, and log likelihoods behave differently for randomized and default labels when performance is above randomly guessing class frequencies. While accuracy can be noisy, differences are clearly visible for probabilistic entropy and log likelihood. We average over 500 random in-context datasets, thin lines are bootstrapped 99 % confidence intervals.

---

Table F.I: Full summary statistics for the randomization experiment of $5. We strongly encourage readers to also view the full training curves across all possible context sizes in $F. Across all metrics, here show the average difference between the default and randomized label scenario. We compute 'Metric(Default) (etric(Random)" such that positive accuracy/log likelihood differences indicate that ICL performs worse with randomized labels. We compute differences at the maximum context size for each task-model combination (cf.ßE and Table E.1). We display experiments where ICL accuracies or log likelihoods on the default labels do not significantly exceed random guessing performance in lightgray. Across models, and metrics, model performance is usually significantly worse when labels are randomized and default performance exceeds random guessing. We display mean differences and standard errors over 500 runs. We bold entries for which mean differences are statistically significant,
A Log Lik.
SST-2
Subj
Phraseb.
Hate Speech
News
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
0.42
0.39 0.02
0.57 0.02
0.18
0.53 + 0.04
0.03 0.01
0.02::0001
0.03
0.02=
LLaMa-2 13B
0.41 1.11:00 0.02
0.62 -.0
0.49 = 0.03
0.24:0.00
0.81 0.81 0.04
0.04 0.01
0.01 0.01
0.06
1.02=0000
LLaMa-2 70B
0.00
0.53
0.02
0.80
0.02
0.02
0.02
0.18
LLaMM 7B
0.39 0.08
0.02
0.36 0.02
0.15
0.30 0.03
0.03 0.01
0.00 1:000:0000
0.03
0.01 0.02
LLaMa 13B
0.03
0.00
0.37 0.02
0.16
0.32
0.04 0.02
-0.01
0.05
0.02 0.02
LLaMa 65B
0.002
0.00
0.02
0.02
0.08
0.14 0.02
0.01
0.08 + 0.02
Falcon 7B
0.00
0.09
0.25 0.01
0.06
0.01
0.01 0.02
0.01 0:002
-0.01 0.02
0.01 0.02
Falcon 7B Instr.
0.01
0.08 +
0.11 0.01
0.03 +03+0.00
0.15
0.03 0.02-002
0.02 0.008
-0.00 0.02
0.00 0.02
Falcon 40B
0.00
0.35 0.02
0.31
0.18
0.90
0.06
0.01
0.01 0.02
0.02 0.02
Falcon 40B Instr.
0.25 + 0.02
0.37 + 0.03
0.27 0.02
0.02 0.00
0.77 0.04
0.06 0.02
0.02 0-02
0.02 0.02
0.04 - 0.02
À Accuracy
SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
28.4 2.1
39.8 + 2.4
30.4 t 2.4
16.2 = 2.3
13.4 + 2.2
3.8 2.2
5.2 2.8
4.0
3.0
LLaMa-2 13B
19.8 2.0
46.0
24.4 + 2.4
19.0 = 2.3
34.6 2.3
3.2
2.8
1.6
3.6
LLaMa-2 70B
34.8 2.3
29.8 2.1
27.0 + 2.2
28.6 2.5
24.4 2.1
19.6 - 2.5
3.6
14.0
13.6
L. 7H
24.6 24.6-2.22
36.4 2.5
2.5
19.8 2.1 2.1
10.8 = 2.2
11.2 2.0
4.6 2.8
-02
2,8 2.6
1.0
LLaMa 13B
31.4
42.4 2.5
26.4 2.7
13.8 2.0
9.2
4.0
220
2.0 2.0224
3.0
LLaMa 65B
47.4
34.4 2.3
18.4 2.1
13.6
2.2
39.2 2.7
8.4 2.0
2.
7.6 1.61..
4.6
Falcon 7B
6.8 6.8 1.8
14.4 2.0
2.0
24.0 1 2.4
7.6 1 1.8
13.2 13.2 2.0
3.0 2.2
2.6
-1. 2.4
0.0 2.1
Falcon Instr.
3.6 1.4
7.6 1.8
5.2
1.5
2.0
8.2
1.6 2.6
3.0
1.0 23
2.8 28818
Falcon 40B
2.1
2.1
10.6
11.2 2.4
00.4
1.2 1.222
Fulcon 40B Instr.
6.8 + 1.4
21.8 1 2.2
10.4 1 1.8
0.8 # 1.6
40.4 - 2.4
4.2 1 2.0
344 044420
3.2 + 1.9
3.6 4.222
A Entropy
SST-2
Subj
E Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
-0.455 + 0.007
-0.177
0.008
-0.529
+ 0.012
-0.099 + 0.006
0.910 0.014
-0.004
-0.005
0.002
0.014:0.004
0.001 0.003
LLaMa-2 13B
+
-0.429
1.407
-0.204
-0.018
0.004
0.006
LLaMa-2 70B
-0.564
0.007
-0.500
0.007
-0.563
00.6330.000 0.010
-0.313 = 0.010
-1.46
-0,242
-0.074 + 0.006
1024660.000
-0.220 0.008
LLaMa 7B
-0.426 0.009
4.150
-0.225
-0.103 0.007
0.007
-0.497
0.003
-0.002
-0.001 0.003
0..00
-0.001 0.001
LLaMa 13B
0..36 0.009
-0.193
0.009
0.218
-0.112 0.007
0.567
-0.08 +.008
-0.014
-..012
0.004
0.001
L.aaMa 65B
-0.422 0.009
-0.283
0.008
-0.502 0.011
-0.273 + 0.009
-1.032
0.013
1.000
-0.040 + 0.005
-0.128 1128=00 0.008
-..553
0.008
Falcon 7B
-0.196 0.007
-0.109
0.006
0.006 -0.099
-0.046
0.005
-0.428 0.015
00,000 0.000
0.003 0.005
-0.002 0.004
-0.002 0.004
Falcon Instr.
0.2207 + 0.007
0.038
0.115
-0.411 0.006
0.200 0.011
0.000 0.000
0,007 0.006
1
-0,003 0.006
Falcon 40B
-0.382 + 0.009
-0.248
0.009
-0.358 0.010
-0.175 0.008
-0.923
0.015
0.002 0.004
0.001 0.006
0.008 0.005
-0.001 0.004
Falcon 40B Instr.
-0.396 0.008
-0.177
0.009
0.3113 +.0100
-0.202 + 0.009
-0.922
0.015
-0.039
0.006
-0.033
0.006
-0.012 0.005

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelihood f
Entropy
0.8
0.6-
I.aMa-2 13B
LLaMa-2 70B
Guessing
0.6
0.4-
0.4
0.2
50
100
50
100
50
100
LLaMa 7B
0.6-
LLaMa 13B
LLMMa
Guessing
0.4-
0.4-
3
0
20
40
20
40
20
40
Falcon 7B
0.8
0.6
Falcon Instr.
Falron 40R
B
Falcon 40B Instr.
0.6
Guessing
2-
0.4
33
0.2
50
No Observed In-Contes Examples
Figure F.13: Few-shot ICL on novel author identification task for all models. Models achieves accuracies significantly better than random guessing novel author identification task. Thus, ICL predictions must depend on the label relationship provided in-context. Thin lines are averages 500 runs, thick lines moving average (window size 5).
35

---

Published as a conference paper at ICLR 2024
Accuracy t
Log Likelihood
Entropy
Replacing default
(negative, positive) with:
I.LaMa-2 7B
(negative, positive)
0.84
(positive, negative)
(bad, good)
1S E
0.44
(good, bod)
B)
(B, A)
(green, blue)
0.7
(blue, green)
LLaMa-2 13B
-0.3
(sea, chair)
(chair, sea)
0.5-
-1.3
1.2
L.aaMa-2 70B
0.8-
0.5-
-1.5
0.4-
0.0
-3.3
Observed In- Co-Coeexxt Examples
Figure F.14: Few-shot ICL with replacement labels SST-2 for LLaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
36

---

Published as a conference paper at ICLR 2024
Accuracy f
Log Likelihood
Entropy
1.2
Replacing defauls
pasitive)
LLAMMa 7B
(aegative, positive)
(positive, negative)
(bad, good)
0.4
0.34
(good, bad)
(A, B)
(B, A)
0.7
(groen, blue)
(blue, green)
I.LaMa 13B
-0.0
(sca, chair)
0.84
0.54
(chair, sea)
m1.34
-0.5
1.2
L.LaMa 651
-1.5
0.4
0-3-
Halcon 7B
0.8
0.1-
1.2
Falcon 7B Instr.
-1.3-
0.4-
0.3
-9.5
Falcon 40B
0.8
0.3-
1.2
Falcon 40B Instr.
-0.3
1.33
0.44
0.3
No Obscrved In-Context Examples
Figure F.15: Few-shot ICL with replacement labels on SST-2 for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size for clarity.
37

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing defauit labels
(objective, subjective) with:
I.LaMa-2 7B
(objective, subjective)
(subjective, objective)
(bad, good)
(gond, bad)
(A, B)
(B,
(green, blue)
1.0
0.7
(blue, green)
LLaMa-2 13B
(sea, chair)
(chair, sea)
0.8-
0.3-
0.04
0.5
1.aaMa-2 70B
-05
0.3
0.4
40
0.14
No Observed In- Context Examples
Figure F.16: Few-shot ICL with replacement labels Subjectivity for LLaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
38

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
1.0
Replacing defanit
LLAMa 7B
00
(objuctivw, subjeesive)
(objective, subjective)
(subjective, objective)
0,6
(bad, good)
(good, bad)
(A, B)
(B,
(groen, blue)
(blue, green)
I.LaMa. 13B
0.0
chair)
(chair, sen)
0.0-
0.4
1.0
L.LaMa 65B
Halcon 7B
1.0
Falcon 7B Instr.
Falcon 40B
1.0
Falcon 40B Instr.
0.6
0.8-
-1.
0.4-
No Observed In- Context Examples
Figure F.17: Few-shot ICL with replacement labels Subjectivity for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
39

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing default labels
1.0-
(negative, neutral, positive) with:
L.- 7B
(negative, neutral, poeitive)
[neutral, positive, negarive)
(positive, negative, neutral)
(positivn, neutral, negative) (bad, ok, good)
(good, oks, bad)
(A, B, C)
1.0
(C, A, B)
LLaMa-2 13B
-0.5
[B, C, A)
0.4
-1.5
0.0
0.2
-2.5
0.2
1.04
Ma-2 70B
-1.5-
0.2
2.55
No Observed In- In-Context Examples
Figure F.18: Few-shot ICL with replacement labels Financial Phrasebank for LLaMa-2 mod- els. In addition to the default labels, we study variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size for clarity.
40

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing default labels
I.LaMs 7B
0.8
1.0
(ongative, nautral, positive)
(negative, neutral, positive)
(neutral, positive, negative)
11.5
(positive, negative, neutral)
(positive, neutral, negative)
(bad, ok, good)
(good, oks bad)
[A, C)
I.IaMaa 13B
-0..5
[C, B)
[B, A)
0.49
0.0
-0.5
0.0
L.aaaa 651
0.8
1.0
-1.5
0.0
-0.5
Falcon 7B
0.0-
0.0
-2.5
0.2
Faleon 7B Instr.
0.8
1.0
0.4
-1.5
-0.5
Falcon 40B
0.44
0.0-
0.0
-2.5
0.2
Falcon 4013 Instr.
0.8-
1.0-
0.4
-1.5
8.6-
20
30
10
20
30
20
30
No Observed In-Context Examples
Figure F.19: Few-shot ICL with replacement labels on Financial Phrasebank for LLaMa and Falcon models. In addition to the default labels, we study variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
41

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing default labels
(aot apocch, hate speech)
1.LaMa-2 7B
0.8-
hate speech, Itate speech)
0.6-
(bate spetch, Itate speech)
0.04
(bad, good)
-1.
(good, had)
0.44
0.49
(A, H]
(B,
0.9
0.5-
(green, blue)
(blue, green)
LLaMa-2 133
-0.5
(sea, chair)
(chair, sea)
1.0
0.5
1.JaMa-2 70B
0.8-
0.6-3
0.6-
0.44
0.4-
0.2
Observed Context Examples
Figure F.20: Few-shot ICL with replacement labels Hate Speech for LLaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
42

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
1.0
0.5
Replacing defanit lahele
-0
speach, spsnch)
LLAMaa 7B
late speech. hate speech)
(hate speech, not hate speech)
0,6
(bad, good)
0.4
(good, bad)
(A, B)
(B,
(green, blue)
(blue, green)
I.LaMa. 13B
(aca, chair)
0.64
(chair,
0.0-
1.0
-0.0
L.LaMa 65B
-10
0.4
Halcon 7B
0.6-
-1.0
1.0
0.5
Falcon 7B Instr.
-0.0
0.4-
Falcon 40B
0.6
1.0
0.5
Falcon 40B Instr.
0.85
0.8-
2.5
0.2
Obscrved In-Context Examples
Figure F.21: Few-shot ICL with replacement labels on Hate Speech for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
43

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing defauls labels
(world, aports, business, acience technology) with:
I.LaMa-2 7B
1.2
technology) (science technology, world, sports, business) (business, scienee and technology, world, sports) (sports, business, science and technology, world) (science and technology, business, sports, world)
(world, sports, business,
D) A)
0.0
LLaMa-2 133
I.aaM-22 70B
0.89
1.99
0.8
0.4-
0.44
0.0
20
0.0
No Observed Context Examples
Figure F22:: Few-shot ICL with replacement labels on AG News for 2LaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
44

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
f
Entropy
1.0
1.0
Replacing labele
technology) (world, sports, business, science and technology) (science and technology, world, sports, business)
sports,
LLAMMa 7B
(business, science and technology. world, sports) (sports, business, science and technology, world) (science and technology, business, sports, world) D) A)
0.2
0.5
I.LMMa 13B
0.0m
1.0
L.LaMa 651
0.2
0.5
Falcon 7B
0.6-
1.0
-0.2
1.0
3.0
Falcon 7B Instr.
0.34
0.84
Falcon 40B
0.6
1.0
1.0
2.8
Falcon 40B Instr.
0.0
No Observed In-Context Examples
Figure F:23: Few-shot ICL with replacement labels on AG News for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
45

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelihood f
Entropy
Replacing default labels (not equivalens; equivalent) with:
La- 7B
(not equivalent, equivalent)
0.7-
-
(equivalent, equivalent)
(bad, good)
(good, bad)
(A, B)
(B, A)
(green, blue)
0.0
0.7-
(blue, green)
LLaMa-2 13B
(sea, chair)
(chair, sea)
0.7
0.84
0.84
0.3
0.5
1a-aMa-2 70B
0.3
No Observed In-Context Examples
Figure F.24: Few-shot ICL with replacement labels Medical Questions Pairs for LLaMa-2 models. In addition to the default labels, we study variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size for clarity.
46

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood f
Entropy
1.0
00.
Replaciog default labela
(bot equivalent; equivalant)
I.LaMs 7B
(not equivalent, equivaleut)
0.0-
(equivalent, equivalent)
(bad, good)
0.4
(good, bad)
(A, B)
(B, A)
1.0
(green, blue)
(blue, groen)
I.IaMa 13B
(sea, chale)
-1.34
(chair, sea)
0.49
-2.3
0.3
-3.3
0.5
0.5
-0.3
L.aMaa 6513
0.6
0.4
00.5
Falcon 7B
0.89
-2.
0.2
0.2
0.0
Faleon 7B Instr.
00.3
-1.3-
0.5
0.4-
00.3
Falcon 40B
-1.3-
0.0
-2.3
0.2
-3.5
0.2
1.0
0.5
Falcon 40B Instr.
-033
0.4-
30
16
20
10
20 15
20
No Observed In-Context Examples
Figure F.25: Few-shot ICL with replacement labels on Medical Questions Pairs for LLaMa and Falcon models. In addition to the default labels, we study variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
47

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing default labels (aot equivalent, equivalent)
1.LaMa-2 7B
equivalent, equivalent)
0.7-
(equivalent, equivalent)
-135
(bad, good) (gnod,
0.54
(A,
A)
0.3
-3.5
0.5
(green, blue)
(blue, green)
LLaMa-2 13B
(eea, chair)
(chair, sea)
0.9
0.0
0.7
1.aaMa-2 70B
0.7-
0.5-
0.8-
0.8
-3.3
0.8
No Observed In-Context Examples
Figure F.26: Few-shot ICL with replacement labels MRPC for LLaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
48

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
H0
0.5
Replacing labele
equivalent, equivalant)
LLLMMa 7B
(not equivalent, equivaleut)
(equivalent, equivaleut)
(bad, good)
0.5
0.4
(good, bad)
(A, B)
(B, A)
(groen,
(blue, green)
I.LaMa 13B
(sea, chair)
0.75
0.61
(chair, sea)
-1.--
-3.
-0.3
L.LaMa 651
0.8
0.4
Halcon 7B
0.7
0.6-
0.0
00.5
0.5
Falcon 7B Instr.
0.8-
0.47
Falcon 40B
0.7
0.5
Falcon 40B Instr.
0.9
-0.3
0.5
0.84
0.2
No Observed In- In-Context Examples
Figure F.27: Few-shot ICL with replacement labels on MRPC for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size for clarity.
49

---

Published as a conference paper at ICLR 2024
Accurscy
Log Likelihood
Entropy
Replacing defauls labels
(entailment, estailment)
I.LaMa-2 7B
0.60
(entailment, entailment)
(not entaihment, entailmentt)
0.354
(bad, good)
(good, bad)
0.45
0.35
(green, blue)
(blue, green)
LLaMa-2 133
0.8
-0.5
(sea, chair)
(chair, sea)
-1.--
0.75
1.aaMa-2 70B
0.8
0.45
0.88
-1.5
0.15
0.49
0.15
0.2
+9.
0.38
Observed In-Context Examples
Figure F.28: Few-shot ICL with replacement labels RTE for LLaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
50

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Replacing defanls labelz
untailment)
LLAMa 7B
(entailment, nct eutailmeut)
(bot entailment, eutailmeut)
(bad, good)
0.44
(good, bad)
B)
A)
(green,
0.8
(blue, green)
I.LaMa 13B
(sca, chair)
0.49
0.0-
(chale,
0.4
L.LaMa 651
0.8
Halcon 7B
0.4
0.0-
0.4
0.8
Falcon 7B Instr.
0.8
Falcon 40B
0.89
0.0
0.4
Falcon 40B Instr.
0..5
No Observed In-Context Examples
Figure F.29: Few-shot ICL with replacement labels RTE for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
51

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
0.8-
005
Replacing defauls labels
(not eatailment. cstailment)
I.LaMa-2 7B
0.60
entailment, entailment)
(entailment, entailmentt)
0.35
(bad, good)
(good, bad)
0.4-
0.45
0.9
0.335
(green, blue)
(blue, green)
LLaMa-2 13B
(sea, chair)
(chair, sea)
0.46
0.75
0.8
-0.5
L.aaM-22 70B
0.48
0.6-
-1.5
0.15
0.15
0.2
-0. 0
20
0.38
Observed In-Context Examples
Figure F.30: Few-shot ICL with replacement labels WNLI for LLaMa-2 models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
52

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
1.0
Replasing defanls labelz antailment,
LLAMa 7B
(not entailment, eutailmeut)
(entailment. eutailmeut)
0,6
(bad, good)
01-4
(good, bad)
B)
A)
(green,
(blue, green)
I.LaMa 13B
(sca, chair)
0.04
(chale,
0.0
1.0
L.LaMa 65B
0..
Falcon 7B
0.0-
1.0
0.88
Falcon 7B Instr.
Falcon 40B
0.0
Falcon 40B Instr.
1.0
0..5
0.8-
No Observed In-Context Examples
Figure F:31: Few-shot ICL with replacement labels on WNLI for LLaMa and Falcon models. In addition to the default labels, we study a variety of replacement labels (see legend). We average over 100 random subsets and then additionally apply moving averages (window size for clarity.
53

---

Table F.2: Full summary statistics for the flipped label experiments of $7. We strongly encourage readers to also view the full training curves across all possible context sizes and adittional replacement labels in $F. Across all metrics, here show the average difference between the default and flipped label scenario. We compute 'Metri(Deefault) Metric(Flipped)' such that positive accuracy/log likelihood differences indicate that ICL performs worse with flipped labels. We compute differences at the maximum context size for each task-mode combination (cf.şE and Table E.1). We display experiments where ICL accuracies log likelihoods on the default labels do not significantly exceed random guessing performance in lightgray. Across models, and metrics, model performance is usually significantly worse when labels are flipped (and default performance exceeds random guessing). In particular for entropies, we observe significant differences between ICI. predictions. We display mean differences and standard errors over 100 runs. We bold entries for which mean differences are statistically significant.
A Log Lik.
SST-2
Subj
Phraseb.
Hate Speech
News
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
0.41 0.02
-0.04 + 0.02
0.36 0.03
0.26 + 0.01
0.87 0.03
0.01 0.00
-0.02
0.22 + 0.01
-0.00 0.01
LLaMa-2 13B
0.31 + 0.02
0.03 = 0.02
0.30 0.02
0.26 0.02
0,90 0.03
0.15 0.01
0.15
0.38 0.02
0.20 0.02
LLaMa-2 70B
0.08
0.90
0.24 0.02
0.15 0.02
+ 0.02
0.08
LLaMM 7B
0.36
0.13
0.53 = 0.02
0.31 1 0.02
1.06 0.03
0.06 0.00
0.06 0.01
0.14
-0.08 0.01
LLaMa 13B
0.40
0.09 0.02
0.51
0
1.18
0
0.16 0.01
+:00
0.05
LLaMa 65B
0.02
0.16
0.09
1.02
0.39 0.02
0.31 0.31
0.41 +
0.20 + 0.02
Falcon 7B
0.48
0.37
0.58 0.02
0.09
1.08
0.05 0.01
0.02 0:00
0.01 0.05 0.01
0.12 0.01
Falcon 7B Instr.
0.02
0.48 - 0.01
0.77 = 0.02
0.52
2.29
-0.07 0.01
0.01
0.13
0.22 0.02
Falcon 40B
0.33
0.25
0.74
0.27
1.00
0.20 0.01
0.19
0.39 0.01
0.09 0.01
Falcon 40B Instr.
0.45 + 0.02
0.37
0.02
0.94 0.03
0.68 0.68 0.02
0.89 1 0.03
0.48 0.02
0.15 0.01
0.70 1 0.02
0.19 - 0.01
à Accuracy
SST-2
Subj
F. Phraseb.
Hate Speech
News
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
34.0 :: 5.3
+ 4.3
15.0 t 4.3
26.0 = 6.9
47.0 5.6
1.0
-5.0 5.9
$2.0 + 7.5
1.0
LLaMa-2 13B
14.0
1. 1.0
12.0 +
18.0 -
37.0
22.1
20.0 4.3
36.0
7.0
27.0
LLaMa-2 70B
1.4
10
13.0 + 3.4
5..
11.04.4
20.0 5.5
7.0
5.0
L. 7H
22.0
3.0++4.. 4.8
36.0 + 5.9
27.0 = 6.6
64.0
6.0 6.6
13.0 6.6
19.0
7.0
0.0 + 8.6
LLaMa 13B
26.0 5.8
7.0 3.5
26.0 +
36.0 5.9
55.0
16.0
12.0
28.0
6.0 -
LLaMa 65B
15.0
8.0 3.7
27.0 5.3
14.0
50.0
29.0
41.0
46.0 6.7
30.0 6.1
Falcon 7B
49.0 5.9
39.0 4.9
34.0 1 6.7
20.0 = 8.4
63.0 6.7
13.0 82
7.0 10-4-00 4.5
10.0
6.1
12.0 + 8.3
Falcon Instr.
63.0 5.8
46.0 7.0
48.0
6.6
42.0 7.4
71.0
-1.00 7.
-9.
11.0
17.0
Falcon 40B
15.0 4.3
17.0 4.5
36.0
58.0
28.0 6.8
32.0 -
25.0
16.0
Fulcon 40B Instr.
29.0 + 5.3
30.0 1 5.2
45.0 1 5.5
53.0 - 6.8
49.0 6.9
45.0 1 7.0
11.0
48.0 + 7.0
22.0 + 7.6
A Entropy
SST-2
Subj
F. Phraseb.
Hate Speech
AG News
MQP
MRPC
RTE
WNLI
LLaMa-2 7B
0.519 0.022
0.010
0.397 + 0.026
-0.100
0.013
0.779
0.030
-0.003 0.005
0.049 0.005
-0.027
0.008
-0.006 0.008
LLaMa-2 13B
-0.479
-0.058
-0.475
-0.194
1.677:00000
0.011
LLaMa-2 70B
-.006
00.00
0.029 -0.398
0.018 -0.255
-0.995 0.024
-0,243 0.019
-0.150 + 0.015
-0.258
0.019
-..209 = 0.017
LLaMa 7B
-0.034
-0.01
-0.355 0.023
-0.645 0.033
0.007 4.0000
-0.032 +: 0.005
-0.005
0.0011
LLaMa 13B
0.424
-0.005 0.020
0..337 0.022
0.151
0.731 0.033
0.008 0.009
00.00 0.008
-0,009
L.aaMa 65B
-0.345
0.020
0.024 -0.228
-0.273
0.020
-1.041 0.029
0.128 0.019
- 0.015
0..005
0.020
-..19 0.019
Falcon 7B
0.276
-0.118
-0.022 0.018
-0.062 0.013
-0.518 = 0.037
0.002 0.009
0.005 0.006
-0.004 0.010
-0.015
Falcon Instr.
0.366 0.020
0.068
0.012
0.327
0.016
0.220 0.028
0.024
0.127 0.015
0.005 0.0015
0.001 0.020
Falcon 40B
-0.392 0.020
-..233
-0.420
0.027
-0.190 0.022
-0.904
0.033
-0.003 0.010
-0.103 0.012
-0.118 0.014
-0.001 0.033
Falcon 40B Instr.
-0.483 0.020
-0.160
-0.430 + 0.028
0.012
-0.916 0.033
-0.098 0.017
-0.017
-0.063 0.017
-0.013

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
D D-F
LLaMa-2 70B
-0.4
0.0-
F
Alternate F D
-00
Changepolat
Changepoint
0.4
Guessing
-1.4
-10
0.2
1.00
0.7
LLaMa 05B
-0.4
-0.0
-1.0
20
60
10
60
40
6.8
-0.2
Falcon 40B
-0.3
-0.4
-0.5
-0.6
-0.7
0.5
0.0
20
N2 Obscrved In-Context Examples
Figure F332: Few-shot ICL on SST-2 when the label relationship changes throughout ICL. For (D F), we start with default labels and change flipped labels at the changepoint, for (F D) we change from flipped to the default labels at the changepoint, and for (Alternating F ++ D) alternate between the two label relationships after every observation. For all setups, '2 Changepoint' the LLMs have observed the same number of examples for both label relations. If, according NH3, ICL treats all in-context information equally, predictions should be equal that point-but they are not. Averages over 500 repetitions, we apply moving averages (window size 3), and thin lines are bootstrapped 99 confidence intervals, thin dashed lines are mean results without moving average smoothing.
55

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
1.44
D D-F
LLaMa-2 70B
-0.23
F
-0.50
Alternate F D
Changepolat Changepoint
0.50
-1.00
0.45-
Guessing
-1.25
0. 40-0
30-
1.95
0.30
0.0
6.9
0.60-
LLaMa 05B
-0.6
0.50
6.4-
-0.7
0.50-
6.4-
-0.9
0.50-
-1.0
6.0
30
10
30
1.0
-0.3
0.660
Falcon 40B
0. 0.570
-0.5
0.550
-0.6
0.528
0.88
1.0
N2 Obscrved In-Context Examples
Figure F.33: Few-shot ICL on Subjectivity when the label relationship changes throughout ICL.
For (D DD+FF,, F), start with default labels and change flipped labels the changepoint, for (F + D) we change from flipped to the default labels at the changepoint, and for (Alternating F <> D) we alternate between the label relationships after every observation. For all setups, at '2 Changepoint' the LLMs have observed the same number of examples for both label relations. If, according NH3, ICL treats all in-context information equally, predictions should be equal that point-but they are not. Averages over 500 repetitions, we apply moving averages (window size 3), and thin lines are bootstrapped 99 confidence intervals, thin dashed lines are mean results without moving average smoothing.
56

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelihood
Entropy
-0.23
D D-F
LLaMa-2 70B
F
Alternate F D
Changepolat
1.00
Changepoint
0.7-
Guessing
-1.75
0.6
-0.25
LLaMa 05B
6.4-
-0.50
0..0
-1.00
6.4-
-1.20
-1.50
1.95
0.65
-5.00
-2.25
0.31
30
10
30
100
0.0-
0.90
Falcon 40B
-1.0
0.50
-1.5
0.50
0.70
20
1.0
N2 Obscrved In-Context Examples
Figure F334: Few-shot ICL on Financial Phrasebank when the label relationship changes throughout ICL. For (D F), start with default labels and change flipped labels at the changepoint, for (F + D) we change from flipped to the default labels at the changepoint, and for (Alternating F F++ D) we alternate between the two label relationships after every observation. For all setups. at '2 Changepoint" the LLMs have observed the same number of examples for both label relations. according NH3, ICL treats all in-context information equally, predictions should be equal at that puint-but they are not. Averages over 500 repetitions, we apply moving averages (window size 3). and thin lines are bootstrapped 99 % confidence intervals, thin dashed lines are mean results without moving average smoothing.
57

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelihood f
Entropy
0.9
D D-F
LLaMa-2 7013
F
Alternate F D
-04-
Changepolat
0.6-
0.5-
Changepoint
Guessing
0.54
-1.0
0.44
0.4-
0.45-
LLaMa 05B
-0.0
+1.
6.55
15
20
10
15
20
20
20
-0.4
0.40
0.40
-0.0
0.00
Falcon 40B
6.75
6.70-
-0.3
6.45-
6.40-
6:40
6.30
20
-30
10
15
0.25
N2 Observed In-Context Examples
Figure F.35: Few-shot ICL on Hate Speech when the label relationship changes throughout ICL.
For (D)F),, start with default labels and change flipped labels the changepoint, for (F + D) we change from flipped to the default labels at the changepoint, and for (Alternating F <> D) we alternate between the label relationships after every observation. For all setups, at '2 Changepoint' the LLMs have observed the same number of examples for both label relations. If, according NH3, ICL treats all in-context information equally, predictions should be equal that point-but they are not. Averages over 500 repetitions, we apply moving averages (window size 3), and thin lines are bootstrapped 99 confidence intervals, thin dashed lines are mean results without moving average smoothing.
58

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
D D-F
LLaMa-2 70B
-0.4
F-D
Alternate F D
-0.0
Changepolat Changepoint
-0.8
0.0-
Guessing
0.4
-1.0
0.44
0.4-
0.5
00.00
1.2-
LLaMa 05B
1.0
-1.23
1.95
-2.00
15
20
15
20
10
20
-0.4
-0.0
Falcon 40B
0-7-
0.4
1..0
1.0-
-12
0.8-
1.5
10
16
20
10
N2 Obscrved In-Context Examples
Figure F.36: Few-shot ICI. on AG News when the label relationship changes throughout ICL.
For (D+FF) F), start with default labels and change flipped labels the changepoint, for (F + D) we change from flipped to the default labels at the changepoint, and for (Alternating F <> D) we alternate between the label relationships after every observation. For all setups, at '2 Changepoint' the LLMs have observed the same number of examples for both label relations. If, according NH3, ICL treats all in-context information equally, predictions should be equal that point-but they are not. Averages over 500 repetitions, we apply moving averages (window size 3), and thin lines are bootstrapped 99 confidence intervals, thin dashed lines are mean results without moving average smoothing.
59

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Flip After 10
6-7-
Alternate e D
Changepoint 2 Changepoint
-0.8
Guessing
0.4
-1.9
10
10
20
10
6.9-
Flip After 20
-0.6
0.5
-1.0
Flip After 30
6-4-
0..0
6.0-
-0.8
6.8
1.2
0.3
6.2
70 40 60
No Observed Tn-Context Examples
Figure F.37: Few-shot ICL on Hate Speech with LLaMa-2-70B when the label relationship changes throughout ICI. We here investigate a three different changepoints, flipping labels af- ter 10, 20, or 30 in-context observations. For (D F), we start with default labels and change to flipped labels at the changepoint, for (F > D) we change from flipped to the default labels at the changepoint, and for (Alternating + D) we alternate between the two label relationships after every observation. For all setups, at '2 Changepoint' the LLMs have observed the same number of examples for both label relations. If, according to NH3, ICL treats all in-context information equally, predictions should be equal that point, regardless of whether the changepoint is after 10, 20, or 30 observations. However, this is not the case. In particular, predictions are significantly different when the changepoint is 30. Averages over 500 repetitions, we apply moving averages (window size 3), thin lines are bootstrapped 99 % confidence intervals, thin dashed lines are mean results without moving average smoothing.
60

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelibood f
Entropy
LLaMa-2 70B
0.75
0.5
0.6
Flipped w/ Prompt
Instruct
0.50
1.0
0.418
Ignore
Invert
0.25
Default
0.2-
Gueeting
50
100
50
100
50
100
LLaMa 65B
0.75
0.5
0.50-
1.0
0.4
0.25-
1.5-
0.2
25
50
25
50
25
50
Faleon 40B
0.75
0.5
0.6
0.50
1.0
0.25
0.2-
25
50
25
50
0
25
50
Falcon 40B Instr.
0
0.64
0.75-
0.50
1100
0.4-
0.25-
-1.5
0.2
25
50
25
50
25
50
No Observed In-Context Examples
Figure F338: Prompted few-shot ICL with flipped labels SST-2. Some prompts are able to im- prove ICL on flipped labels compared to not using prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
61

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelibood f
Entropy
1.00
LLaMa-2 70B
0.6
Flipped w/ Prompt
0.75-
0.5
Instruct
Ignore
Invert
0.50
1.0-
0.2
Default
Gueeting
50
50
0
50
1.00
LLaMa 65B
0.6
0.75-
0.44
0.50
-1.0
0.2
10
20
30
10
20
30
30
1.00
Falcon 40B
0.6
0.75-
0.5
0.50
1.0
0.2-
10
20
30
20
30
10
20
30
Falcon 40B Instr:
1.00
0.6
0.75+e
0.1-
0.50
-1.0-
10
20
30
20
30
10
20
30
No Observed In-Context Examples
Figure F.39; Prompted few-shot ICL with flipped labels Subjectivity. Some prompts are able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
62

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelibood f
Entropy
1.00
LLaMa-2 70B
Flipped w/ Prompt
0.75
Instruct
-1.0
Ignore
0.50
Invert
0.50
Default
Gueeting
50
50
50
1.00
LLaMa 65B
0.5
0.75-
0.75
-1.0-
0.50-
11.5
10
30
10
20
30
10
20
30
1.00
Falcon 40B
005
0.75-
0.75
-1.0-
0.50-
0.50
-1.5
20
40
20
40
20
40
Falcon 40B Instr.
1.00
0.5
0.75-
0.75-
-1.0
0.50-
0.50
-1.5
20
40
20
40
20
40
N Observed In-Context Examples
Figure F.40: Prompted few-shot ICL with flipped labels on Financial Phrasebank. Some prompts are able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
63

---

Published as a conference paper at ICLR 2024
Aocuracy 1
Log Likelibood f
Entropy
LLaMa-2 70B
0.75-
0.6
Flipped w/ Prompt
Instruct
Ignore
0.4+8
Invert
Default
0.25
Gueeting
50
50
0
50
LLaMa 65B
0.6
0.50
0.4
-
0.25
-2
10
20
30
10
20
30
10
20
30
Falcon 40B
0.75-
0.6
0.50
0.4
0.25
10
20
30
10
20
30
10
20
30
Falcon 40B Instr.
0.75
0.6
0.50
0.4
0.25-
10
20
30
20
30
10
20
30
N Observed In-Context Examples
Figure F.41: Prompted few-shot ICL with flipped labels Hate Speech. Some prompts are able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
64

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelibood f
Entropy
LLaMa-2 70B
Plipped w/ Prompt
1.0
Instruct
0.50
Ignore
Invert
0.25
0.5
Default
Guesting
20
40
20
40
20
LLaMa 65B
0.75
1.0-
0.50
0.25
0.5
3-
10
20
20
Faleon 40B
0.75
1.0-
0.50
-2
0.25
0.5
10
20
10
20
10
20
Falcon 40B Instr.
0.75
0.50
0.25
0.5
10
20
20
10
N2 Observed In-Context Examples
Figure F.42: Prompted few-shot ICL with flipped labels AG News. Some prompts able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
65

---

Published as a conference paper at ICLR 2024
Accuracy f
Log Likelibood f
Entropy
LLaMa-2 70B
Plipped w/ Prompt
0.75
Instruct
Ignore
0.50
Invert
0.4-
Default
22
Gueesing
20
40
20
40
20
LLaMa 65B
0.75
0.6
0.50
0.4-
0.25
10
20
10
20
Falcon 40B
0.75
0.50
0.14
0.25
-2
10
20
10
20
20
Falcon 40B Iastr.
0.6
0.50
0.44
0.25-
10
20
10
20
10
20
N Observed In-Context Examples
Prompted few-shot ICL with flipped labels Medical Questions Pairs. Some prompts are able to improve ICL on flipped labels compared to not using prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving aver- ages (window size 5) for clarity.
Figure F.43:
66

---

Published as a conference paper at ICLR 2024
Accuracy f
Log Likelihood f
Entropy
0.8
LLaMa-2 70B
Flipped w/ Prompt
0.6
0.6
Instruct
Ignore
-10-
Invert
0.4-
Default
Gueeting
10
20
30
10
20
30
10
20
30
0.8
0..5
LLaMa 65B
0.6
-1.0-9
0.4
0.4
10
20
20
10
20
0.8
-0.5
Falcon 40B
0.6
-1.0-
0.4
10
20
10
20
10
20
Falcon 40B Instr:
0.8
00.5
0.6-0
0
-1.0-
0.4+%
0.4
10
20
10
20
10
20
N Observed In-Context Examples
Prompted few-shot ICL with flipped labels MRPC. Some prompts able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
Figure F.44:
67

---

Published as a conference paper at ICLR 2024
Accuracy 1
Log Likelibood f
Entropy
LLaMa-2 70B
0.75
Flipped w/ Prompt
Instruct
Ignore
0.50
Invert
Default
0.25
2
Gueeting
10
20
10
20
10
20
LLaMa 65B
0.75
0.6
0.50
0.41
0.25
10
0
10
Falcon 40B
0.75
0.6
8
0.50
0.4
0.25
10
6
10
Falcon 40B Instr.
0.75
0.6-
0.4
0.25
5
10
10
No Observed In-Context Examples
Figure R45: Prompted few-shot ICL with flipped labels on RTE. Some prompts are able to improve ICL on flipped labels compared to not using a prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
68

---

Published as a conference paper at ICLR 2024
Accuracy f
Log Likelibood f
Entropy
LLaMa-2 70B
0.75-
0.5
Flipped w/ Prompt
0.6
Instruct
0.50
-1.0-
Ignore
Invert
-1.5
0.4-
Default
0.25
Gueeting
25
50
25
50
25
50
LLaMa 65B
0.75
0.5
0.6
-1.0-
0.50
-1.5
0.4-
0.25-
10
20
10
20
0.75
0.5
Faleon 40B
0.6
-1.0
0.50
-1.5
0.4
0.25-
10
20
30
10
20
30
10
20
30
Falcon 40B Instr.
0.75
00.5
0.6
-1.0
0.50
-1.5 -1.5
0.14
0.25-
10
20
10
20
10
20
30
No Observed In-Context Examples
Figure F.46: Prompted few-shot ICL with flipped labels WNLI. Some prompts are able to im- prove ICL on flipped labels compared to not using prompt as before (label none in the figure). We average over 100 random subsets and then additionally apply moving averages (window size 5) for clarity.
69

---

Published as a conference paper at ICLR 2024
G ADDITIONAL EXPERIMENTS
In this section, we provide additional experiments and insights into label learning in ICL.
G.1 PROGRESSIVE RANDOMIZATION
Previously, have observed that ICL performance suffers when all labels in the context domized. In this section, we study the changes to ICL predictions when gradually increasing the proportion of in-context examples with random labels. This is of practical relevance noisy labels are a common concern in many applications.
Figure G.1 gives the performance of ICL for LLLMM-2-700 SST-2 at different noise levels. We observe that probabilistic log likelihood immediately and clearly degrades, even at our smallest pro- portion of random labels of 0.2. This makes sense intuitively, as we would expect model predictions to become less certain in the face of noisy label observations. In contrast, accuracy stays relatively constant until the noise level reaches proportion of 0.6, i.e. more than half the in-context points are noisy.
G.2 CALIBRATED LABEL FLIPPING
We here repeat our label flipping experiments from $7 regarding NH2 with probabilities calibrated according to the approach of Zhao et al. (2021) to see calibrated predictions can overcome the model's pre-training preference.
Figure G.2 is a reproduction of Fig. 5 with calibrated probabilities. We observe that differences between the calibrated and uncalibrated versions generally small: find almost differ- ences for SST-2/Falcon-40B and small improvements for all labels for Hate Speech/LLaMa-651 and MQP/LLaMa-2-700B Most importantly, we still observe large differences performance be- tween the different label setups at maximum context size, such that, here, NH2 remains rejected for ICL with calibrated predictions.
We do not necessarily find these results surprising. Zhao et al. (2021) focus on scenarios with small in-context dataset sizes (up to 16). They show that this is where calibration is most important. Already for 16 in-context demonstrations, the gains from calibration are insignificant, see, e.g. their Figure 1. In contrast, we study ICL at much larger numbers of in-context demonstrations, Thus, we are not surprised that our conclusions do not change with calibration.
G.3 LABEL FLIPPING FOR AUTHOR IDENTIFICATION
In Fig. G.3, present results for our label flipping experiments (NH2, $7) with LLaMa-2 models on our novel author identification task ($6). Generally, we observe that accuracy is very similar across different label scenarios. However, small differences clearly for entropy). Interestingly, these differences are much smaller than in most of our previous experiments. This provides additional support for the intuitions underlying $7: we would expect the
visible for the probabilistic metrics (most
Accuracy
Log Likellhhod
Entropy
1.0
Random Propoction
-0.2
0.6
0.0
0.4+
0.4-
-0.6-
0.6
50
100
0
50
100
50
100
Figure G.I: Few-shot ICL with different proportions of randomized labels for SST-2 and LLaMa- 2-70B. As the proportions of random labels increases, predictions become less certain. Accuracies stay relatively constant, until label noise increases above 50 %. We average over 500 random context datasets.
70

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood
Entropy
Falcon 40B
SST-2
- 0.5
0.6
Label Relationship
Default
0.6-
1.0-
Flipped
0.3
0.3-
Arbitrary
25
50
1.5
25
50
25
50
Arhitrary Plipped
MQP Hate Speech
LLaMa-2 70B LLaMa 65B
0.8
0.6
0.5
0.4
2
30
15
30
15
30
0.8
0.5
0.6
1.0
0.5+
1.5
0.4
20
40
20
40
0
20
40
No Observed Tn-Context Examples
Figure G.2: Reproduction of Fig. 5 with calibrated probabilities according Zhao et al. (2021). Few-shot ICL with replacement labels for Falcon-40B on SST-2, LLaMa-2-65B on Hate Speech, and LLaMa-2-70B on MQP. Results are largely similar the original figure Fig. 5 and NH2 remains rejected. Averages over 100 runs and thick lines with moving average (window size 5).
model to only suffer significantly from flipped or replaced in-context labels if the pre-training data actually suggests to prefer particular label setup over another. For the author identification task, which is guaranteed to not be part of the training data (cf. $6), it makes sense that ICL would have a much weaker preference for any particular label setup.
We can also discuss the small differences that we do observe between label setups: (1) Especially initially, the performance for the arbitrary/arbitarry flipped labels, i.e. A/B or B/A, can be lower than the performance for the default/flipedd labels, which are the authors' first names. It seems the model prefers assigning the sentences to 'names' rather than 'symbols'. makes sense that this is a bias that would emerge from the pre-training data. (2) Further, we observe that the larger LLaMa-2 models can slightly prefer the default label direction. We believe that a possible explanation for this is that, sometimes, the sentences in dataset contain the authors name, c.g. 'Hi Author 1' in sentence from Author 2. This could confuse the model for the flipped label direction, in particular, if the model has some general understanding of author identification tasks. Lastly, we note that, to reject NH2, it is sufficient to find any dataset on which it does not hold, such that our conclusions in $7 remain valid regardless.
G.4 ANSWER IN CONTEXT
To better understand how ICI. leverages in-context label information, in this section, we study ICL. predictions when one (or multiple) of the in-context examples exactly match the test query. That is, the model can always achieve perfect accuracy on the test query by looking up the label of the exact match in the in-context training set. We examine this across different label setups to investigate how pre-training preferences affect prediction behavior of the model here.
Figure G.4 shows results for SST-2 at 10 in-context examples averaged across our selection of mod- els. We find the following behavior: (1) A single repetition of the test query (in the training set) increases performance for arbitrary and flipped labels but not for default labels, where performance is high already. Further, absolute performance remains highest for the default labels. We do not observe perfect accuracy, i.c. perfect copying behavior, for any of the label setups. This means ICL does not implement a nearest neighbor predictor, and single observations have but limited influence on the decision function. This behavior seems generally sensible, in particular, if the model expects that single examples can be mislabelled. (2) For multiple (here, four) repetitions of the test query we observe perfect accuracy for all label setups. Variance across the different models is negligible. This behavior reasonable, as we would expect that multiple observations the same input with consistent labeling should lead to a confident prediction of that label for that input.
71

---

Published as a conference paper at ICLR 2024
Accuracy
Log Likelihood f
Entropy
LLAMa-22 70B LLaMa-2 13B LLaMa-2 713
Label Relationship
0.75
0.50
Default
Flipped
0.50
2
0.25-
Arbitrary
Arbitrary Plipped
50
50
50
0.75
0.50
0.50
2-
0.25
50
50
50
0.75
1-
0.50
0.50
0.25-
50
50
50
No Observed In-Context Examples
Figure G.3: Few-shot ICL with replacement labels for LLaMa-2 at different sizes on our novel author identification task. We find that differences between different label setups are much smaller than in most previous experiments. This conforms to our expectations: the author identification task purposefully chosen such that it is novel, and as such, we would not expect the LLMs to have a strong preference towards any particular label relationship. Averages over 100 runs and thick lines with moving average (window size 5).
1.0
Default
Arbitrary
Flip
Accuracy
0.6
0.4
0.2
0.0
None
One
Four
Repetitions
Figure G.4: Few-Shot ICL for answer in context experiment for SST-2 averaged across our usual selection of models and for three different label setups. For each input, we add (No, One, Four) 'repetitions' of the exact test query to the in-context examples. For one or more repetitions, ICI. could achieve perfect accuracy by looking up the exact match and predicting its label. We find that one repetition improves performance for arbitrary or flipped replacement labels, but that robust copying behavior, i.c. perfect accuracy, emerges only for multiple repetitions of the test query.
72


Reviews:
[{"review_id": "VgmtsdLQ6U", "reviewer": "Reviewer_LQGJ", "summary": "This paper investigates different aspect of ICL, including does ICL rely on label information, does ICL view different orders of examples the same, can ICL overcome the pre-training preference. By using extensive study, the authors show that ICL do rely on label information, ICL treat different position differently, and ICL cannot overcome the pre-training preference. Thus reaching the conclusion that ICL is not conventional learning.", "questions": "Please see the weakness part, thanks! In addition, in the NH2 --- the ICL cannot overcome pre-trained preferences-- did you try the calibrated version of ICL, proposed by [1]? I am not sure if NH2 is still rejected if you calibrate the output. \n\n[1] Calibrate Before Use: Improving Few-Shot Performance of Language Models, Zhao et al, ICML 2021.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I really appreciate the number of experiments done in this paper. The authors try lots of parameters and spend lots of time to do the experiments (including ablation studies). These experiments make the paper sound and solid. Besides, the presentation is generally good. The argument that: ICL cannot overcome the pretraining preference is very interesting!", "weaknesses": "My main concern is that: although some of the claims in this paper are interesting, the depth of the explanation/experiment design is not enough. For example, in Min et al, even in their experiment results (just focusing on the accuracy), the random label ICL performance and the true label ICL performance are not the same (I remember in Min et al, claim that many tasks’ acc difference is less than 5%, but it is not a small number of classification tasks), and on some tasks, they are different very different. Thus, one can already reach the conclusion that ICL needs to use the label information **to some extent**. However, people do not know how ICL utilizes this label information. The experiment results in this paper are not enough to tell people how ICL utilizes this label information. It only shows that under another metric (the probability), using true labels and random labels is very different. It seems more like an ablation study of Min et al. Another claim for the position importance of different ICL examples, the author claims that ICL does not treat each in-context example equally. From my side, it seems to be a well-known thing, since people change the order of the ICL examples and can observe huge differences in the prediction accuracy [1]. However, people don’t know how ICL treats these examples. I don’t think the experiment in this section deepens our understanding of how/why ICL treats different in-context examples differently. It seems to use new experiments to support some known or folklore claims.\n\nI suggest the author to emphasize some new/deeper implications from their experiments in the rebuttal session (for NH1 and NH3). NH2 seems new to me and this finding is v. interesting (but i am not sure if other people already know this).\n\n[1] An Explanation of In-context Learning as Implicit Bayesian Inference, Xie et al", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7xmEEPe6vJ", "reviewer": "Reviewer_WYZX", "summary": "In-context learning is an important property of LLMs and is often utilized to improve performance. However, there is disagreement on how in-context learning works, with some papers drawing analogy to SGD, others to kernel regression, Bayesian inference, etc. To understand how ICL works, this paper identifies several assumptions of expected behavior of \"conventional learning algorithms\", e.g., models that learn P(y | x) from x_train, y_train ~ P and tests if ICL follows such behaviors. There are several questions proposed for the classification setting: do model predictions depend on ICL example labels? Can ICL work on tasks that are not seen in the pre-training corpus? Can ICL ignore bias from pre-training data? And does ICL treat all in-context information equally? \n\nThrough a systematic evaluation of several models and classification tasks, the paper concludes that 1) ICL depends on the in-context labels 2) ICL can work on tasks not seen in pre-training corpus 3) the pre-training preference matters, and 4) not all information is regarded equally. This refutes the findings of a previous paper (Min et. al.) and has implications for alignment and robustness of LLMs. More generally, it imposes some conditions on the loose analogy of ICL as a conventional learning algorithm.", "questions": "- This observation that larger models are more sensitive to label randomization is concerning. I would have expected that with more pre-training data and parameters, the model would be more confident in its prediction and less influenced by random noise in the in-context examples. On the other hand, I do buy the argument that for small models and hard tasks, changing the in-context examples significantly will still result in the model doing poorly, thus appearing to not be as impacted by the random labels. Do you have any thoughts on how to interpret this? \n\n- See \"Quality\" section of Weaknesses. Would be curious to see what these behaviors are.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality:\n- Breaking down the understanding of ICL into parallels with conventional learning algorithms and experiments on label relationships is quite novel to me. It reminds me a bit of Zhang et. al (2016)'s famous paper on rethinking generalization. \n\nQuality:\n- Thorough experimental results, well-designed experiments. \n\nClarity:\n- The paper was very clear and I enjoyed reading it overall. \n\nSignificance:\n- The conclusions are pretty valuable and suggest that we should reevaluate our expectations of alignment and robustness of LLMs via ICL.", "weaknesses": "Originality:\n- Some of the results on their own are not surprising, such as the model paying more attention to the last in-context example. However, I think the study overall is quite novel and rigorous. \n\nQuality:\n- For section 5, it would be nice to show how the model performance degrades as randomization increases. Similarly in section 7, you could try gradually mixing in the flipped labels or the arbitrary labels. I think this could help us study the robustness of ICL (in contrast to deep learning models or conventional learning algorithms), as in-context information can often be noisy in general. For instance, I am thinking of an example where the human prompter is giving examples of their preferences to align a model, and fails to accurately articulate one or two examples. \n- Another experiment I'd be interested in seeing: can you study the Lipschitzness/label consistency of ICL? For example, you can come up with a setting like \"x1 y1 x2 y2 .... xtest ?\" where xtest is a perturbation of one of the x_i's (e.g. a slight rewording, or very close by in embedding space). Will the model predict ytest to be equal to y_i? And will this happen even if we scramble the labels? How different from x_i does x_test need to be in order for it to no longer be significantly influenced by a flipped y_i? Overall I think extending your study to consider the relationships among the x_i's and the x_test w.r.t. the label is interesting, since this is a property of conventional learning algorithms (inputs with similar feature vectors have similar labels). \n\nClarity:\n- Minor nit: I'd prefer some more signposting about how a conventional learning algorithm is defined, or some notation of (x_ICL, y_ICL) versus conventional (x_train, y_train). Is a deep learning model considered a conventional learner if it can fit to arbitrary noise? \n- In section 5 I'd appreciate an example of what randomizing the labels exactly means. I had it confused with the experiment in section 7, whose arbitrary (A, B) experiment could be thought of as randomizing the label space but not the label relationships. \n- While I found the experiment in section 6 to be quite clever, I feel like \"novel task we create\" can be made a bit more explicit by specifying it's based on your private data. Without any elaboration, I was surprised about how you were so certain about the task.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IEgvoZeKJi", "reviewer": "Reviewer_tkcu", "summary": "This paper offers an empirical study into characteristics of ICL compared to in-context learning. The authors start with 3 null hypotheses about how ICL works, and empirically address those hypotheses.\nH1. ICL does not learn p(yIx). This Hypothesis is rejected through an experiment with label randomization.\nH2. ICL can overcome model priors that gained through pre-training about label semantics. This hypothesis is rejected because changing label words in ICL examples to have neutral or opposite meanings does affect the performance of the model.\nH3. ICL example order does not matter. This hypothesis is rejected through an experiment where labels of some ICL examples are corrupted in different positions of the input sequence and the performance is shown to depend on which ICL examples were corrupted. \n\nThe paper concludes that ICL is in some ways similar to conventional learning algorithms (H1) but different in other ways (H2 and H3).", "questions": "A hopefully minor concern I have about this work is I see a lot of traditional NLP benchmark tasks e.g. GLUE tasks being used against modern models such as LLAMA. Do we know if GLUE benchmark was present in LLAMA's training data? Could the paper's conclusions be different if newer harder benchmarks were used?\n\nDid you try the novel task of author identification in the label flipping experiments as well? If so, can we see some examples of the author-id prompts as well as results regarding all hypotheses? I am interested in the results using novel tasks because as the authors mentioned in the paper, in that case there is less chance of contamination in the models' pretraining data.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This is clearly written paper that is easy to follow, with plots that tell the story well. The question of how ICL really works is an open question in the field that has not been addressed sufficiently yet. The authors go beyond using only accuracy and use several metrics to measure the model performance before and after ICL (log likelihood and entropy). The results are reported on models with various sizes.", "weaknesses": "The paper is framed as a comparison between ICL and conventional learning and claims that hypotheses 2 and 3 reveal differences between ICL and conventional learning. I do not agree with this for the following reasons:\nH2. The empirical observations corresponding to this hypothesis are not novel and not limited to ICL. For example, in \"Making Pre-trained Language Models Better Few-shot Learners\", Gao et al show that even fine-tuning the models with flipped or neutral labels degrades their performance, so even conventional learning is not able to overcome the pre-training priors.\nH3. Sample order does matter in conventional learning too, that is why we shuffle the training data. In addition, these experiments seem to be a re-discovery of the recency bias in LLMs, for example studied in \"Calibrate Before Use: Improving Few-Shot Performance of Language Models\", Zhao. et al\n\nLastly, one major claim of the paper is that although flipping/randomizing labels of ICL examples sometimes does not affect accuracy of the model much, but probabilistic metrics such as log likelihood or entropy reveal larger gaps in performance. But this claim is not quantified well, because accuracy has a clear and interpretable range but the same cannot be said about loglikelihood.\n\n\n\n\nMinor writing suggestions:\n- \"expresses the sentiment\" --> the word sentiment not appropriate for describing a scientific paper's conclusion\n- \"This is important when the context contains diverging information about a label relationship\" --> unclear", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KUa4zRupt4", "reviewer": "Reviewer_Sjxd", "summary": "This paper explores how label information is used within ICL for LLMs. Experiments suggest that predictions given ICL almost invariably rely on the labels provided in-context. They also suggest that novel tasks can be acquired given this simple technique. In contrast, priors in the model remain hard to overcome, and statistical biases remain in the use of particular subsets of ICL components. Experiments are run on (technically) three families of LLM.", "questions": "- Can you please check your work for grammatical errors (e.g., ‘we rephrase these them as…’) and formatting errors (e.g., capitalization in references can be resolved by adding {} appropriately)?\n- Would it have been more appropriate had Null Hypothesis 2 ($H^{(2)}_0$) been the reverse — that ICL cannot overcome 0-shot prediction preferences. Or is the original version of “NH2” even a null hypothesis in the statistical sense, or merely a proposition that can be proved or disproved by example? It seems like this was set up with the expected outcome in mind.\n- Is it really necessary to examine _all possible_ numbers of demonstrations if the behaviours are so macroscopically clear?\n- Is it really the case that the authorship identification task is truly novel? It could be confidently assumed that neither the particular messages were used in training the LLMs, nor even the particular authors, but author attribution itself is something LLMs learn implicitly, and ‘authorship embeddings’ of the authors of this paper are surely plausibly part of decoding/inference? I.e., this may be a novel dataset (which is not uncommon to see in NLP writ large), but how is this a novel _task_?\n- If you couch your _research questions_ as null hypotheses, why do you not do any significance testing?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The appendices appear very numerous, although it was not possible to read the entirety of the ~60 pages for this review and, since most of these are in the form of (at least superficially) very similar plots, the depth of their informativeness is not clear.\n- A good number (perhaps moderate variety) of LLMs are considered, and a very good number of tasks.\n- The inclusion of observations and discussions directly with the associated questions was appreciated, especially with regards to previous work (e.g., Min et al (2022b)).\n- Section 8, regarding NH3 (which is not really treated as a null hypothesis, admittedly) is convincing.", "weaknesses": "- Several of the outcomes of this paper are, admittedly, already examined in some of the background work described in Sec 2 (and elsewhere), e.g., Zhao et al (2021). It may be advised to add differentiating factors with previous work directly into Sec 2.\n- Additional differences exist between this work and that of Zhao et al (2021; e.g., the models are entirely different) so a direct comparison in Sec 8 of this type is somewhat disingenuous\n- Although not a direct effect of this paper, the extent to which these particular results will generalize to future models is somewhat limited. On a positive note, the methods used to obtain these results may be applicable for some time. \n- Some of the arguments seem to fall to the ‘hasty generalization’ fallacy. E.g., claims are made about ICL in general (at least within the context of LLMs), but really only specific examples are applicable. This is most notable on the experimentation around NH2, which is perhaps the weakest of the three so-called null-hypothesis experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)