Understanding In-Context Learning from Repetitions
Authors: Jianhao Yan
Jin Xu
Chiyu Song
Chenming Wu
Yafu Li
Yue Zhang
Venue: ICLR 2024 poster (2024)
Keywords: In-context learning
Large language models
Repetition
Primary area: general machine learning (i.e., none of the above)

Abstract: This paper explores the elusive mechanism underpinning in-context learning in Large Language Models (LLMs). Our work provides a novel perspective by examining in-context learning via the lens of surface repetitions. We quantitatively investigate the role of surface features in text generation, and empirically establish the existence of \emph{token co-occurrence reinforcement}, a principle that strengthens the relationship between two tokens based on their contextual co-occurrences. Furthermore, we find similar reinforcements lie behind the pretraining corpus, revealing the existence is due to LLMs' efforts to maximize the likelihood. By investigating the dual impacts of these features, our research illuminates the internal workings of in-context learning and expounds on the reasons for its failures. This paper provides an essential contribution to the understanding of in-context learning and its potential limitations, providing a fresh perspective on this exciting capability.

Published as a conference paper at ICLR 2024
UNDERSTANDING IN-CONTEXT LEARNING FROM REP- ETITIONS
Jianhao Yan 1.2 Jin Xu*
Chiyu Song! Chenming Wu Yafu Li Yue Zhang2.3. 2,3,3
School of Engineering, Westlake University Institute of Advanced Technology, Westlake Institute for Advanced Study 4 Tsinghua University Baidu Research elliottyan3766 74 g0aamiil com
'Zhejiang University
ABSTRACT
This paper explores the elusive mechanism underpinning in-context learning in Large Language Models (LLMs). Our work provides a novel perspective by ex- amining in-context learning via the lens of surface repetitions. We quantitatively investigate the role of surface features in text generation, and empirically establish the existence token co-occurrence reinforcement, a principle that strengthens the relationship between two tokens based on their contextual co-occurrences. Further- more, we find similar reinforcements lie behind the pretraining corpus, revealing the existence is due to LLMs' efforts maximize likelihood. By investigating the dual impacts of these features, our research illuminates the internal workings of in-context learning and expounds on the reasons for its failures. This paper provides an essential contribution to the understanding of in-context learning and potential limitations, providing fresh perspective on this exciting capability.
INTRODUCTION
The impressive ability of Large Language Models (LLMs; Touvron et al. 2023a Chowdhery et al.) 2022; [ to execute in-context learning (ICL. is standout characteristic. This behavior mirrors human learning and reasoning from analogy (Winston) 1980), enabling LLMS to rapidly adapt to range of downstream tasks. Without being explicitly pretrained to learn from demonstrations, LLMs can predict responses to unseen test queries from a few demonstrations and without any instruction given (Brown et al. 2020 Zhang et al. 2022 Chowdhery et al. 2022). An example of in-context learning can be found in Figure Ia), where pre-trained LLaMA model is given demonstrations for a binary classification task, and learns to make predictions correctly. Despite the success in applications, the working mechanism of in-context learning is still an open question.
Existing work has investigated input-label mapping (Min al. 2022 Yoo al. 2022 Wei et al.
2023) and demonstration construction (An et al. 2023) Lu et al. [022 Liu et al.) 2022) as underlying factors for ICL. However, little research has focused on the correlation between ICL and textual features. Intuitively, the behavior of ICL depends on the context and can be fragile to its variations. As Figure lbb shows, the same LLaMA model makes the incorrect prediction 'True' given the input "Circulation revenue has decreased by 5% in Finland.", which is likely because of the repeated pattern "Answer:" "True" from the demonstrations. In the same perspective, the success case in Figure D a) can be attributed to learning desired patterns such as "Answer:" -> "TruelFalse" in the demonstrations. Such patterns are apparently used as features the autoregressive inference process by the model.
We take a feature-centric view to understand ICL, analyzing the key patterns the input context that correlate with ICL behavior. The patterns we discussed above can be viewed as generalizations to repetition patterns (Holtzman et al., 2019; Fu et al. (020) and self-reinforced patterns (Xu et al. 2022) which have been discussed in the literature. The where the model tends to perpetuate the generation of sentences that have frequently appeared in its context. These effects regarded as harmful text generation and previous work puts efforts
effect' describes phenomenon

---

Published as a conference paper at ICLR 2024
Demonstrotions
Reinforced Connections (Ours)
Irput: Paying off the national deet will extremely painful. // Answer: False
"Answer:" True Faise
Irput:
Gonerous and subversive artwrrss. Answer:
"Input:" True False
Input: Panostaja did not disclose the purchase price. I/ Answer: Fabe
"Ciroulation revenue.." True
Input: Circulation recenue has increased by 5% Finland.
"Panostaja did not
Palse
Input: Circulation revenue has decreased by 5%% Fisland.
painful False
False
(a) correct prediction of in-context learning.
Demonstrotions
Reinforced Connections (Ours)
paintul. // Answer: False
Input: Paying off the national
estremely
"Paying off... False
Ingut: Ciecuation revenue has icreased by 594 Finlend. Answwer: True
"Answer" -o True
Irput: Croulation revenue has increased by 5%
Finland. // Answer: True
"Circulation" > True
Irput: Cirodation resenue has increased 5% Firland. True
True
Input:
by in
Finland" True
Answar. (?]
(b) An in-correct prediction of in-context learning.
We showease correct and incorrect predictions of in-context learning of LLaMA-65B. The shown task is to identify whether the given sentence presents a positive sentiment. We involve the token reinforced connections from demonstrations. In both cases, LLMs learn connections from the demonstrations and make decisions based on these connections. In the case of in-context learning, the model learns reliable connections and hopefully several of these connections result in the function of sentiment analysis. On the other hand, in repetitive demonstrations, the model gets stuck to spurious connections and misses the key information 'decreased' leading to wrong prediction.
Figure 1:
to mitigate However, they could give a unique view of the ICL behaviors from the angle of text generation.
We quantitatively investigate in in-context learning from the perspective of surface patterns, illustrating the inherent correlation among surface patterns, seff-reinforcemment, and ICL. First, we study the roles of self-reinforce patterns as surface features that guide text generation. We empirically establish the existence of the token co-occurrence reinforcement, where the connection between any two tokens gets reinforced with the number of contextual co-occurrences, a primary principle learning surface- level patterns. We further delve into the reasons and inner workings causing token reinforcement, showing it as an inevitable result of the model's efforts to maximize the likelihood of the training corpus.
Given the existence and reasons behind such patterns, we scrutinize the beneficial and detrimental effects of these surface patterns in-context learning. On the one hand, experiments MMLU and GSM8K show that the reinforcement helps constrain the output space and format outputs to follow demonstrations like outputting 'Let's think step by step.'. On the other hand, experiments with non-informative connections and reordered answers in MMLU demonstrate that intentionally constructed connections make LLMs lean towards specific answers, revealing the risk of unintended. spurious connections. Our work not only reveals the intrinsic workings of in in-context learning to some extent, providing perspective not analyzed the literature also explains the underlying reasons for the failure of in-context learning
2

---

Published as a conference paper at ICLR 2024
Probabilliy belore and ofter repetition.
&
&
1.00
0.
0.00
0.1
OPT
LiaMA
Numberol Repeats
Nobobaik defere Mesciiaor.1s
Figure Left: An example of the self-reinforcement effect. We choose normal sentence ('Answer A'), repeat it several times, and present the probability of the token 'A'. The model used is LLaMA- 7B. Right: Sentence-level reff-reinforeement over LLMs. We plot all sizes of OPT and LLaMA with colors from light dark. All sizes of LLaMA and OPT models demonstrate strong sentence-level self-reinforcement effects.
Our main contributions can be summarized as follows:
We propose a novel perspective to understand ICL with repetitive text generations.
We perform systematic analyses and empirically establish the existence of token reinforce- ment across various L.Mss, alongside with the reason behind.
We show that token reinforcement constrains output space and enables desired patterns for ICI., but is also responsible for spurious connections and possible failure of ICI.
2 ICL AND REPTITIONS
We first illustrate the background. In ICL, given desired task f. we feed an LLM with K input- output pairs {(r Vss,1 n E [1, K]), where ys = f(z)). Each input-output pair formatted demonstration d (FI, TR. Fo, Vr). F1 and Fo denote formatting tokens for inputs and outputs, e.g.. 'Input:' and 'Answer:' and 2y and yı can consist of several tokens.
A pretrained language model M predicts the output y conditioned on the concatenation of demon- strations and the test query x,
Puci.yla k) = M(y Fo, Fo. Mi, FT, Fk, Fo, Uke FI, x., Fo)).
To understand the influence surface repetitions as in [ we fist show a case of sentence repetition and plots the probability of the sentence as the number of repetition in the previous context increases (Figure 2 When we manually repeat the sentence 'Answer A', the probability of generating 'A" after 'Answer is' gets boosted from 0.03 to almost 1.0.
The right part of Figure demonstrates our quantitative study with two families of large language models, namely OPT (125M, 350M, 1.3B, 2.7B, 6.7B, 13B, 30B, /Zhang et al. 2022) and L.LaMA (7B, 13B, 30B, 65B, Touvron et al. 2023a). We plot the average sentence probability after repeats for LLMs of varying sizes, arranged by light dark colors. We use 1,000 sentences from each of the three different datasets Wikitext- 103 (Merity et al. 2016), BookCorpus (Zhu et al. 2015). and sequences of random words. More experimental details can be found in Appendix With 10 repeats, the probability of generating the sentence is significantly increased across all tested LLMs. LLMs amplify the occurrence of previously presented sentences, even sequences of random tokens
The above observations are related to the study self-reinforcement effect (Xu et al. 2022) in literature. Formally, the conditional probability we model here is PREP(w) I..],,, where is the repeating sentence, denotes the number of :, ... "
Xu et al. (2022) reported that sentences with a initial probability such as sentences composed random tokens have smaller self reinforcement effect. In our experiments, even sentences with random tokens (which initially have near-zero probability) become reinforced to probability nearing one. The difference may come from different model sizes (150M vs. maximum 65B) and pretrained corpus size (hundred millions of tokens trillions of tokens).

---

Published as a conference paper at ICLR 2024
Kopt
Kept Tolenns
Kept
0.7
Nama-75
lamart
0.6
Nam
0.5
300
650
0.4
0.3
02
1ama Th
liama-7
1ame 13b
laama-13b
0.1
Tame 20b
Ilama-3te
0.0
Tama 500
Blam--600
9
12
38
Repetison
Repetillos
Repecition
Ropobiton
(A): X.........
..I...X..........
(C):..X..Y.... N...X.X.YY............ X..YZ....
N...Y.Z...........
Figure 3: Token co-occurrence reinforcement. Even if only one token repeats in context, the self-reinforcemment loop triggers. "..X..Y." Y.. denotes 2 tokens are kept unchanged. The mean and variance are computed over 1,000 randomly generated samples.
occurrences, and w; is the i-th token in the sentence Previous research finds the seff-reinforeemment effect, where the probability of generating the sentence s of length I.2 occurred N times context, TPx = I, Σ: PREP(W/#: While the generation of repetitive sentences above can be understood as the influence of a single surface feature, we investigate more sophisticated surface patterns, which can be causes to behaviors of in-context learning.
wi;; = N), almost monotonically increases with the number of N.
3 SEIF-REINFORCED SURAFACE FEATURES FOR IN-CONTEXT LEARNING
Similar to previous definition of the demonstration, we set s Fr; Fo; and have,
times
PREP(Wi K) M ((l]((F1]) (F F1, Fo. Un F1, Fo. V1 F1, 21, Fo)). (ulL, k = K) = MMll (FI, Fo. Un, FI, Fo. FI,x, Fo)).
Comparing PREP (w) (v), find: (1) F1 and Fo both repeated across demonstrations; (2) In repetitive generation, 21 and //1 are repeated, while in ICL, x and y are changing.
To investigate the correlation between surface patterns and the resulting answer y, gradually expand reiforeinforeement patterns toward in-context learning. We achieve this by introducing random perturbations each demonstration, imitating the role of changing while leaving certain components, such as Fo and y, unchanged.
The experiments in this section conducted over the dataset of randomly generated sentences as in Section 2 and with the four LLaMA models. The results on Wikitext-103 and BookCorpus, and results with OPT and various other LLMs can be found in Appendix D For each experiment, repeat the pattern 20 times and report the mean and standard deviation of the probabilities for the kept tokens.
3.1 ANY TWO TOKENS CAN CREATE TOKEN REINFORCED LOOP
We assume that tokens are the base unit of reinforcement. By assessing the self-reinforcement effect among tokens, we understand the self-reinforcement effect from first principles.
Formally, given sentence (wi.... w.,) from corpus D, we construct binary mask sequence
m- (m, m..... ., and define replacement operation R(w, m) Wr: if m 0 that
w, if
replaces w with randomly sampled token w, from the vocabulary if m 0 and keep it unchanged when m- = 1. Note that w, is independently sampled for each sentence and each position. As for mask sequence mi, we randomly sample positions put the Os and Is of the mask sequence vi and control the number of kept tokens. In this way, a sentence is transformed into s (R(w), m.),...................))
where ΣlE(I,L,) my L,, Then, we report the average token probability TP as in the previous section. Suppose have sentence (Apple, there, is, red) and m = (1, 0, 1, 1), and the target token w = red. Then, the demonstrations in this section will be like 'Apple there is red // Apple logo is red // Apple juice is red'.
4

---

Published as a conference paper at ICLR 2024
0.5
0.23
5.20
Ilama-7
020
lama-7o
Th: Te.aedage Wenn
0.4
0.13
230
Iamm--220
Buma-3300
015
0.3
650
0.2
Bame-77
0.
1am
0.05
Namm-
0.0
Namm-555
0.00
6.00
0.00
12
15
38
Repatison
Repetttion
Repesition
Repetitoo
..YYY....YY.
(B): ... ..X.Y....XXXX
(C): ..X.Y.........
X.............Y...
Figure 4: Successive and distant reinforcement. The self-reinforcemmet effect is the strongest when tokens successive, i.e., distance=O. Otherwise, the reinforcement smaller and appears insensitive to the distance. " "..X...." denotes the distance between two tokens is 1.
Number of Tokens As depicted in Figure even only one single token shared across demon- strations elicits relfrreenfrremment The scenario with two tokens kept unchanged is particularly interested (Figure BB)), as it reveals basic case that one token triggers the generation of the other one. We find that the connection between any two tokens gets reinforced and the probability increases monotonically with the number of their contextual co-occurrences. We refer to this base effect the token co-occurrence reinforcement. When we increase the number of preserved tokens from to 4, observe strengthening of the reinforcement effect. This is because each former token forms reinforced connection with all the latter ones.
Distance In-Between We further examine the role of distance in token reinforcement. This analysis confined to tokens. Figure 4 distinctly differentiates between successive tokens (distance= 0) and distant tokens (distance> 1), which we term as successive reinforcement and distans reinforcement, respectively. The successive reinforcement significantly elevates the probability, from O to 0.4, with only several repetitions. Conversely, the distant reinforcement provides moderate boost the probability, from 0 0.2, and appears be indifferent to the distance between tokens.
Across all experiments, we observe a marked increase in reinforcement as model size scales, especially in the distant token reinforcement. This consistent escalation suggests that larger LLMs are more capable of following complex patterns in in-context demonstrations, which is consistent with results from Wei et al. (2033). We provide more supporting evidence of token reinforcement Appendix [
The link we found between any two tokens forms the foundation of sentence-level relforeinforcemmntt. Each token strengthened by the ones before it. In ICL, common elements in demonstrations, such as pattern words, form connections with label words like "A, B, C, D".
3.2 TOKEN REINFORCEMENT IN THE PRE-TRAINING CORPUS
We find that token reinforcement could be a result of the model's effort to maximize likelihood of the training data.
Given a pre-training corpus Dirašns the language modeling task over sequence of tokens {wi, W2, Wr) with length
Whrd twieforeement.io in Petteiiing Corpus
T is defined by max -! log P(w, w.z). We compute the following empirical probabilities over the corpus:
Pa EWEV Σ P(w, w Count(w, wes) =n), Vn,
where Count(.)) counts the appearance of word w in the previous context weii and V is the vocabulary.
Figure 5: The probabilities of next occurrence after several occurrence observed in context.
Intuitively, these probabilities reveal whether a particular word is likely to recur if there have already been n instances of w observed in the context, resonating with our analysis of token reinforcement.
5

---

Published as a conference paper at ICLR 2024
i-th Demonstration
What are the mean and standard deviation of binomial experiment that occurs with probability success times?
Space
repeated
0.07
0.30
114,
0.82
100.5,5.23
0.85
114, 5.23
D. The mean is 114, but there is not enough information given determine standard deviation.
EC, bareliee
0.45
Nasc
Mask Haplace Optan Sarcs
0.42
Nask Replace Answw trdicate
Mumoer Demoostmaters.
Figure 6: Left: An example demonstration from MMLU's high school statisties dataset. Colors indicate the parts to be masked. Right: Probability MMLU's label space. We find that: (I) Masking question contents and answer contents of the demonstration does not influence directing the label space. (2) Both replacing the option names and the answer indicator significantly hurt the ability constrain the label space. The gray shadow denotes the standard deviation across three runs.
We compute these probabilities over commonly used pretraining corpus, sekipedia-englishe 2024 encompassing over 5.7B tokens, using the LLaMA tokenizer to first tokenize corpus and preprocess each context window with T 1024 tokens.
Figure demonstttstes our results. We find that the trend of word probabilities accords to our results in Section 3.1 The more instances seen, the greater the probability of the same word recurring. Therefore, we infer that the token co-occurrence reinforcement stems from the LLMs' optimisation of the training likelihood. The LLMs manage to learn this inherent feature from the training data and generalize it to longer phrases and distant connections. This also elucidates the scaling with reinforcement, where larger models more effectively capture this feature from the training corpus.
3.3 UNDERSTANDING ICL VIA REINFORCEMENTS
Token reinforcement effect discussed in previous sections provides a new perspective to understand in-context learning. Consider the following example with three demonstrations [A,B,C,D A, a.B,C.D: A,b,C,())) Our target is 'D'. In this example, several reinforced connections exist concurrently. 'A->D' is reinforced twice; 'b->D' is reinforced once; 'C->D' is reinforced three times. Here, all three reinforcements reach a consensus and predict 'D' in cooperation.
However, this ideal scenario not always the case. Consider another example [A,B,C,D; A.b.C.E: a.B.C.F A.?).. At this time, 'A->D' reinforced once; 'b->E' reinforced once; 'a->F" is reinforced once and et cetera. These reinforcements create conflicts and compete against each other. Thus, instead of the traditional view of ICL as mapping from input to output, we view ICL from feature angle, as combination of connection of tokens, even though some of them are highly reinforced and some of them are not. The next section shows how the reinforcements play a crucial role in in-context learning.
4 THE EFFECTS OF SURFACE PATTERNS TO IN-CONTEXT LEARNING
We quantitatively study how self-reinforced surface patterns lead to both beneficial functions and detrimental effects in ICL. The experiments in this section are conducted over MMLU (Hendrycks) et al. 1 and GSM8K (Cobbe et al.) 2021). Due to limit of computing resources, sampled 20 samples for each of the 57 tasks of MMLU, resulting in collection of 1 1440 test samples. The demonstrations
randomly
independently drawn for each of the test samples. All experiments conducted across three random seeds. For further experimental details, see Appendix
4.1 BENEFICIAL EFFECTS
Constraining Output Space. important advantage brought by the reinforced effect is that helps constrain the output space with several demonstrations, connections between formatting

---

Published as a conference paper at ICLR 2024
tokens ('Input'' and 'Answer:') and label words in each demonstration ('ABCD') reinforced, through distant and successive reinforcement, respectively. In this way, the LLM learns to predict either one of ABCDD the final answer, instead of continuation sequences such as 'Oh, an interesting question. hope know the answer.'.
We verify this advantage with the MMLU dataset, which is widely used to evaluate the language understanding of real-world large language models. To isolate the effect of seff-reinforcement, we construct masked demonstrations for analyses. An example of how mask demonstrations shown in the left part of Figure 6 Particularly, demonstration in the MMLU dataset can be divided into five parts: question content, option name (e.g., 'A.'). option content (e.g. $114, 27. 27.35'), answer indicator (e.g.. 'Answer:') " and final answer (e.g., 'D'). Based on token reinforcement, we hypothesize that the option names, i.e., "A.", "B." reinforce outputting "A,B,CI via distant reinforcement. The answer indicator, i.e., "Answer: ", reinforces outputting within label space via successive reinforeement. To validate our hypothesis, we first mask all the questions and option contents in all demonstrations and keep the formatting words, final answer, and test query unchanged. Then, further ablate option names and answer indicators by replacing them with semantically equivalent substitutes. More experimental details can be found in the Appendix B
We use 1LaMA-65B and plot the probability of choosing "A, B, C, D" as the predicted answer. The results are shown in the right of Figure We find that masking the question and option contents, typically regarded as the most informative parts, does not influence the ability to direct the label space. Option names and answer indicators repeated several times in the demonstrations fulfill the job. Our conclusion is further solidified after ablating the option names and answer indicators individually. We see huge decrease when replacing both option names and answer indicators, validating our hypothesis.
Learning to Follow Patterns. Another distinctive fea- ture of in-context learning is to follow the patterns of demonstrations. This is exemplified in techniques such as the Few-shot Chain-of-thouu
Let's think step by step.
1.0
taneline
-aak (Qusstann)
(CoT) prompting (Wei) et al. 2022), frequently employed in reasoning tasks of LLMs like the GSMSK (Cobbe et al. 2021).
Prbtiteyy OaT Artown
0.8
Nask OUT Answer)
Here, we illustrate how the reinforced features in Section affect the pattern following of ICL, by showing how demonstrations the GSM8K high school math dataset. Each demonstration in the dataset follows the form "Question: [Question] // Let's think step by step. // [CoT Answer]". We demonstrate how models learn to say the CoT pattern, i.e., "Let's think step by step.". We further discuss the connection between surface patterns and [CoT Answer] in Appendix H.1
LLMs follow the
0.0
Number Demo remtietnioon
Figure 7: Ability to follow patterns from demonstrations.
Based on the findings in previous sections, hypothesize that the common parts in the demon- strations teach the LLM to generate the CoT pattern. More specifically, 'Question:' builds a distant reinforcement, and the new liner '//" builds a successive reinforcement with the CoT pattern. We mask out each part in each demonstration progressively with random tokens to ablate the influences.
The probability of the CoT pattern is shown in Figure 7 After masking "//", the probability gains obtained from demonstrations almost diminish, verifying our hypothesis of successive reinforcement. Another interesting finding is masking [Question] reduces the probability of generating the CoT pattern, indicating the [Question] part to some extent builds connection with the CoT pattern. Since the questions in demonstrations are different but lie the same group of grad math problems, there might be some common patterns among these questions.
4.2 DETRIMENTAL EFFECTS
Token reinforcement is not always helpful. In this section, explore the detrimental consequences that arise from it. As shown in Section[3.] distant and successive reinforcement are activated with two random tokens. This could potentially lead to spurious patterns across demonstrations,
7

---

Published as a conference paper at ICLR 2024
Table Top: Effeet of Mon-Informettve Connections (NC). The accuracy of increases with the cost of A, B and C. Bottom: Effect of Reordered Answers. With more reordered demonstrations, the oulputs are more leaned toward D. The delta values on the superseript denote the improvement compared with zero-shot scenarios. 'Avg. [A,B,C]' denotes the average accuracy of samples whose golden answer A, B or C. 'D' denotes the accuracy of samples whose golden answer is D. "f": significantly different compared to its corresponding ICL baseline (p < 0.05).
Non-informative Connections
# Demos
0
4
5
Avg. [A,B,C]
63.39
63.74"0.3
64.56-
64.17
64.44--.0s
64.37
Avg. [A.B,C] w/ NC
63.04
61.21 1.81
62.57
63.27
63.00 0.4
004t
63.47*043
D DW/NCC
52.28
59.65
60.00
59.30*M0
59.88*760
59,53
49.47
63.63414.51
64.09-14621
63.51+1404)
62.57-13.11
61.64-1216
Reordered Answers
Avg. [A,B,C]
63.39
63.7440.35
64.56+
64.17 04.177
64.44-1.05
64.37-097
Avg. [A, w/ 50% RA
63.39
64.56
64.44*100
64.05
63.94
63.86*947
Avg. [A,B,]] w/ 75% RA
63.39
64.52*1.15
62.65 62.65-9.44
62.53-0.86
61.95-1.441
62.344-1055
Avg. [A,B,]] w/ 100% RA
63.39
64.80*1.901
62.03 1.36
61.17
59.10
58.05 5:3+
D
52.28
59.65*737
60.00
59.30*710
59,8 59.88*7mi
59,53
D w/ 50% RA
52.28
59.30
60.82
61.40
61.75+944
61.64
D w/ 75% RA
52.28
59.06+6.78
62.81 +10.53
65.50+13:22
67.0 67.02+14774
66.90-M422
D w/ 100% RA
52.28
58.71
16.45
66.20*139927
71.35*1906f
75.67 23..9t
77.19*24911
which might be completely unforeseen by end users. We illustrate this point using two experiments where we manually construct spurious patterss in the MMLU dataset.
Non-informative Connections. Our first approach is adding connections between phrase and certain choice. We append a reasonable but non-informative phrase such as Pleass kindly provide your answer.' 'Looking forward to your choice.' right before the template 'Answer:' each time the question's ' answer is 'D'. In testing, we also append the same phrase and check whether the outputs are navigated toward 'D'. By doing so, we construct distant reinforcement loop from the non-informative phrase to the answer 'D', We ensure the test set is balanced with equal numbers of questions having golden answers "A.B.C.D", and we report the accuracies at the top of Table !]
We first see a gap even without demonstrations, where adding non-informative phrases lowers the accuracy of choices D. We further discuss the selection bias (Zheng et al.) (023) of different choices in the Appendix. Then, we see that the non-informative connection overcomes the selection bias and significantly elevates the accuracy choice with a noticeable gap, in the cost of accuracy of A, B, and C. These results show the potential risk of manually injecting spurious connections and directing in-context learning toward unintended outcomes.
Answer Indicator Connections. In our second experiment, we show that reinforcing the connec- tion between 'Answer:' and a certain choice, e.g.. D, navigates the outputs. To this end, we randomly replace r' percent of demonstrated answers with D. Simultaneously, we exchange the option contents of the original golden answer and D, to keep the demonstrations valid. In this way, we gradually reinforce the connection between 'Answer:" and 'D', with successive reinforcement.
The results are presented at the bottom of Table I Our baseline is 25%, where the demonstrations are balanced. With the increased ratio of answer in demonstrations, the accuracy of is largely improved, from 0.52 to 0.78, while the accuracy of A, B, and C decreases from 0.63 to 0.58. Our findings corroborate with [nn et al. (2023), where they show how diversity affects the performance of in-context learning. Our findings demonstrate how unbalanced demonstrations bring an unfair advantage for certain outputs.
Discussion (1) Reinforcements can be the underlying reason for ICL, but also causes its vulner- ability. (2) Our observations guide how to build demonstrations to maximize ICL effectiveness! The demonstrations should be both balanced for all possible output labels, and kept concise enough without introducing any unnecessary reinforcement.
8

---

Published as a conference paper at ICLR 2024
5 RELATED WORK
Explaining In-Context Learning. A range of contributions has deepened our understanding of In-Context Learning (ICI.),Chann et al. (2022) and et al. (2022) explore the emergence of ICL from the perspective of training data and Bayesian inference, respectively. Implicit learning of ICL over demonstrations is further highlighted by Garg et al. (2023), Li et al. (202), and Hahn & Goyal (2023) theoretically show that performance of TCL can be represented by a complexity that repetition structures can be represented by a small PCFG tree. The similarity between gradient descent learner and in-context learner is demonstrated by Akyürek et al. (2023); von Oswald et al. (2023), while Dai ee al. (2023) explain language models as meta-optimizers and likens ICL to implicit finetuning. Our work differs from this line of work with a novel perspective via repetitions and reinforced features, and our findings could potentially explain the mechanism of how LLMs achieve implicit learning. For instance, the step by step reinforcement across demonstrations intuitively resembles the gradient descent process of ICL described in previous work. Olsson et al. (2022) introduce induction heads of copying patterns and provide evidence for their relationship with ICL. Differently, our work investigates the LLM as whole (Anderson] 1972), studies sophisticated patterns, views ICL as a combination of reinforcements, and scrutinizes both the benefits and drawbacks of reinforcements.
Analyzing In-Context Learning. Several studies have analyzed ICL properties. Min et al. (2022);
Yoo al. (022) identify and discuss key factors that influence ICL capability such as input-label mappings. Wei et al. (2023) proposes that learning input-label mapping is an emergent ability.
Factors such as structural similarity, diversity, simplicity (An et al., 2023). , and order or embedding
distribution (Lu et al. 2022. Liu et al.) (002) are also investigated. Pan et al. (2023) partitions
ICL ability into task recognition and task learning, observing different phenomena with varying model sizes. Lastly, Si et al. (023) unveils the presence of inductive biases in ICL by designing
underspecified demonstrations.
Our findings corroborate with multiple previous analyses of in-context learning. For example, the scaling for distant reinforcement echoes Wei et al. (2023); ; Pan et al.] (2023)'s findings of different phenomena when varying model sizes. The importance of demonstration ordering and diversity in An et al. (2023): Lu et al. (2022) can be explained by avoiding spurious connections.
Repetitive Generation and Self- Self-Reinforcement Effect. Repetition is notorious issue in neural text generation, affecting tasks like open-ended and directed generation (Holtzman et al.) 2019 Welleck et al. 2019 [in et al.) 2021] |See et al., 2017] Liu & Lapata| 2019). Maximization-banl
decoding strategies lead to bland, consecutive repetitions at word, phrase, and sentence levels (Holtzman et al.)
Welleck Welleck al. 2019 Li et al. 2016; Karpathy & Fei-Fei 2015 Guan et al., 2021). Despite advancements in large-scale pre- trainrainin with Transformer architecture (Vaswani et al. 2017 Radford
et al. 2019 Lewis al. 2020). unexpected sentence-level repetitions persist (Radford al. 2019
Brown et al. 2020 Fu et al. 2020).
The repetition issue is puzzling given the lack of repetitive sentences the training data. A series of studies investigate the cause, with both from the theoretical perspective (Fu et al. 2020 and empirical findings (Holtzman et al. 2019). Recently, Xu et al. (2022) proposes the selffreimforcemennt effect, suggesting repetitive loop when combined with maximization decoding. Our study extends the effect to large language models and token reinforcement, explains the reasons, and bridges the excellent ability of in-context learning to this notorious issue.
6 CONCLUSION
We have taken a novel feature-centric approach to understanding in-context learning, by exploring its relationship with repetitive generation. We have identified key mechanism, the token reinforcement loop, where any two tokens can form a strong connection through multiple co-occurrences. We delve into the reasons and inner-working of token reinforcement, demonstrating it to be an inevitable result of maximizing likelihood. Based on our findings, we view in-context learning as a combination of token reinforcements with different level of strength instead of input-label mapping. Furthermore, we conduct experiments to demonstrate that token reinforcement plays crucial role in shaping the output space and following patterns in in-context learning. We also illustrate through various studies how token reinforcement leads to spurious connections in in-context learning, highlighting the role of in-context learning as a double-edged sword, where informed demonstrations can maximize ICL effect.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
This publication has emanated from research conducted with the financial support of both the National Natural Science Foundation China Key Program under Grant Number 62336006 and the Pioneer and "Leading Goose" R&D Program of Zhejiang under Grant Number 2022SDXHDX0003.
REFERENCES
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models, 2023.
Shengnan An, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Jian-Guang Lou, and Dongmei Zhang. How do in-context examples affect compositional generalization?, 2023.
Philip Anderson. More is different: Broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047):393-396 1972.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems,
2020.
Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya Singh, Pierre H. Richemond, Jay McClelland, and Felix Hill. Data distributional properties drive emergent in- context learning in transformers, 2022.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.0233 2022.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, al. Training verifiers to solve math word problems. arXiv preprint arXiv22110.14166 2021.
Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers, 2023.
Zihao Fu, Wai Lam, Anthony Man-Cho So, and Bei Shi. A theoretical analysis of the repetition problem in text generation. arXiv preprint 2rXiv:2012.146600 2020.
Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes, 2023.
Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wenbiao Ding, and Minlie Huang. Long text generation by modeling sentence-level and discourse-level coherence. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume I: Long Papers), pp. 6379-6399 2021.
Michael Hahn and Navin Goyal. A theory of emergent in-context learning implicit structure induction, 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Confer- ence on Learning Representations, 2021. URL URlhhtts:://pppeeeiee: net. wwwpteett.oooiteoeooooo
d7KB jmI 3GmQ.
hiyouga. Llama factory. httpss//gthhbb.ccm//yyyogg/aaaap
2023.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. International Conference on Learning Representations, 2019.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux. Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023.
10

---

Published as a conference paper at ICLR 2024
Andrej Karpathy and Li Fei-Fei. Deep visual-semante aligmments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2015.
Mike Lewis. Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-tor se-uennecce pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871-7880, 2020.
Jiwei Li, Will Monroe, and Dan Jurafsky. A simple, fast diverse decoding algorithm for neural generation. arXiv preprint a6 1611::1111..000000 1.08562, 2016.
Yingcong Li, M. Emrullah Ildiz, Dimitris Papailiopoulos, and Samet Oymak. Transformers as algorithms: Generalization and stability in in-context learning, 2023.
Xiang Lin, Simeng Han, and Shafiq Joty. Straight to the gradient: Learning to use novel tokens for neural text generation. In International Conference on Machine Learning, pp. 6642-6653, PMLR, 2021.
Jachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100-1 14, Dublin, Ireland and Online, May 2022. Association for Computational Linguistics. doi: 10. 11613/11/2022222222 deelio- 1.10. URL htttps://ae http://accaaanoooo..ooooooo deelio-1.11
Yang Liu and Mirella Lapata. Text summarization with pretrained encoders. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3730-3740, 2019.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In I Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8086-8098 Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.118663/1122022.ac-- lon.....56.... URL wttttt::///// httpss////aaaannooogoooooooo wtt.lllogooooor/2222222 ac long
556
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2016.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work?, 2022.
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. arXiv preprint arXiv: 1195, 2022.
OpenAl. Gpt-4 technical report, 2023.
Jane Pan, Tianyu Gao, Howard Chen, and Danqi Chen. What in-context learning "learns" in-context: Disentangling task recognition and task learning, 2023.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073-1083, 2017.
Chenglei Si, Dan Friedman, Nitish Joshi, Shi Feng, Danqi Chen, and He He. Mcasuring inductive biases of in-context learning with underspecified demonstrations, 2023.
11

---

Published as a conference paper at ICLR 2024
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1397. 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodrigue7, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent, 2023.
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. treesh-traaffoommee jax May 2021.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of thain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837 2022.
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. Larger language models do in-context learning differently, 2023.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. In International Conference Learning Representations, 2019.
Patrick H Winston. Learning and reasoning by analogy. Communications of the ACM, 23(12): 689-703, 1980.
Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. An explanation of in-context learning implicit bayesian inference. In International Conference on Learning Representations, 2022. httttps://oeenrevieww..... ott.://e//.oummmmmoodddo/////11
Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li, and Jian Li. Learning break the loop: Analyzing and mitigating repetitions for neural text generation, 2022.
Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang goo Lee, and Taeuk Kim. Ground-truth labels matter: A deeper look into input-label demonstrations, 2022.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 2022.
Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. On large language models' selection bias in multi-choice questions. arXiv preprint arXiv:2309.0388 03382, 2023.
12

---

Published as a conference paper at ICLR 2024
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In The IEEE International Conference on Computer Vision (ICCV), December 2015.
13

---

Published as a conference paper at ICLR 2024
Appendix
Table of Contents
Limitations
B Experimental Details
14
B.I Datasel Descriptions
14
B.2 Experimental Details
16
C Further Investigation of Reasons and Inner-workings for Token Co-occurrence Rein- forcemen!
17
D
Supporting Experiments of Token Co-occurrence Reinforcement
18
DJ
Phrase Co-occurrence Reinforcement
18
D.2
Token Reinforcement of OPT Models
19
D33
Token Reintorcement of Other LLMS
D
Token Reinforcement on Other Datasets
20
D.5
Semantic Relationships Token Reinforcement
20
D.6
Improved Ratio Token Reinforcement
21
Other Experiments on Defrimental Effects
21
Generation of Cot Answer
21
Selection Bias of LLMs
22
A LIMITATIONS
While this study provides some insightful findings in the field of in-context learning, there are some limitations that should be noted. First, the experiments in this work constrain themselves to repeating surface patterns. More sophisticated patterns are not discussed. Second, our work mainly focuses revealing the token co-occurencc reinforcement and understanding its influence on in-context learning. Its detrimental influences on in-context learning suggest that resolving the spurious connections would be helpful to either thain-of-thouuhh in-context learning.
B EXPERIMENTAL DETAILS
B.1 DATASET DESCRIPTIONS
this paper, we mainly use the following five datasets, we introduce each of them and describe preprocess of these datasets individually. We present cases from each dataset demonstrate their characteristics in T
Wikitext-103 The Wikitext- 103 dataset, introduced by Merity et al. 22016 a language modeling dataset that contains a collection over 100 million tokens extracted from the set verified Good and Featured articles
Wikipedia. We use randomly sampled collection of 1000 sentences provided by Xu et al. (2022 The provided version is pre-tokenizer to words and we use the moses detokenize restore the untokenized version for compatibility with the tokenizer of transformers.
cokeniieer/ddccceeiiee....
14

---

Published as a conference paper at ICLR 2024
Table 2: Datasets and Cases
Dataset
Number of Cases
Examples
The Bintulu Government Secondary School was built in 1964.
Wikitext-103
1000
BookCorpus
1000
"A littl, "he admitted.
Green Ou incarcer hijab ura na Edmonton regardless iken Mayor
Random
1000
Question: Janet's ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
GSM8K
1000
Let's think step by step.
Janet selly 3- 3 <16-3-4=9m9 duck eggs a day. She makes 9 2 = $=9*2=18=18 every day at the farmer's market, ### 18
As more lamps are connected in parallel in circuit, the current in the power source
A. increases
MMLU
1140
B. decreases
C. remains the same
D. Not enough information say Answer:
15

---

Published as a conference paper at ICLR 2024
Table 3: Semantically equivalent substitutes.
Category
Original
Substitutes
II, III.; IV.
E.; R; G.; H.
Option Names
A.; B.; C.
2.; (a).; (b).; (c).; (d).
Solution
Reply
Answer Indicator
Answer:
Response
Result
Choice
Book CorkCsssus BookCorpus (Zhu et al. 2015) originally introduced align the books their movie releases in order to provide rich descriptive explanations for visual content. It contains 74M sentences from various sources of books, and we randomly sample 1000 sentences and also detokenize with moses.
Random Generated Sentences Here, follow Xu et al. (2022) and construct 1000 randomly generated sentences. For each sentence, we first sample a random length from tokens to 10 tokens, and then sample the tokens from the whole vocabulary. All random sentences generated by uniformly sampled from the vocabulary. All the symbols are equally likely to appear, except the special tokens.
GSM8K GSMSK (Cobbe et al. 2021) is dataset of high-quality grade school math problems created by human problem writers. The dataset contains 7500 training problems and 1000 test problems. All the problems are answered with between 2 and steps to solve.
a frequently
used benchmark to evaluate the reasoning ability of large language models (Touvron et al.] 2023a
Chowdhery et al. (2022). To analyze the chain-of-the of-thought (Wei et al.) (022)) effect of LLMs, we add LLet's think step by step" followed by the question and right before the answer.
The MMLU (Hendrycks et al. 2021) dataset another commonly used benchmark to evaluate the knowledge of large language models. It covers 57 subjects across STEM, the humanities, the social sciences, and more. It ranges difficulty from an elementary level to an advanced professional level, and it tests both world knowledge and problem-solving ability. We randomly sample 20 test cases from each task, resulting in 1140 test queries in total. In the 'undesired" section, we uniformly redistribute the answers and options to isolate the selection bias of LLMs.
MMLU
B.2 EXPERIMENTAL DETAILS
Throughout the paper, we adopt random substitutes to isolate the effects of tokens, specific formats, and other components. The random substitutions conducted in the following manner. To avoid the effect of different sequence lengths, we first tokenize the original sentence demonstration using the Transformers tokenizer. Then, replace the part to substitute with random tokens from the corresponding vocabulary. With the substitutes, we exclude the special tokens to ensure that all random tokens valid.
Random Substitutes
Self-Reinforcement For each experiment, we report the averaged results across three runs with different seeds. We mainly conduct our experiments over two model families, LLaMA and OPT. For LLaMA models, we use models sized [7b, 13b, 30b, 65b]. For OPT models, we use models with sizes ranging in [125m, 350m, 1.3b, 2.7b, 6.7b, 13b, and 30b]. Each sentence is repeated 20 times in our experiments. Following Xu et al. (202)) we randomly concatenate a prefix before all repeated sentences.
16

---

Published as a conference paper at ICLR 2024
Losses Over Time
3.4
Full Context
Mask Random
3.2
Repeat
2.0
2.6
2.4
2.0
10000
20000
30000
40000
50000
Steps
Figure 8: Pretraining losses when masking or not masking the repetitive features.
In-Context Learning the MMLU experiments, we replace option names and answer indicators to study the importance of token reinforcement in directing output space. Specifically, for each replacement, we choose semantically equivalent substitutes from a pool and randomly replace the original option name/answer indicator with the chosen substitute. In this way, break the token reinforcement from the demonstrations, We put the pool of our replacement in Table
Significance test We conduct the significance test using paired t-tests, where we randomly split the test into 5 folds and compute the significance over accuracies these 5 subsets. In Table T compute the significance levels for [A, B, C] and D separately.
C FURTHER INVESTIGATION OF REASONS AND INNER-WORKINGS FOR TOKEN CO-OCCURRENCE REINFORCEMENT
In Section 3.2
investigate the reason of token reinforcement it is inherently embedded within the pretraining corpus. Here, we further investigate token reinforcement with two more experiments, from the learning process and attention mechanism, respectively.
Leveraging repetitive features helps maximizing likelihood. Our next experiment shows that the utlization of repetitive features a natural choice of LLMs as it helps in the pre-training stage.
To illustrate this, we focus on a conditional language modeling task that predicts tokens based on given prompt. Formally, model the probability P(wr,:r (err,)) where T and Tp are the total context length and the number of tokens to condition on, respectively.
We then consider three types prompts. The first one serves as our baseline prompt with W<T,, In the second type, mask the tokens in the prompt that appear in Wr, For the third type, we mask the same quantity of tokens as in the second type, but we select the tokens randomly.
For each type of prompts, we pretrain a language model. The base architecture of our model is the same LLaMA. Due to limitation of computational cost, we make several changes in hyperparameters for smaller size. We set the hidden size to 1024 and the FFN size to 4096, and incorporate 12 layers. The tokenizer is the same as LLaMA and the vocabulary size is 32000. This configuration results in a model with 267M trainable parameters. The dataset we use is the and the code base we use is ALaMA-Factonn
(hiyouga 2023). We pretrain each model for 50k steps, with a total batch size of 160 sentences per step. Each sentence contains 1024 tokens. Prompt length T, is 0.75 1024 768 tokens. We run our experiments on NVIDIA A100 GPUs, and each experiment takes about 30 hours to finish.
Aikipedia-english-2200
Our results are shown Figure Masking repetitive features lead to a worse converged loss compared with masking random tokens and masking. It indicates that repetitive feature is favorable in optimizing training loss, and thus making natural for language models to use repetitive features.
17

---

Published as a conference paper at ICLR 2024
Distance
Distance
llama 656 wro mask
Prob
0.4
0.20
llama 63b w/ mast
0.1
0.15
6M
lamm--33 we mask
0.2
8am mark
0.10
0.1
18
Reepetition
Repelittion
Figure 9: Left: Attention example of successive token reinforcement. "_iif" and "ute" are the two kept tokens randomly chosen. Right: Ablation study of attending preceding token. The results are based on LLaMA-65B.
Phrase Longth =
Phrase Length 3
Length
LLaMA-65B
0.6
1h keorgg Takes hdb
0.6
0.4
15
18
12
38
Repstilon
Nasotiiin
(AI: XY.
(B)....XYZZ....xx
(C): IC):
(D): Scaling
Figure 10: Phrase co-occurrence reinforcement. When disrupting the reinforcement loop with random prefixs, the self-reinforcement effect still retains. (A)-(C) plot the scaling of LLaMA models with a certain phrase length (e.g., "XY" denotes length of 2). (D) plots LLaMA-65BB probability varying phrase length. The gray shadow denotes the standard deviation across 1,000 samples.
Attending the preceding token is responsible for reinforcement. Another important question is what inner-working of LLMs responsible for token reinforcement? We start our investigation with visualization of attention weights. An example is shown on the left of Figure ? In this example, each token wy attends its preceding tokens wi-1 with relatively high attention weights. Thus, our intuition is that attending the preceding token is the key to reinforcing the connection.
To validate this hypothesis, reproduce the successive reinforcement experiment in Section 3.1 with an attention mask preventing each token from attending its preceding token. Our results are shown in the right of Figure 9 We observe that upon masking the adjacent token, the strength of reinforcement is reduced by a factor of ten. Hence, the reinforcement is not simply realized by attending to similar tokens. Rather, the information propagated through a process wherein each token iteratively attends to its adjacent token.
D SUPPORTING EXPERIMENTS OF TOKEN CO-OCCURRENCE REINFORCEMENT
In this section, we provide more experimental evidence related to the token co-occurrence reinforce- ment. The experimental settings in this section follow the same setting as in Section 3.1 except for models to use, datasets to use, and the choices of tokens.
D.1 PHRASE CO-OCCURRENCE REINFORCEMENT
We first construct consecutive tokens to create phrase reinforcement in this section. Specifically, we Without loss of generality, we place the phrase the end of each demonstration.
L, L,-L,
We construct a binary mask sequence m O..... 0. where Lp is the length of the kept phrase. Then, we can define randomly perturbed sentence š
18

---

Published as a conference paper at ICLR 2024
(R(w1, m1), (R,,, mL,),, Effectively,
keep phrase of length Lp unchanged for each sentence, and replace other tokens with random tokens. We compute PREP-P(W) = M(wl[3'; ........................ ; :-n=1; R(w,, mi ) -.,, m,))) and the average token probability that only considers the kept tokens TPN Lp Li. PREP w = Wi, n N) m,- As concrete example, we take a sentence (Apple, is, red) and m = (0. 1, and the target token w = red. Then, the demonstrations discussed will be like 'Apple red // Orange red // Banana red'. The pattern kept unchanged is in color red. In the context of in-context learning, this pattern corresponds to demonstrations like ...Answer: D' or ...Rossoose: Positive'.
Figure depicts our experimental results for repeated and discontinuous phrases. We use randomly generated sentences here. We see that disrupting the sentence-level loop with random prefixes does not break the self-reinforcement effect. The effect persists across various LLaMA models. In addition, we observe a stronger reinforcement when increasing the phrase length. More tokens are kept unchanged, higher probability the model assigns to the phrase. Particularly, Figure TT serves ablation study, cach line progressively adds one more unchanged token. A repeating phrase of 'Y.'YZ' gets reinforced more compared to the phrase "...XY, indicating that each repetitive token bolsters its subsequent tokens.
Kept Tokens
# Kept Tokens
Kept Tokens 3
Kept Tokens
opm-123m
opmm22mm
opt-125m
2oogogg Takan
opc 350m
9p: 350m
opt-330m
1.30
9p:11.b0
opt-1 30
0.4
2070
900-276
out 70
omt-125m
0.3
opt
opc 5.70
oph-33b
op: 6,76
opt 70
opt:33
opt-).28
a2t-350m
1pt-10e
epn-10 100
ro1-336
02t-2.76
o3t-6 6.70
0.01
ogt 130
o2t 30b
0.0
(B)
(C)
(D)
Figure 11: Token co-occurrence reinforcement of OPT models.
D.2 TOKEN REINFORCEMENT OF OPT MODELS
In Figure II we plot the token reinforcement of all 7 OPT models. The results are consistent with our results of LLaMA in the main context, validating the generality of token reinforcement across different families of large language models.
D.3 TOKEN REINFORCEMENT OF OTHER LLMS
We present token reinforcement on LLaMA-2 Touvron et al. 2023b) (7B, 13B, 70B), Mistral (Jiang
al. 2023)(7B), GPT-J (Wang& & Komatsuzaki 2021 (6B). Figure 12 demonstrates that all these LLMs exhibit similar token reinforcement effect, with various strength. These results consolidate our findings.
Distance
Distance
gptj-ab
Mistral-78.v..1
Lama-2-70--t
Limm
0.3
Llamm-2-70b--f
0.2
Mistral-8Bvv0.1
0.10
U1a-a-2-7b-if
Uama 2 2:13b
Usmo 700
12
15
12 Ropetttoo
Figure 12: Token reinforcement on Various LLMs.
19

---

Published as a conference paper at ICLR 2024
Kopt
Kept Tokens
Kept
Nama-75
lamara
Tama-
1am
lama
300
300
Toma
044
0.3
02
llama-7
lamm-130
0.
Itmm-330
12
38
Repatiton
Repetiiion
Ropeziion
Repobitin
(C)
Figure 13: Token co-occurrence reinforcement on Wikitext-103.
Kept
Kept
Kept
0.8
Imma77
llan--7
Iaama-33b
Th: arduge Tolan
0.6
0.4
llam--7
llam--13b
0.0
Repatison
Repetition
Mon
Repatition
(C)
Figure 14: Token co-occurrence reinforcement on BookCorpus.
Ilama-70
llama-133
Ilama-30b
Ilama-65b
0.20
rendom sanee
rendom
same
neighbour
Intattity
random
random
0.00
nelghbour
neighbour
10
15
20
5
1.0
15
20
10
35
20
15
20
Number of Regeats.
Trumber Regeets
Number of Repests.
Number of Repests
Figure 15: Token reinforcement against semantic relationships.
D.4 TOKEN REINFORCEMENT ON OTHER DATASETS
Here, we plot the token reinforcement on Wikitext- 103 and BookCorpus. As we can see, the probability of reinforced tokens is quite different in the two datasets. In BookCorpus, with 4 kept tokens, the probability can be boosted to about 0.8, whereas in Wikitext-103. the probability can only reach about 0.5. Note that compared to the results on randomly generated sentences, tokens in these datasets are more likely to co-occur in the pre-training data. This indicates the semantic relationship between tokens affects how token reinforcement performs.
D.5 SEMANTIC RELATIONSHIPS OF TOKEN REINFORCEMENT
We further conduct experiments to investigate how the semantic relationship between tokens affects the token co-occurrence reinforcement. We choose three relationships: (1) Random two tokens. (2) The same two tokens. (3) Two tokens that are similar in the embedding space.
Figure 15 plots our results for different LLaMA models. We observe clear gaps among different semantic relationships. Two tokens that are the same can build a strong connection stronger than two tokens that similar in the embedding space. Further investigating the reasons behind this is interesting and may unravel the internal biases large language models. We leave as the future work.
20

---

Published as a conference paper at ICLR 2024
Random
BookCorpus
1.0
Inberve hallo TP 1900
0.3
0.5
0.4
Tama-1h
lamr- 7b
llama-7
0.2
Lame-13s 130
[amra133
laama115b
Tama 300
lamr 300
laam--30
0.0
Tuma 600
lame 550
l1am-660
15
Repatiiion
Repesttion
Ripptition
(A)
Figure 16: Improved Ratio on three datasets.
Geneer sling CoT Ansswr.
Generating First the
bazeline Music [Qusation)
(Question)
Privocitt el Cot Nocser.
0.10
Protebsttyll
0.4
0.04
0.2
Figure 17: Left: The probability of generating the CoT answer. Right: The probability of generating the first token in CoT answer.
D.6 IMPROVED RATIO OF TOKEN REINFORCEMENT
In this section, solidify our findings with the ratio of improved token probability. Formally, the improved ratio (IR: Xu et al. 2022)) defined as follows:
IR. TP TP.
Our metric of improved ratio is defined over the setting in Section 3.1 Figure 16 plots our results of IR on three datasets. We only consider the case of two tokens.
As see, the improved ratios for all three datasets quickly reach almost 1.0 with only several repetitions, indicating token reinforcement exists in most the cases. In addition, we observe that IRs in Wikitext-100 and BookCorpus have larger variances than those in randomly generated sentences, because tokens in Wikitext- 103 and BookCorpus are more likely to co-occur in the same context and thus get larger probabilities without reinforcement.
E OTHER EXPERIMENTS ON DETRIMENTAL EFFECTS
E.1 GENERATION OF COT ANSWER
In this section, we study how the CoT answer is generated with respect our discovered surface patterns. We study the problem on the GSM8K dataset. Recall the demonstrations GMS8K, i.e., "Question: [Question] // Let's think step by step. // |CoT Answer]". We plot the probability of [CoT Answer] in Figure 17 We would expect the [Question] to have huge effect on generatin the COT answer. Thus, we involve the probability with random substitution of the [Question] part. Interestingly, we find that even without the [Question] part, the probability of [CoT Answer] still increases with demonstrations, suggesting the patterns play a crucial role in learning how generate the CoT answers.
So what does the [Question] part do in in-context learning of GSMSK? We further plot the probability of the first token of [CoT Answer]. The probability of the first token significantly decreases when mask out [uesstion]. Hence, even though the [Question] part does not affect much the probability of [CoT Answer], it substantiates the generation of [CoT Answer] at the very beginning.
21

---

Published as a conference paper at ICLR 2024
E.2 SELECTION BIAS OF LLMS
In experiments illustrated in table observe selection bias of LLaMA-65B. We show the results for each class in Table 4 Note that we randomly permute the test set to make sure all questions are balanced with [A,B,C,D].
First, we see that the zero-shot performances in different class are quite different. Class 'A' gets an accuracy as high as 71.58% and class 'D' only gets 52.28%. Second, with more demonstrations, the overall accuracy is improved, from 60.61% % to 63.16% demonstrating the effectiveness of ICL. However, the accuracy of class 'A' largely decreases, while the accuracies for 'B', 'C', and 'D' all increase. The above findings indicate that LLaMA-655 has selection bias in both zero-shot and few-shot scenarios.
Table 4: Accuracies for each class on our balanced MMLU. We use the LLaMA-65B and vary the number demonstrations from 0 to 5.
Class
0
2
A
71.58
57.31
58.25
57.31
56.96
54.74
B
59.65
66.90
69.94
70.41
69.59
70.76
C
58.95
67.02
65.50
64.80
66.78
67.60
D
52.28
59.65
60.00
59.30
59.88
59.53
Avg.
60.61
62.72
63.42
62.95
63.30
63.16
22


Reviews:
[{"review_id": "rZ2puYTm2z", "reviewer": "Reviewer_d42T", "summary": "This paper quantitatively investigates in-context learning in terms of surface patterns. \nIt shows that there is an inherent correlation between surface patterns, self-reinforcement, iterative generation and their important role in text generation.\nIn particular, it shows the role of surface surface patterns in text generation and the existence of token co-occurrence reinforcement that strengthens the relationship between two tokens based on their contextual co-occurrences.\nThe experiments on MMLU and GSM8K show that the reinforcement helps to constrain the output space and format the output according to a  demonstration such as ‘Let’s think step by step’ .", "questions": "Is it possible to provide hypothesis about the reason for the token reinforcement phenomenon and check it, even experimentally?\nThe authors may argue that analyzing reasons is the next step, i.e., outside the scope of this paper; however, it is fandamental in machine learning research.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper provide a novel framework to understanding in-context learning via  the notion of token co-occurrence reinforcement.\nThrough various experiments, the authors have shown how token reinforcement causes spurious correlations in in-context learning.", "weaknesses": "Althogh  there is a novelty in showing experimentally that token reinforcement can cause some problems in in-context learning,\nthe paper lacks the important perspective of analyzing why token reinforcement exists and causes problems. \nFor example, the following paper, which is only briefly mentioned in this paper, analyzes the impact of repetition structures in a corpus on in-context learning from an information-theoretic perspective.\nA Theory of Emergent In-Context Learning as Implicit Structure Induction\nMichael Hahn, Navin Goyal\nThey showed that the performance of in-context learning is represented by a complexity that repetition structures can be represented by a small PCFG tree, and  experimentally investigated theoretical finindings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7c5jw4rOUw", "reviewer": "Reviewer_iqcg", "summary": "This paper studies the emergent in-context learning (ICL) ability of LLM. The authors try to probe the ICL performance through surface repetitions, and establish a theory called token co-occurrence reinforcement, which explains the reasons for possible failures.", "questions": "As mentioned above, how could me mitigate the biased effect in ICL demonstrations?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This work provides meaningful explanations on the possible failures of ICL. \n- The experiments seem comprehensive and convincing.", "weaknesses": "- This work tries to understand the inherent ICL behavior of LLMs, yet is in lack of theoretical analysis. For example, how is such token co-occurrence reinforcement established? This may involve the detailed interactions between prompts and self-attention mechanism, etc., which I would like the authors to delve into.\n- As an experiment-oriented work, the authors should examine their assumptions on more LLMs; otherwise, it's hard to reach a common conclusion.\n- The findings of this work are not completely new. As far as I'm concerned, the findings are based on the distributional bias in the demonstrations. The impact of spurious correlations are widely discussed in out-of-distribution generalization literature. In this regard, more insights are welcome. Also, the authors could discuss on how to address such (inevitable) distributional bias in the demonstrations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4lUOdM5jGX", "reviewer": "Reviewer_5Dh6", "summary": "The authors delve into an investigation of the impact of token repetition on in-context learning (ICL) performance, revealing both beneficial and detrimental effects through empirical analysis. On the positive side, repetition aids in narrowing down the output space, fostering consistency in subsequent predictions. However, the downside is evident when non-informative or incorrect messages are repeated in the demonstrations, resulting in diminished prediction accuracy.", "questions": "While I do not have any specific questions at this moment, I would appreciate it if the authors could address the concerns raised in the Weakness section, particularly regarding the need for clearer motivation and enhanced clarification of certain terms and concepts.", "limitations": "", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The authors conducted extensive experiments to explore the nuances of token repetition and its effects on ICL.", "weaknesses": "1. **Lack of Clear Motivation**: The rationale behind investigating token repetition patterns is not adequately clear, leaving the reader uncertain about the study's purpose. Additionally, the abstract does not effectively capture the paper's core idea, necessitating further clarification to provide a concise summary of the work.\n2. **Need for Enhanced Clarity**: The manuscript's presentation is complex, making it challenging for readers to navigate through the content. A more detailed explanation of certain terms and concepts, such as *surface pattern*, *self-reinforcement effect*, and *token reinforcement*, would significantly enhance comprehension, particularly if these have been previously established in prior literature.\n3. **Specificity of Experiments**: The experiments are overly specific and do not sufficiently connect to real-world applications. Demonstrating how the findings on repetition patterns can be applied in practical scenarios would strengthen the paper's relevance and impact.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5AEnrBVGF0", "reviewer": "Reviewer_Ydk8", "summary": "The paper examines the phenomenon of token co-occurrence reinforcement, whereby tokens occurring in the context have a higher chance of being predicted by an LM.\nAn interesting consequence is that In-Context Learning (ICL) can go wrong when a label that is different from the target label is repeated often in the in-context examples, potentially helping explain some failure modes of ICL.\n\nIt first shows that self-reinforcement occurs with randomly constructed sequences of words. The longer the repeated part, the stronger the reinforcement effect (Figure 3).\nIt then shows that even discontinuous repeated subsequences give rise to the reinforcement effect.\nSection 4 studies the effect of the reinforcement effect on ICL. The effect can help constrain the output space and the desired pattern, but it can also lead to incorrect results.", "questions": "- Clarity: Section 3.1 uses the term \"sentence\" for the examples, but in the experiment they are \"randomly generated\". How are they sampled -- just as random sequences of symbols sampled i.i.d. from the vocabulary? Are all symbols in the vocabulary equally like to appear? The same question about how masked subsequences are resampled also applies to the other experiments.\n\n- Section 4.1, \"Learning to follow patterns\" -- when masking the Question, does this mean that the questions in the demonstrations are masked (i.e., replaced with random word sequences) but the question in the final element of the prompt (the one to respond to) is not masked?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- systematic study using openly available LLM family\n\n- interesting results about the factors impacting the success and failure of ICL. For instance, in Figure 7, the authors seem to show that that in a specific CoT prompting setup (GSM8K), replacing the questions and CoT answers in the demonstrations with random tokens does not hurt ICL performance nearly as much as replacing a separator with random tokens.", "weaknesses": "- The paper remains somewhat unclear regarding the overall contributions and implications. The paper argues that the results helps understand both limitations of ICL and the inner workings of ICL.\n To the extent that the paper aims to illuminate the \"inner workings\" of ICL, the contribution is left somewhat unclear. Do the authors argue that co-occurrence reinforcement is implicated in LMs' ability to pick up input-label mappings? And what \"inner workings\" are responsible for the co-occurrence effect? A range of recent research discussed in Section 5 aims to explain how ICL works, and observations on the self-reinforcement effect could help shed light -- and indeed the paper hints at this repeatedly (e.g., end of page 4, \"In the context of ICL, this pattern corresponds to demonstrations like\" -- it seems that the experiment is understood as some kind of simple prompt-like structure, but this idea and its implications are then not made explicit).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)