What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023)

Abstract: Large language models (LLMs) exploit in-context learning (ICL) to solve tasks with only a few demonstrations, but its mechanisms are not yet well-understood. Some works suggest that LLMs only recall already learned concepts from pre-training, while others hint that ICL performs implicit learning over demonstrations. We characterize two ways through which ICL leverages demonstrations. Task recognition (TR) captures the extent to which LLMs can recognize a task through demonstrations – even without ground-truth labels – and apply their pre-trained priors, whereas task learning (TL) is the ability to capture new input-label mappings unseen in pre-training. Using a wide range of classification datasets and three LLM families (GPT-3, LLaMA and OPT), we design controlled experiments to disentangle the roles of TR and TL in ICL. We show that (1) models can achieve non-trivial performance with only TR, and TR does not further improve with larger models or more demonstrations; (2) LLMs acquire TL as the model scales, and TL’s performance consistently improves with more demonstrations in context. Our findings unravel two different forces behind ICL and we advocate for discriminating them in future ICL research due to their distinct nature.

## introduction large language models (llms) have demonstrated the ability to perform in-context learning (icl), i.e., "learning" to perform a task purely from examples in the context without any parameter updates @xcite . this powerful and flexible phenomenon enables llms to be used as general-purpose models that can perform any task with a small set of labeled examples. however, there is still no consensus on how incontext learning works. some previous work hy1 our code is publicly available at https://github.com/ princeton-nlp/whaticllearns . ## results figure 2 shows our main results with gpt-3, llama, and opt with our 3 settings: gold, random, and abstract. below we summarize the trends of tr and tl across different conditions. ## related work many works have attempted to deepen empirical or theoretical understanding of icl since its emergence in @xcite . for instance, xie et al. (2022) present a theoretical framework where latent "concepts" parameterize each document in pre-training. they posit that all concepts have been learned in pre-training; thus, icl is the result of implicit bayesian inference, where the lm uses incontext demonstrations as evidence to identify the correct concept. @xcite present empirical evidence for this framework by showing that only limited information, rather than true input-label mappings, is needed to perform icl. other works investigate the impact of the pretraining corpus on icl. @xcite identify properties of the pre-training distribution that enable icl behavior, including burstiness, label multiplicity, and a long-tailed class distribution -all of which are satisfied by natural language. @xcite show that the frequencies of terms in the pre-training corpora is positively correlated with model performance. @xcite show that both a rich training distribution and a sufficiently large model are critical to the development of in-context learning abilities. more recently, several works have explored theoretical frameworks in which icl can be seen as im-plicit gradient descent, treating a forward pass over the in-context demonstrations as an "update" to an implicit internal model. @xcite @xcite . for mechanistic perspectives on icl, @xcite and @xcite identify induction heads (subnetworks that perform in-context pattern recognition) in small and large models, respectively. while our conclusions align with aspects of previous studies, our work contributes novel insights on multiple axes. @xcite also show that even small models can perform tr and argue that the performance gap between gold and random is consistently small, but most of their experiments are @xmath0; we show that as model sizes scale, gold tends to improve while random does not. thus, the performance deficit of random grows as models become larger. yoo et al. (2022) also perform similar experiments to random and abstract, but they do not deeply investigate the effect of model sizes or numbers of demonstrations. contemporary work @xcite obtain similar results; additionally, they show that instruction-tuning strengthens the model's semantic priors more than it improves tl. however, they primarily focus on closed-source models, whereas we also conduct experiments on public models such as llama and opt. collectively, our findings offer a comprehensive understanding of how icl works across scales. ## conclusion while previous work often studies icl as an umbrella term, regardless of model sizes and numbers of examples, we argue that there are two distinct characterizations of icl -task recognition and task learning -and demonstrate that they emerge under different conditions. even small models are capable of performing tr, but this ability does not scale. on the other hand, tl is an emergent ability of large models; small models are unable to perform tl even when provided with more demonstrations, whereas large models can leverage more demonstations to improve their tl performance. we suggest that future work on icl should distinguish the two phenomena and clearly state the conditions under which the experiments are conducted.