Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism (2024)

Abstract: Large language models (LLMs) exhibit remarkable in-context learning (ICL) capabilities. However, the underlying working mechanism of ICL remains poorly understood. Recent research presents two conflicting views on ICL: One emphasizes the impact of similar examples in the demonstrations, stressing the need for label correctness and more shots. The other attributes it to LLMs’ inherent ability of task recognition, deeming label correctness and shot numbers of demonstrations as not crucial. In this work, we provide a Two-Dimensional Coordinate System that unifies both views into a systematic framework. The framework explains the behavior of ICL through two orthogonal variables: whether similar examples are presented in the demonstrations (perception) and whether LLMs can recognize the task (cognition). We propose the peak inverse rank metric to detect the task recognition ability of LLMs and study LLMs’ reactions to different definitions of similarity. Based on these, we conduct extensive experiments to elucidate how ICL functions across each quadrant on multiple representative classification tasks. Finally, we extend our analyses to generation tasks, showing that our coordinate system can also be used to interpret ICL for generation tasks effectively.

## introduction large language models (llms) have demonstrated impressive in-context learning (icl) capabilities @xcite , i.e., when provided with fewshot examples, llms can effectively perform a broad range of tasks without requiring parameter updates @xcite @xcite . the simplicity of this method, combined with its zero training cost and the versatility of applying a single model across 1 our code is publicly available at: https://github.com/ plclmezboss/2d-coordinate-system-for-icl . various tasks, has made icl a promising approach to fully leveraging the potential of llms. however, the underlying working mechanism of icl remains an open question @xcite @xcite . existing works hold two conflicting views to explain icl: the first view argues llms explicitly learn from similar examples in the demonstrations during the inference stage @xcite . selecting demonstrations similar to the test sample and ensuring their label correctness, or increasing the number of demonstration shots can both improve the performance @xcite @xcite @xcite . the second view, on the contrary, suggests that llms implicitly learn tasks required for downstream applications during the pre-training stage, and in-context demonstrations simply provide cues for them to recognize the task @xcite . there have been empirical supports for this hypothesis which show that the performance of icl is insensitive to label correctness of demonstrations, or the number of demon-stration shots @xcite @xcite . although both of the above views hold in specific cases, neither can fully explain the working mechanism of icl from a holistic perspective. in this work, we seek a unified framework encompassing both views to systematically unveil icl. to do so, we first introduce the peak inverse rank metric to accurately identify the task recognition capability of llms. based on this metric, we observe that llms do not always recognize tasks during icl, even when correct labels and similar examples are provided in the demonstrations. conversely, successful task recognition also does not necessarily require the presence of similar examples and correct labels. hence, we suggest that the effectiveness of icl should be described through the interactions of these two orthogonal variables, resulting in four distinct icl scenarios. to conceptualize this intuitively, we represent each variable along two axes: the x-axis denotes perception, indicating the model's dependence on similar examples for reference. this mirrors human perception, where recognizing similar observed patterns and drawing analogies is crucial for interpreting new information. the y-axis represents cognition, reflecting the model's task recognition ability. similar to human cognition, this involves recognizing the right logic learnt from past tasks and reasoning over it to draw the answer, rather than simply replicating observed patterns. consequently, all icl scenarios are visualized within a two-dimensional coordinate system, as depicted in figure 1 . in each quadrant of the coordinate system, we systematically analyze 8 models spanning up to 40b parameters across three major llm families. we examine a wide range of classification tasks and have the following main findings: in general, our findings indicate that for similar examples, their label correctness has a consistent and significant impact on icl, especially in scenarios where the model cannot recognize the task and relies heavily on similar examples for inference (below the x-axis). increasing the number of demonstration shots substantially improves icl, as it raises the likelihood of matching similar examples. for dissimilar examples, label correctness primarily affects the model's confidence in task recognition (above the x-axis), but once the task is properly recognized, the effect becomes marginal. finally, considering the wide application of icl in generation tasks @xcite @xcite , we extend our analyses beyond classification tasks by conducting a thorough case study on a machine translation task. this study demonstrates that our coordinate system can also effectively capture the behavior of icl in generation tasks. in summary, our proposed coordinate system provides a principled and universal way to understand the working mechanism of icl. ## extension to generation tasks given the recent success of icl in generation tasks @xcite @xcite , we aim for our two-dimensional coordinate system to enhance the understanding of icl behavior not only in classification tasks but also in generation tasks. this is non-trivial, as almost no prior work has conducted an in-depth analysis of in-context generation tasks. to extend our coordinate system to generation tasks, we face two main challenges: (1) in classification tasks, predicting a single label token is sufficient, whereas generation tasks require predicting an entire sentence. (2) we determine whether icl recognizes the task by examining if the hidden states of label tokens at internal layers possess task semantics. however, for generation tasks, the label for each example in the demonstrations is not a single word but a complete sentence. to address these challenges, we formulate a hypothesis, h: we treat a generation task as multiple smaller sub-classification tasks focused on predicting each token. in each sub-classification task, every label sentence contains a token that serves as the label token. however, the position of the label token in each label sentence changes when predicting different tokens. according to this hypothesis, each icl subclassification task in generation tasks is equivalent to the standard icl in classification tasks we previously studied, as both involve predicting a single token, with the label token still represented by a single token. however, verifying our hypothesis h for most generative tasks is challenging. in classification tasks, the task of predicting each token (i.e., a label token) is very clear and specific. for instance, in sst-2, it is a "sentiment" task. in contrast, for generative tasks, predicting a token may involve mimicking the abstract style of a segment figure 11: for the in-context translation task with a strict subject-verb-object structure: "sentence: 她喝水。 label: she drinks water. sentence: 我们吃米饭。 label: we eat rice. sentence: 他们看电视。 label: they watch tv. sentence: 他读 书。 label:" , the pir of "subject," "verb," "object" at each token of the final label sentence when respectively predicting "he," "reads," "book," on llama-2-7b. therefore, by decomposing the entire generative task into multiple sub-classification tasks, our coordinate system can help understand the working mechanisms of in-context generative tasks. ## related work current research on icl mechanisms mainly falls into the following categories: theoretical framework. recently, numerous studies have employed theoretical frameworks to enhance the understanding of icl. @xcite describe icl as implicit bayesian inference. @xcite demonstrate that transformers can learn linear functions through icl. additionally, several studies conceptualize icl as gradient descent on an implicit internal model @xcite @xcite . @xcite and pan (2023) disentangle icl into task recognition and task learning. empiricism. various factors affecting icl have been studied, such as the order of examples @xcite , the choice of label words @xcite , and the selection of demonstrations @xcite . effective demonstration strategies @xcite can notably boost icl performance. logit lens. recently, some studies use the logit lens technique @xcite to project complex feature representations into the vocabulary space to study the mechanisms of icl. @xcite discover three distinct stages of processing by decoding the next token prediction at each layer. @xcite find that a small number of attention heads transport a compact representation of the demonstrated task. @xcite provide insights into the mechanisms of icl in the context of task learning. ## conclusion in this paper, we map two variables whether llms can recognize the task and the presence of similar examples in the demonstrations onto the y-axis and x-axis of a 2d coordinate system, to visualize icl scenarios. first, for classification tasks, we conduct a systematic study of the proposed coordinate system and describe in detail the working mechanisms of icl in each quadrant. then, we extend our analyses beyond classification tasks through a thorough case study on a machine translation task. our proposed coordinate system offers a universal framework to better understand icl.