Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning (2024)

Abstract: Fine-tuning and in-context learning (ICL) are two prevalent methods in imbuing large language models with task-specific knowledge. It is commonly believed that fine-tuning can surpass ICL given sufficient training samples as it allows the model to adjust its internal parameters based on the data. However, this paper presents a counterintuitive finding: For tasks with implicit patterns, ICL captures these patterns significantly better than fine-tuning. We developed several datasets featuring implicit patterns, such as sequences determining answers through parity or identifying reducible terms in calculations. We then evaluated the models’ understanding of these patterns under both fine-tuning and ICL across models ranging from 0.5B to 7B parameters. The results indicate that models employing ICL can quickly grasp deep patterns and significantly improve accuracy. In contrast, fine-tuning, despite utilizing thousands of times more training samples than ICL, achieved only limited improvements. We also proposed circuit shift theory from a mechanistic interpretability’s view to explain why ICL wins.

## introduction adapting pre-trained models to specific tasks or domains is commonly achieved through fine-tuning @xcite or in-context learning @xcite . fine-tuning, a well-established method, involves further training a pre-trained model on a smaller, domain-specific dataset, directly updating the model's parameters to retain improvements across various contexts and scenarios. in contrast, in-context learning (icl) enhances task performance by incorporating taskspecific examples into prompts, guiding the model in task completion without altering its parameters during training. figure 1: (a) a simple example of an implicit pattern detection task. the given problem (arithmetic expression calculation task in this figure) can be solved in either a formal way, e.g., directly calculating, or by exploiting the detected implicit pattern as a shortcut. (b) illustration of implicit pattern detection for in-context learning and fine-tuning. for icl, several examples with answers are given in context, and a further new question is used to test accuracy. for fine-tuning, llm learns from single examples using parameter update methods like full-parameter fine-tuning or peft methods. there has been much debate about the pros and cons of fine-tuning and in-context learning. finetuning is praised for its ability to bring permanent memorization to models @xcite , and it can perform well even with a small amount of training data @xcite . however, critics argue that fine-tuning demands substantial computational resources @xcite and can encounter issues such as catastrophic forgetting @xcite . this conserves computational resources but necessitates longer prompts and incurs higher inference costs. how about icl? it is favored for its trainingfree nature @xcite , allowing prompts to be easily changed for adaptation to other domains without re-training @xcite . other works @xcite showed that icl can help the model uniquely identify a discrete function sample-efficiently. reseach @xcite showed that icl is driven by the abrupt emergence of an induction head, which subsequently competes with in-weights learning. other works @xcite observed that icl and gradient descent modify the output distribution of language models differently. despite these advantages, icl is limited by context length restrictions and incurs higher costs during each inference stage due to the longer prompts required. essentially, the primary distinction between finetuning and icl lies in parameter updating; all finetuning methods modify the model's parameters. it might seem, therefore, that icl's impact is less profound. however, our research reveals a counterintuitive finding: for datasets with implicit patterns, icl is more adept at uncovering these latent patterns than fine-tuning. to investigate this phenomenon, we designed datasets containing implicit patterns across various domains, including two mathematical tasks: expression calculation and boolean function. one textual task: relation reasoning, and one code reading task. these domains share a common trait: the presence of implicit patterns that can simplify problemsolving. we evaluated llms' capability to recognize such patterns with these datasets. our findings include: (1) both fine-tuning and icl could detect and utilize implicit patterns, resulting in increased test accuracy. (2) icl performed much better than fine-tuning in implicit pattern detection, e.g., iclbased models enjoyed higher test accuracy. (3) icl also showed strong performance in robustness tests and ood data tests. our experiments demonstrate that the ability of llms to leverage implicit patterns significantly enhances their problem-solving capabilities, providing a clear advantage for tasks involving complex data structures. understanding the operational principles of llms is crucial for their safety and ethical implications and can further promote improvements. therefore, we delved deeper into the mechanisms behind this phenomenon. from a mechanistic interpretability perspective @xcite , we proposed the circuit shift theory. circuits are certain groups of attention heads and mlp layers @xcite . a shift in circuits typically represents the model adopting a different method in problemsolving. our findings indicated that icl resulted in a larger-scale circuit shift compared to fine-tuning, which means that with icl, the model changed its problem-solving method more significantly for implicit pattern detection and utilization. we also provided a visualized heatmap of circuits for detailed observation. in summary, our contributions are threefold: implicit pattern detection dataset. we defined and illustrated the implicit pattern detection task, then developed a dataset across mathematics (expression calculation, boolean function), textual reasoning (relation test) and code (output guessing). ## background transformer. transformer @xcite is the cornerstone architecture for llms nowadays, with its breathtaking ability in parallel training and sota performance. one transformer model f trf usually consists multiple of transformer layers f layer and an embedding layer f emb . for an input sequence (typically ids after tokenization) x 0 ∈ r n×1 with length n, it first passes through an embedding layer f emb with hidden state size d, then passes all the transformer layers, and finally gets an output o @xmath0: fine-tuning. fine-tuning is a process where a pre-trained llm is further trained on a specific task or dataset to improve its performance for that particular application. suppose there exists a pre-trained transformer model f trf with learnable parameters in llms is an emergent capability where the model uses the provided context to perform tasks. given a special task f and a series of prompt inputs x 1 , • • • , x n , icl happens when these inputs and their answers y 1 = f (x 1 ) are given in multi-shot, i.e., (x 1 , y 1 , • • • , y n , x n+1 ). in this scenario, the goal for llm to do icl is to learn the task f and accurately predict y n+1 . this phenomenon allows the model to adaptively handle a variety of tasks, such as translation, question-answering, and more, simply through appropriate prompt engineering. icl happens in inference-stage without explicit re-training, thus resulting in more friendly requirements for gpus @xcite . even llama-3 70b could run on a single 3090 gpu with powerinfer @xcite . ## implicit pattern detection test through detailed observation and thinking, humans can detect some underlying, non-explicit patterns within the data. this enables us to solve problems more efficiently. implicit pattern detection refers to the ability of models to recognize underlying, non-explicit patterns within data, enabling them to solve problems more efficiently. this concept is illustrated through tasks such as arithmetic calculations, where the model can bypass complex operations by identifying simplifying patterns. for instance, in mathematical expressions (see figure 1 and figure 2 ), a model might detect that certain terms have negligible impact and can be ignored, leading to quicker computations. we will give a detailed description of our dataset design and experimental settings in the following sections. ## results and analysis in this section, we present our results for the implicit pattern finding tasks following the experimental setting in section 3.2. we show that icl achieved an overall higher level of accuracy over fine-tuning on these four tasks. we also show that the improvement of accuracy with icl mainly comes from the detection of those implicit patterns in section 5 and refsec:circuit. ## explanation of icl's victory: circuits shift theory understanding the inner mechanisms of llms greatly benefits their ethical use and safety. we have found that icl performs much better than fine-tuning on implicit pattern detection, and in this section, we try to explain why. from a mechanistic interpretability perspective, we investigate this problem using circuits. circuits are specific pathways (typically combinations of attention heads and mlp layers) within a model responsible for processing and interpreting particular patterns or tasks. the change in circuits for llms represents a shift in their inner mechanisms, revealing that llms choose different ways to solve problems. based on this viewpoint, we propose a theory: circuits shift, to explain this phenomenon. we will first provide a method for probing circuits, explaining what they are and the types of circuits we found in icl-based and fine-tuning-based models. then we will show that the reason icl performs better than fine-tuning is that the circuits in models experience a more significant shift. a detailed explanation of circuits and experimental settings can be found in appendix d. ## conclusion in conclusion, our research demonstrates that in-context learning (icl) significantly outperforms fine-tuning in capturing implicit patterns within specific tasks. through our experimental evaluations, we observed that icl not only enhances task performance more effectively but also exhibits greater adaptability in problem-solving approaches, as evidenced by the notable shifts in model circuits.