Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023)

Abstract: Large pretrained language models have shown surprising in-context learning (ICL) ability. With a few demonstration input-label pairs, they can predict the label for an unseen input without parameter updates. Despite the great success in performance, its working mechanism still remains an open question. In this paper, we explain language models as meta-optimizers and understand in-context learning as implicit finetuning. Theoretically, we figure out that Transformer attention has a dual form of gradient descent. On top of it, we understand ICL as follows: GPT first produces meta-gradients according to the demonstration examples, and then these meta-gradients are applied to the original GPT to build an ICL model. We comprehensively compare the behaviors of in-context learning and explicit finetuning on real tasks to provide empirical evidence that supports our understanding. Experimental results show that in-context learning behaves similarly to explicit finetuning from multiple perspectives. Inspired by the dual form between Transformer attention and gradient descent, we design a momentum-based attention by analogy with gradient descent with momentum. The improved performance over vanilla attention further supports our understanding from another perspective, and more importantly, shows the potential to utilize our understanding for future model design. The code is available at https://aka.ms/icl .

## introduction in recent years, large pretrained language models, especially in transformer-based architectures (e.g., @xcite , have shown strong emergent in-context learning (icl) ability @xcite . different from finetuning which needs additional parameter updates, icl just needs several demonstration examples prepended * contribution during internship at microsoft research. before the query input, and then the model can predict labels for unseen inputs. on numerous downstream tasks, large gpt models can achieve surprising performance, which even exceeds smaller models with supervised finetuning. however, although icl has achieved great performance, its working mechanism is still an open question to be investigated. in this paper, we explain in-context learning as a process of meta-optimization and analyze connections between gpt-based in-context learning and finetuning. concentrating on the attention modules, we figure out that the transformer attention has a dual form of gradient descent. on top of it, we propose a novel perspective to explain in-context learning: (1) a pretrained gpt serves as a meta-optimizer; (2) it produces meta-gradients according to the demonstration examples through forward computation; (3) the meta-gradients are applied to the original language model through attention to build an icl model. as illustrated in figure 1 , in-context learning and explicit finetuning share a dual view of gradient descent, where icl produces meta-gradients through forward computation, while finetuning computes gradients by back-propagation. therefore, it is reasonable to understand in-context learning as implicit finetuning. in order to provide empirical evidence to support our understanding, we conduct comprehensive experiments based on real tasks. on six classification tasks, we compare the model predictions, attention outputs, attention weights to query tokens, and attention weights to training tokens between in-context learning and finetuning. experimental results validate that the behavior of in-context learning is similar to explicit finetuning from multiple perspectives. these results are strong evidence to prove the reasonability of our understanding of in-context learning as implicit finetuning. further, inspired by the dual form between transformer attention and gradient descent, we design a momentum-based attention, which regards the attention values as meta-gradients and applies the momentum mechanism @xcite to them. experiments on both language modeling and in-context learning show that our momentum-based attention consistently outperforms vanilla attention, which supports our understanding of meta-optimization again from another perspective. we note that beyond this preliminary attempt, our understanding may have more potential to enlighten model design, which is worth investigating in the future. our contributions are summarized as follows: ## understanding in-context learning (icl) as implicit finetuning we first qualitatively analyze the transformer attention under a relaxed linear attention form to figure out a dual form between it and gradient descent. then, we compare in-context learning with explicit finetuning to analyze connections between these two optimization forms. based on these theoretical findings, we propose to understand in-context learning as implicit finetuning. ## momentum-based attention inspired by dual form of transformer attention we have figured out the dual form between transformer attention and gradient descent. as illustrated in figure 2 , inspired by this dual view, we investigate whether we can utilize momentum @xcite , a widely used technique for optimization algorithms, to improve transformer attention. figure 2 : inspired by the dual form between attention and gradient descent, we introduce the momentum mechanism into transformer attention by analogy with gradient descent with momentum. experiments on language modeling first, we evaluate the effect of momentum-based attention on language modeling. we train two gpt models with 350m parameters from scratch, where one is the vanilla transformer, and another applies momentum to attention. more training details are provided in appendix c. we evaluate the perplexity ## related work recently, some pieces of work have attempted to understand the inference mechanism of in-context learning. xie et al. (2022) explain in-context learning as implicit bayesian inference. they state that in-context learning emerges when language models can infer the shared latent concept among the demonstration examples, which is learned during pretraining. on another aspect, @xcite focus on specific modules in transformers. they find some induction heads in transformers that refer to abstract patterns in previous sequences to help predict the next token. they indicate that the induction heads drive the ability of in-context learning. different from them, we concentrate on the learning algorithm of icl and explain it as a process of meta-optimization. some other work also studies the learning algorithm of icl. as a case study, @xcite show that transformers can be trained to in-context learn a class of linear functions and the performance is comparable to the least squares estimator. based on linear regression, @xcite prove that they can construct parameters of transformers to implement gradient-descent-based learning algorithms. further, they show that models trained with an in-context learning objective tend to match the behavior of models computed by explicit learning algorithms. also based on regression tasks, von @xcite show that linear attention-only transformers with constructed parameters that implement gradient descent and models learned by an in-context learning objective are highly related. compared with them, we are the first ones to explain in-context learning in real scenarios. to be specific, (1) we analyze in-context learning for off-the-shelf gpt models, instead of models trained from scratch by an icl objective; ## conclusion in this paper, we aim to explain the working mechanism of gpt-based icl. theoretically, we figure out a dual form between transformer attention and gradient descent, and propose to understand icl as a process of meta-optimization. further, we analyze connections between icl and explicit finetuning and show the reasonability to regard icl as implicit finetuning. empirically, we comprehensively compare icl and finetuning based on six real nlp tasks. the results prove that icl behaves similarly to explicit finetuning from multiple perspectives. further, inspired by our understanding of meta-optimization, we design a momentum-based attention that achieves consistent performance improvements over vanilla attention. we believe our understanding will have more potential to enlighten icl applications and model design in the future.