In-context Learning and Gradient Descent Revisited (2024)

Abstract: In-context learning (ICL) has shown impressive results in few-shot learning tasks, yet its underlying mechanism is still not fully understood. A recent line of work suggests that ICL performs gradient descent (GD)-based optimization implicitly. While appealing, much of the research focuses on simplified settings, where the parameters of a shallow model are optimized. In this work, we revisit evidence for ICL-GD correspondence on realistic NLP tasks and models. We find gaps in evaluation, both in terms of problematic metrics and insufficient baselines. We show that surprisingly, even untrained models achieve comparable ICL-GD similarity scores despite not exhibiting ICL.Next, we explore a major discrepancy in the flow of information throughout the model between ICL and GD, which we term Layer Causality . We propose a simple GD-based optimization procedure that respects layer causality, and show it improves similarity scores significantly.

## introduction in recent years, large language models have shown strong emergent in-context learning abilities @xcite -where a pretrained model's performance significantly improves on a task by conditioning the language model on a small set of input-label pairs (demonstrations). despite substantial research, the inner workings of icl remain elusive. at face value, in-context learning and gradient descent-based finetuning have very little in common. nevertheless, a series of recent studies discuss apparent similarities between icl and gradient descent-based optimization, mostly in synthetic scenarios @xcite @xcite . the claim this body of research aims to make is that * equal contribution icl can implement implicit gd, using in-context demonstrations as training examples. while most of the synthetic setups concern: (1) restricted transformers, (2) simplified regression tasks, and (3) direct training for icl -the work of @xcite stands out in its ability to demonstrate an ostensible similarity between icl and gd optimization in (1) full-fledged transformers, (2) for realistic nlp tasks, (3) naturally occurring in models trained only on causal text generation. we call the hypothesis that icl mimics finetuning on the model itself -as is analyzed in @xcite -the strong icl-gd correspondence. we will later discuss how this diverges from the icl-gd correspondence other works consider. in this paper, we make two main complementary contributions. we perform a careful re-analysis of the work of @xcite and show how seemingly mild problems in evaluation lead to a significant overestimation of similarity between the two procedures. surprisingly, we find that untrained models achieve similarity scores at least as good as trained ones. this result provides strong evidence against the strong icl-gd correspondence. 1 secondly, in an attempt to relax the strong icl-gd correspondence hypothesis, we suggest a rectified version of gd that we show aligns better with icl. to do this, we first identify a core discrepancy in the flow of information throughout the model between in-context learning and vanilla gradient descent, which we call layer causality. in icl, the information that influences the hidden state comes from the output of shallow layers ("earlier layers") alone. in gd, however, the update to the weights of a layer depends on gradients, which come from all of the model layers including deeper ("later layers"). we showcase the importance of this simple observation by suggesting a simple variant of gd that incorporates layer causality. this simple modification, layer causal gradient descent (lcgd), consistently improves upon vanilla gradient descent on the similarity metrics. notably, it outperforms the trained transformer significantly in terms of both similarity metrics. in comparison to the untrained baselines, it significantly surpasses them in attention map similarity (simam ∆ ) and is consistently on the high end in terms of hidden state similarity (simaou). in spite of that, the scores are still low. this can be due to a suboptimal choice of hyperparameters but likely has to do with inherent problems in the strong icl-gd correspondence hypothesis, even with the layer causal version we propose. we leave this for future work to explore. lastly, we dedicate a short discussion to the line of work on synthetic settings that builds on insights from von @xcite . we observe terminology differences with @xcite that might cause confusion. "gradient descent" is used differently in both cases. while synthetic settings usually consider gradients of shallow implicit functions, @xcite consider complex gradients with respect to the model itself. in the synthetic setting, layer causality is often trivially satisfied. our contributions are the following: ## preliminaries in this work, we build on the benchmark proposed by @xcite . we focus on its setting using the same datasets and examine the same similarity metrics to compare the behavior of icl and finetuning. this section provides details on the benchmark they use. in the next section, we will address problems in the metrics described below. ## conflation of terms in icl-gd correspondence works rooted in the work of von oswald et al. ## discussion in this work, we provide different perspectives on the icl-gd correspondence. we show evidence against it but also show that it might be fixed. we find that previous work does not justify the strong icl-gd correspondence, and instead discusses a weaker notion of a shallow gd. this should also apply to layer-causal gd, as it is designed as a modification of the strong icl-gd correspondence. still, we see it outperforms untrained transformers in terms of attention map similarity (and fares well in terms of hidden state similarity). this can be due to irrelevant causes (see limitations below). however, it is worth noting that the layer-causal variant can be justified by its similarity to the kernel regression and functional gd variants that have been addressed in the literature on synthetic settings @xcite . future work can use the (corrected) similarity metrics suggested in @xcite to gauge the similarity of shallow gd methods to icl. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 7 limitations ▷ similarity metrics: the similarity metrics we use only consider a very specific correspondence between icl and gd, where each layer applies gd to the model. however, it is possible that the exact mechanism is different (e.g. not all layers do gd). to make a more general claim, other models should be tested too. ## related work many works consider synthetic settings @xcite @xcite . they are mostly concerned with icl implementing gd of a shallow model, mostly variants of linear models or kernel regression. unlike these works, @xcite , which we are heavily influenced by, study large gpt transformers on structured language classification tasks. gradient descent in @xcite is with respect to the transformer itself, which is also a significant departure. @xcite show how a transformer can implement the backward pass of another (smaller) transformer in its forward pass. as far as we know, there is no indication that this process is happening in real-world models. recently, new works have emerged @xcite suggesting a different approach to interpreting icl as an algorithm that compresses training demonstrations into a function/task vector that steers the model to perform the task. other perspectives of icl include induction heads @xcite and bayesian inference @xcite . the work of @xcite points to another discrepancy between full-batch gd and icl. they show that vanilla full-batch gd and icl cannot be reconciled due to icl's sensitivity to the order of the demonstrations, while full-batch gd is invariant to it. however, this discrepancy can be mitigated easily by applying gd sequentially, as was done in the work of @xcite that we compare to. layer causal gd is similar to @xcite , where a similar idea was proposed to accelerate training by finding a good starting point using a greedy layer-wise approach. ## conclusions inspired by recent work, we explore the relationship between in-context learning and gradient descent-based finetuning in practical settings. we show problems with the strong version of the icl-gd correspondence. we correct the similarity metrics used in prior work and propose alternatives. furthermore, we show that a simple baseline of untrained models has higher similarity scores compared to trained models. our work suggests considering the possibility that only a weak version of icl-gd holds. we rely on layer causality to further justify this view. we study a potential workaround to this problem (lcgd) that does not violate layer causality and get mixed results. the study of lcgd is not comprehensive enough to make a definite statement for or against layercausal gd mesa-optimizers. we note a potential connection to kernel regression and functional gd, that come up in works on synthetic setups that uphold the weak icl-gd correspondence. we leave for future work to elucidate the nature of these connections, as well as propose better layer-causal variants.