Re-examining learning linear functions in context

Authors: Omar NAIM, Guilhem Fouilhé, Nicholas Asher
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: In context learning, GPT, limitations

Abstract:
In context learning (ICL) is an attractive method of solving a wide range of problems.  Inspired by Garg et al., we look closely at ICL in a variety of train and test settings for several transformer models of different sizes trained from scratch.  Our study complements prior work by pointing out several systematic failures of these  models to generalize to data not in the training distribution, thereby showing some limitations of ICL. We find that models adopt a strategy for this task that is very different from standard solutions.

Decision: reject
Reviewer scores: 5, 1, 3, 5

Reviews and discussion:
<thread>
creation date:
1730361115636

thread_type:
official_review

<thread_object>
creator: reviewer_i5ec

<content>
summary:
The paper investigates in-context learning (ICL) across various training and testing scenarios using different sizes of transformer models trained from scratch. Building on previous work, it highlights systematic failures in these models' ability to generalize to data outside the training distribution, revealing some limitations of ICL.

strengths:
1. The paper focuses on an important and challenging problem: understanding the in-context ability of language models.
2. The writing is clear and easy to understand.
3. The authors provide code and detailed instructions for reproduction.

weaknesses:
1. The models and empirical studies in the paper differ significantly from current large language models, potentially creating a gap between the claims and reality.
2. The findings of the paper have been previously proposed in other works.
3. The paper is missing some key references.

questions:
Can you explain how your findings differ from the following paper? In particular, [1] also discusses how distribution influences the in-context learning ability to learn linear functions.

[1] Trained Transformers Learn Linear Models In-Context.
[2] Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection
</content>
</thread_object>

</thread>
<thread>
creation date:
1730491434075

thread_type:
official_review

<thread_object>
creator: reviewer_2uvx

<content>
summary:
The study investigates in-context learning (ICL) in transformer models, focusing on their ability to learn and generalize linear functions from contextual prompts. Inspired by previous work, the authors examine various transformer models, including small ones trained from scratch, to explore whether they can learn linear functions and generalize beyond the training distribution.

However, there are two main problems in this paper:

### 1. The writing problem: There are many typos, e.g., in ``line 047'', there should be a ''.'' after ''training data''.
### 2. Novelty: The paper indeed provides robust experiments to show the main point, but it lacks novelty, such as how to improve this problem.

strengths:
The paper has the following strengths:

### 1. Clear Motivation: The paper begins with a well-defined motivation, addressing gaps in the current understanding of in-context learning (ICL) in transformer models, especially for generalization.

### 2. Comprehensive Experiments: The experiments cover various transformer architectures and test them on various distributions.

weaknesses:
The paper has the following weaknesses:

### 1. Clarify Terminology and Notation: The writing is a little poor. For example, in ``line 047'', there should be a ''.'' after ''training data''. Furthermore, the table should be in a more beautiful structure.

### 2. Explanation for the Problem: Although the paper provides various experiments, it should explain the failures of these models to generalize to data not in the training distribution.

### 3. Novelty: The paper provides robust experiments to show the main point but lacks novelty, such as how to improve this problem.

questions:
1.. Explanation for the Problem: Could you please explain the failures of these models to generalize to data not in the training distribution?
2. Could you please provide some methods to improve this problem?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730678727833

thread_type:
official_review

<thread_object>
creator: reviewer_zjmu

<content>
summary:
The paper investigates Transformer behavior when trained from scratch to perform linear regression. It examines out-of-distribution (OOD) generalization across various settings, such as different ranges and distributions of linear functions.

strengths:
The paper conducts thorough experiments across various scales and settings, providing a comprehensive analysis of Transformer behavior.

weaknesses:
1. The related work could benefit from a more comprehensive review. The paper primarily discusses the works of Garg et al., Akyürek et al., and Von Oswald et al. on regression for in-context learning (ICL), but there are additional relevant studies in this area that are not cited. A more thorough literature review, covering empirical and theoretical works on regression in ICL, would enhance the paper’s context. Checking recent citations in this line of research may help identify key studies to include.

2. The notation in Section 4 could be clarified, as some symbols are difficult to interpret. For example, it’s not immediately clear what $\sigma$ represents in the context of $f_{i, \sigma}$. Additional explanations could help improve readability.

3. The organization of the paper could be refined to improve the overall flow. At times, the presentation feels somewhat informal, with experiments presented in sequence without clear connections, motivations, or in-depth analyses. For instance, it would be helpful if the authors could clarify the rationale for studying models of different scales and discuss what insights are gained from these comparisons. Additionally, mixing experiments on different scales and distributions makes it challenging to understand the primary conclusions. This structure could make it clearer to the reader what the authors aim to convey.

In general, I appreciate that the authors highlight the out-of-distribution (OOD) generalization issue for Transformers trained on linear regression, as initially noted by Garg et al. However, the experimental findings in Section 4 could be more impactful with clearer motivations and discussions. The hypothesis regarding induction heads and their role in OOD performance is somewhat interesting, though it could be strengthened with supporting theoretical insights or experimental validations, such as through mechanical interpolation. Presenting this hypothesis with additional rigor could provide more substantial contributions to the community.

questions:
1. While it seems intuitive that, for instance, "9L6AH" refers to a model with 9 layers and 6 attention heads, this notation is somewhat non-standard. It would be helpful if the authors could define this notation explicitly before using it. Many other notations in the paper also follow this informal style, though I haven’t listed each instance. It would be beneficial if the authors could standardize and define these terms clearly at the outset.

2. In line 193, could the authors clarify whether it is $D^T_F$ or $D^t_F$? There are also several other typos throughout the paper that I haven't enumerated. Clarifying these would improve overall readability and precision.

3. In line 190, it’s unclear why the authors mention that the coefficients are in the range $[-1, 1]$, as this differs from the $N(0, 1)$ distribution. Additionally, there is no supporting figure or result indicating that coefficients within $[-1, 1]$ lead to zero MSE error. Given that I generally observe non-zero but small MSE error, it would be helpful if the authors could clarify this paragraph, particularly regarding the model size required to achieve zero average MSE error.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730933597655

thread_type:
official_review

<thread_object>
creator: reviewer_jbsd

<content>
summary:
This paper studies experimentally the setting of in-context learning linear regression . The authors reproduce the experiments of Garg et al and at inference time test the models with 1) different distributions for the input/weight vectors 2) larger values for the input/weight vectors.
Based on the observations of these results the authors argue that these models do not learn some type of algorithm.

strengths:
Understanding what these models learn even in the setting of linear regression can significantly enhance our understanding of their capabilities limitations. Indeed it has been observed that the models do not generalize in out-of-distribution samples and thus it is unclear whether these models learn some type of algorithm.

weaknesses:
1. The provided experimental study does not explain what these models are actually learning. For example it can be the case that the model are learning a tailor-made preconditioned gradient descent type of algorithm, with the preconditioned matrix being optimal for the in-distribution values and sub-optimal for out-of-distribution values.
2. It cannot be excluded that the current training methods are not optimal, since we know that these models do have the capability of representing these algorithms. 
3. Some of these results have already been observed experimentally for example see [1] (Figures 5,6).  In these experiments consider multi-dimensional linear regression, they keep all expect for one dimension fixed and plot how the function changes when varying one dimension from [-B,B] similar to the authors' experiments for one dimensional linear regression.

In general the main weakness of this paper is that it does not make a convincing argument towards what these models are actually learning. 
[1]: Giannou, Angeliki, et al. "How Well Can Transformers Emulate In-context Newton's Method?." arXiv preprint arXiv:2403.03183 (2024).

questions:
I agree with the authors that these models do not exactly learn some type of algorithm but I think that the main question is why these models do not do so while they have the expressivity ? One possible explanation is that there exist  parameters that better interpolate the specific distributions, while existing algorithms work for any type of distribution. 

Did the authors try to train the models with multiple distributions? It could be the case that then the models are able to perform some type of algorithm by not fine-tuning their weights to fit a specific distribution. Furthermore, considering the second point above, did the authors perform a search over the hyperparameters for training?
</content>
</thread_object>

</thread>


Metareview: