Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors pointed out that the reason why in-context learning (few-shot learning without parameter updates) can be achieved with large left-to-right (causal) language models is due to the data distribution.

Compared with in-weights learning (learning with parameter updates), the authors found that in-context learning is superior when the training data has the following characteristics: (1) burstiness (degree of unevenness as time series data), (2) large number of classes or heavy-tailed distribution (3) tokens have dynamic meaning. Also, they found that these characteristics are not valid in the recurrent architecture, but are more pronounced in the Transformer architecture.
strengths_and_weaknesses: ### Strengths

It is fascinating to see research that provides an analysis from a new perspective on distributional properties of the training data to the important issue of realizing few-shot learning with causal language models.

### Weaknesses

The experiments are not controlled.

The three dependent variables are different in in-context learning and in-weights learning: (1) binary classification <-> multi-class classification (2) no model update <-> model update (3) classification for holdout classes <-> classification for trained (or common) classes. Therefore, we do not know which explanatory variable (burstiness, heavy-tail, dynamic semantics) works for which dependent variable. It must be said that the scientific findings are unclear.
questions: I assume "burstiness" refers to the so-called Taylor's law (bias as time series data), but can it be expressed in short training data of eight examples? Isn't it simply the uniformity of the frequency (cf. time series) that is changing...?
limitations: Limitations (future directions) are carefully described in Discussion section.
review_title: null

final_score: rating: 9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work performs a detailed investigation of the factors that promote different types of learning in transformers:  in-context vs. in-weights. The results delineate which aspects of the training distribution promote one form of learning or the other, or even both together. Further experiments find that recurrent models do not demonstrate the same learning abilities as the transformers studied here.
strengths_and_weaknesses: This is an elegant and important paper. The investigations and their results are presented in a clear and compelling way, making for a real page-turner! 

The work makes several contributions. The two types of learning studied here (in-context and in-weights) are neatly defined and explained. The experiments demonstrate how in-context learning is promoted by several specific properties of the training data:  burstiness, number of classes, and the Zipfian exponent of their marginal distributions. Especially exciting are the results indicating that a Zipfian marginal distribution with exponent of 1 over classes promotes a combination of both in-context and in-weights learning, which are otherwise at odds with each other in most experiments.

The comparisons between transformers and RNNs is of added benefit. The broader implications of these results, such as for application to RL, are especially exciting.

Overall, this work begins to dispel the mysteries behind the excellent empirical performance of transformers, and should lead to many more enlightening studies.

questions: How might the critical effects of the Zipfian exponent of the distributions be explained?

What are the color codings of the curves in Fig 7 & Fig 8? Are they just plotting the individual runs for different seeds, or perhaps for different hyperparameter settings?

limitations: Line 232 says:  “We performed a comprehensive hyperparameter search for all models (see Appendix for details).” But the appendix mentions only “15 runs for each architecture” over three hyperparameters with 2 values each. This seems like a fairly shallow hyperparameter sweep. Unless perhaps each hyperparameter combination was counted as a separate architecture, making 8 * 15 = 120 runs for the transformer alone, etc.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper asks a range of questions around what affects in-context vs in-weight learning in modern neural networks, focusing on Transformers, with RNN and LSTMs serving as comparisons. In totality, the experiments evaluate:

1. In-context vs In-weight based on burstiness of the data.
2. In-context vs In-weight based on number of training classes in the data.
3. In-context learning based on the number of labels per class.
4. In-context vs In-weight based on within-class variation.
5. How a Zipfian distribution in the data affects In-context and In-weight learning.
6. How In-context learning compares across Transformers, RNNs, and LSTMs.

The paper asks these questions in context of the recent strong LLM results. How can it be that these models seem to be exhibiting both in-context and in-weight learning? They then proceed to examine why this is and try to parse apart the causes by examining artificial data and comparing model classes.
strengths_and_weaknesses: Originality:

This paper is original wrt the questions it asks. The methods to answering them (omniglot, simple plots, etc) are not new, but they don't need to be. 

Clarity:

While the writing was good, the paper was surprisingly tougher to follow than I would have expected. I think it would get a huge improvement from a graphical interpretation of in-context vs in-weight learning. These are not terms I've seen before (and I've worked in FSL) and in-weight learning wasn't really ever properly defined in the paper. As far as I can tell, in-context learning was only roughly defined and that was on page 1 in relation to meta-learning. I had to piece together what I think is the definition from the rest of the work. I'm not sure if others you've shown this to have had the same problem, but just setting aside some space near the beginning to define what you mean by this would help a lot. Otherwise, I was left to wonder what actually fits into this view and what doesn't.

Quality:

The quality is high. Once I understood what was going on with their interpretation of in-context vs in-weight learning, the results were pretty interesting. I liked the Zipfian distribution experiment a lot because it was one of the questions I would have had towards doing this on language. I am genuinely surprised that the LSTM and RNN models failed so terribly and am curious if the authors have a take on this.

Significance:

The paper's experiments are intriguing and point to some insights into why LLMs are working so well. I'm not intimately familiar with this part of the literature to know whether these align with other results, but to me this is new and good stuff asking worthwhile questions with thought-provoking results. This strikes me as significant for computer vision as well because of the questions about whether it's worthwhile to shift to different paradigms of data than the prevailing uniform IID approach.
questions: 1. What is your actual definition of in-context vs in-weight learning?
2. How would you construct a language dataset to test some of these hypotheses instead of a dummy omniglot one? That feels like a missing experiment here because you do lots of variations, but do not include anything that gets at the environment that the LLMs are operating in. Could you create a language dataset that wasn't bursty? In my mind, this would be testing what happens when you use a real-world dataset that followed the uniform IID concept.
3. What would it take to do the same experiments in vision like you point to in the discussion?
limitations: As evidenced by the lengthy discussion, the authors understand the paper's limitations and what it points to.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: Recently, the emergence of in-context learning raises a question about the cause of this learning behavior. This paper considers in-context few-shot learning for image data. In the setting, this paper finds that in-context learning emerges when the training data is bursty. In addition, when the data follows a skewed Zipfian distribution, both in-context learning and in-weight learning can co-exist in a single model.

Updates after seeing other reviews: I under-estimated the core contributions of this paper, both in the novelty of the perspective and the setting of empirical evaluations. However, I still think this paper will benefit from more rigorous descriptions of some key terms. I increased the score to 5 (confidence rating 3).
strengths_and_weaknesses: **Strengths**

- The study of interaction between in-context learning and in-weight learning is novel.
- The inquiry into the reason that lead to in-context learning is important.
- The experiments show some interesting findings.

**Weaknesses**

- Formatting: The page limit is 9 pages right? Lines 314-317 appear in page 10.
- Three key concepts discussed in this paper, in-context learning, in-weight learning, and burstiness, lack quantified definitions. Please refer to my comments below.
questions: - It appears that burstiness is treated as a binary feature (defined in Figure 1 caption, lines 5-7). This is not a rigorous definition, and would definitely benefit from additional elaborations. 
- Similarly, "in-weight learning" and "in-context learning" are also defined in Figure 1 caption (lines 6-7), which are not quantitative definitions. The in-weight vs. in-context learning is an effect mentioned a lot in this paper. I think a less ambiguous definition than just "learning labels across sequences" or "referring back to the context" would be necessary.
- "Bursty sequences can be solved by in-weights learning or in-context learning". So the non-bursty sequences can not be solved by either in-weights learning or in-context learning?
- How does in-context learning work together with few-shot learning? Did you sample multiple demonstration sequences and run few-shot learning? Compared to few-shot learning (using just one image as input), this setting has a lot of computational overheads. Compared to in-context learning (without changing model parameters), this approach is limited to much smaller models. What are the benefits of in-context few-shot learning?
- Sec 3.1: Doesn't bustiness overlap to some extent with "a large number of rarely occurring classes"? 
limitations: The limitations are adequately addressed.
review_title: null

Meta-review: metareview: This paper poses and analyses an interesting question -- do statistical properties of the training data affect emergent behavior (e.g. in-context learning) in Transformers? The study is novel since it probes the properties of the data itself, as opposed to most existing work that has studied the effect of model architectures and training algorithms on model capabilities. The paper shows that properties like Zipfian distribution and burstiness are useful for eliciting in-context learning (two properties that are widely found in natural languages, where in-context learning has had a rapid emergence). All the reviewers appreciated the originality and timeliness of this paper, and experiments provide interesting conclusions. Overall, I believe the paper will spark future inquiry into data distributional properties.

I recommend the authors pay attention to Reviewer D2K5's comment about clarifying the definition of "burstiness" in the writing and making it more consistent with the experiments performed. 
confidence: Certain
Decision: Accept