SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 7: Induced emergent reconstruction ability in shallow nonlinear autoencoders. (A) A published emergent ability at the BIG-Bench Periodic Elements task [33]. (B) Shallow nonlinear autoencoders trained on CIFAR100 [21] display smoothly decreasing mean squared reconstruction error. (C) Using a newly defined Reconstruction $_c$  metric (Eqn. 1) induces an unpredictable change.
Figure 8: Induced emergent classification ability in autoregressive Transformers. (A) A published emergent ability on the MMLU benchmark [9]. (B) Autoregressive transformers trained to classify Omniglot images display increasing accuracy with increasing scale. (C) When accuracy is redefined as classifying all images correctly, a seemingly emergent ability appears.
This paper has several limitations. First, nothing in this paper should be interpreted as claiming that large language models cannot display emergent abilities; rather, our message is that some previously claimed emergent abilities appear to be mirages induced by researcher analyses. Second, our experiments and analyses are limited because some LLMs with claimed emergent abilities (e.g., PaLM 1, Gopher, Chinchilla) are private and not queryable at the time of our analysis. Lastly, the best metric(s) arguably depends on human preferences, which may exhibit qualitatively different behavior; we are unaware of studies quantifying whether human judgment is thresholded in an "emergent" way.
Srivastava et al. [33] observed that while accuracy at a particular task can empirically appear sharp and unpredictable, cross-entropy does not appear so; the authors then discussed whether emergent abilities may be partially attributed to the metric. Our paper converts their discussion into precise predictions, then quantitatively tests the predictions to reveal metric choice is possibly responsible for some claimed emergent abilities; well-known and widely-used metrics (including metrics used by [33]) capture graded improvements; emergent abilities do not appear only on tasks involving multiple steps, such as the discontinuous Multiple Choice Grade; metric choice can be used to induce emergent abilities in a novel domain (vision) in diverse architectures and tasks.
Alternative explanations exist for the origin of emergent abilities. Caballero et al. [5] explain emergence by assuming a piece-wise power law functional form; under this view, emergent abilities are real, caused by a "break" (or possibly multiple breaks) in the governing power law. In contrast, our work suggests that emergent abilities can be induced by the researcher under a single power law. Both
explanations could be true: some emergent abilities might genuinely be abruptly appearing, whereas some emergent abilities might be attributable to the metric. Michaud et al. [28] posits that language modeling data might be comprised of discrete subtasks ("quanta") that networks learn; if larger networks have greater capacity, and are thus more capable of learning more of these quanta, then if some downstream task requires a network to learn some combination of quanta, larger networks are more likely to have all the requisite capabilities and thus are capable of performing this downstream task. We think that this is a very interesting hypothesis. Whether language modeling data can or should be understood from this quantization perspective, and whether these quanta indeed are the origin of emergent abilities, are really exciting questions that we think merit more study.
Our paper presents an alternative explanation for the claimed emergent abilities of large language models. For a fixed task and a fixed model family, the researcher can choose a metric to create an emergent ability or choose a metric to ablate an emergent ability. Ergo, emergent abilities may be creations of the researcher's choices, not a fundamental property of the model family on the specific task.
Our work has several implications. Firstly, a task and a metric are distinct and meaningful choices when constructing a benchmark. Secondly, when choosing metric(s), if the goal is to accurately predict scaling behavior, then one should consider the interplay between cross-entropy, transformations, and resolution-limited evaluations so that one isn't surprised. As a corollary, continuous/linear metrics are probably better for accurate scaling forecasts, but if discontinuous/nonlinear metrics are preferred, then one may need a lot of data for sufficient resolution to accurately measure performance. The key is thinking through the consequences of one's choices! Thirdly, when making claims about capabilities of large models, including proper controls is critical. In this particular setting, emergent abilities claims are possibly infected by a failure to control for multiple comparisons. In BIG-Bench alone, there are  $\geq 220$  tasks,  $\sim 40$  metrics per task,  $\sim 10$  model families, for a total of  $\sim 10^{6}$  task-metric-model family triplets, meaning the probability that no task-metric-model family triplet exhibits an emergent ability by random chance might be small. Fourthly, scientific progress can be hampered when models and their outputs are not made available for independent scientific investigation.
9 Contributions
RS conceived of the research direction collected data, ran experiments, and analyzed results. SK supervised and guided the project. BM also provided guidance. All authors helped write the manuscript.
10 Acknowledgements
This work is partially supported by the National Science Foundation under grants No. 2046795, 1934986, 2205329, NIH 1R01MH116226-01A, NIFA award 2020-67021-32799, the Alfred P. Sloan Foundation, and Google Inc. RS is partially supported by a Stanford Data Science Scholarship and BM is partially supported by a Stanford School of Engineering Fellowship and a Stanford EDGE Scholar Fellowship. We thank our colleagues Professor Tatsunori Hashimoto, Eric Han, Max Lamparth, Mikail Khona, Kateryna Pistunova, Victor Lecomte, and Zane Durante for discussing our findings with us and providing much appreciated feedback.
[38] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022.
[39] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12104–12113, 2022.
A Approximate Behavior of Metrics on Sequential Data
How do different metrics behave when used to measure autoregressive model outputs? Precisely answering this question is tricky and possibly analytically unsolvable, so we provide an approximate answer here.
Notationally, we consider  $N$  test data of length  $L$  (here, length is measured in tokens) with targets denoted  $t_n \stackrel{\mathrm{def}}{=} (t_{n1}, t_{n2}, \ldots t_{nL})$ , the autoregressive model has a true-but-unknown per-token error probability of  $\epsilon \in [0, 1]$  and the model outputs prediction  $\hat{t}_n \stackrel{\mathrm{def}}{=} (\hat{t}_{n1}, \hat{t}_{n2}, \ldots \hat{t}_{nL})$ . This assumes that the model's per-token error probability is constant, which is empirically false, but modeling the complex dependencies of errors is beyond our scope.
A.1 Per-Token Error Probability is Resolution-Limited
Note that because we have  $N$  test data, each of length  $L$ , our resolution for viewing the per-token error probability  $\epsilon$  is limited by  $1 / NL$ . Here, resolution refers to "the smallest interval measurable by a scientific instrument; the resolving power." To explain what resolution means via an example, suppose one wants to measure a coin's probability of yielding heads. After a single coin flip, only two outcomes are possible (H, T), so the resolution-limited probability of heads is either 0 or 1. After two coin flips, four outcomes are possible (HH, HT, TH, TT), so the resolution-limited probability of heads is now one of 0, 0.5, 1. After  $F$  coin flips, we can only resolve the coin's probability of yielding heads up to  $1 / F$ . Consequently, we introduce a resolution-limited notation:
$$
a _ {b} \stackrel {\text {d e f}} {=} a \text {r o u n d e d t o t h e n e a r e s t i n t e g e r m u l t i p l e o f} 1 / b \tag {2}
$$
A.2 Token Edit Distance
We first consider an adaptation of the Levenshtein (string edit) distance for models that function on tokens rather than characters, an adaptation we term the token edit distance. The token edit distance between two token sequences  $t_n$ ,  $\hat{t_n}$  is defined as the integer number of additions, deletions or substitutions necessary to transform  $t_n$  into  $\hat{t_n}$  (or vice versa).
$$
\begin{array}{l} \operatorname {T o k e n E d i t D i s t a n c e} \left(t _ {n}, \hat {t} _ {n}\right) \stackrel {\text {d e f}} {=} \operatorname {N u m S u b s t i t u t i o n s} + \operatorname {N u m}. (3) \\ = \sum_ {\ell = 1} ^ {L} \mathbb {I} \left[ t _ {n \ell} \neq \hat {t} _ {n \ell} \right] + \text {N u m . A d d i t i o n s} + \text {N u m . D e l t o t i o n s} (4) \\ \geq \sum_ {\ell = 1} ^ {L} \mathbb {I} \left[ t _ {n \ell} \neq \hat {t} _ {n \ell} \right] (5) \\ \end{array}
$$
The expected token edit distance is therefore:
$$
\begin{array}{l} \mathbb {E} [ \text {T o k e n E d i t D i s t a n c e} (t _ {n}, \hat {t} _ {n}) ] \geq \mathbb {E} [ \sum_ {\ell = 1} ^ {L} \mathbb {I} [ t _ {n \ell} \neq \hat {t} _ {n \ell} ] ] (6) \\ = \sum_ {\ell = 1} ^ {L} p \left(t _ {n \ell} \neq \hat {t} _ {n \ell}\right) (7) \\ \approx L (1 - \epsilon) (8) \\ \end{array}
$$
The resolution-limited expected token edit distance is therefore:
$$
\mathbb {E} [ \text {T o k e n E d i t D i s t a n c e} (t _ {n}, \hat {t} _ {n}) ] _ {N L} \geq L \left(1 - \epsilon_ {N L}\right) \tag {9}
$$
From this, we see that the expected token edit distance scales approximately linearly with the resolution-limited per-token probability. The real rate is slightly higher than linear because additions
and deletions contribute an additional non-negative cost, but modeling this requires a model of how likely the model is to overproduce or underproduce tokens, which is something we do not currently possess.
A.3 Accuracy
$$
\begin{array}{l} \operatorname {A c c u r a c y} \left(t _ {n}, \hat {t} _ {n}\right) \stackrel {\text {d e f}} {=} \mathbb {I} [ \text {N o a d d i t i o n s} ] \mathbb {I} [ \text {N o d e l e t i o n s} ] \prod_ {l = 1} ^ {L} \mathbb {I} \left[ t _ {n l} = \hat {t} _ {n l} \right] (10) \\ \approx \prod_ {l = 1} ^ {L} \mathbb {I} [ t _ {n l} = \hat {t} _ {n l} ] (11) \\ \end{array}
$$
As with the Token Edit Distance (App. A.2), we ignore how likely the language model is to overproduce or underproduce tokens because we do not have a good model of this process. Continuing along,
$$
\begin{array}{l} \mathbb {E} [ \log \text {A c c u r a c y} ] = \sum_ {l} \mathbb {E} [ \log \mathbb {I} [ t _ {n l} = \hat {t} _ {n l} ] ] (12) \\ \leq \sum_ {l} \log \mathbb {E} [ \mathbb {I} [ t _ {n l} = \hat {t} _ {n l} ] ] (13) \\ \approx L \log (1 - \epsilon) (14) \\ \end{array}
$$
Taking an approximation that would make most mathematicians cry:
$$
\begin{array}{l} \mathbb {E} [ \text {A c c u r a c y} ] \approx \exp (\mathbb {E} [ \log \text {A c c u r a c y} ]) (15) \\ = (1 - \epsilon) ^ {L} (16) \\ \end{array}
$$
This reveals that accuracy approximately falls geometrically with target token length. The resolution-limited expected accuracy is therefore:
$$
\mathbb {E} [ \text {A c c u r a c y} ] _ {N L} = (1 - \epsilon) ^ {L} _ {N L} \tag {18}
$$
From this we can see that choosing a nonlinear metric like Accuracy is affected significantly more than a linear metric by limited resolution because Accuracy forces one to distinguish quantities that decay rapidly.
A.4 ROUGE-L-Sum
Another BIG-Bench metric [33] is ROUGE-L-Sum [25], a metric based on the longest common subsequence (LCS) between two sequences. Section 3.2 of [25] gives the exact definition, but the key property is that ROUGE-L-Sum measures the "union" LCS, which means "stitching" together LCSs across the candidate and multiple references. As explained in the original paper [25]: if the candidate sequence is  $c = w_{1}w_{2}w_{3}w_{4}w_{5}$ , and if there are two reference sequences  $r_1 = w_1w_2w_6w_7w_8$  and  $r_2 = w_1w_3w_8w_9w_5$ , then  $LCS(r_1,c) = w_1w_2$  and  $LCS(r_2,c) = w_1w_3w_5$ , then the union LCS of  $c, r_1, r_2$  is  $w_{1}w_{2}w_{3}w_{5}$ , with length 4. Intuitively, this disproportionately benefits models with smaller error rates because their mistakes can be "stitched" across multiple references; this is confirmed in Monte Carlo simulation (Fig. 9).
A.5 BLEU
Yet another BIG-Bench metric [33] is BLEU [30], a metric based on shared n-grams between the generated string and reference strings. BLEU is also a discontinuous and nonlinear metric for several
Figure 9: ROUGE-L-Sum is a sharp metric. Simulations show that as the per-token error probability slightly increases (e.g. from 0.05 to 0.1), the ROUGE-L-Sum metric falls sharply.
Figure 10: BLEU is a sharp metric. Simulations show that as the per-token error probability slightly increases (e.g. from 0.05 to 0.1), the ROUGE-L-Sum metric falls sharply.
reasons. For an explanation of its discontinuity, consider bleu.compute(predictions=["hello there general"], references=[["hello there general"]]). At first glance, this might seem like it should also result in a BLEU score of 1.0 since the prediction matches the reference. However, the issue here is the absence of longer n-grams. For the unigrams, bigrams, and trigrams, the precision is 1.0 since they match perfectly. However, for the 4-grams, there are none in both the candidate and the reference. This results in a precision of 0 for the 4-grams because the BLEU score takes the geometric mean of the n-gram precisions, meaning any 0 in the set will make the entire product 0. Hence, despite the match in unigrams, bigrams, and trigrams, the absence of 4-grams results in a BLEU score of 0.0. This behavior of BLEU has been a point of criticism, as short sentences or those with fewer n-grams than the maximum considered (often 4) can yield scores that are counter-intuitive. This is confirmed in Monte Carlo simulations (Fig. 10)
B Inducing Emergent Abilities in Networks on Vision Tasks
B.1 Emergent Classification of MNIST Handwritten Digits by Convolutional Networks
We begin by inducing an emergent classification ability in a LeNet convolutional neural network family [24], trained on the MNIST handwritten digits dataset [23]. This family displays smoothly increasing test accuracy as the number of parameters increases (Fig. 11B). To emulate the accuracy metric used by emergence papers [9, 38, 33], we use subset accuracy: 1 if the network classifies  $K$  out of  $K$  (independent) test data correctly, 0 otherwise. Under this definition of accuracy, the model family displays an "emergent" ability to correctly classify sets of MNIST digits as  $K$  increases from 1 to 5, especially when combined with sparse sampling of model sizes (Fig. 11C). This convolutional
Figure 11: Induced emergent MNIST classification ability in convolutional networks. (A) A published emergent ability from the BIG-Bench Grounded Mappings task [38]. (B) LeNet trained on MNIST [23] displays a predictable, commonplace sigmoidal increase in test accuracy as model parameters increase. (C) When accuracy is redefined as correctly classifying  $K$  out of  $K$  independent test data, this newly defined metric induces a seemingly unpredictable change.
family's emergent classification ability qualitatively matches published emergent abilities, e.g., at the BIG-Bench Grounded Mappings task [38] (Fig. 11A).
C Relationship Between Emergent Abilities and Grokking
Emergent abilities [4, 9, 33, 38] are sometimes compared with grokking [31, 26, 2, 11], a phenomenon whereby a single model will, over the course of learning, achieve high training accuracy and only much later achieve high test accuracy. There are several differences between grokking and emergent abilities:
1. Grokking is primarily studied within a single model, whereas emergent abilities are studied within a model family (i.e., multiple models).
2. Grokking occurs with increasing gradient steps, whereas emergent abilities occur with increasing model scale, typically measured in parameters or effective parameters (although more recently compute).
3. Grokking explicitly studies a discrepancy between the model's train and test behavior, whereas emergent abilities (to the best of our knowledge) do not present separate train & test curves.
4. Grokking is primarily studied on toy "algorithmic" tasks in small networks, whereas emergent abilities are often studied on benchmark NLP tasks in large language models.