Are Emergent Abilities of Large Language Models a Mirage?
Authors: Rylan Schaeffer
Brando Miranda
Sanmi Koyejo
Venue: NeurIPS 2023 oral (2023)
Keywords: large language models
foundation models
natural language processing
language modeling
emergent abilities
TL;DR: Some have argued that some improvements in model capabilities are unpredictable; we argue that many claimed emergent capabilities are predictable, either using better statistics or alternative metrics

Abstract: Recent work claims that large language models display \textit{emergent abilities}, abilities not present in smaller-scale models that are present in larger-scale models.
What makes emergent abilities intriguing is two-fold: their \textit{sharpness}, transitioning seemingly instantaneously from not present to present, and their \textit{unpredictability}, appearing at seemingly unforeseeable model scales.
Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due the researcher’s choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous, predictable changes in model performance.
We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities, (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show how to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks.
Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.

Are Emergent Abilities of Large Language Models a Mirage?
Rylan Schaeffer
Brando Miranda
Sanmi Koyejo
Computer Science
Computer Science
Computer Science
Stanford University
Stanford University
Stanford University sanmi @csmi@cs..r stanf .
rschaeffca stanford.edd
brando98cs. stanford.edu
Abstract
Recent work claims that large language models display emergent abilities: abilities not present in smaller-scale models that present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present present, and their unpredictabilityy appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: for a particular task and model family, when ana- lyzing fixed model outputs, emergent abilities appear due to the researcher's 's choice of metric rather than due to fundamental changes models with scale. Specifically, nonlinear or discontinuous metrics produce seemingly emergent abilities, whereas linear continuous metries produce smooth, continuous, predictable changes in model performance. We present our alternative explanation in a simple mathemati- in three complementary ways: we (I) make, test and confirm
cal model, then three predictions on the effect of metric choice using the InstructGPT/GPT-3 family tasks with claimed emergent abilities; (2) make, test and confirm two predictions about metric choices in meta- meta-analysis of emergent abilities on the Beyond the Imitation Game Benchmark (BIG-Bench); and (3) show how choose metrics produce never- rever-before-see seemingly emergent abilities in multiple vision tasks across diverse deep network architectures. Via all three analyses, we provide evidence that emergent abilities disappear with different metrics or with better statistics, and may not be a fundamental property of scaling Al models.
Introduction
Emergent properties of complex systems have long been studied across disciplines, from physics to biology to mathematics. The idea of emergence was popularized by Nobel Prize-winning physicist P.W. Anderson's "More Is Different" (I], which argues that the complexity of a system increases, new properties may materialize that canmot be predicted even from a precise quantitative understand- ing of the system's microscopic details. Recently, the idea of emergence gained significant attention in machine learning due to observations that large language models (LLMs) such as GPT (4, PaLM (Fig.
7 and LaMDA
35 exhibit so-called "emergent abilities" [8 9 33
The term "emergent abilities LLMs" was recently and crisply defined as "abilities that are not present in smaller-scale models but are present in large-scale models; thus they cannot be predicted by simply extrapolating the performance improvements on smaller-scale models" [38]. Such emergent abilities were first discovered in the GPT-3 family 4 Subsequent work emphasized the discovery, writing that "[although model] performance predictable at a general level, performance on specific task can sometimes emerge quite unpredictably and abruptly scale" [ These quotations collectively identify the two defining properties of emergent abilities in LLMss:
1. Sharpness, transitioning seemingly instantaneously from not present to present
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

LaMDA
GPT-3
Gopher
Chinchilla
PaLM
Random
(A) Mod. arithmetic
(B)
IPA transliterate
(C)
Word unscramble
(D) Persian QA
50
50
40
40
Exact match (%)
Exact match (%)
BLEU (%)
30
30
30
30
Accuracy
20
20
20
20
10
10
10
10
10.00
1022
1024
10
10.0
10a2
10
10
1000
10ah
10
1018
1020
10
1024
(E) TruthfulQA
(F)
Grounded mappings
(G) Multi-task NLU
(H)
Word in context
70
70
70
70
Accuracy (%)
50
Accuracy (%)
50
Accuracy (%)
50
Accuracy (%)
50
40
30
30
30
30
20
10
10
10
10
1020
10P2
1024
1020
1022
1024
1020
10a2
1024
1020
1022
1024
Model scale (training FLOPs)
Figure 1: Emergent abilities of large language models. Model families display sharp and unpre- dictable increases performance at specific tasks as scale increases. Source: Fig. from [38].
2. Unpredictability. transitioning at seemingly unforesecable model seales
These emergent abilities have garnered significant interest, raising questions such as: What controls which abilities will emerge? What controls when abilities will emerge? How can we make desirable abilities emerge faster, and ensure undesirable abilities never emerge? These questions are especially pertinent to Al safety and alignment, as emergent abilities forewarn that larger models might one day, without warning, acquire undesired mastery over dangerous capabilities 34
19 20
In this paper, call into question the claim that LLMs possess emergent abilities, by which specifically mean sharp and unpredictable changes in model outputs as a function of model scale on specific tasks. Our doubt stems from the observation that emergent abilities seem appear only under metrics that nonlinearly or discontinuously scale any model's per-token error rate. For instance, as we later show, > 92% of emergent abilities on BIG-Bench tasks 33 (hand-annotated by [37] appear under either of these two metrics:
Multiple Choice Grade def if highest probability mass on correct option otherwise
Exact String Match def 1 if output string exactly matches target string otherwise
This raises the possibility of an alternative explanation for the origin of LLMs' emergent abilities: sharp and unpredictable changes might be induced by the researcher's choice of measurement, though the model family's per-token error rate changes smoothly, continuously and predictably with increasing scale. Specifically, our alternative posits that emergent abilities primarily by the researcher choosing a metric that nonlinearly or discontinuously deforms per-token error rates, and secondarily by possessing too few test data to accurately estimate the performance of smaller models, thereby causing smaller models to appear wholly unable to perform the task.
mirage caused
To communicate our alternative explanation, we present it as a simple mathematical model and demonstrate how it quantitatively reproduces the evidence offered in support of emergent abilities of LLMs. We then test our alternative explanation in three complementary ways:

---

A
Emergent Abilities
No Emergent Abilities
Rar
Target Str Len
samte moo momeest Thaess
Nonlinearly score LLM outputs
Linearly score outputs
30t
10t7
30
0.0
Nodel Parkmeters
Model Parmmeherg
stsings (anee)) tap 4ostir)
Choices
2.0
Brie Scom
Discontinuously score LLM outputs
Phos
0.2
Continuously score LLM outputs
50
107
10th
139
Hedel Parameters
Nedal
Figure 2: Emergent abilities of large language models are created by the researcher's chosen metries, not unpredictable changes in model behavior with scale. (A) Suppose the per-token cross-entropy loss decreases monotonically with model scale, e.g., Lox scales as power law. (B) The per-token probability of selecting the correct token asymptotes towards 1. (C) If the researcher scores models' outputs using nonlinear metric such as Accuracy (which requires a sequence of tokens to all be correct), the metric choice nonlinearly scales performance, causing performance to change sharply and unpredictably in manner that qualitatively matches published emergent abilities (inset). (D) If the researcher instead scores models' outputs using discontinuous metric such as Multiple Choice Grade (akin to a step function), the metric choice discontinuously scales performance, again causing performance to change sharply and unpredictably. (E) Changing from nonlinear metric to a linear metric such as Token Edit Distance, scaling shows smooth, continuous and predictable improvements, ablating the emergent ability. (F) Changing from discontinuous metric to continuous metric such as Brier Score again reveals smooth, continuous and predictable improvements in task performance. Consequently, the observation of "emergent abilities" can be explained by the researcher's choice of metrics, and does not require fundamental changes in model family behavior on specific tasks with scale.
1. We make, test and confirm three predictions based on our alternative hypotheses using the InstructGPT 27 GPT-3 model family.
2. We meta-analyze published benchmarks for specific metrics, not for model families on particular tasks, and that changing the metric causes the emergence phenomenon disappear.
33
38 to reveal that emergent abilities only appear
3. We induce never-before-seae seemingly emergent abilities multiple architectures across various vision tasks by intentionally changing the metries used for evaluation.
2 Alternative Explanation for Emergent Abiltties
How might smooth, continuous, predictable changes in model family performance appear sharp and unpredictable? The answer is that the researcher's choice of nonlinear or discontinuous metric can distort the model family's performance appear sharp and unpredictable.
To expound, suppose that within a model family, the test loss falls smoothly, continuously, and predictably with the number of model parameters. One reason to believe this is the phenomenon known as neural scaling laws: empirical observations that deep networks exhibit power law scaling in the test loss function of training dataset size, number of parameters compute
[5
32
3 18

---

TO 14 17
8 29 For concreteness, suppose we have model family of different numbers of parameters N and assume that each model's per-token cross entropy falls as a power law with the number of parameters N for constants > 0, a < 0 (Fig. AA):
39
16
(NE )
To be clear, we do not require this particular functional form to hold; rather, we use it for illustrative purposes. Let V denote the of possible tokens, p denote the true but unknown probability distribution, and PN denote the N-parameter model's predicted probability distribution. The per- token cross entropy as function of number of parameters N is:
(CE(N) def
Ep(() log pNg()(
eEV
In practice, is unknown, so we substitute a one-hot distribution of the observed token ex:
(cE(N) = - log (Nv
model with N parameters then has a per-token probability of selecting the correct token (Fig. 23):
p(single token correct) = exp ( (- (CEE(N)) = exp - (N//)))
Suppose the researcher then chooses metric that requires selecting tokens correctly. For example, our task might be I-digit integer addition, and model's output is scored 1 if all L output digits exactly match all target digits with no additions, deletions or substitutions, otherwise. If the probability each token is correct is independent the probability of scoring is:
Accuracy(N) pvv (single token correct) of tokens exp (NT)))a)*
This choice of metric nonlinearly scales performance with increasing token sequence length. When plotting performance on a linear-log plot, one sees sharp, unpredictable emergent ability on longer sequences (Fig. (C) that closely matches claimed emergent abilities (inset). What happens the researcher switches from a nonlinear metric like Accuracy, under which the per-token error rate scales geometrically in target length (App. A.3). to approximately linear metric like Token Edit Distance, under which the per-token error rate scales quasi-linearly in target length (App. A.2
Token Edit Distance(NN) ~e L (single token correct) = e1x (-((0(())))))
The linear metric reveals smooth, continuous, predictable changes in model performance (Fig. 2). Similarly, if the researcher uses discontinuous metric like Multiple Choice Grade, the researcher can find emergent abilities (Fig. ())), but switching to continuous metric like Brier Score removes such abilities (Fig. ) In summary, sharp and unpredictable changes with increasing scale can be fully explained by three interpretable factors: (I) the researcher choosing metric that nonlinearly or discontinuously scales the per-token error rate, (2) having insufficient resolution to estimate model performance in the smaller parameter regime, with resolution set by 1 1/test dataset size, and (3) insufficiently sampling the larger parameter regime.
3 Analyzing InstructGPT/GPT.S Emergent Arithmetic Abilities
Previous papers prominently claimed the GPT 4 27 family displays emergent abilities at integer arithmetic tasks
9 33 38 (Fig. A).. We chose these tasks they were prominently presented
'While the independence assumption is not true, the approximation yields results qualitatively matching the observed emergence claims.
2 Resoluuionn defined as "The smallest interval measurable hy a scientific instrument; the resolving power." As of 2023-03-15, 4 models with 350M, 1.3B, 6.7B, 175B parameters available via the OpenAl API.

---

Mathematical Model
2- 2-Integer 2- 2-Digit Multiplication
2 2-Integer 4-Digit Addition
1.0
Bareett Lein
10
Sigas Ine
10
0.0
0.0
3.0
1,0
Model Parameters
102
12th
201
1011
Trye So lee
3ryd St
Edit Detancce
-2
3.0
Target Str
1
10"
1060 Parameters
Goeal Terumaters
CFT3 Medel Rarumaters
Figure 3: Claimed emergent abilities evaporate upon changing the metric. Top: When perfor-
mance is measured by a nonlinear metric (c.g., Accuracy), the InstructGPT/GPT- [4] family's s performance appears sharp and unpredictable longer target lengths. Bottom: When performance instead measured by a linear metric (e.g., Token Edit Distance), the family exhibits smooth, pre- dictable performance improvements.
Mathematical Model
2- 2-Integer 2-Digit Multiplication
2-Integer 4-Digit Addition
Sugel
Toree
10
10
109
0.0
Target
10"1
10"1
16a
10a
Model Parameters
10
2011
122 10PF
230
10P 2091
Figure Claimed emergent abilities evaporate upon using better statistics. Based on the predictable effect Accuracy has on performance, measuring performance requires high resolution. Generating additional test data increases the resolution and reveals that even on Accuracy, the InstrucIGPT/GPT-3 family's [4. [77] performance is above chance and improves in a smooth, continu- ous, predictable manner that qualitatively matches the mathematical model.
and we focused on the GPT family due to it being publicly queryable. As explained mathematically and visually in Sec. 2 our alternative explanation makes three predictions:
33 38
1. Changing the metric from a nonlinear discontinuous metric (Fig. 2CD) linear or continuous metric (Fig. 2 EF) should reveal smooth, continuous, predictable performance improvement with model scale.
2. For nonlinear metrics, increasing the resolution of measured model performance by increas- ing the test dataset size should reveal smooth, continuous, predictable model improvements commensurate with the predictable nonlinear effect of the chosen metric.
3. Regardless of metric, increasing the target string length should predictably affect the model's performance as a function of the length-1 target performance: approximately geometrically for accuracy and approximately quasilinearly for token edit distance.
5

---

To test these predictions, we collected outputs from the InstructGPT/GPT-3 family on two tasks: 2-shot multiplication between two 2-digit integers and 2-shot addition between two 4-digit integers.
Emergent Abilities Disappear With Different Metries On both arithmetic tasks, the GPT family displays emergent abilities if the target has or digits and if the metric is Accuracy (Fig. 3 top) 44. 2 38. However, if one changes from nonlinear Accuracy to linear Token Edit Distance while keeping the models' ouiputs fixed, the family's performance smoothly, continuously and predictably improves with increasing scale (Fig. 3 bottom). This confirms first prediction and supports our alternative explanation that the observation of emergent abilities can be explained by the researcher's 's choice of metric, not changes in the model family's 's outputs. We also observe that under Token Edit Distance, increasing the length of the target string from to predictably decreases the family's performance in approximately quasilinear manner, confirming the first half of our third prediction.
Prediction:
Prediction: Emergent Abilities Disappear With Better Statistics We next tested our second prediction: that even on nonlinear metrics such as accuracy, smaller models do not have zero accuracy, but rather have non-zero above-chance accuracy commensurate with choosing to use accuracy as the metric. In order to accurately measure models' accuracy. we increased the resolution by generating additional test data, and found that both arithmetic tasks, all models in the InstructGPT/GPT- family achieve above-chance accuracy (Fig. 4. This confirms our second prediction. We also observe that as the target string length increases, the accuracy falls approximately geometrically with the length of the target string, confirming the second half of our third prediction. These results additionally demonstrate that the researcher's choice of metric has the effect that one should predict accuracy to have, i.e., geometric decay with the target length.
4 Meta-Analysis of Claimed Emergent Abilities
Analyzing the GPT family is possible because the models are publicly queryable. However, at the time of this analysis, other model families claimed exhibit emergent abilities not publicly queryable, nor are their generated outputs publicly available, meaning we are limited to analyzing the Our alternative explanation makes two predictions.
published results themselves g 38 37].
1. At the "population level" of T Model Family triplets, emergent abilities should appear predominantly on specific metrics, not task-model family pairs, and specifically with nonlinear and/or discontinuous metrics.
2. On individual the metric to a linear and/or continuous metric should remove the emergent ability.
Family triplets that display an emergent ability, changing
To test these predictions, we used claimed emergent abilities on BIG-Bench benchmark being pertinent and publicly available.
33
38
due to the
Emergent Abilities Should Appear with Metrics, not Task-Model Families If emergent abilities are real, one should expect task-model family pairs to show emergence for all reasonable metrics. However, if our alternative explanation is correct, abilities to appear only under certain metrics. To test this, we analyzed on which metrics emergent abilities appear. To determine whether task-metric-mee modell family triplet exhibits a possible emergent ability, we used a metric from previous work 33 Letting y: E denote model performance at model scales 2; R. sorted such that 2; 2s+1, the emergence score is:
Prediction:
should expect emergent
Emergence Score (f
(((2 (# yn) )
sign(arg max; Vi argmin; v1) (max, yi min; ys)
/ Mediaa 1)-))
We found that most metrics used in BIG-Bench have zero task-model family pairs that exhibit emergent abilities: of the 39 preferred metrics in BIG-Bench, at most 5 display emergence (Fig. 5A). Many of the 5 are nonlinear and/or discontinuous, e.g., Exact String Match, Multiple Choice Grade, ROUGE-L-Sum (App. A.4). Notably, because BIG-Bench often scores models on tasks using multiple metrics, the lack of emergent abilities under other metries suggests that emergent abilities do not appear when model outputs are scored using other metrics.

---

aigiman swrrer
light p0r
rull pol thoie pode upptypii
comrat poht,oy
Sareti peacted
100
203
303
403
S20
600
Emergaace briwssaioo ctui 2012) Over A
Exhibiting Emergent Abilities
Exhibiting Emergent
Metrics Model-Task
16.12% (11)
10.26% (4)
str
Emergent
Not Emergent
normelized ed aggregate score
4.48% (. (3)
multiple_cher
89.74% (35)
76.12% (SI
Figure 5: Emergent abilities appear only for specific metrics, not task-model families. (A) Possible emergent abilities appear with most 5 out of BIG-Bench metrics. (B) Hand-annotated data by [37 reveal emergent abilities appear only under 4 preferred metrics. (C) > 92% of emergent abilities appear under one of two metries: Multiple Choice Grade and Exact String Match.
Because emergence score only suggests emergence, we also analyzed hand-annotated task-metric- model family triplets (37). which revealed emergent abilities appear with 4/39 metrics (Fig. (BB). and 2 metrics account for > 92% of claimed emergent abilities (Fig. 5C) Multiple Choice Grade and Exact String Match. Multiple Choice Grade is discontinuous, and Exact String Match is nonlinear.
Prediction: Changing Metric Removes Emergent Abilities To test our second prediction, we focused the LaMDA family [3 because its outputs are available through BIG-Bench. We identified tasks on which LaMDA displays emergent abilities with Multiple Choice Grade, then asked whether LaMDA still displays emergent abilities on the same tasks with different BIG-Bench metric: Brier Score B. Brier Score is a strictly proper scoring rule for predictions of mutually exclusive outcomes; for a binary outcome, the Brier Score simplifies to squared error between and the model's probability mass the outcome. LaMDA's emergent abilities on the discontinuous Multiple Choice Grade disappeared when we changed the metric to the continuous Brier Score (Fig- ( These results support alternative explanation that emergent abilities induced by the chosen metric.

---

Published Abiitties with Choice Grade
EL Lengltat conwerts
0.6
0.5
0.
1pt
Effectho Hocel Size
Effective Model Site
No Emergent Abillies with Netric Brier Score
shot
Stot
Lageenagge
spert novttinndntenddnn
-0.12
-1335
-0.2
-0.3
-0.35
-0.4
127
10tt
2221
1.2
30ta
1251
Effectvve Madel Size
Effective Madel Size
Figure 6: Changing the metric when evaluating task-model family pairs causes emergent abilities to disappear. Top: The LaMDA model family displays emergent abilities when measured under the discontinuous Multiple Choice Grade. Bottom: The LaMDA model family's s emergent abilities disappear when measured under continuous BIG-Bench metrie: Brier Score.
Inducing Emergent Abilities in Networks on Vision Tasks
To demonstrate how emergent abilities be induced by the researcher's choice of metric, show how to produce emergent abilities in deep networks of various architectures: fully connected, convolutional, self-attentional. We focus vision tasks because abrupt transitions in vision models' capabilities have not been observed to the best of our knowledge; this is one reason why emergence in large language models is considered interesting. For the convolutional example, see App. B
Emergent Reconstruction
CIFAR100 Natural Images by Nonlinear Autoencoders We first induce an emergent ability to reconstruct images in shallow (i.e., single hidden layer) nonlinear autoencoders trained on CIFAR 100 10 natural images 21.. To emphasize that the sharpness of the metric is responsible for emergent abilities, and to show that sharpness extends to metrics beyond Accuracy, we intentionally define discontinuous metric that measures a network's 's ability to reconstruct a dataset the average number of test data with squared reconstruction error below cutoff
Reconstruction,
(
def
Eu[lEn N
11
2, <
(1)
where I[()) denotes an indicator variable and i, is the autoencoder's reconstruction of Jn. The autoencoder family displays smoothly decreasing squared reconstruction error as the number of bottleneck units increases (Fig. BB) Under our newly defined Reconstruction, metric and for particular choices of c, the autoencoder family exhibits a sharp and seemingly unpredictable image reconstruction ability (Fig. (C) that qualitatively matches published emergent abilities (Fig. (7).
Emergent Classification of Omniglot Characters by Autoregressive Transformers We next induce emergent abilities in Transformers [36] trained to autoregressively classify Omniglot hand- written characters [22] in a setup inspired by recent work [6]: Omniglot images are embedded by convolutional layers, then sequences of embedded image-image class label pairs are fed into decoder-only transformers. We measure image classification performance on sequences of length E [1, 5). again via subset accuracy: 1 if all 7. images are classified correctly (Fig. 83). O otherwise. Causal transformers display seemingly emergent ability to correctly classify Omniglot handwritten characters (Fig. g that qualitatively matches published emergent abilities (Fig. 8A).

---

wms.ggrrr.cccoc Aa.ly
Mall unsduce kinageriAe
19.4
Aigu NOCIE Bilactioo
Inalica Autoncade Nocil
Figure 7: Induced emergent reconstruction ability in shallow nonlinear autoencoders. (A) A published emergent ability the BIG-Bench Periodic Elements task 33 (B) Shallow nonlinear autoencoders trained on CIFARION 100 (1 display smoothly decreasing mean squared reconstruction error. (C) Using newly defined Reconstruction. metric (Eqn. D induces an unpredictable change.
Mastcradued
035
3.45
0.4
0.44
0.14
0.52
Figure Induced emergent classification ability in autoregressive Transformers. (A) A published emergent ability on the MMLU benchmark (9). (B) Autoregressive transformers trained to classify Omniglot images display increasing accuracy with increasing scale. (C) When accuracy redefined as classifying all images correctly, seemingly emergent ability appears.
Limitations
This paper has several limitations. First, nothing in this paper should be interpreted claiming that large language models cannot display emergent abilities; rather, our message is that some previously claimed emergent abilities appear to be mirages induced by researcher analyses. Second, our experiments and analyses are limited because some LLMs with claimed emergent abilities (e.g., PaLM I, Gopher, Chinchilla) are private and not queryable the time of our analysis. Lastly, the best metric(s) arguably depends on human preferences, which may exhibit qualitatively different behavior; we are unaware of studies quantifying whether human judgment thresholded in "emergent" way.
Related Work
Srivastava al. 33 observed that while accuracy a particular task can empirically appear sharp and unpredictable, cross-entropy does not appear so; the authors then discussed whether emergent abilities may be partially attributed to the metric. Our paper converts their discussion into precise predictions, then quantitatively tests the predictions reveal metric choice is possibly responsible for some claimed emergent abilities; well-known and widely-used metrics (including metrics used by (B3)) capture graded improvements; emergent abilities do not appear only tasks involving multiple steps, such as the discontinuous Multiple Choice Grade; metric choice can be used to induce emergent abilities in novel domain (vision) in diverse architectures and tasks.
Alternative explanations exist for the origin of emergent abilities. Caballero et al. explain emergence by assuming piece-wise power law functional form; under this view, emergent abilities are real, caused by a "break" (or possibly multiple breaks) in the governing power law. contrast, our work suggests that emergent abilities can be induced by the researcher under a single power law. Both

---

explanations could be true: some emergent abilities might genuinely be abruptly appearing, whereas some emergent abilities might be attributable the metric. Michaud al. [28] posits that language modeling data might be comprised of discrete subtasks ("quanta") that networks learn; if larger networks have greater capacity, and are thus more capable of learning more of these quanta, then some downstream task requires network to learn some combination of quanta, larger networks are more likely to have all the requisite capabilities and thus are capable of performing this downstream task. We think that this is very interesting hypothesis. Whether language modeling data can or should be understood from this quantization perspective, and whether these quanta indeed are the origin of emergent abilities, are really exciting questions that think merit more study.
8 Discussion
Our paper presents an alternative explanation for the claimed emergent abilities of large language models. For a fixed task and fixed model family, the researcher can choose metric create an emergent ability or choose a metric to ablate an emergent ability. Ergo, emergent abilities may be creations of the researcher's choices, not a fundamental property of the model family on the specific task.
Our work has several implications. Firstly, task and metric are distinct and meaningful choices when constructing a benchmark. Secondly, when choosing metric(s), if the goal is to accurately predict scaling behavior, then should consider the interplay between cross-entropy, transformations, and resolution-limiie evaluations so that one isn't 't surprised. As a corollary. continuous/linear metrics are probably better for accurate scaling forecasts, but if discontinuous/noolineai metrics are preferred, then one may need a lot of data for sufficient resolution to accurately measure performance. The key is thinking through the consequences of one's choices! Thirdly, when making claims about capabilities of large models, including proper controls is critical. In this particular setting, emergent abilities claims are possibly infected by failure control for multiple comparisons. BIG-Bench alone, there are > 220 tasks, ~ 40 metrics per task, ~ 10 model families, for a total of ~ 10th task- metric-model family triplets, meaning the probability that no mask-metric-model family triplet exhibits an emergent ability by random chance might be small. Fourthly, scientific progress can be hampered when models and their outputs are not made available for independent scientific investigation.
Contributions
RS conceived of the research direction collected data, experiments, and analyzed results. SK supervised and guided the project. BM also provided guidance. All authors helped write the manuscript.
10 Acknowledgements
This work is partially supported by the National Science Foundation under grants No. 2046795, 1934986, 2205329, NIH |R01MH116226-011 A, NIFA award 2020-67021-32799, the Alfred P. Sloan Foundation, and Google Inc. RS is partially supported by a Stanford Data Science Scholarship and BM is partially supported by Stanford School of Engineering Fellowship and a Stanford EDGE Scholar Fellowship. We thank our colleagues Professor Tatsunori Hashimoto, Eric Han, Max Lamparth, Mikail Khona, Kateryna Pistunova, Victor Lecomte, and Zane Durante for discussing our findings with us and providing much- anch-apppeciated feedback.
10

---

References
[1] Philip W Anderson. More is different: broken symmetry and the nature of the hierarchical structure of science. Science, 177(4047):393-39,, 1972.
[2] Boaz Barak, Benjamin Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang Hidden progress in deep learning: Sgd learns parities near the computational limit. Advances in Neural Information Processing Systems, 15:21750-21766 2022.
[3] Glenn W Brier et al. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
[4] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 13:1877-1901, 2020.
[5] Ethan Caballero, Kshitij Gupta, Irina Rish, and David Krueger. Broken neural scaling laws. arXiv preprini urXiv:2210. 14891, 2022.
[6] Stephanie CY Chan, Adam Santoro, Andrew Kyle Lampinen, Jane X Wang, Aaditya K Singh, Pierre Harvey Richemond, James McClelland, and Felix Hill. Data distributional properties drive emergent in-context learning in transformers. In Advances in Neural Information Processing Systems, 2022.
[7] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sehastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022.
[8] Aidan Clark, Diego De Las Casas, Aurelia Guy, Arthur Mensch, Michela Paganini, Jordan Hoffimann, Bogdan Damoc, Blake Hechtman, Trevor Cai, Sebastian Borgeaud, et al. Unified scaling laws for routed language models. In International Conference on Machine Learning, pages 4057-4086. PMLR, 2022.
[9] Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, et al. Predictability and surprise in large generative models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1747-1764, 2022.
[10] Mitchell A Gordon, Kevin Duh, and Jared Kaplan. Data and parameter scaling laws for neural machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5915-5922, 2021.
[11] Andrey Gromov. Grokking modular arithmetic. arXiv preprint arXiv:2301.02679 2023.
[12] Dan Hendrycks. Detecting emergent behavior. 2022.
[13] Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701, 2020.
[14] Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint
2021.
[15] Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Patwary, Mostofa Ali, Yang Yang. and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.0044 2017.
[16] Jordan Hoffmann. Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203. 15556, 2022.
[17] Andy L Jones. Scaling scaling laws with board games. arXiv preprint arXiv:2104.0311 2021.

---

[18] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprini arXiv:201100831, 2020.
[19] Victoria Krakovna, Vikrant Varma, Ramana Kumar, and Mary Phuong. Refining the sharp left turn threat model, part 1: claims and mechanisms. 2022.
[20] Victoria Krakovna, Vikrant Varma, Ramana Kumar, and Mary Phuong. Refining the sharp left turn threat model, part 2: applying alignment techniques. 2022.
[21] Alex Krizhevsky. Learning multiple layers features from tiny images. Technical report, 2009.
[22] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266) 2015.
[23] Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/nmistl, 1998.
[24] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
[25] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.
[26] Ziming Liu, Eric J Michaud, and Max Tegmark. Omnigrok: Grokking beyond algorithmic data. arXiv preprint arXiv:2210001117, 2022.
[27] Ryan Lowe and Jan Leike. Aligning language models to follow instructions. 2022.
[28] Eric J. Michaud, Ziming Liu, Uzay Girit, and Max Tegmark. The quantization model of neural scaling, 2023.
[29] Oren Neumann and Claudius Gros. Scaling laws for a multi-agent reinforcement learning model. arXiv preprini arXiv:22 2022.
[30] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computational Linguistics.
[31] Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Gen- eralization beyond overfitting on small algorithmic datasets. arXiv preprint arXiv:2201.02177 2022.
[32] Jonathan S Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit. A constructive prediction of the generalization error across scales. In International Conference on Learning Representations, 2019.
[33] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonss et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint 2:XX::2200604615. 2022.
[34] Jacob Steinhardt. Future ml systems will be qualitatively different. 2022.
[35] Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng- Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.0822 08239, 2022.
[36] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[37] Jason Wei. 137 emergent abilities of large language models. 2022.
12

---

[38] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud. Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, al. Emergent abilities of large language models. arXiv preprint urXiv:2206.07682, 2022.
[39] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12104-12111 2022.
13

---

A Approximate Behavior of Metrics on Sequential Data
How do different metries behave when used to measure autoregressive model outputs? Precisely answering this question is tricky and possibly analytically unsolvable, so we provide an approximate answer here.
Notationally, we consider N test data length L (here, length is measured tokens) with targets denoted fn def (tm11 (222,.. the autoregressive model has true-but-unknown per-token error
probability of E [0, 1] and the model outputs prediction & (fm1, ...z). This assumes that the model's per-token error probability is constant, which empirically false, but modeling the complex dependencies of errors is beyond scope.
A.1 Per-Token Error Probability is Resolution-Limit
Note that because we have N test data, each of length L. our resolution for viewing the per-token eπror probability is limited by 1/N L. Here, resolution refers to "the smallest interval measurable by scientific instrument; the resolving power. " To explain what resolution means via an example, suppose one wants to measure coin's 's probability of yielding heads. After a single coin flip, only two outcomes are possible (H, T), the resolution-limite two coin flips, four outcomes are possible (HH, HT, TH, TT), so the resolution-liminn limited probability of heads now one of 0, 0.5, 1. After F coin flips, we can only resolve the coin's probability of yielding heads up to 1/F. Consequently, we introduce a resolution-limited notation:
probability of heads is either or After
rounded to the nearest integer multiple of 1/b
A.2 Token Edit Distance
We first consider an adaptation of the Levenshtein (string edit) distance for models that function on tokens rather than characters, adaptation we term the token edit distance. The token edit distance between two token sequences txx fn is defined as the integer number of additions, deletions or substitutions necessary to transform t, into f,, (or vice versa).
Token Edit Distance in) def Num Substitutions Num. Additions Num, Deletions
(3)
Thue F line + Num. Additions + Num. Deletions
(4)
t=1
Tlnt # inn)
(5)
E=1
The expected token edit distance is therefore:
E Token Edit Distance (f,,, {n)] 2 E
I[tne F Enell
(6)
8=1
plnnt Ine)
T(1 -)
The resolution-limitet expected token edit distance is therefore:
E Token Edit Distance(In, (n)l 2 L(1 (11-EN))
From this, we see that the expected token edit distance scales approximately linearly with the resolution-limited per-token probability. The real rate is slightly higher than lincar because additions
14

---

and deletions contribute an additional non negative cost, but modeling this requires a model of how likely the model is overproduce or underproduce tokens, which is something we do not currently possess.
A.3 Accuracy
Accuracy(fa,
def I[No additions] No deletions) II [tt fnt]
(10)
Il [n]]
(11)
As with the Token Edit Distance (App. we ignore how likely the language model is to overproduce or underproduce tokens because not have good model of this process. Continuing along,
Bloog Accuracy]
Ellog T(mm = [nl]
(12)
log [[[ = End]]
(13)
L Log 1 - c)
(14)
Taking an approximation that would make most mathematicians cry:
E Accuracy] exp(EElog Accuracy))
(15)
= -11-ep
(16)
(17)
This reveals that accuracy approximately falls geometrically with target token length. The resolution- limited expected accuracy is therefore:
E [Accuracy) = (1 =) NL
(18)
From this we can see that choosing a nonlinear metric like Accuracy affected significantly more than linear metric by limited resolution because Accuracy forces one to distinguish quantities that decay rapidly.
A.4 ROUGE-L-Sum
Another BIG-Bench metric 33 is ROUGE-L-Sum 25 metric based on the longest common subsequence (LCS) between two sequences. Section 3.2 (2 gives the exact definition, but the key property is that ROUGE-L-Sum measures the "union" LCS, which means "stitching" together LCSs across the candidate and multiple references. As explained the original paper 25]: if the candidate sequence is
two reference sequences 71 wi WyWgWy and 72 = Wi Wh Wig WyWss then LCS((11, c) = wi W2 and LCS( c) = wi Wg then the union LCS of 7, is wi W5, with length 4. Intuitively, this disproportionately benefits models with smaller error rates because their mistakes can be "stitched" across multiple references; this is confirmed in Monte Carlo simulation (Fig. 9)
Wi We WyWW and if there
A.5 BLEU
Yet another BIG-Bench metric generated string and reference strings. BLEU is also discontinuous and nonlinear metric for several
3 is BLEU 30 a metric based on shared n-grams between the
15

---

1.0
10a
Target String Length
20
0.8
36
ROUGE-L-Sum Sum
ROUGE-L-SUN Lumm
Num. References
0.6
0.4
10
0.2
0.0
10a
10a
10:0
10
10a
10a
10:0
10a2
Model Parameters
Model Parameters
Figure 9: ROUGE-L-Sum is a sharp metric. Simulations show that as the per-token error probability slightly increases (e.g. from 0.05 to 0.1), the ROUGE-L-Sum metric falls sharply.
1.0
10a
Target String Length
0.8
16
20
0.6
10
Num. References
BLEU
0.4
10-2
0.2
0.0
10a
10a
10.0
10a2
10a
10a
1030
10
Model Parameters
Model Parameters
Figure 10: BLEU is sharp metric. Simulations show that as the per-token emor probability slightly increases (e.g. from 0.05 0.1), the ROUGE-L-Sum metric falls sharply.
reasons. For an explanation of its discontinuity, consider proccccooopppprrrrcccooossss"""" general"], references=[["heloo there general"[]). At first glance, this might seem like it should also result BLEU score of 1.0 since the prediction matches the reference. However, the issue here is the absence of longer n-grams. For the unigrams, bigrams, and trigrams, the precision is 1.0 since they match perfectly. However, for the 4-grams, there are none in both the candidate and the reference. This results in a precision of ( for the 4-grams because the BLEU score takes the geometric mean of the n-gram precisions, meaning any 0 in the set will make the entire product O. Hence, despite the match in unigrams, bigrams, and trigrams, the absence of 4-grams results in BLEU score of 0.0. This behavior of BLEU has been a point of criticism, as short sentences or those with fewer n-grams than the maximum considered (often 4) can yield scores that counter-intuitive. This is confirmed in Monte Carlo simulations (Fig.
there
B Inducing Emergent Abilities in Networks on Vision Tasks
B.1 Emergent Classification of MNIST Handwritten Digits by Convolutional Networks
We begin by inducing an emergent classification ability in a LeNet convolutional neural network
family (24). trained on the MNIST handwritten digits dataset (23 This family displays smoothly increasing test accuracy as the number of parameters increases (Fig. BTB). To emulate the accuracy we use subvet accuracy: if the network classifies K out of K (independent) test data correctly, O otherwise. Under this definition of accuracy, the model family displays an "emergent" ability to correctly classify sets of MNIST digits as increases from 1 to 5, especially when combined with sparse sampling of model sizes (Fig. ITC). This convolutional
metric used by emergence papers 9 38 33
16

---

Adtinnl Ensiggrr Melly
Nebreegeett.o
Metic vokkcee Anirpes
3.6
32
62
23n
Made PaPmeeens
Interresis Paivrsttee
Leker Roaeetes
Figure 11: Induced emergent MNIST classification ability in convolutional networks. (A) A published emergent ability from the BIG-Bench Grounded Mappings task 38 (B) LeNet trained on MNIST 23 displays a predictable, commonplace sigmoidal increase in test accuracy as model parameters increase. (C) When accuracy redefined as correctly classifying K out of K independent
test data, this newly defined metric induces seemingly unpredictable change.
family's 's emergent classification ability qualitatively matches published emergent abilities, e.g., the BIG-Bende Grounded Mappings task [38 (Fig. ITA).
C Relationship Between Emergent Abilities and Grokking
Emergent abilities 3 38 are sometimes compared with grokking 26 III a phenomenon whereby single model will, over the course of learning, achieve high training accuracy and only much later achieve high test accuracy. There are several differences between grokking and emergent abilities:
1. Grokking is primarily studied within a single model, whereas emergent abilities are studied within model family (i.e., multiple models).
2. Grokking occurs with increasing gradient steps, whereas emergent abilities occur with increasing model scale, typically measured in parameters or effective parameters (although more recently compute).
3. Grokking explicitly studies a discrepancy between the model's train and test behavior, whereas emergent abilities (to the best of our knowledge) do not present separate train & test curves.
4. Grokking is primarily studied on toy "algorithmic" tasks in small networks, whereas emer- gent abilities often studied on benchmark NLP tasks in large language models.
17


Reviews:
[{"review_id": "GEroM6Kd2X", "reviewer": "Reviewer_Cfbn", "summary": "The paper shows that the apparent suddenness with which capabilities of language models seem to emerge with scale may be an artifact of the metrics by which they are measured. Theoretically, emergence should be more gradual if the metric scales linearly with log loss on some test set. Three sources of empirical evidence are used to support this claim. First, the authors study the scaling of the GPT-3 family on arithmetic tasks and show directly that emergence is smoother when a linear metric is used. Second, they conduct a meta-analysis of BIG-Bench, and show that gradual emergence only occurs with non-linear metrics, and that the effect is reduced when switching to a linear metric. Third, they artificially induce sudden emergence by using non-linear metrics for autoencoders on CIFAR100 and for transformers on Omniglot.", "questions": "No questions other than those given above", "limitations": "No limitations other than those given above", "rating": 7, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "The basic point made by the paper is an important one that has not been fully appreciated by researchers. It is something that everyone studying scaling laws should understand and incorporate into their future work.", "weaknesses": "I think there are notable weaknesses in the empirical evidence given to support the basic point of the paper, and insufficient acknowledgement of the limitations of the evidence and other possibilities that could still give rise to sudden emergence.\n\nOn the empirical evidence:\n- The main direct evidence that previous claims of sudden emergence are an artifact of the metric comes from studying those previous claims directly, which is done in Section 3. However, the evidence here is somewhat inconclusive: for example for 2-digit multiplication with target str len 4 (which is presumably most 2-digit multiplications?) the jumps from 350M->1.3B->6.7B are gradual while there is a much bigger jump from 6.7B->175B. Moreover, this bigger jump is not predicted by the theoretical model. So it seems like there is still some relatively sudden emergence (even though it is reduced by using the right metric).\n- The BIG-Bench analysis is helpful, but again on logical_args it seems like there may still be an unpredictable jump. And on the other tasks, the emergence is not that sudden even with the original metric.\n- The induced emergence experiments are also helpful, but they show that sudden emergence can be induced by using a non-linear metric, not that it is always induced by a using a non-linear metric. So this evidence is more indirect.\n- Overall I think the paper would have come to more correct (and probably more nuanced) conclusions if it had studied all of the tasks from Figure 1, where sudden emergence had previously been claimed, in detail, instead of studying just one of these tasks combined with other, weaker sources of evidence.\n- Nevertheless, for an initial exploration of the topic, I think combining multiple source of evidence, even if somewhat weak, is helpful, and it is reasonable to defer a more careful analysis to future work.\n\nOn other theoretical possibilities:\n- There is no guarantee that because validation loss scales smoothly, cross-entropy loss on a specific sub-distribution must also scale smoothly. This depends on how the model allocates capacity to that sub-distribution. Therefore there is still an important theoretical possibility of sudden emergence, even with a linear metric. Indeed, this may explain why the data did not match the theoretical model in Section 3.\n- In some cases, it may not be possible to construct a linear metric. For example, for program synthesis, whether the program passes the unit tests cannot be easily converted to something like edit distance or log loss.\n- It is argued that log accuracy can be used in this case. But this is not necessarily feasible. For example, if current models pass unit tests with probability 1 in a trillion, it is not feasible to measure this accurately enough to perform any useful extrapolation. Therefore there may be sudden emergence for all practical purposes.\n\nOverall, I thought that the evidence was strong enough to support a conclusion that sudden emergence is partially explained by the choice of metric, and that researchers should prefer metrics that scale linearly with log loss when available. I also think that this conclusion is itself enough for the paper to be an important contribution. But I think the paper should take more care not to overstate its claims and acknowledge the limitations of the evidence and, space permitting, other theoretical possibilities. In particular, claims like \"metric choice is likely wholly responsible for emergent abilities\" (p. 9 line 204) are not justified and should be removed or reworded.\n\nI am recommending acceptance of this paper because of the importance of the contribution, but I think that some effort should be made to revise the paper prior to acceptance. However, I am confident in the ability of the authors to do this.\n\nSome presentation points:\n- I think it would be clearer to talk about \"sudden emergence\" and \"gradual emergence\" rather than \"emergence\" and \"no emergence\", since \"emergence\" can also be interpreted as \"appearing despite not being explicitly trained for\". However, I understand that the current terminology is taken from the existing literature and the paper cannot be blamed for this.\n- I suggest changing the title of the paper, because the current title can be misinterpreted as asking whether the _abilities_ are mirages, rather than whether the emergence (or sudden emergence as I prefer to put it) is a mirage. Any researcher who reads the abstract should not be misled, but unfortunately provocative titles that can easily be misinterpreted can be picked up on by the media and end up misleading the public. However, I acknowledge that choosing titles is hard and no title is perfect.\n\nMore minor points:\n- p. 4 line 78 - I am not familiar with this Delta notation (or I have a vague recollection of it). It is clear from the context so probably fine, but consider alternative notation/clarifying the definition.\n- p. 5 line 115 - Why choose this task from among all the tasks in Figure 1, and why 2-shot / 4-digit? (Would be even nicer to include results on other tasks/other numbers of digits, but absent that, it would be good to clarify how you chose, to make it clear it was not cherry-picked.)\n- Figure 3: Target str len 1: how do you get a 1-digit answer from multiplying two 2-digit numbers? I assume you are counting things like \"01 * 01\"? Maybe clarify.\n- Figure 3: I think it would be clearer to include titles at the top of each column saying \"Mathematical Model\", \"2-Integer 2-Digit Multiplication Task\", \"2-Integer 4-Digit Addition Task\", instead of putting this in the caption. That way people can easily see at a glance what each plot is showing, without having to dig into the caption.\n- Figure 4: Same comment.\n- Figure 5: It would be nicer to group the metrics based on their mathematical form. E.g. it is unclear what \"gender_bias_score\" is. This also might help spot if there are more examples with emergence that didn't have high emergence score.\n- Figure 5B is a bit redundant. Maybe just point out in figure 5A which of the 5 did not count as emergence. It's also a bit unclear why this didn't count despite the emergence score.\n- p. 9 line 209 - I don't think this work really contradicts broken neural scaling laws. Even if sudden emergence goes away when you change the metric, one might still be interested in forecasting the original metric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for time & feedback! You made clear effort to think critically about how to improve our paper! 6k space limited -> dense response; try our best.\n\n> Why few-shot int arithmetic\n\n1. Few-shot int arithmetic featured prominently in multiple prev papers L102-104.\n2. To make point about sufficient resolution, needed to be able to generate more samples; generating samples easier for arithmetic than other tasks, e.g., Persian QA. We’ll add.\n\n> Better to study all tasks from Fig 1\n\nAgreed but not fully feasible: 4/5 models private (LaMDA, Gopher, Chinchilla, PaLM). PaLM2 recently became queryable, but to best of knowledge, PaLM 1 spanning several scales is not. Before submitting, tried obtaining networks’ outputs via priv comms with authors but unsuccessful. Thanks to your comment, we realized we can study GPT-3 fam on IPA Transliterate & Grounded Mappings. Aren’t sure about MMLU since MMLU=57 tasks; may be beyond our budget. How important would collect+analyze IPA Transliterate & Grounded Mappings be?\n\n\n> main direct evidence [...] is somewhat inconclusive\n\nOur work follows ethos: all models are wrong, but some are useful. Role of mathematical model: explain quantitatively what scaling behavior we should qualitatively expect in LLMs & why. As stated, math model relies on approximations (per-token error rates are approximately constant and independent) that are only ~true. Despite, math model usefully predicts effects of: varying length of target string, changing metric, & sampling additional data to incr resolution.\n\n> [BIG-Bench] logical_args seems like there may still be unpredictable jump\n\n> And on the other tasks, emergence is not that sudden even with original metric.\n\nBIG-Bench’s data sometimes confusing. Example: Swahili English Proverbs (red): Multiple Choice Grade improved for largest model, but Brier Score regressed; inconsistent? Data from private model (LaMDA) -> unable to investigate. Unsatisfying answer is claims of emergent abilities involve subjective judgment.\n\n> The induced emergence experiments are also helpful, but [...] this evidence is more indirect.\n\nAgreed. Point: show that metric choices can make well-known networks appear to possess emergent abilities on commonplace tasks.\n\n> There is no guarantee that because validation loss scales smoothly, cross-entropy loss on a specific sub-distribution must also scale smoothly [...] Therefore there is still an important theoretical possibility of sudden emergence\n\nAgree. L218: “We emphasize that nothing in this paper should be interpreted as claiming that large language models _cannot_ display emergent abilities.” **Will add Limitations section** including this. Interesting hypotheses exist & are cited e.g., Michaud et al. 2023.\n\n> It may not be possible to construct a linear metric, e.g., program synthesis\n\nIndeed, linear metric not always possible, but to clarify, we aren’t advocating linear metrics, nor criticizing any metrics. Main point: **if** goal to predict scaling behavior of model family, chosen metric may have effect, so take that effect into account so you aren’t surprised. Corollary: to make accurate scaling forecasts, continuous & linear metrics are probably more advantageous, but if discontinuous & nonlinear metrics preferred, then may need lots of data to have sufficient resolution to accurately measure performance. Choose whichever metric is best for your setting - just make sure to think through consequences of metric choice!\n\nLinear metrics might be more broadly applicable; Program Synthesis: count number of unit tests passed e.g. HackerRank coding interviews\n\n> [log accuracy] is not necessarily feasible.\n\nNot claiming log accuracy necessarily feasible. Rather, claim: if one wishes to use a particular metric when forecasting scaling behavior, then one should consider whether they possess sufficient data to have sufficient resolution to accurately estimate performance under their metric. In some cases, this might be feasible, but in other cases, this might not be. Will clarify.\n\n> Therefore there may be sudden emergence for all practical purposes.\n\nThis involves a hidden step that we think is unlikely. We are skeptical that accuracy could transition from ~1e-12 to ~1 without detectably passing through intermediate orders of magnitude at intermediate model scales. While not impossible, to best of knowledge, such an abrupt transition has not been observed.\n\n> the paper should take more care not to overstate its claims and acknowledge the limitations\n\nAgreed. Will add Limitations sec & replace strong claims e.g., “metric choice is likely wholly responsible for emergent abilities” with measured statements e.g., “metric choice might possibly be responsible for some claimed emergent abilities.”\n\n> p. 4 line 78\n\nDelta = prob simplex. Will clarify.\n\n> p. 9 line 209 - doesn’t contradict broken neural scaling laws\n\nDon’t contradict, but dissimilar. BNSL attributes emergent abilities to changepoint in scaling behavior -> $\\geq$2 scaling law vs  our paper says _even with 1 scaling law_, emergent abilities might be due to the metric.\n\n> suggest changing the title of the paper\n\n> clearer to talk about \"sudden\" & \"gradual emergence\"\n\nTerminology tricky & attempts resulted in dissatisfied colleagues. Difficulty with “sudden” vs “gradual emergence” is emergence has rich history in physics, typically at phase transitions; “gradual emergence” could be viewed as oxymoron. We have also been told not to use “phase transition;” one colleague: “Unless you can write down a partition function and show that it's non-analytic, I strongly suggest not doing this.” Also tried “Emergent Abilities” in quotation marks to make clear referencing prior terminology; quotation marks interpreted as mocking prior work. Have more stories. After trying repeatedly, we feel current terminology is best available compromise. Tempering our claims & adding Limitations sec will help. Happy to hear additional suggestions.\n\nOther sugg changes will be made."}, {"review_id": "6WYkq0cn3G", "reviewer": "Reviewer_GXCv", "summary": "This paper shows that the widely-discussed phenomenon of \"emergent abilities\" in large language models is largely due to the choice of (discontinuous/nonlinear) metrics and underpowered analyses.\n", "questions": "- l227: Not sure what is meant by \"multiple controls\" - is the implication that the community should have more aggregate / multi-metric benchmarks like BIG-Bench?\n- comment: I found myself wanting to compare Fig 4 side-by-side with Fig 3 to understand the change from additional test data, but it was difficult to do so (especially with the diff y-axis scaling).", "limitations": "Yes", "rating": 9, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- Well-written paper studying an important/commonly discussed phenomenon\n- Comprehensive, well-designed, and convincing experiments + analysis\n", "weaknesses": "No major weaknesses - the paper was a pleasure to read!\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer GXCv for their time and feedback, and we’re overjoyed to hear you write that the paper was a pleasure to read! Replying to each comment, suggestion and/or question in turn:\n\n> l227: Not sure what is meant by \"multiple controls\" - is the implication that the community should have more aggregate / multi-metric benchmarks like BIG-Bench?\n\nTo confirm, the sentence you’re referring to is “In this particular setting, emergent abilities claims are possibly infected by a failure to control for *multiple comparisons*”? Assuming the answer is yes, thank you for giving us a chance to clarify! Here, “multiple comparisons” doesn’t refer to aggregate (sometimes called holistic) evaluation like BIG-Bench or HELM.\n\n\nRather, “multiple comparisons” refers to a phenomenon where someone can draw an incorrect conclusion after considering many attempts because they forget to normalize by the number of attempts. The point is that if one runs many tests, the probability of at least one false positive increases. XKCD #882 explains this through a parable of jelly beans causing acne.\n\nHere, in the context of emergent abilities, the risk is that if one looks at the scaling behavior of 10^6 task-model family-metric triplets, one probably should expect to find at least a few seemingly unpredictable changes due to random chance alone. The solution to multiple comparisons (oftentimes referred to as “controlling for multiple comparisons”) is to raise one’s threshold for declaring a discovery. For instance, we might say, “Because we’re looking at 10^6 task-model family-metric triplets, in order for us to be confident that emergent abilities are a phenomenon (and not just due to random chance), we need to see emergent abilities on at least 1% of the tests.” The statistical hypothesis testing community has ways of making this precise and statistically justified; a good starting point might be the [family-wise error rate](https://en.wikipedia.org/wiki/Family-wise_error_rate) page. Hopefully, this helps clarify!\n\n> comment: I found myself wanting to compare Fig 4 side-by-side with Fig 3 to understand the change from additional test data, but it was difficult to do so (especially with the diff y-axis scaling).\n\nVery understandable! We need the y-axes to have different scaling to make our point that by changing the y-axis and by using more data to increase the resolution, the seemingly zero-accuracy values are actually non-zero accuracies. Would you recommend visualizing the effect of sample size in a different way? For instance, we could show estimation error at different sample sizes by subsampling the larger dataset at different subsample sizes. We have these plots and could submit them as part of the 1 page PDF addendum.\n"}, {"review_id": "26t8GFtXBy", "reviewer": "Reviewer_eK8N", "summary": "This paper aims to debunk a possibly myth about the emergent properties of language models with regard to scale. The emergent behaviors are defined as abilities that are not demonstrated with smaller models, but emerge suddenly with an increase in the model capacity, and cannot be explained by smoothly extrapolating from the model capacity -- i.e., emergent abilities are sharp and unpredictable. The claim that the authors seek to validate is that emergent abilities are actually artifacts of the fact that the choice of evaluation metric, and that continuous metrics (instead of discrete) show smooth increases in performance with scale -- i.e., that \"emergent\" abilities are in fact smooth and predictable. ", "questions": "- Why are smooth evaluation metrics necessarily desirable if they are sometimes a relaxation of the real objective, which may actually be discrete (e.g. the 0-1 loss)? \n- NOTE: please switch to the NeurIPS 2023 template, I noticed that you are using the 2022 version of the template. ", "limitations": "The authors address this. ", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This paper has many strengths:\n- First off, it is well-written. \n- The paper is timely -- large language models have only become moreso in the center of attention in the past year with abundant claims surrounding emergent abilities. \n- The claim that discrete metrics are a source of so-called \"emergence\" of new abilities is reasonable and makes intuitive sense. \n- The empirical evaluation is thorough, with evaluations of GPT-3/InstructGPT models, a meta-analysis using BigBench, and using discrete metrics to artificially induce \"emergent behavior\" in existing vision models. ", "weaknesses": "I may have misunderstood parts of the argument, but I am not entirely convinced that emergent abilities might not be real, when it is clear that they are real with respect to discrete metrics (that often have practical interpretations). The paper seems to question the basic premise of emergent abilities, i.e., whether or not they are real. The authors cite the following definition early on -- emergent abilities are \"abilities that are not present in smaller-scale models but are present in large-scale models; thus they cannot be predicted by simply extrapolating the performance improvements on smaller-scale models,” which itself depends on an evaluation metric. Since the definition depends on an evaluation metric, why does changing the metric make emergent abilities a mirage? Shouldn't emergent abilities only hold, then, with respect to a metric? \n\nPer my understanding of this work, it seems to me that incorrect conclusions have been drawn in prior work by *only* considering discrete evaluation metrics, which is not the full picture -- pointing this out is the main contribution of this paper, and this is an important point to make. However, it still seems that these properties are real with respect to discrete metrics, and these discrete metrics are often desirable. Does this make the choice of metric bad, and emergence a mirage as a result? Or does this render prior analyses incomplete, since they do not consider metrics in which performance **does** smoothly change? ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer eK8N for their time and feedback! Replying to each comment, suggestion and/or question in turn:\n\n\n> The authors cite the following definition early on -- emergent abilities are \"abilities that are not present in smaller-scale models but are present in large-scale models; thus they cannot be predicted by simply extrapolating the performance improvements on smaller-scale models,” which itself depends on an evaluation metric. Since the definition depends on an evaluation metric, why does changing the metric make emergent abilities a mirage? Shouldn't emergent abilities only hold, then, with respect to a metric?\n\n\nThis is a good question! Our view is that the answer is no. An ability is a talent, skill, or proficiency in a particular area, whereas a metric is one way of quantitatively measuring that ability. Many different metrics can be used for measuring a particular ability. For example, if we want to quantify how good someone is at playing basketball in the NBA, there are many metrics we might turn to: number of won championships, true shooting percentage, effective field goal percentage, player efficiency rating, etc.\n\n\nTo explain another way, emergent abilities are discussed as profound changes of a model family with scale. But if indeed fundamental changes are occurring in the model family, then the changes should appear under many metrics. What we show is that this doesn’t seem to appear in the claimed emergent abilities we analyze.\n\n\n> I may have misunderstood parts of the argument, but I am not entirely convinced that emergent abilities might not be real, when it is clear that they are real with respect to discrete metrics (that often have practical interpretations).\n\n\nRespectfully, “real” and “not real” is terminology we try to avoid. Rather, the claim of “emergent abilities” is that as models get larger, their behavior changes sharply and unpredictably. Our paper argues that previously observed sharp and unpredictable changes might be better attributed to the metric than to fundamental changes in the model family with scale. We use the term “mirage” because, from a distance, models appear to be changing abruptly, but upon closer investigation, the abrupt changes can disappear and become more predictable. \n\n\n> However, it still seems that these properties are real with respect to discrete metrics, and these discrete metrics are often desirable.\n\n\nWe agree; indeed, we show this in our Figure 3 Row 1! Our paper is not saying previous papers’ experimental results are wrong or that the metrics they use should not be used. Rather, our point is that previous papers might have been surprised by scaling behavior because of the metrics they used or because of limited data. In addition, we show that even for such discrete metrics, the jump is an expected behavior and provide evidence against labeling this as \"emergence,\" i.e., an unpredictable jump. Furthermore, by changing the metric or increasing the resolution using more data, they might have discovered smoother, more predictable trends with scale.\n\n\n> Or does this render prior analyses incomplete, since they do not consider metrics in which performance does smoothly change?\n\n\nThis statement is closer to what we have in mind, but we think our paper goes beyond merely saying previous papers are “incomplete”. We think a conceptual blind spot existed in multiple papers that our paper identified. To explain, “Emergent abilities” doesn’t mean that bigger models perform better than smaller models. Rather, “emergent abilities” means that bigger models _suddenly and unexpectedly_ perform better than smaller models. To our knowledge, previous papers did not try changing metrics or collecting more data to increase resolution. Our work qualitatively predicts the effects (and interaction!) that metric and sample size should have, then confirms those predictions.\n\n\n> Does this make the choice of metric bad, and emergence a mirage as a result?\n\n> Why are smooth evaluation metrics necessarily desirable if they are sometimes a relaxation of the real objective, which may actually be discrete (e.g. the 0-1 loss)?\n\n\nThank you for giving us a chance to clarify! We are not criticizing any metric, nor recommending one metric over another metric. Our main takeaway is that *if* you want to be able to predict the scaling behavior of a model family, the metric you choose may have an effect on your results, and so you should take into account the effect your chosen metric will have so you aren’t surprised. A corollary is that if you want to make accurate scaling forecasts, continuous & linear metrics might be more advantageous, but if you do want to use discontinuous or nonlinear metrics (which is totally fine!), then you may need lots of data to have sufficient resolution to accurately estimate model performance under your chosen metric. You should of course choose whichever metric is best suited for your setting - just make sure to think through what effects your metric choice will have! We will make these points more clearly in our Discussion. Hopefully that helps!\n\n\n> NOTE: please switch to the NeurIPS 2023 template, I noticed that you are using the 2022 version of the template.\n\nGreat catch - thank you for letting us know! We will fix this!"}, {"review_id": "IHydEpdJy5", "reviewer": "Reviewer_ccWP", "summary": "The authors argue that purported emergent abilities of language models are mostly explained by the use of specific metrics, including non-linear or discontinuous scoring, and evaporate when linear or continuous scoring metrics are applied. The authors (1) present a toy model to explain emergence, (2) demonstrate that most abilities previously reported as emergent no longer appear emergent under an alternative metric, and then synthetically cause the appearance of emergence in an image reconstruction task.", "questions": "See weaknesses", "limitations": "The authors adequately address the limitations.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Simply and clearly addresses the enigmatic phenomenon of emergence\n\n- Formalizes and quantitatively tests the hypothesis of [1] that emergence is due in part to metric\n\n- Compelling toy model to illustrate how emergence can appear with non-linear metric (accuracy) but vanishes with linear metric (token edit distance)\n\n- Convincingly demonstrates that emergence is (at least often) a result of the metric", "weaknesses": "- As the authors point out in their related work, the idea that emergence appears due to the metric is not novel, and was proposed by [1], so the novelty of this paper lies in the quantitative evaluation of this hypothesis\n\n- There has been work on examining emergence and progress measures in similar contexts, e.g. grokking, which should be addressed in the related work [2–5]\n\n- There was no discussion of the relationship to alternative hypotheses. The authors mention in the related work multiple papers that suggest that abilities do, in fact, emerge with scale, but the authors should explain how their hypothesis relates\n\n- There was no discussion of the BLEU score metric, even though there are many tasks that have a very high score with this metric\n\n- Minor: a lot of the figures are small and therefore hard to read\n\n[1] Srivastava, Aarohi, et al. \"Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.\" arXiv preprint arXiv:2206.04615 (2022).\n\n[2] Power, Alethea, et al. \"Grokking: Generalization beyond overfitting on small algorithmic datasets.\" arXiv preprint arXiv:2201.02177 (2022).\n\n[3] Liu, Ziming, Eric J. Michaud, and Max Tegmark. \"Omnigrok: Grokking beyond algorithmic data.\" arXiv preprint arXiv:2210.01117 (2022).\n\n[4] Nanda, Neel, et al. \"Progress measures for grokking via mechanistic interpretability.\" arXiv preprint arXiv:2301.05217 (2023).\n\n[5] Barak, Boaz, et al. \"Hidden progress in deep learning: Sgd learns parities near the computational limit.\" Advances in Neural Information Processing Systems 35 (2022): 21750-21764.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer ccWP for their time and feedback! Replying to each comment, suggestion and/or question in turn:\n\n> As the authors point out in their related work, the idea that emergence appears due to the metric is not novel, and was proposed by [1], so the novelty of this paper lies in the quantitative evaluation of this hypothesis\n\nSrivastava et al. 2022 indeed questioned whether emergent abilities of LLMs may be due to the metric. However, respectfully, we feel the reviewer undervalues our paper’s novelty & contributions. Our work converts a few sentences into precise conceptual understanding, which in turn generates empirically testable predictions in LLMs, suggests new meta-analyses of previous publications’ results, and inspires new experiments that we implement. Specifically: Firstly, we converted Srivastava et al. 2022’s question into a novel simple mathematical model that is both intuitive and quantitatively testable. Secondly, we used our mathematical model to generate empirically testable predictions and then tested those predictions, confirming them. Thirdly, we proposed and executed a novel meta-analysis of BIG-Bench to reveal additional supporting evidence. Fourthly, we identified a class of metrics that Srivastava et al. did not discuss (i.e., discontinuous metrics such as Multiple Choice Grade). Lastly, we showed how metric choice can induce seemingly emergent abilities in a different modality that had never before been shown, providing an explanation for why emergent abilities have (to the best of our knowledge) not been observed in vision. We also mention additional causes for skepticism, such as previous papers’ possible failure to control for multiple comparisons.\n\n> There has been work on examining emergence and progress measures in similar contexts, e.g. grokking, which should be addressed in the related work [2–5]\n\nWe see several significant differences between grokking and emergent abilities of LLMs:\n1. Grokking is primarily studied within a single model, whereas emergent abilities are studied within a model family\n2. Grokking occurs with increasing gradient steps, whereas emergent abilities occur with increasing model scale typically measured in parameters or effective parameters (although more recently compute).\n3. Grokking explicitly studies a discrepancy between the model’s train and test behavior, whereas emergent abilities (to the best of our knowledge) do not present separate train & test curves.\n4. Grokking is primarily studied on toy tasks in small networks, whereas emergent abilities are studied on benchmark tasks in large language models.\n\nWhile we personally appreciate the grokking literature, and while we may be mistaken, we respectfully feel that grokking is too distant from this work. If requested, we would be happy to add a comparison with grokking to our Appendix.\n\n> There was no discussion of the relationship to alternative hypotheses. The authors mention in the related work multiple papers that suggest that abilities do, in fact, emerge with scale, but the authors should explain how their hypothesis relates.\n\nRespectfully, we do discuss alternative hypotheses and how they relate to our paper on Lines 209-212: “Caballero et al. [4] explain emergence by assuming a piece-wise power law functional form; under this view, emergent abilities are real, caused by a change in the governing power law. In contrast, our work suggests that emergent abilities are induced by the researcher, even under a single power law. Michaud et al. [25] posit that emergent abilities may be real under strong data assumptions.”\n\nAre you requesting that we discuss the differences in greater depth? We would be happy to expound here, as well as in the paper.\n\nCaballero et al. 2023 contrasts with our work in that their hypothesized explanation for emergent abilities is that the governing power law changes at specific break points. Our explanation for emergent abilities is that the researcher’s analyses create the seemingly emergent abilities. Both explanations could be true: some emergent abilities might genuinely be abruptly appearing, whereas some emergent abilities might be attributable to the metric. \n\nMichaud et al. [25] posits that language modeling data might be comprised of discrete subtasks (“quanta”) that networks learn. Larger networks have greater capacity, and thus are more capable of learning more of these quanta. If some downstream task requires a network to learn some combination of quanta, then larger networks are more likely to have all the requisite capabilities and thus are capable of performing this downstream task. We think that this is a very interesting hypothesis. Whether language modeling data can or should be understood from this quantization perspective, and whether these quanta indeed are the origin of emergent abilities, are really exciting questions that we think merit more study! \n\n> There was no discussion of the BLEU score metric, even though there are many tasks that have a very high score with this metric\n\nPrevious papers have not made many claims about emergent abilities under BLEU score; to the best of our knowledge, we know of only one task (IPA transliterate) on which at least one model family exhibits emergent abilities. In contrast, the hand-annotated emergent abilities by https://www.jasonwei.net/blog/emergence almost all seem to occur under other metrics (Multiple Choice Grade, Accuracy, Exact Match). Due to a dearth of claimed emergent abilities under BLEU, we chose to focus on the more prevalent metrics.\n\nThat said, we actually had an appendix subsection on BLEU score that we cut due to the relative rareness of BLEU and due to the subsection’s length (explaining BLEU requires more work than Exact Match). We would be happy to put the appendix subsection back in if requested. The answer is that BLEU is also a sharp nonlinear metric. \n\n> Minor: a lot of the figures are small and therefore hard to read\n\nWe will fix this. Thank you!"}]

Author rebuttal:

Meta-review: This work provides a closer look at the nature of increased capabilities of machine learning models at scale, arguing that prior evidence for unpredictable and qualitative leaps in capability is confounded by the metrics employed to measure such capabilities. This has broad implications on how we describe and evaluate the capabilities of model.

Reviewers found the support provided for this claim to be thorough and appreciated the additional nuance resulting from the discussion period. The critical perspective provided by this work is timely and will be of broad interest at the conference, and also just provides an important lesson.
Decision: Accept (oral)