The Quantization Model of Neural Scaling
Authors: Eric J Michaud
Ziming Liu
Uzay Girit
Max Tegmark
Venue: NeurIPS 2023 poster (2023)
Keywords: scaling laws
emergence
language models
science of deep learning
TL;DR: Develops a model of neural scaling where smooth scaling laws average over small emergent changes in model performance on subtasks.

Abstract: We propose the Quantization Model of neural scaling laws, explaining both the observed power law dropoff of loss with model and data size, and also the sudden emergence of new capabilities with scale.  We derive this model from what we call the Quantization Hypothesis, 
where network knowledge and skills are "quantized" into discrete chunks (quanta). We show that when quanta are learned in order of decreasing use frequency, then a power law in use frequencies explains observed power law scaling of loss. We validate this prediction on toy datasets, then study how scaling curves decompose for large language models.  Using language model gradients, we automatically decompose model behavior into a diverse set of skills (quanta). We tentatively find that the frequency at which these quanta are used in the training distribution roughly follows a power law corresponding with the empirical scaling exponent for language models, a prediction of our theory.

The Quantization Model of Neural Scaling
Zichng Eric J. Michaud, Ziming Liu, Uzay Girit, and Max Tegmark MIT & IAIFI
Abstract
We propose the Quantization Model of neural scaling laws, explaining both the observed power law dropoff of loss with model and data size, and also the sudden emergence of new capabilities with scale. We derive this model from what we call the Quantization Hypothesis, where network knowledge and skills are "quantized" into discrete chunks (quanta). We show that when quanta are learned in order of decreasing use frequency, then power law in use frequencies explains observed power law scaling of loss. We validate this prediction on toy datasets, then study how scaling curves decompose for large language models. Using language model gradients, we automatically decompose model behavior into a diverse set of skills (quanta). We tentatively find that the frequency at which these quanta are used in the training distribution roughly follows a power law corresponding with the empirical scaling exponent for language models, prediction of our theory.
Introduction
In the aggregate, larger neural networks trained on more data perform better than smaller neural networks trained on less data, in predictable way. Across a range of studies, mean test loss has been observed to decrease as a power law in both the number of network parameters (L x N- and the number of training samples (L x D- [1, 2, 3, 4, 5, 6, 7]. Although aggregate performance changes smoothly with scale, when particular capabilities are examined, larger models often have emergent abilities, i.e., qualitatively different performance than smaller models [8, 9]. Understanding and reconciling both facets of scaling the predictable power law decrease in loss and the emergence of new capabilities at scale is of both theoretical and practical interest [10]. Understanding how scaling changes what neural networks learn is entangled with core questions: what are deep neural networks doing internally, and will they will continue to improve with scale?
Recent studies of the internals of neural networks have found a variety of impressive algorithms learned by gradient descent [1 12, 13, 14, 15]. As more work is put into understanding the structures learned by neural networks (the task of mechanistic interpretability). we may find more and more circuits [11. 16] in models, intelligible internal algorithms for accomplishing prediction in specific contexts. Can such analysis be scaled up to frontier models [17]? Two assumptions which, true, would make us more optimistic about mechanistically understanding large models include (1) hatommosaabiittymmmuuur/////////
[18, 19, 20, 21] that large models are decomposable into parts, and only small number of these parts are relevant to the model's behavior on any given sample and (2) universality [22, 11, 23, 24] - that similar structures recur across models of increasing size. Recently, Olsson et al. [25] found encouraging evidence for universality of "induction heads" across LLMs and found that these emerge a discrete transition during training.
In this paper, we articulate the Quantization Hypothesis, set of informal conjectures about the decomposability of networks into smaller parts, the universality of computations performed across model scales, the discreteness of what models lear, and about how properties of the data distribution
"eriçjm@ crriji@@mi.eddu mit.edu
Project code can be found
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

LLM skill "quanta" auto-discovered in text
Cluster 50: incrementing numerical sequences
Cluster 100: predicting newlines in line length limited text
D1- Mi Querencia (Simón Diazia
TO PERFORM QUADRATIC REGRESSIONN
02- Tonidd De Luma Llena (Simén Diazimm
ON THE T184, GRAPEING CALCULATOR., DETERMINE HOW WELL THE tw REGRESSION MODEL FITS THE DATA
Sabinat (Josk Salazar/Sim Di
04- Caballo Vieja (Sinda Diazya
05- Toda Este Campu Es Mio (Simán Dian)/n
AND THEN MAKE PREDICTIONS 'n USING THE RECRR ESSION FQUATION,
06- La Pera Del Becenere (Simjón Diaz.jon
STATISTICS,
REGRESSION ANALYSIS INCLUDESS US ANY TECHNIQUES USED FOR MODELING
from opening a through moad or sncet for public use acoss said public park in the Park of The City of River ton v. (Fiphhaii suplliad.) Appealing that order.
plaintifis have standnng action: (2) that the propased nod was in unded
right maistain
crod damp free sottware: you can rodisrabute andror modifyln
isated purt, of the parks (3) that the proposed road was an access road and
it under the terms of the GNU General Publis License as publisshd by/rt
not through street punt of the city street system; (#
the Free Seffware Foundation, either version 3 of the License, orlnt
your option) any later varsion
4. Introduction Im
A'n
S. Chapter 1:
What 1s Trass
cred damp distrbuted in the hope that will be usrfaal tn
6. Chapter 2. Trust Brings Resiin
Chapler
Truss
8. Chipter The Foily of Rell-Rell
<I-mm
9. Chapter S: Trust God and Do Good (Pan) 1)
10. Chupler 6: Trust God and Do Good, (Par) 2)
Copyright (c) 2019, The Android Open, Source Projectio
Chapter 7)
12. Chapter
Licensedd under the Apache License, Vensoon 0 (the "License" I:'n you, may not use this lile except in compliance with the License Ant
achieved The chosen sites were rocanled ne) it sound (* =I 13) first visible sign noncav iisted lesion seen only when wooth dried: = visible moneavitated losion soen when wet and dry: 3 microcas iusiin in coamel; = noncavitatted lesion estending into dentine seen an under mining shallw:5 15 small cavisated lesion with visible dentine; than surface;
*'m
Pursuant to 5'11 CIR, R. 47,5, the court has deserminedio that this opinion should not be published and is prevedemi'n except under limised circumstances forth in
OCBlockLishy
= =0x00am
Geil ateest StauuMigg
=0006in
children have lack of maturit and in undrreeeeoewdli sense responsiblliy, leading recklesnness impul sivity, and hredlless risk taking ... Secund, children'a in are more vulnerable. to negative influences andia outside pressures. including from their family and'n peers: they have limmted controf]] over own
Latest
Oxx0c'n
Prepure Block Bacckl
= Ox0dda
Get ChaggeMag
Uxciela
PingMsg
0x00
Figure 1: We auto-discover quanta basic units of model knowledge/skill - for a language model. Here we show collections of next-token prediction samples which our method clustered together, each corresponding to some coherent model behavior. We indicate the token which was predicted from the context before it with a red highlight. We indicate newlines using "\n". See Section 5 for explanation.
produce power law neural scaling. In particular, we hypothesize that many prediction problems, there corresponds a particular enumerable set of indivisible pieces of knowledge or skills that models must learn, and that model performance is determined by which of these elements models successfully learn. We call these basic building blocks of model performance the quanta:
Quantum (plural quanta): An indivisible computational module that, for example, retrieves a fact, implements an algorithm, or more generally corresponds to some basic skill possessed by model.
We use this terminology in analogy to Max Planck's assumption in 1900 that energy is quantized into discrete chunks (quanta) - here we imagine that knowledge/skills are quantized into discrete chunks (quanta). Since "quantization" is commonly used in machine learning in the context of low-precision arithmetic, we suggest "knowledge quantization" " or "skill quantization" " to refer to our notion of quantization. We will see that Zipfian distribution governing the "use frequency" of the quanta produces power law neural scaling, where the effect of scaling is to learn an increasing number of discrete quanta, and smooth scaling laws average over small discrete jumps in model performance.
This paper is organized as follows: in Section 2 we give a theoretical model of power law neural scaling from the Quantization Hypothesis. In Section 3 we construct toy datasets satisfying the hypothesis, where smooth power laws average over many discrete jumps in model performance. In Section 4 we then analyze how power law scaling decomposes for real LLMs. In Section 5, we develop a method for automatically discovering quanta in language models by clustering their behavior into basic coherent skills, and analyze the statisties of these clusters, concluding in Section 7.

---

2 Theory
Consider the task of modeling the distribution of text the internet. Successful prediction requires an immense amount of knowledge, and the ability to perform diverse computations, due to the immense complexity and diversity of the world and therefore of human language. For instance, in order to predict what word will come next in a conversation between two physicists, one must "know" much about physics. In order to continue the text "2534 + 7261 arithmetic (for large enough numbers, memorization becomes highly inefficient strategy) [26]. A great many distinct types of computations are present the world in the processes that produce text, and so predicting text requíres those computations to be present in our models.
one must be able to perform
In this paper, we conjecture the Quantization (or Quanta) Hypothesis:
QH1 Many natural prediction problems decompose into an enumerable set of com- putations, pieces of knowledge, or skills, which models must learn to reduce loss. We call these quanta, and model them as being discrete, they are either learned or not learned. Model performance is determined by which quanta have been learned.
QH2 Some quanta more useful for reducing loss than others, leading to a natural ordering of the quanta. We call the ordered quanta the Q Sequence. Optimally trained networks should therefore learn the quanta in that order. The effect of scaling is to learn more of the quanta in the Q Sequence, so scaling performance is simply determined by how many quanta successfully learned.
QH3 The frequencies at which the quanta used for prediction follow power law.
Together these result in power law neural scaling. We model the Quantization Hypothesis as follows, referring to the below as the "Quantization (or Quanta) Model". Let q denote a bit string whose 8th bit qı 1 if the 8 quantum in the Q Sequence has been learned, and qı 0 otherwise. QH1 implies that the mean loss I. is simply a function of q. QH2 implies that when n = Σx quanta have been learned, have qı for l < n. Let I., denote the mean loss in this case. From QH3, we have that the 1tth quantum benefits prediction on a randomly chosen sample with probability
5(a + 1:- (c++) x (a++1)
1
Pk =
for a Zipf power law a 0, where ((s) Ek= k= Let us also assume that learning the hth quantum reduces average loss from by before it is learned to a after it is learned on the samples where it is utilized. ag and by are k-independent (a; a, by = b), then model that has learned the first n quanta will have expected loss
Ln =
Σ apk + k=
Σ k=n+1
bpk =
Σ apk +
Σ
(b - a)pk
k=1
k=n+1
b -
b -
k: (a+ dk + aC(a +
+
ς(a 1)
In other words, Los = a and (In - Loo) x n a is a power law.
In Appendix A, we provide analogous derivations for other assumptions for ag and by and find that range of assumptions produce curves that are exact or approximate power laws the latter include a small logarithmic correction.
In the derivation above, we assumed that all samples are what we will refer to as monogenic, meaning that prediction relies on at most single quantum, akin to how monogenic traits in biology (e.g. cystic fibrosis) depend on single gene. By assuming that all samples are monogenic, we can write the expected loss as a sum over quanta, weighted by the fraction of samples which rely on that quanta Pk. We further explore the idea of monogenic vs. polygenic samples in Section 4.2. So far we have seen how the Quantization Hypothesis can produce power law scaling function of the number of quanta learned n. We will now give one possible mechanism by which this can translate into power law scaling in parameters, data, etc.:
3

---

Parameter scaling: In networks of finite size, network capacity can bottleneck how many quanta are learned. we assume that all quanta require the same capacity of C' network parameters, then network with N parameters can learn roughly = N/C quanta. Therefore [(NN) Tox x n n=na R (N/C)-a x N-a, we so we get power law scaling in N with exponent aN = a.
Data scaling (multi-epoch); For data scaling, we assume that for each quantum, threshold of T examples utilizing quantum are needed in the training set for quantum
be learned". With D training samples, approximately Dps samples relying on quantum & will be present, and solving for Dp.. = 7 we get the last quantum to be learned will be n < (D/7)1/(at1) since Pk x k (a(11) Under this model, we get scaling in data samples L(D) L na (D/r))
D D-a/(a+t), and ap aa(a + 1). From our earlier result that aN a, would therefore predict that ap = aan/aa/// (aN + 1). We discuss whether this relationship holds empirically for data and parameter scaling exponents observed across variety of studies in Appendix F.
Data scaling (single-epoch): In multi-epoch training. the information contained in the training dataset can bottleneck which quanta are learned. However, the rate of convergence of SGD can also bottleneck performance. For single-epoch training, a greater number of training samples allows one to train for longer. In our model, the amount that each quantum reduces mean loss by follows a power law. If the magnitude of the gradients for learning these quanta also follow a power law, then the convergence time for each quanta may follow power law too. If the number of steps to learn quantum i is x 1/Pk. then if the first quantum requires T steps to be learned, quantum n will require T steps, and so n = (S/T) quanta can be learned in steps. This gives scaling in training steps L(S) - Los x n x (S/T) (S/T))a/(a+t) aa((+11) x S-a/(a+)), and so as = a/(a + 1). Under this model, multi-epoch and single-epoch data scaling exponents coincide: ap- ag.
3 Proof of concept: a toy dataset
In this section, will deseribe a toy dataset consisting of distinct subtasks which are power law distributed in frequency. We observe power law neural scaling in data and parameters on this task, and find that the mechanism of neural scaling coincides with our theory from Section 2. It is therefore possible for scaling laws to arise from the Quantization Model for data with the right structure. We leave study of whether natural datasets (e.g. natural modeling) possess such structure to Section 4.
3.1 The "multitask sparse parity" dataset
The toy task we will construct consists of many subtasks distinct types of inputs which each require corresponding distinct computations (quanta). For each subtask, we choose variant of the "sparse parity" problem, recently studied in [28]. The sparse parity prediction problem is simple: given bit string of length n, compute the parity (sum mod 2) of fixed subset of ie of those bits. We introduce an extension of this task, which we call "multitask sparse parity".. Beyond n and k. multitask sparse parity adds an additional parameter Nusss the number of subtasks (number of distinct versions of sparse parity) present in the dataset. To construct the task, we first choose Tlssk random subsets S; of indices from {1, 2, , n): S, (1, n and |S = k. where i = 1,2,.. 1,2..... .... Trakks Input bit strings are length Tlasks If control bit is active, then the parity is computed from the subset S, of the task bits. The control bits hhoo encode the task number: are zero. For the sample shown below, since control bit 2 is active, the answer is the parity of the task bits S2 {2, 7}, which is 0 for this input:
We call the first Resks bits the control bits and the last n bits the task bits.
given input, only control bit is set to a time the rest
Subtack ①
Subtask
Subtask
0100... 00
T...1110011
"control bits"
"task bits"
Thiis type of threshold has precedent, e.g. for the algorithmic tasks where "grokking" occurs [27].

---

0.8
2x 4"
L S-u/lu
Mcan best cross-ontropy (bits
0.6
0.8
Mean best cross roos-eebropy
Mean bagt CroSs-en entrroyyyyyy....
0.6
0.4
0.4
0.3
0.4
0.2
0.3
L D male
0.2
10a
10a Parameters (N)
10a
10
10
10
10a
10a
10
Steps (5)
Training samples (D)
41
Subtask: decr free
31
08
31
0.8
decr free
31
0.7
550T 159t
decr
550T 459
L507 15RL
21
21
21
Subtask
Subtask:
11
0.2
0.2
0.3
10*
10a Parameters (N)
10a
10 10s Steps (S)
104
10a
10c
Training samples (D)
Figure 2: Top: Neural networks exhibit power law scaling in loss w.r.t. parameters N, training time S, and training samples D (for multi-epoch training) when trained on the multitask sparse parity dataset. Here a 0.4 and plot lines & N~a, x S-a/(a+1), x D-a/(a+1), Bottom: neural scaling broken down by subtask. Scaling behavior on individual subtasks exhibits emergence, where subtasks suddenly learned above particular scale. Power law neural scaling of mean test loss averages over large number of qualitative changes in network performance (when broken down by subtask), with loss being driven to zero on an increasing number of subtasks which are power law distributed in frequency, a realization of the mechanism of neural scaling discussed in Section 2.
We impose a uniform distribution over the task bits. On the control bits, we impose a Zipfian distribution: the probability that a sample has control bit active (and therefore the parity must be computed from the subset S, of the task bits) is 3/4 (a+()))) where 7 > Tinda (a++1) This imposes a power law distribution over subtasks in data. Since answers are parities, this task can be treated as binary classification problem on the subset bit strings {0, 10,11"%a [1laatt where for each string all but one bit of the first Thasks bits are zero.
3.2 Power law scaling and emergence
We train ReLU MLPs with single hidden layer to solve this task with cross-entropy loss. The input dimension is Reass + n. We use the Adam optimizer with learning rate 10-3 To study scaling train networks of varying width by sampling with respect to the number of model parameters, batches online. Within an individual single-epoch training run, we can study scaling in steps S. To study scaling with respect to multi-epoch training dataset size D, for capacity to not be bottleneck, and for varying / we sample a training set of D samples and train for multiple epochs. recording model performance when mean test loss is lowest (carly-stopping).
network of sufficient width
Training dynamics on the multitask sparse parity problem are highly nontrivial - on each individual subtask, loss follows reverse-S curve, dropping after an initial plateau. This transition happens at different times for different subtasks, so the overall loss decreases smoothly. averaging over these transitions. See Appendix B for more discussion of training dynamics.
Figure shows scaling curves on the multitask sparse parity problem. For results shown, we used 100, & = 3, le4 to 5e6 and vary hidden-layer width from to 500 neurons. We train for 2e5 steps. In line with the theory from Section 2, we find that scale training data and parameters, networks learn more
Plasks = 500,
0.4, and batch size of 20000. We vary training dataset size from
5

---

Mean loss scaling
Distribution p(L) scaling
Integrand Lp(L) scaling
4.2
L x
0..083
10
19m
Expected loss integrang Lp(L)
Test cross-entroph (bits)
4.0
Probability Density P(L)
85m
0.3
85m
300m
3.8
8
300m
3.6
800m
1.26
6
1.2b
6.4b
2.5b
0.2
3.2
4
6.4b
3.0
0.1
2
2.8
2.6
10a
10a
0.01
0.1
1
10
5
15
Non-embedding parameters
Test cross coosspyy (bits)
Test cross-entropy (bits)
Scaling of mean test loss w.r.t. non-embedding parameters for the Pythia models [29]. The parameter scaling exponent ony is measured be = 0.083 from the first six points along the curve (the seventh model appears to break the trend). Center: the distribution p(L) over losses on individual samples for models of different size. Losses Rs 0 are by far the most common, and larger models achieve x 0 loss on an increasing fraction of samples. Right: the expected loss integrand Lp(L) for models of different sizes. Low-loss samples contribute minimal mass to the mean loss, which is instead dominated by samples with much higher loss of 5-10 bits (depending on scale).
Figure 3: Left:
and more quanta (reducing loss more and more subtasks), roughly in order of their frequency, and that this is what drives neural scaling. We see that scaling w.r.t, parameters is noisier than data scaling, possibly due model initialization having some influence which quanta learned (for data scaling experiments, we use the same seed and same model size for all runs). We also see that for scaling individual subtasks, there rough scale of data parameters below which networks do not learn the task, and above which they do. Smooth power law scaling therefore averages over a large number of emergent changes in model performance when properly decomposed by subtask, a proof of concept that the Quantization Model can be the mechanism of neural scaling for data with the right structure. See Appendix B for additional results and discussion on how the scaling exponents aN, as, op relate to the subtask distribution power law exponent a + empirically.
Decomposing LLM scaling laws
We now study how scaling curves for large language models decompose. For our experiments, we the Pythia model suite from Eleuther AI [29], set of decoder-only transformers of varying size trained on approximately 300 billion tokens of The Pile [30]. We evaluate several models in the suite (ranging from 19m 6.4b non-embedding parameters) on approximately 10 million tokens from the test set of The Pile. We record cross-entropy loss on every token, enabling us to study how loss on individual tokens, well how the distribution over losses, changes with model scale.
4.1 The distribution over per-token losses
In Figure 3, we show how the distribution over losses scales with model size. First, we find that for the first six models in the Pythia sequence, the mean loss scales as power law with exponent aN = 0.083, roughly in line with the parameter scaling exponent of 0.076 measured in [3]. The 6.4b model does not fit the scaling curve well, so we excluded its loss when measuring the scaling exponent. Next, we plot the probability distribution over per- por-token losses p(L). We find that losses close to zero are by far the most common, and that scaling increases the portion of approximately- losses. We also plot Lp(L), the probability density over losses weighted by loss. The mean loss the area under this curve. We see that despite approximately-zero most common, they do not contribute much mass to the mean loss. See Figure 11 for how these distributions change over training steps rather than across model size. note that neural scaling in the wild is much more complicated than for multitask sparse parity notably, the distribution over
tokens being by far

---

Monogenic sample
Polygenic sample
12
cross-entropy (bits)
Cross: entropy (bits)
6
3
10
10
10
109
Parameters (non- (non-embedding)
Parameters (non-embeddling)
accused Jagdish Tytler a Congress where Sheila kik took charge as party's Delhi Shic Akali Dal MILA Manjinder Singh Sir aa alleged that the Congress...
The big disappeintment this summer was that despite my plum trees fruiting abund antly. beyond expectations, the fruit was mostly spoiled by an inf estation of worms and several...
Figure 4: Per-sample scaling curves can have diverse behavior: Here we show extreme examples where scaling (of loss on predicting the highlighted token) abrupt versus smooth. If the Quantization Hypothesis describes language modeling, then samples with sharp scaling would be monogenic, displaying sharp emergence at a particular model scale when the relevant quantum is learned. Samples with gradual scaling would be polygenic, where many quanta, emerging at different scales, marginally improve the loss. We show additional examples in Figure 12.
losses is not bimodal. We leave detailed study of whether the statistics of neural scaling in LLMs are compatible with prior models of neural scaling to future work.
4.2 Monogenic versus polygenic scaling curves
In our introduction of the Quantization Hypothesis in Section 2 and our multitask sparse parity study in Section 3 we modeled network performance on individual samples as benefitting from a single quantum all samples belong to a single subtask, which either solved or not solved binary fashion. In our model and on multitask sparse parity, scaling curves on individual examples all exhibit emergence loss on individual examples undergoes a sharp transition at a particular scale of parameters or data. Do we observe this in large language models?
Inspecting large number of per-token (per-sample) scaling curves, observe variety of scaling behaviors. On some samples, loss drops at a particular scale. More typically though, loss improves multiple scales. If the Quantization Hypothesis true and the effect of scaling to simply add new quanta to the model, then for per-sample loss curves to show progress at multiple scales, those samples must benefit from multiple quanta additively. As first mentioned in Section we borrow terminology from genetics and refer to prediction problems for which the model's performance is determined by single quantum as monogenic (akin to when single gene determines a trait) and as polygenic when multiple quanta influence performance (in analogy to when multiple genes contribute trait). In multitask sparse parity, all prediction problems are monogenic. In natural language, we observe that model performance most tokens are polygenic, but we can find tokens for which loss drops as a single phase transition in scale. Polygenicity forms spectrum: the smoothness of the loss curve can vary substantially between examples, presumably with some prediction problems using few quanta and others using many. In Figure 4, we show extreme examples of both monogenic and polygenic samples.
most tokens improves at multiple scales, suggesting that
Note that our monogenic/polygeni taxonomy of model behaviors assumes that QHI and QH2 are true. However, it could be the case that there isn't 't an underlying discreteness to what is learned, or that scaling totally changes what networks learn, rather than simply adding additional quanta. Whether scaling truly has the effect we described will have to be investigated in future studies of the internals of neural networks. We also note that is possible that sharp transitions in the per-token loss curves could be due to noise- - if we had multiple runs with different random seeds for each model scale, we could better test whether the mean loss across seeds decreases smoothly or there genuine discreteness where gradual progress is impossible for apparently "monogenic" tokens.
7

---

5 The quanta of language modeling
We have conjectured that the internals and behavior of language models are decomposable into an enumerable set of modules and associated skills (quanta). What might these basic building blocks of LLMs be? In this section, we develop a preliminary method to discover quanta. In particular, we will attempt to cluster tokens in language corpus according to what knowledge to predict those tokens from their context. Our goal is to find coherent clusters of language model behavior that each reveal some distinct skill that the model has learned. Note that in clustering tokens to discover quanta, we are making the likely unrealistic assumption that these tokens are monogenic that there is only one quantum involved in predicting each token. Not also that these clusters of behavior will not give us mechanistic understanding of the quanta, but simply provide examples of LLM skills which could be studied further in future work.
skill LLMs use
We propose the use of gradients to cluster next-token prediction samples, where a "sample" consists of token and its context in some document. Given some model, will cluster two samples together the gradient of the model's 's loss on each sample w.r.t. the model's parameters is similar for the two samples. The intuition for using gradients is follows: if model uses the same internal module to generate its prediction on two samples, then the gradients for parameters within the module may be nonzero and similar for the two samples (and possibly Rs 0 for parameters in irrelevant modules). If a model uses different modules to generate its prediction on different samples, then the gradients may not overlap. We therefore use gradient similarity as a proxy for mechanistic similarity whether a model uses similar mechanisms/modulee to generate its prediction on distinct samples. While crude, we find that gradients contain enough information to allow us to automatically discover many coherent clusters of LLM behavior using the following algorithm:
Quanta Discovery from Gradients (QDG): We will use spectral clustering on gradients to find clusters of samples whose gradient has nonzero cosine similarity. Given a set of samples (z,, y1) and model fo, we compute gradients for each sample gi (oT.(f((z)), yi). We then normalize these gradients
> 0J so that 0; 0; = 1. Let A be a matrix whose rows are the normalized gradients: A., = 9t- If we are clustering d samples and our model has parameters, A has shape (d, n). We compute an affinity matrix C = AAT, matrix of shape (d, d) where Cjj- ف 9j. the cosine similarity between gradients Ii, My. From this, we compute an affinity matrix of the angular similarities Ö (which take values in [0, 1J) via Cj.- = 1 arccos( (Ci))((i/)))/) n. We then perform spectral clustering with C' to cluster samples.
QDG is expensive to compute for large models and for large numbers of samples. We therefore only apply it to the smallest model in the Pythia suite, which has 19m non-embedding parameters. We cluster 10000 tokens on which this model is confident and correct in its prediction, achieving less than 0.1 nats of cross-entropy. See Appendix C. for more detail.
We find that many, though not all, QDG clusters reveal some coherent model behavior. We show examples from clusters in Figure and Figure 13. These clusters were found with the spectral clustering hyperparameter n_ clusters = 400. While most clusters involve the prediction of the same token, manually inspecting these clusters we find that they usually involve predicting the same token for a coherent reason, rather than being based merely on having the same output. We also find clusters for more abstract prediction rules. For instance, the quantum shown the left column of Figure 1 is the skill of incrementing a numerical sequence, and the examples involve predicting a variety of different tokens representing numbers.
5.1 The natural distribution over language modeling quanta
In our model, some quanta are more fiequently used than others. If these frequencies follow a power law in accordance with the Quantization Hypothesis, then we may expect QDG cluster sizes to be governed by power law. The measured scaling exponent of any 0.083 from Figure 3 implies power law distribution over quanta with exponent -10883. Do the cluster sizes follow this?
Figure 5 shows rank-frequeen freeueeeeunc curves for clusters discovered with QDG for varying choices of n_ clusters. These curves sort the clusters according to their size and then plot size against cluster index (rank). We plot rank-frequency curves for many choices of n_ clusters since it is unclear a priori which n_ clusters to use. When we measure the slope of the rank-frequency curve, we measure it from the envelope formed by the many rank rank-frequency curves, a practice which
8

---

Similarity Matrix
rank-frequency of clusters
104
mueleen Leoeacn Bnma Juybbee
y x x Los
1000
yax 1.277 envelope
cluster frequency
10a
800
number of clusters
0.7
0.6
10a
600
400
10
100
5225
5450
10"
10a
10a
10a
token index
cluster rank
Figure 5: Left: angular similarity between model gradients for a variety of natural language samples. Samples are reordered according to their QDG cluster (with 400 clusters) reveal the block-diagonal structure of the similarity matrix. We visualize a small part of the overall similarity matrix this plot note that not all clusters of QDG clusters. We measure the slope of the envelope of the hank-freqeenyy curves from cluster rank 100-1000 to be s 11224, which steeper than the slope of -1.08 expected from the measured parameter-scaling exponent from Figure 3, though within the margin of error given the uncertainty of clustering methodology. See Appendix for discussion of the bias/uncertainty of method.
visibly distinct the ones shown. Right: rank-frequency plot
discuss Appendix F. Biases the clustering algorithm and inherent noise model gradients make clustering imperfect, and lead high uncertainty of our the measured power law exponent. From our analysis in Appendix E, we think that extracting the power law exponent over quanta utilization frequency by measuring the slope of the mank-frequence curve should have uncertainty of least 0.2. We also note that some rank- rank-frequency curves don' 't look like clean power law. In Appendix E, Figure 16 we find that we can get aimilar-looking curves in a toy model of this clustering process when the dimension and noise is high. Between ranks 100- 1000, we measure slope of ~= -1224, about off our expected slope of 11008 and so within the margin of error. We are encouraged that the size of our discovered clusters seem to decay rate (very roughly) compatible with observed neural scaling exponents, in line with our theory. However, less naive clustering schemes, operating on more samples with more clusters, could be useful to sharpen this measurement.
6 Related Work
Models of neural scaling: Several models of neural scaling laws have been proposed in prior work. Sharma and Kaplan [31] explain power law scaling w.r.t. model parameters using an argument from approximation theory, which relates neural scaling exponents to the dimension of the data manifold d. Michaud et al. [32] point out that effective dimension d could be generalized to the maximum arity of the target function's computation graph for sparse compositional problems. Bahri et al. [33] generalized the model of Sharma and Kaplan [31] to sealing w.r.t. dataset size, additionally relating scaling exponents to the power law spectrum of certain kernels. Maloney et al. [34] develop an exactly solvable muudom-fcatur model of scaling, from which they derive a joint parameter-dam scaling law. Bordelon et al. [35] develop model of data scaling for kernels, decomposing the generalization error into a sum over eigenmodes, whereas we decompose error into sum over quanta. Arguably the closest prior work to ours is Hutter [36], who develops model of data scaling wherein discrete set of "features" must be learned. In this model, feature is learned it occurs at least once in the Zipfian distributed, this produces power law scaling in expectation but
training set. If the features with high variance. In our model, using a data threshold v > 1 lowers the variance in the scaling curve, and also considered scaling w.r.t. parameters and applied the model real networks.
Understanding emergent abilities: Wei al. [8] and Srivastava et al. [37] document examples of emergent abilitics in large language models, though Schaeffer et al. [38] suggest that these examples

---

are an artifact of the metric used to evaluate performance. Arora and Goyal [39] develop framework for the emergence of "skills", where predicting text requires combining multiple different skills from an underlying set of language skills.
The topic of phase transitions in machine learning is not new [40], but our work was strongly influenced by the recent work of Olsson et al. [25] who observe a phase change from the formation of induction heads and especially Nanda et al. [13] who conjecture that phase changes may be ubiquitous. Simon et al. [41] also exhibit a task where learning proceeds as a series of discrete steps. Chen et al. [42] develop a framework for understanding LLM "skills" in hierarchy and for choosing data to more efficiently learn desired skills. Chan et al. [43] study how Zipfian data distribution influences in-context learning.
Miscellaneous:
7 Discussion
Summary: The Quantization Hypothesis posits that for some types of prediction problems, models must learn discrete (quantized) set of modules/nnowledge/kkll (quanta). When data is distributed these quanta follow a power law, then power law neural
in such a way that the "use frequencies" scaling can arise as models learn more and more quanta, with smooth scaling curves averaging over many small cases of emergence. We presented a toy dataset where neural scaling exhibits these properties. We then documented how language model scaling curves decompose, beyond simply how the mean loss scales. Lastly, we developed a method to discover quanta from the internal structure of trained models, from which we were able to enumerate a large number of skills of small language model. The frequencies at which the quanta we discover are used for prediction in natural text seem to roughly track the power law our theory would predict, though this measurement is quite imprecise.
Limitations: While the Quantization Hypothesis appears to hold for our toy datasets, much work remains in investigating to what extent it holds for natural tasks like language modeling. Probably our riskiest assumption was that there is an underlying discreteness to everything that models learn. Gradual scaling seems typical in LLMs [38], and it could be more parsimonious to model neural scaling as an underlying smooth process rather than to assume that most tasks are highly polygenic with underlying discrete quanta. Note also that in model of scaling w.r.t. parameters N, having more parameters merely increases the capacity of the network. In practice however, larger networks more efficient learners [7], and one trade off between parameters and data, whereas in model parameters and data independently bottleneck the number of quanta that can be learned. Additionally, modeled the quanta being independent, where learning order is given just by the use frequencies, but it could make more sense to think of the quanta as living in a hierarchical dependency graph. Lastly, our QDG method is neither very principled nor scalable, and much better methods could likely be developed to discover quanta and study their statistics for larger models and across more samples.
Implications for emergence and forecasting: Srivastava et al. [37] find that on some tasks, neu- ral scaling has high "lincarity"", with gradual improvements to scale, with other tasks displaying "breakthroughness"" where performance improves sharply at some scale. In our model, high linearly would result from a task's relevant quanta being widely spread along the Q Sequence, and high breakthroughness would result from a task being monogenic or from the relevant quanta being close together in the Q Sequence. Our model also suggests that future capabilities could be forecasted if one could estimate the frequency at which that skill would benefit prediction in the training corpus.
Implications for mechanistic interpretability: If the Quantization Hypothesis is correct, then understanding a network reduces to enumerating its quanta. Having done this, the quanta could perhaps then be translated into more interpretable format (something like code), studied in this format, and eventually executed in this format, rather than via the operation of the network.
Outlook:
Lastly, our decomposition of networks into quanta is reminiscent of Minsky's 's Society of Mind [44] perspective that minds are decomposable into individually mindless "agents". If this decomposition is indeed possible, then the quanta (agents) become natural objects of study within networks. This mexoscale understanding of networks, in terms of the internal modules which collectively constitute their performance, could perhaps act like statistical physics for deep learning, allowing us to bridge our microscale understanding of low-level training dynamics and our macroscale understanding of model performance.
10

---

Acknowledgments and Disclosure of Funding
We thank Tamay Besiroglu, Neel Nanda, Tony Wang, David Bau, Ben Edelman, Brian Cheung. Wes Gurnee, Stephen Casper, Peter Hase, Davis Brown, Eleni Shor, Max Nadeau, and Xander Davies for helpful conversations and feedback. We thank Lauro Langosco for helping with code to visualize samples from The Pile. This work was supported by the Foundational Questions Institute, Rothberg Family Fund for Cognitive Science, the NSF Graduate Research Fellowship (Grant No. 2141064), and IAIFI through NSF grant PHY-2019786.
References
[1] Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Patwary, Mostofa Ali, Yang Yang, and Yanqi Zhou. "Deep learning scaling is predictable, empirically". In: arXiv preprint arXiv:1712. 10:::1112000099 (2017).
[2] Jonathan S Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit. "A constructive prediction of the generalization error across scales". In: arXiv preprint arXiv: 11199999.227 (2019).
[3] Jared Kaplan. Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. "Scaling laws for neural language models". In: arXiv preprint arXiv:2001. 003661 (2020).
[4] Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Hee- woo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. "Scaling laws for autoregressive generative modeling". In: arXiv preprint FrxXiv:2110.14000 (2020).
[5] Mitchell A Gordon, Kevin Duh, and Jared Kaplan. "Data and parameter scaling laws for neural machine translation". In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021, pp. 5915-5922.
[6] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. "Scaling vision trans- formers". In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 12104-12113.
[7] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. "Training compute-optimal large language models". In: arXiv preprint arXiv:2203. 155.6 (2022).
[8] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi. Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. "Emergent Abilities of Large Lan- guage Models". In: Transactions on Machine Learning Research (2022). Survey Certification. ISSN: 2835-8856. URI.: https: /tttps://openee openreview net/forumiidgyye sirrum/?ddyy2kUU52dd...
[9] Jacob Steinhardt. Future ML Systems Will Be Qualitatively Different. Accessed: 2023-10-26. Jan. 2022. URL: htt.ttpppee//oodddddeeeeeee www.t........................... gualitatively-ddiffeeenliii
[10] Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, et al. "Predictability and surprise in large generative models". In: 2022 ACM Conference on Fairness, Accountability, and Transparency. 2022, pp. 1747-1764.
[11] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. "Zoom In: An Introduction to Circuits". In: Distill (2020). https::/diise htt///issii1...pp/22000/ pub/2020/ 00004.000444044 001.
circuits/zoom- in. DOI: 10. 23995//istsiii
[12] Nick Cammarata, Gabriel Goh. Shan Carter, Ludwig Schubert, Michael Petrov, and Chris Olah. "Curve detectors"'. In: Distill 5,6 (2020), c00024-003.
[13] Neel Nanda, Lawrence Chan, Tom Liberum. Jess Smith, and Jacob Steinhardt, "Progress measures for grokking via mechanistic interpretabilityy. In: arXiv preprint arXiv:2301. 105217 (2023).
[14] Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. "Emergent world representations: Exploring a sequence model trained on synthetic task". In: arXiv preprint arXiv:2210.13382 (2022).

---

[15] Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. "Interpretability in the Wild: Circuit for Indirect Object Identification in GPT-2 small". In: arXiv preprint arXiv:221 1.05933 (2022).
[16] Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Doddk Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. "A Mathematical Framework for Transformer Circuits". In: Transformer Circuits Thread (2021). https: https:://ttaafffitee orner circiits......... http:/irtttttrarp///eeeeeeeee
html..
[17] Tom Lieberum, Matthew Rahtz, János Kramár, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. "Does circuit analysis interpretability scale? evidence from multiple choice capabili- ties in chinchilla". In: arXiv preprint arXiv:2307.094 (2023).
[18] Stephen Casper, Shlomi Hod, Daniel Filan, Cody Wild, Andrew Critch, and Stuart Russell. "Graphical clusterability and local specialization in deep neural networks". In: ICLR 2022 Work- shop PAIR ( [extasciiiiirum } 2Struct: Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data. 2022.
[19] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. "Towards Monosemanticity: Decomposing Language Mod- els With Dictionary Learning", " In: Transformer Circuits Thread (2023), https://trassormerr- htmm.
[20] Jonathan Frankle and Michael Carbin. "The lottery ticket hypothesis: Finding sparse, trainable neural networks". In: arXiv preprint arXiv: / 000.0003333333 (2018).
[21] Deniz Bayazit. Negar Foroutan, Zeming Chen, Gail Weiss, and Antoine Bosselut. "Discov- ering Knowledge-Criin Critical Subnetworks Pretrained Language Models". In: arXiv preprint arXiv:23 03003044044444 (2023).
[22] Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John Hopcroft. Convergent Learning: Do different neural networks learn the same representations? ? 12Xiv:15110075:: [cs]. Feb. 2016. URL: http:/ aarxivvvvvvv.aa///aaa/aaaaaaaa./ org/ https/151..070
(visited on 08/19/2022)
[23] Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth. KKiv:2010.1532. [cs]. Apr. 2021. URL: http: / /ttt:///rxivv org/ abt///a0/..0..////////////////// 15327.
[24] Amil Dravid, Yossi Gandelsman, Alexei A. Efros, and Assaf Shocher. Rosetta Neurons: Mining the Common Units in a Model Zoo ar..a:23vv00660 09346 [cs]. June 2023. DO1: 10.
09346. URL: http::/ www.ariii.g...oooooooooooa www.:ogg///...ee.eooooooooees /a/////s///0/ 09346.
[25] Catherine Olsson, Nelson Elhage. Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Haffield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jack- son Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. "In-context Learning and Induction Heads", In: Transformer Circuits Thread (2022). https: http///aaaaffooeeereee///e circuits, pubb/022/in. context-learning and-
index.idxxx/// html.
[26] Gwern Branwen. The scaling hypothesis. 2021. URL: https: // gwern net / scaling bypothesis.
[27] Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. "Grokking: Generalization beyond overfitting on small algorithmic datasets". In: arXiv preprint arXiv:2201.000 002177 (2022).
[28) Boaz. Barak, Benjamin I. Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang. "Hidden progress in deep learning: Sgd learns parities near the computational limit". In: arXiv preprint arXiv:2207 0.7999 (2022).
[29] Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O' O'Brin,, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. "Pythia: A suite for analyzing large language models across training and scaling". In: arXiv preprint 2rXiv:2304.01333 (2023).
12

---

[30] Leo Gao, Stella Biderman. Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, al. "The pile: An 800gb dataset of diverse text for language modeling". . In: arXiv preprint arXiv: M0Ni::2010000022 101.00027 (2020).
[31] Utkarsh Sharma and Jared Kaplan. "Scaling Laws from the Data Manifold Dimension". In: Journal of Machine Learning Research 23.9 (2022). pp. 1-34. URL: http: / jmmrr org/
papers/v23 22//22211121 1111. .ttml.
[32] Eric J. Michaud, Ziming Liu, and Max Tegmark. "Precision Machine Learning.. In: Entropy 25.1 (2023). ISSN: 1099-4300. DOI: . $390/e250000 URL: https: httpp:///www.wdee com/
1099-4300/22 4090-4300/2/111 2571/
[33] Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jachoon Lee, and Utkarsh Sharma. "Explaining (2021).
neural scaling laws". In: arXiv preprint
[34] Alexander Maloney, Daniel A Roberts, and James Sully. "A Solvable Model of Neural Scaling
Laws". In: arXiv preprini arXiv:2210. 16859 (2022).
[35] Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan. "Spectrum dependent learning curves in kernel regression and wide neural networks". In: International Conference on Machine Learning. PMLR. 2020, pp. 1024-1034.
[36]
Marcus Hutter. "Learning curve theory". In: arXiv preprint arXiv:2102.04074 (2021).
[37]
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga- Alonso, et al. "Beyond the imitation game: Quantifying and extrapolating the capabilities of language models". In: arXiv preprint arXiv: 04615 (2022).
[38] Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. "Are Emergent Abilities of Large Language Models a Mirage?" In: arXiv preprint vxXiv2234.15500 (2023).
[39] Sanjeev Arora and Anirudh Goyal. "A theory for emergence of complex skills in language models". In: arXiv preprint arXiv:2307. (tt.:23377.1333 15936 (2023).
[40] Lorenza Saitta, Attilio Giordana, and Antoine Cornuejols. Phase transitions in machine learning. Cambridge University Press, 2011.
[41] James В Simon, Maksis Knutins, Liu Ziyin, Daniel Geisz, Abraham J Fetterman, and Joshua Albrecht. "On the stepwise nature of self-supervised learning". In: arXiv preprint arXiv:2303. /5438 (2023).
[42] Mayee F Chen, Nicholas Roberts, Kush Bhatia, Jue Wang, Ce Zhang, Frederic Sala, and Christopher Ré. "Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models". In: arXiv preprint arXiv:2307.144: (2023).
[43] Stephanie Chan, Adam Santoro, Andrew Lampinen, Jane Wang, Aaditya Singh, Pierre Richemond, James McClelland, and Felix Hill. "Data distributional properties drive emergent in-context learning in transformers". In: Advances in Neural Information Processing Systems
35 (2022), pp. 18878-18891
[44]
Marvin Minsky. Society of mind. Simon and Schuster, 1988.
[45] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. "Scikit-learn: Machine Learning in Python". In: Journal of Machine Learning Research 12 (2011), pp. 2825-2830.
[46] Ronen Eldan and Yuanzhi Li. "TinyStories, How Small Can Language Models Be and Still Speak Coherent English?" In: arXiv preprint arXiv:2305.07753 (2023).
[47] Pablo Villalobos. Scaling Laws Literature Review. :/tttp::://pphhti laws were--iteratuur--reiiee Accessed: 2023-01-31 2023.
[48] Jasha Droppo and Oguz Elibol. "Scaling laws for acoustic models". In: arXiv preprint
arXiv:2 09488 (2021).
[49] Newsha Ardalani, Carole-Jean Wu, Zeliang Chen, Bhargav Bhushanam, and Adnan Aziz. "Un- derstanding Scaling Laws for Recommendation Models", In: arXiv preprint 07X44 (2022).
13

---

Appendix
A More general scaling laws
one learns the first n quanta, reducing the loss from by to ag (1 $ l S n)) while the loss remains by for & n,, the expected loss is given by:
Ln
a&Pk
byPk-
(3)
1=
In the main text, we used ag = a and by for our model. However, one can imagine a variety of other choices for o₁ and by-
Case by -log ps and ag = 0. where ph k-(at)))(((0 + 1). This baseline for by the error of model which outputs the token frequencies, independent of the context (assuming that quanta involve the prediction particular token). The expected loss is given by:
1 a alogš(a n
a+1 ac(a + n "log n, (4)
I O.
(-log pk) Pk
a²ç(a +1)
R=1
R==+1
which contains power law term n- plus log term "log For very large n, the log term can be ignored, so L is still approximately a power law of n with exponent -a, shown in Figure 6.
100
10-1
0.05 (includig the log factor)
law)
10-4
8 (incussing the log factor)
(power law)
(rrluding the log factor)
0.4 (power lawó
10:4
(Inclusing factor)
107
10a
10'
107
log
Figure 6: Comparing different scaling laws. Setting as = O, we compare by = log pa (solid lines) and (dashed lines) for different alphas. Although the by llog pk case would cause extra loss term n logn in additional to the power law term n a, the loss becomes a power law asymptotically when n becomes large.
Case bb llo pa and aş llo (Cps) (C' 1), where pa = k -aa+)) (att))/(((a+ 1). The expected loss is given by:
logC aC(a +
n- - + a + alogç(a + 1) a² + 1)
Ln (-log (Cp+))
Σ
(-log Pk
km
(5)
which is power law n plus a constant.
B Additional results on multitask sparse parity
Training dynamics:
When loss is broken down by subtask on multitask sparse parity, learning curves consist of many reverse-S shaped curves, and mean loss decreases smoothly as an average over these curves. In Figure 7, we show loss versus time for each subtask for training runs in both the single-epoch and multi-epoch regimes. Figure we show how convergence time for each subtask relates to the frequency of that subtask.
14

---

Single-epoce training on "infinite" data
Cross-entropy Test Loss (bits)
500
1.0
400
Subtask
300
0.5
200
100
loss
0.0
102
10a
104
10a
Optimization Steps
Multi-epoch training 5m samples
Cross-entrops Test Loss (bits)
500
400
Subtask
300
200
100
102
10a
104
10s
Optimization Steps
Figure 7: Training dynamics on the multitask sparse parity dataset consist of many "phase transitions" when decomposed by subtask the loss curve for each subtask drops following an initial plateau of no apparent progress, in line with [28]. The mean loss decreases smoothly, averaging over these phase transitions in the model's performance on subtasks. We show curves for single-epoch training (top) and multi-epoch training on 5 million samples (bottom). The dashed red line indicates the early stopping point where mean test loss minimized. For these runs, a = 0.4.
10a
1.0
Steps to convergence
0.8
0.6
Mean loss
0.5
104
0.4
0.3
103
0.2
fitted 1ope==0.81
loss
slope-.1
fitted slope--0.45
0.1
10-3
10-2
10 -1
102
10a
104
105
Subtask frequency
Optimization steps
Figure 8: Convergence time for each subtask versus the frequency of that subtask. We that convergence time S₃ on subtask is S₁ x Py 0.81 rather than S₃ x Py we had expected. This leads to steeper scaling w.r.t. S than expected from theory. For these experiments, we used 0.4, and so we would have predicted as z 0.29 but instead we get as z 0.45. We consider the model to have converged on a subtask once gets mean test loss less than 0.1 bits on that subtask.
Scaling for varying a: In Figure 10 we show scaling curves on multitask sparse parity in N, N, D for variety of quanta distribution parameters While all scaling curves appear to be power laws, relationship between ax, a a'p and a not precisely as predicted by theory:
1. Parameter scaling: We observe that the relationship between and a deviates bit from the prediction a = a, with a, for small a and ax a large a. Perhaps model size does not influence learning just by changing capacity, but also by affecting optimization.
15

---

(No. subtasks learned)
1.80
1.75
1.0
empirical exponent
1.70
scaling exponent for
theory; y= 1/a++
10a
1.50
1.55
1.50
0.8
1.45
1.35
1.25
0.6
1.20
100
1.15
1.10
10*
10
10
0.0
0.2
0.4
0.6
0.8
Training samples (D)
quanta distribution
Figure 9: Number of subtasks learned (n), including subtasks learned after early-stopping would have terminated the training run, versus training samples for variety of a. We see that the relation & ()/(++1) approximately holds, in line with theory. Deviation from theory for the scaling exponent of loss L w.r.t. D therefore likely originates from our failure to regularize network training, leading to early-stopping ending training before some subtasks be learned.
2. Step scaling: We observe that 0gg is consistently higher than the theoretical prediction a/(a + 1). In Figure we saw that the number of steps to convergence for each subtask did not precisely follow S& P but was closer Sş Pk. 0081 This means that many subtasks converge faster than we would expect, producing a steeper scaling curve.
3. Data scalaing: We observe that ap is substantially higher than the theoretical prediction a/(a + 1) for small We think this may be related the fact that early-stopping cuts off training before all subtasks are learned as observed in Figure In Figure 9, we show how the number of subtasks learned n, when include subtasks learned after early- stoppig, seems to be in line with theory: 1)/(a+1),
Better understanding the precise nature of power law scaling multitask sparse parity is an interesting avenue for future work.
16

---

1.0
1.8
Mean test cross-entrop (bit
0.8
empirical exponent theory
0.7
1.7
quanta distribution exponent:
0.5
0.4
0.6
0.3
1.5
AN
0.2
0.4
1.4
1.3
0.1
0.2
1.1
0.0
10
10
0.0
0.2
0.6
0.8
Number of parameters (M)
quanta distribution
1.8
Mean test cross-entropy (bits)
1.0
0.8
empirical exponent
0.7
1.7
quanta distribbtuuoooooo exponent:
theory
0.6
0.4
0.3
1.5
as
0.2
0.4
1.4
1.3
0.1
0.2
1.1
0.0
103
10 Steps (S)
10
0.0
0.2
0.4
0.6
0.8
quanta distribution
1.80
Mean test cross-entropy (bits)
1.0
1.75
0.8
empirical exponent theory
1.70 8
0.7
0.6
0.5
1.55
1.50
0.3
1.45
zion
0.4
1.40
1.35
0.2
0.2
1.20
1.15
0.1
1.10
0.0
10
10
10c
0.0
0.2
0.4
0.6
0.8
Training data (D)
quanta distribution
Figure 10: Scaling in parameters (N), single-epoch training time (S), and multi-epoch training samples (D) for varying quanta power law distribution parameter multitask sparse parity. We notice that scaling curves steps S are typically steeper than the os = a/(a + 1) predicted from theory, and that for low the scaling curves in / also deviate from theory substantially.
C Additional results on language models
In Figure 11 show how the distribution losses changes across time during training run, rather than across model scales like in Figure 3.
In Figure 13 we show additional examples from clusters discovered with QDG.

---

Mean loss dynamics
p(L) dynamics (1.3b)
L-p(L) dynamics (1.3b)
0.6
7.0
15m
20
Mean loss integrand p()))
Test cross- cross-entropy (bits)
6.5
1x2-500
123
Density p(L)
0.5
6.0
350m
5.5
1x5 s0>
850m
15
0.4
10
5.0
1w5-404
(5) 5d915
4.5
Frobabilg D
10
0.3
4.0
0.2
3.5
0.1
3.0
104
10a
103
10
10
10 Test cross-entropy (bits)
0.01
0.1
1
5
10
15
Steps (S)
Test cross-entropy L (bits)
Training curves (scaling w.r.t. steps S) of mean test loss for Pythia models. We measure exponents ag between 0.037 and 0.06. Center: the distribution p(L) over time. Over time, models achieve x 0 loss on an increasing fraction of tokens, similar to scaling in model size. Right: The distribution L p()) over time.
Figure 11: Left:
C.1 Details of application of QDG to LLMs
When applying QDG to language models, we use gradients within self-attention and MLP layers, do not include embed, unembed, layer norm gradients when we flatten and concatenate gradients into a vector g.* We choose samples (I; ys) for which our 19m-parameter model achieves cross- entropy loss less than 0.1 nats. We filter based on this criteria since (1) we cannot cluster samples based on model mechanism if the model does not have such a mechanism for performing prediction correctly those samples and (2) intuition that samples with particularly low loss more likely to monogenic. We further exclude samples which can be solved via induction on the context since such samples are quite common (possibly interfering with task of finding diverse quanta) and since early experiments indicated that QDG had trouble clustering such samples together. We choose 10000 such samples perform clustering from the test set of The Pile. After computing the affinity matrix Ć, we the spectral clustering implementation from seikit-learn [45] with labels assigned via k-means.
D Quanta discovery on TinyStories
We also apply QDG TinyStories-33M. language model trained on the TinyStories dataset [46]. We consider only tokens on which TinyStories-33M achieves a loss less than bit of cross- cross-entropy. We apply QDG to 10000 such samples, clustering their gradients with spectral clustering with n_ clusters = 400. We show some samples from the resulting clusters in Figure 14. Many of these clusters reflect some simple recurring pattern
TinyStories dataset, like predicting time" after "Once upon a"", which many documents in the dataset start with. Some other clusters are more interesting however, like Cluster 11, which seems to involve predicting the correct noun a sentence where that noun referred to earlier in the sentence in previous sentences.
"We exclude gradients for embed and unembed parameters because they are high dimensional and also because they contain information more about the input and output rather than the computations the model performs internally. We exclude layer gradients because they appeared to contain less information about clusters in toy experiments.
*We filter (copying) induction problems by excluding samples where the token which is to be predicted is the last token in trigram which occurred earlier in the context. This is not very comprehensive filtering scheme.
18

---

Monogenic samples
Polygenic samples
9
Cross-entropy ross-entropy (bits)
Cross entropy (bits)
6
6
3
3
0
10
10
10
10
Parameters (non-embedding)
Parameters (non-embeddling
"The law of unintended consequences and the history, previous military interventions in the region is not a recipe for political and cconomic stability." said Neil Mac Kinnon, global macro
In general the lesions thoraco-cervvial level were difficult to detect, because the appearance rate of SSEP peaks are reduced over the thoraco -cervical spine even in normal controls In cases with....
strategist at V TB Capital
12
Cross-entropy (bits)
Cross entropy (bits)
9
6
10a
10
10
10a
Parameters (non-embedding)
Parameters (non-embeddding)
Opinion filed March 25, 1988.An
airline by revenue, dropped $2.15, or 7.2 percent, to $27.71 and Delta Air Lines lost S1. 16, or 5.7 percent, to $19.11.An \n
W.Y. Chalfant, of Branine, Chalfant & Hill. of Hutchinson, argued the cause and was on the brief for appellant. Hesston State Bank. An Kenneth C. Jones, of Watson, Ess. Marshall Englas
Stone said oil prices could start weighing on consumer spending down the road
Figure 12: Additional LLM scaling curves on individual samples which exhibit sharp vs smooth improvement. If the Quantization Hypothesis is true for language modeling, then we would interpret samples with sharp drops as "monogenic" and samples with gradual progress as "polygenic".
E The difficulty of estimating the power law exponent from clusters
In Section 5.1, when we looked at the distribution over elements in each cluster, we did not perfectly recover Zipf distribution with exponent = 1.08 that we expected from our theory. In this section, we describe the difficulty of accurately estimating such an exponent with our method.
E.1 QDG on multitask sparse parity
As a first experiment, we performed QDG on multitask sparse parity, where there is a known, artificaall-immpsedd power law distribution over subtasks. We train a width-500 single-hidden-layer ReLU MLP on multitask sparse parity with a = 0.4 and with n 100, k = 3, and Tlaaks = 500. We then took 10000 samples which the network achieves 0 loss on (sampled from the Zipf distribution over subtasks with exponent 1.4). We compute gradients of cross- cross-entropy loss w.r.t. all model parameters for these samples, and then perform QDG just like for LLMs. We show results in Figure 15. We plot the full similarity matrix where samples are ordered according to their priori known subtask, rather than their cluster from QDG, and see clear pattern where elements from same subtask have on average higher angular similarity than elements between subtasks. However, from the rank- ank-frequency plot of the clusters, we not recover slope of -1.4, but rather lower slope of e -1.1. This shows that even when there is an exact decompasition of inputs into subtasks with
19

---

Cluster 146: comma after day of month
Cluster 269: "s" after starting year of decade
Sam Willardha
Romford Ice Arenaln
Samael Steven Willird (been September d]
Romford lor Arema was an ice rink located in Rom ford the London Borough of Haaeriigg England The venue was built in the 1980g
215 U.S. 437 (19407 MECHANICAL APPLIANCE COMPANY'a
Although novel comtinee to be the dominant medium the crime my stery descctiive narrative, short stoties by these contemporary authors may be found in numerous unthologies of the geare published during the 1990g
Vile
CA
Armed
Shen SW26010 chips, surged to the top of the 5OP500 list; Sanway TribuLight It nearly three times as fast us lianhe 2. being benchmarked in Lingack as being abis to perfocm 93 quadrillion calculations each second (93 petaFLOPS) To achievement in contest, moders desktop PC's already more powertul than the top ranked supercomputer: from the curyy 1950g
supercomputer
Supreme Court of United States
Argued December 3. 1909, 'n
Decided January 3)
Predecick W. Keator'a la Fredecick W. Keator (Cetober 22, 1855- Junusry 31]
Early activisn''n
He began lifetime involvement with covolutionary politics the lase 1930g
United States Patent No. 0.073.124 (isoued June 6, 2000) (' the 124 pasent": ". Microsofft in tum asserted counterctaims against NCI for infringmment three of patents United Simes Patent Nos, 822.525 5 999.9 5,794, (066 Only tonss the 2124 patent uc presently before the Court: interpectation of claums in Microsoft's patconi will be interpessed in separate Markmaan hearing to be held on November 15]
Yestenday; visitar Lavgg Davaddoee commensed that he was searching for songs played on the local forecust back in late '80g
In 1954, the couple published Living the Good Life which inspiced many young. educated Americans to creane simpler, rural licestyles and the back- to--he-land land mnovement of 1968
Cluster 278: colon after CSS property
Cluster 292: protocol separator in URLs
rickshuw praph stetail (Te
A'n
poiiter-eventt none
position; absolute:
N'n
topl (1,30
For questions plense refer to]
z-indox: 2
background: rghaic, 0, O. 13ol
botto:: (1.4
widdig
When cun the below codo, I am getting an error; Mougo Eiror) 'The dollar (5) prefixcd field 'ppush in 'Spush" is not valid for sterago) 'ni put logether based
importt 1, /assetss//sssssspii
docs; hrspell
app- hender (i
Grither Martin A. Views, of the Nationa Zaelogical Park in Washingtori DC, showing Exhitir, 1919, Retrieved from the Digital Public Library of America, hap
min- hwigtt 100
display]
But os citizens, our responsibillit is to look beyond the aneedione, We journalists try make sune that if free Gills the forest, it won't don't forest.
See http:/jelillle.ccommm /mpp://flldllleee/mmmmmm met/////// WFCZZ///
body margin: 0,3n puldingj
notiedd Still per to by the IIll all be loat
Rex Smith cdlttrr of the Times Unionn (Shrr your thoughts at hetlll
Figure 13: Additional examples of clusters of inputs discovered by QDG, Like in Figure 1, we used 10000 samples and n_ clusterss of 400.
known Zipf distribution over these subtasks, that we do not perfectly recover this Zipf distribution from QDG.
E.2 A toy model of QDG uncertainty and bias
A toy model:
To understand the bias of spectral clustering, develop the following toy model. We assume the dataset has N = 1000 subtasks, each subtask containing n, = 1 [1 < i N) tokens (A= 1000). We use a Gaussian distribution A ((m;, o'Iaxa) to model gradients within a subtask é, where d the embedding dimension, o is the noise level, and m, is the Gaussian mean. m, itself is drawn from the standard Gaussian distribution m, ~ (O0, Laxa). We define the similarity between
two vectors x, y to be sim 1 + | x We compute pairwise similarity between all ΣN n. tokens, and input the similarity matrix to the spectral clustering algorithm. We also need to specify the number of clusters k.
We have two hyperparameters in the toy model, the embedding dimension d and the noise level a. We need to determine them such that this toy model can decently imitate LLM results (Figure 5). We fix = 1, sweeping d = (30, 100, 10003. a {0, 0.5, 2.0}. and k {100, 200, 500). As shown in Figure 16, the high- dithnsimmniionn (d = 1000) large-noise (a = 2.0) scheme seem to best agree with the LLM results, since the k 200 curve can reproduce the sag and the cliff present in LLM curves.
Estimating a from the frequency curve is hard, in fact, the slope depends on l and the region used to estimate it. However, we observe that different k curves form a clear envelope, whose slope is
20

---

Example "quanta" for the TinyStories dataset
Cluster 11: predicting the currect noun
Cluster 31: "time" after "Once upon a"
Lill asked her mom. "Cin I tonch the senflowerr" mom replied) 'No,
Once upon of Eime
Tilly: The sunilower is not for lonching. It's for looking ae") (n
Once upunt timme
Lily but understond smn
day, Laly her past
Once upon oy Eime
Once upor time, there was tigg hous with door: The dooe browni
Once lilie
move when people opened day, bay came house and saw the impressive door. wanted to upen and see what was inside,, Si he moved the doot
Once upun af time
Once upunt time
had found wagon and put in sack keep safe; Tim was happy have his wagon back and thanked the man Le put his wagon
they yot park..
man with sack.
Once opon time
Once tiime
Lily and went
park with Mom. 'Tbey saw bug pond with many ducks and sw ans Lily liked the swans, They were white and graceful, Shhe wanted to feed them some broad. in
give some
Lily and Den nod, They promisse to be curefull, They zskk mom to read the letter them, Mom smiles. She reacs the letter. It is from grandma She says she loves them lot. are happy. They send kisses und hugs back to grandma, They thank, mom for the letier
them
bugs. Lily and
Cluster 75: comma after temporal phrase
Cluster 77: beginning of quote
One day. Jack wanted to show off his cool sunplases. Hi spotsed nice patch grass the paric, and he decided to sit down and enjor the suri just
Jack showed het the big blued ticket and said) "This is my ticket, L'mi going to it down. "M The girl asked. "Whore will you dowa? "lai
He pot his sunglasses
special In
answered.
A fowl momeents lates]
When she found her mom she said, 'Mom, have news!" Her mom said," "I something up and make
Jack happily agreed and they started the gamel first, hit tricky for them get the aquash the other. after tries,, was a poo. Hi laughed and cheered as he ran back and dorth to get the squisii kn
What June"!
tun!"
Her moms said,
Momimy Jack played until the sun
pai dows. Theal
twin jumped back in surprise. Shee never heard a dlower talk before? She asked the flower, "Who are you "I
Hob loves to gai adventures, He went for walk along the beach,, looking for exciting things,
"My name Pinky," said the flower Jai
suddens
The twin was now even more smpriisdd Shhe acked
Linda was a limlo girl who loved wandering around in nature: She was eager to explore and find what she could. One day, inda uas wandering around the woods when scoopie the flwwe up and inspected more closely..
He askid Iill, "Whlat's in this jar Jill smiled and said, ' "RD' supart Woold you like
yellow flower. She excited that
in
over to Soon]
Inck shook his head and soid, "Noi thank) youl, Edoe'") think T. should) cut sugar My momi won't allow at"
Next, her momi told Emma wipe floor Emmai grabbed cloth wipel the flone When she was finished, was clean new pasuy Ju
Jill nadded said,
Nearhy, her mom was watching and called out, "Lacyy come here! What's you have there?"
Finally]
Tuicy proudly held up the hoopi and annouced.)
Figure 14: Examples of clusters within the TinyStories dataset, discovered by QDG on the TinyStoriie-33M. model. Here we just show samples from four out of 400 n_ clusters
robust in reasonably wide region. The envelope slope seems to indicate We fix d = 1000 and 2.0, sweeping a {0.8. 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5). For each a, we estimate the slope of the envelope. Although there is clear correlation between the estimated envelope and a, if we use the envelope slope to estimate a, the error on the order of 0.2, as shown in Figure 17.
21

---

Similarity Matrix
rank-frequency of clusters
draaients daeleau smmiant JujnBlee
y a x
y yxx
500
cluster frequency
10a
envelope
number clusters
0.6
10a
300
200
0.2
10a
100
2000
4000
6000
8000
100
10
102
10a
input index
cluster rank
Figure 15: Similarity matrix and rank funk-frequency plots from QDG on multitask sparse parity. Despite sparse parity having known decomposition into subtasks which are power law distributed in frequency, we do not recover this same power law from samples. We used a = 0.4 for the frequency distribution for expected aunk-frequency power law exponent of -1,4, but measure rank-frequency envelope slope closer to -1.1.
F Parameter and data scaling exponents across studies
In Figure 18, we show an and ap (or possibly as, depending on the study) for a variety of prior studies of deep learning scaling, compiled by Villalobos [47]. While the data is messy, it is intriguing that most the Rosenfeld et al. [2] samples lie below the ap = a' line, as our model would predict. The scaling exponents from Hoffmann al. [7] are slightly closer to prediction than the relation ap = aN, which has been proposed by other models of neural scaling laws. Overall though, the existing empirical results too messy to definitively support contradict our model.
G Estimates of compute used for our experiments
Multitask sparse parity: Our training script takes roughly 1-4 hours (depending on network size) to perform single-epoch training
GPU. When training multi-epoch on fixed dataset, take typically between 3-10 minutes, with some outliers taking much longer. Our largest experiment was for Figure 10, where we trained networks of varying width on data with varying distributions over subtasks (with different power law exponents). 467 runs completed with total running time of approximately 1450 hours. These experiments were run on a cluster with heterogeneous hardware. Availble GPUs include NVIDIA A A100, RTXA6000, QUADRORTX6000, GHFORCERTX2080TT, GEFORCERTX2080, GEFORCEGTX 1080TI, titan-x, and tesla-v100
Pythia model scaling evaluations: We evaluated Pythia models on NVIDIA A 110 80GBs. We do not have available the running time used when computing loss on approximately ten million tokens (for which we reported scaling statistics on), although it was likely less than an hour per model. The most expensive experiments were for Figure 11. where evaluated the first four models in the Pythia suite across 143 checkpoints for a total of 572 evaluations. We likely used some hundreds of A100-hours for this, though possibly less than 100 hours.
QDG: We ran our QDG experiments on an NVIDIA A100 80GB. For the smallest Pythia model and for 10000 samples, it takes few hours compute the similarity matrix. We performed this computation only a handful of times.
22

---

10a
d-30, a-0.0
d- 100 0-0.0
d- 1-1000, 0.0
cluster size
10a
k=100
k-200
10a
k-500
truth
10a
10a
d=30, 0=0.5
d=100, 0=0.5
d=1000, 0=0.5
cluster size
10a
10a
d=30,
d=100,
d=1000,
cluster size
102
10a
sag
cliff
10
10c
10a 10a 10
10
10a 10
101
10a
10a
cluster rank
cluster rank
cluster rank
Figure 16: To understand the bias of spectral clustering, we apply spectral clustering a toy model with different embedding dimension d, noise scale and number of cluster k. The high- nigh-dimension (d 1000) large-noise (a 2.0) scheme seems best agree with the LLM results (Figure 5).
knndipe 1008 3.206
10F
335
1007
648
383
3077
265
3074
3332
320
201
1007
3323
100h
11.2h
100
3200
3-33.00
3.0000
3500
2.385
1,40
2.0
180
100
107
clunter ranic
10a
10a
Skatter mank
15a
10
clusterra
10a
101
ranz
10
12
Figure 17: The difficulty of measuring from curves We apply spectral clustering to a toy model with different and number of clusters k. For fixed different curves define envelope. One could use the envelope slope to infer a, but this incurs errors around 0.2.
23

---

1.2
Resenfeld Kaplan Hoffmann et Ardalani Gardon
0.8
Droppo et all
8
0.6
dg da
0.4
0.2
0.2
0.4
0.6
0.8
1.0
1.2
aN
Figure 18: Parameter and data scaling exponents from various studies of deep learning scaling, compiled from the database of neural scaling laws from [47]. Our model of scaling predicts that aD = a + 1). indicated with the solid black line. Visible points are from 32, 5, 48). [49] above the visible window of the figure.
24


Reviews:
[{"review_id": "BaqixbVyKA", "reviewer": "Reviewer_7ZcN", "summary": "The paper proposes that the capabilities of a neural network are composed of discrete “quanta” that each help reduce loss on a subset of training examples, and that are learned in approximately decreasing order of frequency. If the quanta are present in a Zipfian (power-law) frequency distribution in the training data, and each quantum is approximately equally valuable in the examples where it’s present, such a theory would provide a mechanistic explanation for power law scaling of loss with respect to model and dataset size.\n\nFirst the authors validate this hypothesis on a toy dataset constructed to have this quantized structure (by being composed of several independent tasks each with a different prefix key). Then they find similar patterns in real language models, and use an algorithm they call “quanta discovery with gradients” to identify potential quanta in these models and their data.\n\nTheir results are supportive of the quantization hypothesis, but with a high degree of uncertainty.", "questions": "How much do you believe the hypothesis overall? Might there also be a significant role for “interaction terms” between quanta or are they close to being fully independent?\n\nDo you have early ideas about alternatives to QDG, especially alternatives that might be more computationally tractable?\n", "limitations": "The authors acknowledge the limitations of their methods and experiments.\n", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The hypothesis is a beautiful and important claim if true, and the paper provides a well presented and solid chunk of evidence that it is. The toy dataset experiment is simple and straightforward and demonstrates that the quantization hypothesis works under ideal conditions, and the LM experiment shows that similar phenomena are also present in real models, while proposing a reasonable initial approach to quanta discovery. The results on monogenic vs. polygenic samples also point pretty clearly towards the validity of quantization. \n\nGiven QDG or any other model of what specifically the quanta might be, the hypothesis also enables testable predictions about scaling laws.\n", "weaknesses": "The multitask parity setting seems too obviously likely to lead to quanta, so it doesn’t seem to prove very much (although it’s useful to set up the framing, almost like a thought experiment).\n\nThe QDG methodology is also a bit disappointing in a few ways. By clustering the gradients, it assumes that quanta (conceptually defined without reference to model structure) will be localized in the model—which is probably true, but might otherwise have been a testable hypothesis. It’s also too slow for the authors to have applied it to models larger than the smallest Pythia LM, greatly limiting its applicability.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback and questions! We’re glad you found the paper to be interesting and valuable. We’ll comment on the weaknesses and respond to your questions below:\n\n**The multitask parity setting seems too obviously likely to lead to quanta, so it doesn’t seem to prove very much (although it’s useful to set up the framing, almost like a thought experiment).**\n\n\nWe agree that the multitask sparse parity dataset is a bit contrived. One of the ways in which it’s valuable is that it shows that the mechanism of power law neural scaling depends on the structure of the data. For instance, in Sharma and Kaplan (2022) “Scaling Laws from the Data Manifold Dimension”, they train neural networks on a set of toy regression problems of varying input dimension and observe power law scaling where the scaling exponent is determined by the dimension of the space that the function is defined on. For those sorts of tasks, their model of neural scaling, which views neural networks as approximating a function defined on some manifold with better and better precision with increasing scale, seems to hold. But our experiments on multitask sparse parity show that one can also get power law scaling in accordance with the Quantization Model when the data is fundamentally discrete, and where neural networks learn an increasing number of discrete computations with increasing scale, rather than globally approximating a function on a manifold with higher precision. So the mechanism of power law scaling depends on the structure of the data. Now when we observe power law scaling in the wild, we can ask whether the Sharma and Kaplan model of scaling or the Quantization Model of scaling, or maybe something else, best describes what’s going on.\n\n\n**The QDG methodology is also a bit disappointing in a few ways. By clustering the gradients, it assumes that quanta (conceptually defined without reference to model structure) will be localized in the model—which is probably true, but might otherwise have been a testable hypothesis. It’s also too slow for the authors to have applied it to models larger than the smallest Pythia LM, greatly limiting its applicability.**\n\nSince we cluster using the (almost) full model gradients (across all layers of the network) our method might still work even if computations are spread across large parts of the model. We just assume that gradients will have some consistency among samples where prediction relies on the same pieces of knowledge/computation. We hope that more principled and efficient methods could be developed in future work.\n\n\n**How much do you believe the hypothesis overall? Might there also be a significant role for “interaction terms” between quanta or are they close to being fully independent?**\n\nIndeed it might be unrealistic to think of the quanta as being fully independent in language modeling -- great point! In the current draft of the paper, when we talk about samples being polygenic, we imagine something like different circuits pushing the logits in a good direction independently. But realistically, polygenicity is probably more complicated than this. On some samples, a model’s computation might be more integrated, where loss is only lowered if multiple quanta are present simultaneously. In the final draft, we will clarify that polygenicity can be complicated, and the quanta might not lower the loss entirely independently of each other.\n\n\nOne interesting biological analogy is the fact that in humans, blue eyes vs. brown eyes is a monogenic trait. However, this gene only matters in the context of many other genes, e.g. the genes that are responsible for humans having eyes in the first place! So it’s sort of implicitly polygenic. Many quanta in LLMs could be similar. A quantum might depend on other quanta, and so would be polygenic in some sense, but deleting it would sharply remove some capability of the model.\n\n\n**Do you have early ideas about alternatives to QDG, especially alternatives that might be more computationally tractable?**\n\nUsing activations instead of gradients could lower the dimension considerably. Also using random projections of the gradient instead of the full gradients approximately preserves cosine similarity via the Johnson-Lindenstrauss lemma. Furthermore, the idea of clustering samples to discover quanta only makes sense when samples are monogenic. Ultimately, we’d like a scheme which enumerates the quanta and then for a given sample identifies which quanta (possibly many of them) were relevant for prediction on that sample.\n"}, {"review_id": "yae8triZUC", "reviewer": "Reviewer_2SwD", "summary": "The authors investigate neural scaling laws and propose an explanation for the power law scaling that is observed in addition to the emergence of new behaviours. ", "questions": "Could you elaborate on why you feel that this demonstrates emergent capabilities in language models? \n\nYou mention the spectral clustering hyperparameters: how sensitive are other parts of this work to broad choices of hyperparameters? For example how are you setting batch size / LR for the LM experiments?\n\nIn Figure 5 right panel there is some interesting shape in the yellow curves-- I'm curious what you think of those?\n", "limitations": "Yes, the limitations are mostly well addressed.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The ideas are interesting and I found some of the experiments such as per token losses on the language models to be quite interesting.\n\n", "weaknesses": "I found the presentation of Section 5 to be a bit confusing.\n\nI also found the connection to emergent behaviours to be a bit speculative. Specifically I think the idea of looking at per token losses/etc to be quite interesting, but I wouldn't necessarily say that I am convinced that this is emergence in the same way I'd think of emergence on some BigBench tasks, for example. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and questions! We’re glad you found our ideas and some of our experiments interesting. Before responding to your questions point by point, we’d like to apologize for not making our most interesting contribution more clear, namely our notion of quantization and of quanta. Essentially we argue that large neural networks are implicitly ensembles of indivisible computational modules (quanta). We view our work as a step toward validating Marvin Minsky’s Society of Mind theory, that intelligent systems can be thought of as collections of many smaller systems performing specialized jobs. Intriguingly, we were able to reconcile this view of neural networks with scaling laws while also providing a framework for understanding emergence, although understanding emergence is a less important point of the paper. We will edit the paper to better emphasize our notion of quantization and its relation to Minsky. \n\n\n**I found the presentation of Section 5 to be a bit confusing.**\n\nWe’re sorry about this! Section 5 could definitely be better motivated and explained. The first paragraph in particular jumps into some confusing discussion about partitioning the problem of language modeling into subtasks without motivating that. We will edit the paper to improve the clarity of this section. In Section 5, we were ultimately interested in trying to discover what some of the computational modules (quanta) in LLMs are.\n\n**I also found the connection to emergent behaviours to be a bit speculative. Specifically I think the idea of looking at per token losses/etc to be quite interesting, but I wouldn't necessarily say that I am convinced that this is emergence in the same way I'd think of emergence on some BigBench tasks, for example.**\n\nOne advantage of looking at per-token loss is that it avoids some of the critiques of emergent abilities brought up by Schaeffer et al. (2023) “Are Emergent Abilities of Large Language Models a Mirage?” -- some examples of emergence in BigBench may be due to the choice of metric. In contrast, the loss on a single token is a smooth function of the model’s output, so if we see a sharp drop in loss, this suggests that there is some genuine qualitative difference in the model. \n\n**Could you elaborate on why you feel that this demonstrates emergent capabilities in language models?**\n\nOne of the core questions we’re interested in with this paper is: how does scaling change what neural networks learn? Our hypothesis is that scaling adds new computational modules (quanta) to the network that weren’t present before. \n\nOne prominent prior model of neural scaling is Sharma and Kaplan (2022) “Scaling Laws from the Data Manifold Dimension”. In this model, neural networks are understood as approximating a function defined on some “data manifold”, and the effect of scaling is to smoothly increase the resolution/precision at which this function is approximated. This can explain power law scaling, but there seems to be some tension between this view, where scaling leads to a smooth change in what models learn, and the sharp improvements with scale we see in network performance on some tasks (e.g. BigBench tasks with high “breakthroughness”). Now, one way of resolving this tension is to argue that emergence is an artifact of one’s choice of metric like Schaeffer et al. (2023) do. But we propose an alternative view with our model, where smooth power law scaling in mean loss averages over many discrete changes as indivisible modules (quanta) are added to the model. \n\nIn the multitask sparse parity dataset we construct, this model of scaling holds -- emergence is real: performance on subsets of the data sharply transitions from random-guess performance to perfect performance with increasing scale. But we still get smooth power laws in mean loss. So a very strong notion of emergence can occur for data with the right structure. \n\nIf this story described language modeling, then we could understand emergent abilities in language models as the result of new quanta being learned. If a benchmark exhibits emergence, we’d say that the quanta relevant to solving that task are learned at a similar scale. At that scale, models transition to good performance on the task since the necessary quanta are now present, and weren’t present at smaller scale.\n\n**You mention the spectral clustering hyperparameters: how sensitive are other parts of this work to broad choices of hyperparameters? For example how are you setting batch size / LR for the LM experiments?**\n\nFor the LM experiments, we didn’t train any LMs ourselves but instead just used the open source Pythia suite from EleutherAI. For the multitask sparse parity experiments, we just used a large batch size and an AdamW learning rate of 1e-3 that worked well in practice. We’d be happy to run some grid searches over batch size and LR and include these results in the appendix!\n\n**In Figure 5 right panel there is some interesting shape in the yellow curves-- I'm curious what you think of those?**\n\nWe’re not quite sure what’s causing the deviations from a clean power law that we see in the rank-frequency curves in Figure 5. In Appendix D.2 we experimented with a toy model of clustering and found that we could get similar looking curves when the dimension is high and when the noise was high -- see the lower right panel of Figure 15 if interested. We will edit the paper to point this out and refer readers to the appendix.\n"}, {"review_id": "QZpCdwtmHE", "reviewer": "Reviewer_7dsB", "summary": "This paper proposes a new way of understanding scaling laws. Namely, it shows that capabilities can be thought of as being broken into discrete units (quanta) which themselves follow a power law and have an inherent ordering--called the Q Sequence. This combined with the fact that 1) an increasing subset of quanta are learned over various scales and 2) model and data sizes can be related to the number of quanta gives rise to the scaling laws shown in previous works. The paper also proposes a method to cluster examples by unit-normalized gradient, and empirically confirms within margin of error that theory agrees with observed data. The quantization model explains a sudden emergence of new capabilities with scale: certain quanta must be learned before the model can carry out certain tasks.", "questions": "Everything is clear.", "limitations": "Not that I am aware of", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Simple, effective model to understand scaling laws", "weaknesses": "While the paper shows that the model scaling trend agrees with theory on real data, it does not empirically validate whether the same alpha of 0.083 matches the data scaling theory. Also, it would be interesting to show this holds on non LLM tasks like image classification, since the (titular) claim is \"neural scaling\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We’re glad you found the paper compelling!\n\n**While the paper shows that the model scaling trend agrees with theory on real data, it does not empirically validate whether the same alpha of 0.083 matches the data scaling theory. Also, it would be interesting to show this holds on non LLM tasks like image classification, since the (titular) claim is \"neural scaling\".**\n\nIt would indeed be great to evaluate whether the Quantization Model holds on other types of data, e.g. for vision tasks like you suggest. Perhaps this could be done in future work. \n\nFor data scaling, Figure 11 and Figure 17 of the Appendix (Supplementary Material) may partially address this. For the Pythia LLM scaling suite, all models are trained on the same amount of data, so we can’t study multi-epoch data scaling. However we do have intermediate training checkpoints for these models, so we can study scaling in training steps. In Figure 17 we see that the training curve is not obviously a clean power law, so it is not clear where to fit a line to the curve. For the lines we do fit, we get a slope which is less negative than -0.083, which is what we’d expect from our theory, although the measured slopes of between -0.037 and -0.06 are shallower than we’d expect from theory: -0.083 / (1 + 0.083) = -0.077. This could be due to a suboptimal choice of hyperparameters when EleutherAI trained these models. Figure 17 shows empirical scaling exponents in parameters and data from a variety of scaling papers. The most reliable point is probably the green point from the Chinchilla paper (Hoffmann et al.), which lies slightly above our curve $\\alpha_D = \\alpha_N / (\\alpha_N + 1)$. \n"}, {"review_id": "skkAN0JAsZ", "reviewer": "Reviewer_S4dM", "summary": "This paper proposes a hypothesis that there exists a universal and crucial discrete set of computations (quanta) for reducing loss in various prediction problems, and the model's performance is determined by successfully learning these computations. Through this hypothesis, it demonstrates the relationship between power law neural scaling and the effective learning of more quanta, which cannot be solved solely by the memorization ability of the existing model, particularly in complex problems. Additionally, the paper proposes \"Quanta Discovery from Gradients\" as a method to auto-discover quanta. ", "questions": "1. In Figure 1, what decoding method was used for the auto-discovered quanta?\n2. The experiments on the toy dataset appear to be influenced by the decoding algorithm (such as greedy, beam search, random). Is there any consideration given to this aspect?\n3. In Figure 4, what is the relationship between the highlighted red section in the prompt and each graph?\n4. In Section 5, it would be beneficial to include details about the dimensional of the gradients used in the Quanta Discovery from Gradients (QDG) method. The absence of explicit notation indicating the shape of each symbol for explaining QDG makes it challenging to ascertain the computational complexity involved in implementing QDG.", "limitations": "There is a need to demonstrate the effectiveness of the proposed Quanta Discovery from Gradients (QDG) method in identifying quanta in various Language Models (LLM) and datasets.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Based on the assumption that there exists a discrete set of computations, known as \"quanta,\" which determines the performance of the model, the paper provides hints for understanding the emergent behavior observed in the scaling law of the model.\n2. The objective of the study is to interpret power law scaling and emergent behavior phenomena by using a multitask sparse parity dataset that cannot be solved solely by the model's memorization ability.\n3. Through experiments conducted on the multitask sparse parity dataset, the paper shows that when the frequency of using quanta follows a power law distribution, power law neural scaling can occur as the model learns more quanta.", "weaknesses": "1. The paper lacks a clear definition of quanta, which could benefit from further elaboration and clarification.\n2. Insufficient explanation is provided regarding the criteria used to determine quanta through the Quanta Discovery from Gradients (QDG) method.\n3. Extending the experiments on the Quantization Hypotheses, using the proposed toy dataset, to Language Models (LLM) is hindered by the lack of clarity regarding the relationship between LLM tokens and quanta.\n4. Section 5 claims that \"Clustering based on inputs or outputs therefore seems unlikely to discover quanta in language modeling,\" but lacks substantial empirical evidence to support this statement.\n5. The explanation concerning the relationship between quanta and gradients, which forms the basis for determining quanta in the QDG method, needs to be further developed and elaborated upon.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and questions, and for highlighting the importance of clarifying our definitions in the paper! We think the clarifications we’ll make, prompted by your review, will significantly improve the paper. We’ll respond point by point:\n\n### Weaknesses:\n**1. The paper lacks a clear definition of quanta, which could benefit from further elaboration and clarification.**\n\nHere is the definition of quanta which we’ll add to the paper:\n\n_Definition of quantum (plural quanta)_: An indivisible computational module that, for example, retrieves a fact or implements an algorithm.\n\nAlthough quanta cannot be learned instantaneously in practice, we expect their formation to correspond with a quick drop in loss akin to an upside-down sigmoid. This was observed for the formation of “induction heads” in Olsson et al. (2022), “In-context Learning and Induction Heads”. We interpret the induction circuit described in that paper as one such quantum, and propose that LLMs can be understood as an ensemble of such modules.\n\n**2. Insufficient explanation is provided regarding the criteria used to determine quanta through the Quanta Discovery from Gradients (QDG) method.**\n\nThanks again for pushing us toward a clearer presentation! \n\n_QDG_: Operationally, we identify quanta as clusters of next-token prediction samples (found by spectral clustering of gradients) that pass subsequent vetting for task coherence.\n\nWhile QDG does not on its own produce a mechanistic understanding of how the quanta work, it does suggest the function that these modules have. For instance, in one QDG cluster, all samples involve predicting a number to continue a numerical sequence, and we think that this suggests that there is a module in the network that is responsible for this capability/behavior.\n\n**3. Extending the experiments on the Quantization Hypotheses, using the proposed toy dataset, to Language Models (LLM) is hindered by the lack of clarity regarding the relationship between LLM tokens and quanta.**\n\nThe quantity being measured in LLM scaling laws is the mean cross-entropy loss for next-token prediction in the training distribution of text. In our model, a quantum in an LLM is a computational module which improves the network’s ability to do next-token prediction on some fraction of tokens in the training distribution of text. To recover LLM scaling laws, we claim that these fractions (frequencies at which the quanta are used) follow a power law.\n\n**4. Section 5 claims that \"Clustering based on inputs or outputs therefore seems unlikely to discover quanta in language modeling,\" but lacks substantial empirical evidence to support this statement.**\n\nWhen we tried to cluster next-token prediction samples just based on what the next token was, or based on the last tokens of the input, we didn’t get interesting clusters. This is a minor point, but admittedly not well supported by any experiments we showed in the paper. We’ll either remove this claim or support it with experiments.\n\n**5. The explanation concerning the relationship between quanta and gradients, ..., needs to be further developed and elaborated upon.**\n\nWhen we clustered samples according to gradients, we intended that this would cluster them according to whether the model used similar knowledge or circuitry to perform prediction on those samples. Consider some quantum (module) in the network. If it is important for prediction on a pair of samples, then the subset of network parameters that are part of the module will perhaps have nonzero and similar gradients for those samples. If on a different sample that module is irrelevant to prediction, then the module’s parameters might have gradients of close to zero (or at least different gradients from those samples which relied on the module). Therefore gradients on samples which rely on the same computational module will have higher cosine similarity than between samples which don’t rely on the same module. Thank you for pointing out that we didn’t justify this at all in the current version of the paper! We’ll add some text like this motivating the use of gradients to identify quanta in Section 5 when we introduce QDG.\n\n### Questions\n**1. In Figure 1, what decoding method was used for the auto-discovered quanta?**\n\nWith quanta, we’re interested in the capabilities present in a single forward pass of a model. The samples shown in Figure 1 are next-token prediction samples, where the token to be predicted is highlighted in red. We only applied clustering to samples (where a sample involves predicting a single token from its context) if the model’s cross-entropy loss on them was low (we chose a threshold of 0.1 nats). These are samples where the model would correctly predict the next token if greedy decoding was used.\n\n**2. The experiments on the toy dataset appear to be influenced by the decoding algorithm (such as greedy, beam search, random). Is there any consideration given to this aspect?**\n\nThe multitask sparse parity task is a supervised binary classification problem which we train MLPs to solve, so we don’t think that decoding is relevant. Categorical cross-entropy loss seems like a natural metric of model performance on this task.\n\n**3. In Figure 4, what is the relationship between the highlighted red section in the prompt and each graph?**\n\nIn Figure 1 and Figure 4, the highlighted red text indicates the token that the LM was predicting from the text shown before it. So in Figure 4, we show how cross-entropy loss changes with model scale in predicting the highlighted red token from its context. These examples are from the test set of The Pile corpus.\n\n**4. …The absence of explicit notation indicating the shape of each symbol for explaining QDG makes it challenging to ascertain the computational complexity involved in implementing QDG.**\n\nGood point, sorry for the omission! We’ll add details about the dimensionality of the model and the complexity of running QDG to Section 5. \n"}, {"review_id": "ok9fW2UCsi", "reviewer": "Reviewer_Kziq", "summary": "This paper proposes a possible mechanism that explains both the phenomenon where the cross-entropy loss of large language models (LLMs) decreases as a power law with respect to the training corpus size, and the phenomenon in which certain capabilities of LLMs emerge spontaneously as the loss becomes low enough. The authors empirically demonstrate that in a toy problem called \"multitask sparse parity\", in which their assumptions explicitly hold, an MLP network indeed obeys both the scaling laws and the emergence phenomena. Finally, the authors propose an empirical method to auto-discover skills in LLMs and apply this method to provide empirical evidence that their assumptions also hold in LLMs. ", "questions": "Could you provide a practical demonstration of the Quantization Hypothesis in a controlled environment that is neither overly simplistic nor complex, while ensuring a clearer definition of natural language quantas? For instance, perhaps you could utilize a pre-trained language model like GPT-4 to create datasets with well defined quantas in a similar manner to the TinyStories dataset [1].\n\n[1] Ronen Eldan, Yuanzhi Li (2023). TinyStories: How Small Can Language Models Be and Still Speak Coherent English?", "limitations": "The authors were very honest in acknowledging less plausible alternative explanations for their empirical findings.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper proposes a novel and timely explanation for both the scaling laws phenomenon and the emergence phenomenon in large language models (LLMs). \n\n2. The authors support their explanation with a clear demonstration on a toy task of multitask sparse parity. Essentially, they trained a simple MLP network on multiple sparse parity tasks and demonstrate both the scaling laws phenomenon and the emergence phenomenon when the distribution of the different tasks follows a power law.\n\n3. In addition, the authors demonstrate the relevance of their explanation for LLMs by proposing a novel method for auto-discovering skills in LLMs and show that the discovered skills obey power law frequencies. Using and scaling this method might be of independent interest for both the mechanistic interpretability community and for designing better pre-training datasets.\n\n4. The paper is clearly written and accessible to readers with varying backgrounds.", "weaknesses": "There is insufficient evidence to conclusively prove that the Quantization Model accurately depicts the training of large language models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and question/suggestion! We agree that more work could be done in evaluating to what extent scaling on real-world datasets satisfies the Quantization Hypothesis.\n\n**Could you provide a practical demonstration of the Quantization Hypothesis in a controlled environment that is neither overly simplistic nor complex, while ensuring a clearer definition of natural language quantas? For instance, perhaps you could utilize a pre-trained language model like GPT-4 to create datasets with well defined quantas in a similar manner to the TinyStories dataset [1].**\n\nThis is a cool idea! Based on this suggestion we tried running QDG on the TinyStories-33M model. We get some interesting clusters, although the most interesting TinyStories quanta seem less interesting than the most interesting quanta on The Pile. Some clusters in TinyStories include:\n- Predicting a question mark at the end of some dialogue that asks a question\n- Predicting the correct pronoun based on the name of the subject\n- Predicting quotation marks after ‘said, ‘\n- Predicting a comma after “Suddenly” or “Just then”\n- Predicting a newline after a dedicated space token\n\nMany of the clusters we get seem to reflect very simple repeated patterns in the TinyStories dataset, rather than reflecting sophisticated reasoning. We’ll add examples of some of these clusters to the appendix.\n"}]

Author rebuttal:

Meta-review: In this paper, the authors develop a model of neural scaling where smooth scaling laws average over small emergent changes in model performance on subtasks denoted as "Quantization Model of Neural Scaling Laws". This is a novel theoretical model of understanding scaling laws which have verifiable predictions in practical LLMs. 

Reviewers pointed out some of strengths and weakness as:

Strength
- novel and timely explanations for scaling laws and emergence in large language models
- compelling case by first clearly demonstrating their theory on a toy model
- provides results showing theory's applicability to real-world LLMs
- the hypothesis also enables testable predictions about scaling laws.
- simple, effective model to understand scaling laws
- paper is clearly written and accessible to readers with varying backgrounds. 


Weakness
- quanta defined in the paper is difficult to identify in practical LLMs and too slow to scale 
- results are supportive of the quantization hypothesis, but with a high degree of uncertainty.
- multitask parity setting not sufficient for validation of hypothesis
- connection to emergent behaviors to be a bit speculative
- insufficient evidence to support the hypotheses about the proposed quantization model showing consistent results in large-scale language models 

Reviewers highlighted that "the paper's novel contributions and rigorous approach warrant publication". "It will spark valuable discussion and inspire further research on these topics".  All the reviewers recommended accepting the paper (8, 8, 7, 5, 5, showing varying degree of support) saying that the paper is technically strong with novel ideas and excellent impact. While there are some concerns on speculative nature and how properly the proposed model explains scaling law and emergent behavior of LLM, it definitely provides a step towards providing theory for these phenomena and is worth sharing with the broad NeurIPS audience.
Decision: Accept (poster)