Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process
Authors: Tian Ye
Zicheng Xu
Yuanzhi Li
Zeyuan Allen-Zhu
Venue: ICLR 2025 Poster (2025)
Keywords: linear probing
language model
grade math problems
logic following
reasoning
Primary area: interpretability and explainable AI
TL;DR: Probing reveals that LLMs develop some "level-2" reasoning skill beyond Humans on solving grade-school level math problems

Abstract: Recent advances in language models have demonstrated their capability to solve mathematical reasoning problems, achieving near-perfect accuracy on grade-school level math benchmarks like GSM8K. In this paper, we formally study how language models solve these problems. We design a series of controlled experiments to address several fundamental questions: (1) Can language models truly develop reasoning skills, or do they simply memorize templates? (2) What is the model's hidden (mental) reasoning process? (3) Do models solve math questions using skills similar to or different from humans? (4) Do models trained on GSM8K-like datasets develop reasoning skills beyond those necessary for solving GSM8K problems? (5) What mental process causes models to make reasoning mistakes? (6) How large or deep must a model be to effectively solve GSM8K-level math questions?

Our study uncovers many hidden mechanisms by which language models solve mathematical questions, providing insights that extend beyond current understandings of LLMs.

Published as a conference paper at ICLR 2025
PHYSICS OF LANGUAGE MODELS: PART 2.1, GRADE-SCHOOL MATH AND THE HIDDEN REASON- ING PROCESS
EXXENDED ABSTRACT]*
Tian Ye
FAIR Meta and Carnegie Mellon University sigma 64 a com and
cmu, edu
Zicheng Xu
FAIR Meta (now Google Research) zi si.hhnnxxu@meta com (now zi zichengxxu@goggee.com
Yuanzhi Li
Mohamed bin Zayed University of AI Yuanzhi Li@mmbzaii.......m..mm.. ac.
Zeyuan Allen-Zhu
FAIR at Meta
zeyuanal lenzhu@metae com
ABSTRACT
Recent advances in language models have demonstrated their capability to solve mathematical reasoning problems, achieving near-perfect accuracy grade- school level math benchmarks like GSMSK. In this paper, we formally study how language models solve these problems. We design a series of controlled experi- ments to address several fundamental questions: (1) Can language models truly develop reasoning skills, do they simply memorize templates?
(2) What is the model's hidden (mental) reasoning process? (3) Do models solve math ques- tions using skills similar to different from humans? (4) Do models trained GSM8K-like datasets develop reasoning skills beyond those necessary for solving GSM8K problems? (5) What mental process causes models to make reasoning mistakes? (6) How large or deep must model be to effectively solve GSM8K- level math questions?
Our study uncovers many hidden mechanisms by which language models solve mathematical questions, providing insights that extend beyond current under- standings of L.MMs.
INTRODUCTION
The field of language models has made significant progress in recent years. Large models like GPT- 4 (OpenAI, 2023) have shown initial signs of general intelligence (Bubeck et al. 2023) even argued to have reached L2 or L3-level of intelligence (Allen-Zhu & Xu 2025), while smaller models have demonstrated good reasoning abilities by solving challenging coding/math problems (Li al.) 2023 Gunasekar et al. 2023 Liu et al. 2023).
'The first six papers in the Physics of Language Models series were presented as two-hour tutorial at ICML 2024 in Austria fyoutu. be betp////yykgggda
A one-hour deep dive into Part 2.1 is available at youtu coutu.bee/bppeee betppbbbbbbbppppee2yyye Full and future editions of Part 2.1, including code release, are available
physics all.n-aallllappa. zhu. com g tithbb.e com and

---

Published as a conference paper at ICLR 2025
In this paper, we focus the ability of small language models to solve grade-school math prob- lems. Unlike previous works that empirically push the accuracy of models on grade-school math benchmarks like GSM8K (Cobbe et al. 2021) and its augmentations (e.g., Liu et al. (2023); Zhang et al. (2024)), we take a more principled approach. We aim to understand the following fundamental questions:
1. How do language models learn solve grade-school level math problems? Do they just mem- orize templates, or do they learn reasoning skills similar to humans? Or do they discover new skills solve the problems?
2. Do models trained solely on grade-school math problems only learn to solve these problems, or do they develop some more general intelligence?
3. How small can language model be while still solving grade-school math problems? Is depth (number of layers) more important than width (number of neurons per layer), or does only size matter as suggested by practitioners (Kaplan et al. 2020)?
These questions are fundamental to understanding the intelligence of language models. To study them, it might seem tempting to start with a pre-trained model and fine-tune it on existing datasets like GSM8K or GPT-4 augmented ones (e.g.. Liu et al. (2023); Zhang et al. (2024)). However, this approach has significant limitations:
The pretrain data of existing models mostly come from publicly available internet (Gao et al. 2020), which is a pile of mess. We do not know how many math problems are included or their structures. There is significant concern regarding whether the GSM8K benchmark has been leaked to language models' training datasets (Zhang et al., 2024). Even if the exact data is not, the pre-trained model might have seen almost identical questions (e.g., the same problem with different numbers). Thus, this approach cannot answer questions 1-3, We do not know whether a model truly learns the reasoning skills it sim- ply memorizes problem templates during training. Therefore, we need full control over the model's pretrain data and must train language model from scratch. This point has been reiterated recently in (Allen-Zhu & Li 2024, 2025a).
DATA CONTAMINATION.
SOLUTION DIVERSITY. The existing fine-tuning data, such as the GSM8K training set, con- tains only 7.5K grade-school math problems, which insufficient to train model from scratch. Although recent works use GPT- to augment GSM8K, this is not enough for our purpose. GPT-4 augmented problems might be biased towards a small number of solution templates, since the original GSM8K data has very few (obviously, at most 8K) solution tem- plates. We need a much larger, more diverse set of grade-school math problems.
With these points in mind, we introduce our framework to generate a large set of diverse grade- school math (GSM) problems and use the dataset to train (from scratch) and test GPT2-like lan- guage model. In the framework, we focus on the "logical reasoning" aspect of grade-school math problems, which involves the dependency of parameters in the problem statement, such as "Alice's apple is three times the sum of Bob's s orange and Charles's 's banana." " We use synthetic sentences to reduce the difficulty arising from Common Sense, like "a candle burned for 12 hours at I inch per hour" (implying the candle is reducing in length). We also intentionally remove the difficulty from pure arithmetic: we only consider integers mod23."
Moreover, our framework ensures that the generated math problems are highly diverse and do not come from a small subset of templates. Even ignoring all the arithmetic. English. variable names, and unused parameters, our problems still have more than 90 trillion solution templates (see Proposition 2.2), much larger than the size of GPTl-small (100M). Thus, language models cannot solve the math problems in our case by simply memorizing the solution templates.
'The conclusions of this paper remain valid if one replaces 23 with, for instance, 2003. However, for better-controlled experiment,
wish separate reasoning from arithmetic.. For instance, if model fails, we want to ensure is not due to an arithmetic error after all, memorizing the multiplication table 23 integers is trivial for even a small model. On a separate note, there is a rich literature studying how well language models can learn arithmetic and length generalization; see Zhou et al. (2023); Jelassi et al. (2023); : Lee et al. (2023) and the references therein. Modern language models are also equipped with calculator plugins. Thus, might not be the language model's joh to perform complex arithmetic (such as 20-digit multiplications) anyway.
2

---

Published as a conference paper at ICLR 2025
In this paper, the GPT2 model (Radford et al., 2019), but replace its positional embedding with rotary embedding (RoPE) (Su et al. 2021; Black et al., 2022). We still call it GPT2 for brevity. We summarize main contributions:
synthetic dataset, not only achieves 99% accuracy in solving math problems from the same distribution but also generalizes to out-of-distributtin problems, such those requiring longer reasoning lengths than any seen during training. This is similar to length generalization in arithmetic (Anil et al., al.,,2022 2022 Jelassi al., 2023), however, in our case, the model has never seen any training example of such reasoning length. This signifies that the model can genuinely learn reasoning skills instead of memorizing solution templates.
RESULT 2. We demonstrate that the GPT2 model, pretrained
RESULT 3. Crucially, the model can learn generate shortest solutions, almost always avoid- ing unnecessary computations. This suggests that the model formulates plan before it gen- erates, avoiding computing any quantities not needed towards solving the underlying math problem.
formulaass
RESULT 4. We examine the model's internal states through probing, introducing six probing tasks to elucidate how the model solves math problems. For instance, we discover the model (mentally!) preprocesses the full set of necessary parameters before it starts any generation. Likewise, humans also do this preprocess although we write this down on scratch pads.
RESULT 5. Surprisingly, the model also learns unnecessary, yel important skills after pretrain- ing, such as all-pair dependency. Before any question asked, it already (mentally!) computes with good accuracy which parameters depend on which, even though some are not needed for solving the math problem. Note that computing all- pair dependency a skill not needed fit all the solutions in the training data. To the best of our knowledge, this is the first evidence that language model can learn useful skills beyond those necessary to its pretraining data This may be preliminary signal of where the G in AGI can come from 3
RESULT 6. We explain why mistakes occur. For instance, the model makes systematic er- rors that can be explained by probing its internal states. Sometimes, these mistakes can be predicted before the model generates answers, making them independent of the random gen- eration process. We connect this to practice, noting that GPT-4/4o also makes similar errors (though cannot probe their internal states).
The depth of the language model is crucial for its reasoning ability. For ex- ample, 16-layer, 576-dim transformer solves harder problems (in reasoning length) than a 4-layer, 1920-dim one, despite the latter being twice as large. This holds even when Chain- of- ohoThhtgh (CoT) is used. We explain this necessity in depth by the complexity of the mental processes involved. We advocate for the use of controlled, synthetic data more principled approach to derive such claims, contrasting with predictions like "only size matters" based on training loss using internet pretrain data (Kaplan et al. 2020).
RESULT 7+8.
While we refrain from overstating that our findings directly apply to foundation models like GPT-4 or more challenging mathematical reasoning tasks, we believe our work significantly advances the understanding of how language models develop their mathematical reasoning skills, and this has to be done in way different from pushing benchmarks.
RESULT 1: DATA GENERATION
Motivation. Recall a standard grade-school math problem in the GSM8K dataset (Cobbe et al., 2021) looks like:
Betty is saving money new wallet which 100. Betty has only half of money she needs. Her parents decided give her 15 for that purpose, and her grandparents twice as much as her parents. How much more money does Betty need to buy the wallet?
II our case, one can solve all the math problems without computing all all-pair dependency. Our pretraining data never includes such information - all the solutions only compute necessary variables.
IIndeed, the skill to sort relationships among in on-context objects is a general skill, which may lead to via instruction fine-tuning skills for solving other tasks, such discovering causal relationships, determining the influence of parameter changes, etc.
3

---

Published as a conference paper at ICLR 2025
Structure Graph
Dependency Graph
Teckpack
necowary ENDRAN
unordd FROOEE iraianes depanderey
quecton FR0WIN
absnrct dipenduk
Figure 1: Structure and dependency graph corresponding to the op 7 casy example in (2.1) and (2.2) De- pendencies from abstract parameters are drawn in red, and from instance parameters in black.
This problem involves multiple parameters whose values are connected through various equalities, such as "Betty's current money = 0.5 x cost of the wallet" and "money given by grandparents 2 x money given by parents." Motivated by this, we build GSM8K- math dataset through synthetic generation pipeline that captures the dependencies of parameters. We wish to capture at least the following three types of dependencies.
1. Direct dependency (V): such A = x (X (Y) so A can be computed after X and Y.
2. Instance dependency ): such "every classroom has X chairs, and there are Y classrooms." Here, the model must infer the total number chairs by multiplying X by Y.
3. Implicit dependency (o)): ): such as "Bob has 3 times more fruits than Alice. Alice has 3 apples, eggs and 2 bananas." Here, the model must learn that apples and bananas are fruits and egg is not, and "Alice's fruits" is an abstract parameter derived from the problem statement.
2.1 STEP GRAPH CONSTRUCTION AND PROBLEM GENERATION
Hierarchical categorization. We layered structure of categories, each contains possible items For instance, categories = (School, Classroom, Backpack) has three layers; category School (Central High, Riverview High, ...): category Classroom {Dance Studio, Film Studio, )..): category Backpack = (School Daypack, Messenger Backpack, ...) ). We prepare 4 predefined hierar- chical categorizations, each with 4 layers and 100 items/layer; this represents the world knowledge.
Structure graph. In each math problem, only specific items exist, leading to structure graph that outlines what sub-items can appear under what item, see Figure I (left), For instance,
Connecting Dance Studio and School Daypack with an edge signifies an instance parameter, "the number of school daypacks in each dance studio," which is a quantifiable variable that as mentioned above.
can assigned This captures the instance dependency
Abstract parameters, like "the total number of classrooms in Central High," " cannot be assigned and are excluded from the structure graph. They reflect implicity dependency
Remark 2.1. Rather than using simple objects like Alice's apple or fake items like Items A/B/C/D, this structure allows us to describe abstract parameters and adds 2 levels of complexity to the data:
The model must implicitly learn English concepts, such as a classroom category includes 100 different classroom types. These concepts cannot be derived from individual math problems, as only limited selection of classrooms will be mentioned in each problem.
The model is required to hierarchically access multiple items calculate abstract parameters, as opposed to a straightforward retrieval of "Alice's 's apple" in the context
*Even though Central High and Rivierside High can both have (possibly multiple) Dance Studios, for sim- plicity, we assume that each Dance Studio has the same number of School Daypacks.
*For example, the total number of backpacks in Riverview High in Figure "Riverview High's number of Dance Studios", ip2
calculated
as ip: x api + ip2 x ap2 where ip; "Riverview High's number of Film Studios". apı = "each Dance Studio's 's number of Backpacks", " and ap2 "each Film Studio's 's number of Backpacks", with ip1, ip2 heing instance parameters and op1, ap2 abstract pa- rameters. Here, the model must only retrieve ip1, ip2 but also compute ap1, ap2 hierarchically.

---

Published as a conference paper at ICLR 2025
Dependency graph. The dependency graph is directed acyelic graph that outlines the dependency among parameters. For each instance parameter, we choose a random set of (up to 4) parameters can depend including possibly special vertex RNG representing a random number generator. For instance, if "[param AJ is more than the difference of [param B] and [param CJ" for X being randomly generated, then draw edges from B, C and RNG to parameter A. The dependency of abstract parameters is implied by the dependency of instance parameters. This captures direct dependency (V) as mentioned above. We give an examples on the right side of Figure and details for how we randomly generate such dependency graph are in the full paper.
Problem generation. The problem is articulated by describing the dependency graphs in English, one sentence for each instance parameter (Abstract parameters are not described because they are inherited by the structure graph.) We randomly permute the sentence ordering to further increase difficulty. A parameter is selected and asked with a question in the end (or at the beginning). Below an easy example corresponding to Figure 1, harder example in Figure 5
(Problem Easy) The number of each Riverview High's Film Studio equals times much as the sum each Film Studio's Backpack and cach Dance Studio's 's School Daypack. The number of cach Film Studio's School Daypack equals 12 more than the sum of cach Film Studio's Messenger Backpack and Central High's Film Studio.
mumber each Central High's Film Studio equals the sum each Dunce Studio's 's School Daypack and each Film Studio's Messenger Backpuck. The number of each Riverview ligh's Dance Studio equals Studio's 's Backpack,
Studio's 's Messenger Rackpack, each Film Studio's School Daypack and Central High's Backpack. The number of cach Dance Studio's School Daypack equals 17. The number of each Film Studio's 's Messenger Backpack equals 13, How many Backpack does Central High have?
sum of
(2.1)
2.2 STEP 2: SOLUTION CONSTRUCTION (CoT)
Let solution be a sequence of sentences describing the necessary steps towards solving the given problem, where the sentences follow any topological order also known as Chain-of-Thern TfoThhht COT. For each parameter necessary towards answering the final question, we assign to it a random letter among the 52 choices (a..z A..Z), and use sentence to describe its computation?
Define [param] as X; [intermediate steps]; so =
Throughout this paper, we consider arithmetics mod 23 to avoid errors from computation involving large numbers. It is perhaps the easiest directly see solution example (corresponding to (2.1)), and a more involved example is in Figure 5
(Solution Easy) Define Donce Studio's 's School Daypack as so 17. Define Film Studio's Messenger Backpack as W; so W 13.
Define Central High's Film Studio B; p W 17+ =3=7. Define Film Studio's School Daypack g. R= W =3 13+7. =2: so = 12 R= 12 + 20 9. Define Film Studio's is Backpack w: =g W = 13 =2. Define Central High's Backpuck as c. so == Answer: 16.
(2.2)
We emphasize that:
The solution only contain parameters necessary towards calculating the final query parameter. The solution follows the correct logical order: i.e. all the parameters used in the calculation must have appeared and been computed beforehand.
broken into 9 12 + and R = 13 + 7 in the above solution. The number of semicolons ":" equals the number of operations. This reduces the arithmetic complexity of the solution, which is not the focus of this paper 8
We break computations to binary ops: 9 12 + 13
Any (abstract) parameter that wasn't mentioned in the problem statement is by default zero.
'We use simple English sentence templates to describe the problem, and did not worry about grammar mistakes such as singular vs plural forms. There are other sources of randomness besides the dependency graph, such as when parameter A depends B, C' it could be A B or A A- B.
"There are different ways to format the CoT solution. We noted that starting with "Define [param] as X" instead of fintermediate steps] improves the model's accuracy, so we have adhered to this CoT format. EEvn GPT-4 make mistakes calculating "3 * (4+10)+ 12 # (5+6)" without using external calculator.
5

---

Although deferring all the data-generation pseudocode to the full paper, we summarize below the main randomness used in the data generation process. This includes the random choice of a hi- Published as a conference paper at ICLR 2025
Cff-itaroo
+4810
-2000
01001
01408
+34
+1400
-1:01
-1:10:
-1400
16.70
200%
10.0%
13.%%
85
16.3%
13.0%
15.0%
50.6
-1299
+2000
48.3%
25.75
3..7%
45.7%
05
23.54
28.%%
20.7%
30.35
22.0%
3244
30.7%
35.05
26.5%
202h
28,95 8%
32.2%
23.5%
26.4
32.35
22.6%
25.0%
23.3%
34.5%
6212
2p14
9915
20-3
6p-7
2015
20-23
6p-12
0p-22
4p-24
42-25
6p-16
4ph27
69:18
69-22
Figure 2: GPT-4 (OpenAl 2023) few-shot accuracies iGSM-med (with mod5 arithmetics). each op we tested 30 problems; and guessing ans = E {0, 2,3, 4} gives a baseline accuracy around 32%. Details are in the full paper, where we also showcase how GPT-4/4o make mistakes.
2.3 DIFFICULTY CONTROL
erarchical categorization (i.e., the English part); structure graph (i.e., the instance parameters); a dependency graph; arithmetic computations on the dependency graph; integer numbers (i.e., the RNG); problem sentence permutation; and the query parameter.
We use two parameters to control data's difficulty: ip is the number of instance parameters, and op
the number of solution operations; the data's difficulty is an increasing function over them. We call our dataset iGSM, to reflect the nature that such synthetic dataset can be of infinite size. We oSp. to denote the data generated with constraint op op and ip S ip, and use iGSM
denote those restricting to op op.
2.4
TRAIN AND TEST DATASETS
We consider two families of datasets.
In the iGSM-med data family we use < 20.
The training data iGSM-med def iGSM99515.ps20 We evaluate the pretrained model both in-distribution, on iGSM- and iGSM-med and distiddisttbiinnn Here,
(OOD), on iGSM-med00=0p for op {20, 21, 22, 23} and iGSM-med reask denotes first generating a problem from iGSM-med op and then resampling query parameter (D)
In the iGSM-hard data family we use ip < 28.
The training data is PSS1 def iGSM9921.ipsas We evaluate the pretrained model both in-distribution, iGSM-hard for op E {28, 29, 30, 31, 32) and iGSM-hard op
ooS and iGSM-hard and OOD on
Additionally, we use iGSM-med to indicate placing the question after the problem and iGSM-med,p the other way (similarly for iGSM-hard). The difficulty of iGSM-med is already quite non-trivial to humans (at least not solvable with few-shot learning using GPT-4/4o, see Figure 2).
Proposition 2.2. Ignoring unused parameters, numerics, sentence orderings, English words, a- z and A-Z letter choices, iGSM-med still has at least billion solution templates, and iGSM-haro has at least 90 trillion solution templates."
WW choose op non-uniformly; for instance, let op min(to, } for two random draws fo, t E [op]- This ensures that the dataset has more easy data - which makes training faster. (See also similar behavior arithmetics (Jelassi et al. 2023).)
"Due the topological nature of data/solution generation process, reask greatly changes the data distribution the number operations needed. It provides an excellent OOD sample for evaluation. Details are in the full paper.
"1 A solution template created by replacing all numbers with '0', substituting variables (a-z A-Z) with letters their appearance order, and changing parameters to their types (instance or abstract). For instance, "Detine Owl Forest's Elephant as y; so y 11. Define Parrot Paradise's Raccoon as t; so t y 11." becomes "Define Inst as a: so = O. Define Inst as b: so b = 0." We use birthday paradox to estimate the num- her of solution templates. If M randomly generated problems yield distinct templates, it suggests with good probability that the total number of templates exceeds (2M2).

---

Published as a conference paper at ICLR 2025
n-dst
ou (000)
of (OCD)
ww-f--611000 (008)
ndst
out-lldstioo 1O00)
beaml insommll
69. 99.3
92.8 87.9 B40 76 93.6
99.3 99.362-1 89.9 BEE
91
100 90.4 96.8 92.0 90.8
100 592 9332 $1.0 80 05.
boant decample
29.
$2. 88.4
5e. 91.6
120 29 12.4 59.5 84.7
300 22.3
24.2
92.2 ao.3 82.5 21. ead
29.2 p4u4 23.3 20.8 an 85
tg ap
goedy
spodt Peart
opera
ope ao
ope (meaa)
wo
ape
op--28 (reas)
23
opec
op W
opw Sa (reatt)
Figure 3: Test accuracies on the model (tran)trained from the iGSM-med, pq/ /pp and iGSM-hard pa/ap datascts.
S'WNCCe sporation
200
200 TEO
120 E00 EN
TAT
030
316
300
30
T0
5.0
000
0:30
1.00
aar 6,35
wwwceesi wwseermaant masck)
0.99
032
234
033
0.03
130
235
635
6:5
034
336
0443
746
3.07
030
6332
1.31
640
Figure 4: Number of unnecessary params/ / operations used per generated correct solution. Details in full paper.
No data contamination. A goal in synthetic math data generation is to prevent data contamination in internet- based math datasets, as noted in Zhang et al. (2024). While it may be impossible to certify that models trained on internet data are free from contamination, in our setting, we can certify this:
1. We perform OOD evaluation such op > 28 while providing only op S 21 training samples.
2. We train with data whose hash value of solution template (see Footnote 11) is < 17 (mod 23), and test with those 2 17. This ensures no template-level overlap between training and testing-
RESULT 2-3: SUMMARIZE MODEL'S 's BEHAVIOR PROCESS
We use the GPT2 architecture (Radford et al. 2019) but replacing its absolute positional embedding with rotary embedding (Su et al. 2021; Black et al. 2022), yet still referring to it as GPT2 for short 12 We mostly stick to the 12-layer, 12-head, 768-dim GPT2 (a.k.a. GPT2-small) for experiments, but explore larger models in Results 7-8. We use context length of 768 1024 for pretraining on iGSM-med/ /irdM- and 2048 for evaluation. More details are in the full paper.
Result 2: accuracy. After sufficient pre-training, we give the model a problem from the test set (without solution) and let it continue to generate (allegedly a solution followed by an answer). Because we have restricted ourselves to a fixed solution format, language models can learn the format easily, allowing us write a solution parser to check if the solution is fully correct. 13
Figure 3 shows that GPT2 performs well when pretrained using iGSM-med or iGSM-hard data, even when evaluated out-of-distribution on harder (i.e., larger op) math problems. Thus, the model can truly learn some reasoning skill instead of memorizing solution templates."" |4 This could be reminiscent of language models' length generalization capability on arithmetics (Zhou et al. 2023), Jelassi et al., 2023);; however, in our case, op captures the "reasoning length" in grade-school math, and our model has never seen any training example of the same reasoning length as in test time! 15
We also tested with Llama architecture (esp. with gated MLP layers) and did not see major change. GPT2- rotary performs no worse than Llama/Mistral for knowledge tasks (Allen-Zhu & Li, 2025b). We are bounded by resources repeat all experiments in this paper with other architectures that have small differences from GPT2-rotary.
We check not only the correctness of the final answer 0..22 but also the calculations and parameter depen- dencies. Language models can learn very complex syntactics, (Allen-Zhh & Li 2023) and the references therein.
"Llama (of the same model size) gives similar performance, but we refrain from repeating all the experi- ments with another model. We not interested in small model differences in this theoretical study; instead, we care more about the general hehavior of (autoregressive) language models.
''Some others such as Anil et al. (2022) start with transformer pre- are-trained on internet data; while the transformer may not have seen the same task during training, it's possible that the model has seen other tasks with the same (or even longer) length and learned transfer from there.
7

---

Published as a conference paper at ICLR 2025
Such accuracies also indicate that our iGSM data families are indeed good for pretraining purpose, allowing us to investigate further how LLMs can solve grade-school math problems.
Remark 3.1. Our controlled experiment distinguishes between "reasoning length generalization" and "token length generalization". When designing our test data, we ensured that the test data have similar token length compared to the training data (though with longer "reasoning length", see full paper for details). Thus, Figure 3 primarily addresses the model's "reasoning length generalization". For readers interested in "token length generalization" we include this also in the full paper.
Result 3: solution redundancy. We examine whether GPT2 achieves high aceuracy by
brute-forcedly computing all the parameters during generation (a "level-0" reasoning skill), or
computing only necessary parameters to give shortest solutions (a "level-1" reasoning skill).
Recall our iGSM (pretrain) data only contains necessary solution steps (i.e., CoTT to simulate what in textbook solutions for math problems. For instance, if a problem describes X=3+2, E=3+X. Y=X+2 and asks for the value of Y. then a shortest solution would be "X=3+2=5 and Y=X+2=7" without ever computing E.
Figure 4 shows that GPT2 predominantly solves the iGSM problems with a "level-1" reasoning skill, avoiding unnecessary computations, even when evaluated diit----iitiittioonn This finding is significant as it suggests that, unlike humans who usually rely on "backward reasoning" and a scratch pad to write down necessary parameters by backtracking the dependencies from the question (Rips, 1994), the language model can directly generate shortest solutions without using a scratch pad. But, how does achieve so? We shall investigate the next section.
4 RESULTS 4-8?
Due to space limitation, we omit the technical details for Results 4-8 in this ICLR version to en- courage readers to refer our full paper at ssrnn ccm/ abst r2ct= We remark that the full paper underwent the ICLR 2025 review process, but we elected to present this camera-ready version as extended abstract, aligning with the tradition in the theory community.
CONCLUSION
We use a synthetic setting to demonstrate that language models can learn to solve grade-school math problems through true generalization, rather than relying on data contamination or template memorization. We develop probing techniques to examine the models' hidden reasoning processes. Our findings reveal that these models can learn math skills aligned with human cognitive processes, as well as "new thinking processes" not present in the training data. Additionally, we propose a method to predict model's errors before it begins to solve problem and to explain why models make mistakes when they occur. Based on this discovery, we write a separate paper to improve language models' math reasoning accuracy (Ye et al., 2025). We also provide principled approach to connect the model's 's depth to its capable reasoning length. We believe this research opens doors to study the mathematical reasoning skills of language models from a different angle compared to pushing math benchmarks.
One may argue that iGSM may be very different from the pretrain data that modern LLMs use. While this may be true, we are looking into the future. Recall, even GPT-4/4o of today cannot few- shot learn to solve iGSM-meg /iguree 2). From this perspective, it is reasonable to believe that future versions of LLMS will rely on synthetic math pretrain data to improve their reasoning skills. While one may not directly use iGSM, is tempting to turn iGSM into more natural formats while keeping the logical chains. On the other hand, we have discovered that models trained purely the iGSM data make similar mistakes compared to GPT-4/4o (see full paper). This further confirms that our findings do connect to practice, regarding the model's hidden reasoning process.
existing LLMs (such Llama-3)
Finally, Part 2 of this work series focuses on how language models solve grade-school math prob- lems (including Part 2.2 (Ye al. 2025)). We also cover how language models learn language
8

---

Published as a conference paper at ICLR 2025
structures in Part 1 (Allen & Li 2023) (in particular, how they mentally perform dynamical programming), learn world knowledge in Part (Allen-Zhu & Li 2024 2025a b), and how these may impact architecture design (Allen-Zhi) 2025).
6 AN EXAMPLE IN IGSM-HARD WITH OP = 21
(Problem A Hard Esample) The number of ench Jungle Fim's Internttionl Market's Choese equals the sum of each Parmeson Cheese's Pear and each The Fresh Ice Cream's Pincupple equals more than each Gost Cheese equals the sum of each Residential College District's Jungle Jim's International Markel, each Jungle Jim International Market's is Parmesan Cheese and each Residental College District's Supermarket. The cumber each Ans Campus's 5 New Seasons Market equals each Cheese's à Pincupple. The number of each Goat Cheese's Banana equals each Vocational School District's Product. The number of each Ressdenial College Distnt's Jungle Jim's Internaiional Market equals more each Cream's Grape. The number each Parmesan Cheese's P'incupple equals
Market's Cream. The
Grape.
each New Seasons Market's Gout
Residentaal College District's The Fresh Market equals each Arts Cumpus' s Trader Joc's. s. The number of each Arts Campus's Trader Joe's equils each Parmesan Cheese's Ingredient. The number of each Goal Cheese's Grupe equals 0. The number of each The Fresh Market's s Ice Cream equals 13 more than the difference of each Residential each New Semson Market's Product, College District's The Fresh Market and each Parmeson Cheewr's i Grape. The mumber of each Gont Cheese's Pincapple The Fresh Market's Cheese. The sumber Cheese
The mumber
equals of each Trader Ioe's i's Cheese equals The number of each The Fresh Market's Chrese equass 3. The number of each Jungle Jim's Insernational Market's loe Cromm equils the difference of each lor Cream's x Banana and esch Guat Chrese's is Grape. The number of each lungle Jim i Internaiional Markelss 'i Parmesan Choese equals Ice Cream's Pincupole. The number of each Parmesin Cheese's 's Peur equil difference each Goat Cheese's is International
number of each Vocational School District's
Fresh
sum of each Trnder
and each lce Cream's Graps. number
Grape equils each The Fresh Market's ss Parmesan Cheese equals each The Fresh Market's Cheese, The number of each Ice Cream's Hanana equiss the sum of each Parmesan Cheese's Pineupple and each Ice Cream's à Pincupple. The number of each School Districtss Jungle Jim's s Internatonal Market equis each The Fresh Market's Ice Cream. The number each Cheese's ' Pineupple equals 20 more than sum of each Trader loe's s's Cheese and each The Fresh Market's Cheese. The number of Ice Cream's Pear equals The number each lee Cream's Grape equals each Goat Cheese's
The number each Parmesan
as much each Residential College District's Jungle
each Trader Joe's loe'ss Pummesan Cheese equals 16. number
Grape. How mony Product does School Distric have?
(Solution- Hard Example) Define Goot Cheese's Grnge sau Define Cream's Grupe as it so mu D. Define Residential Cilllege District's langle
Jim's International Market as N: so 5 + 5 O= S. Defie Parmesan Cheese's Dear as G; so = u- ==0===0- 0= Delie Parmeson Cheese's as Grape us f. so (= 12 2 N 12 Define Campus's Trnder Joe's
Deline Parmessn Choese's Pineupple C. C G Define Purmosan Choese's Ingredient
= f+ = 14 0=
Z
ss soj = 14. Deline Creum's Pinespple ts X: == 2=2. Define loe Cream's Danana K: so K +C 2=2 Deline The Fresh Market's loe Cream as P: P.i=j- ==
s 7- Define Resiemial College Distret's The Fresh
14- 14 = so P= 13 13 13 13, Deline Jungle lim's Internaioonl Market's is's Toe Cream as R: so = un 2--=2222 2-0= 2. Duline School Distrit's is Jingle Jim's
International Market V: so V VPPPP --- 13, Define lingle Jim's Interational Marke's ' Choese as so G P-D D+ 13- Define lingle Jin's 's Intemational Market's Parmesan
so
lungle Internaioonal Market's Product U= R=25+===+2 2= =
17, Define
School Disrict's Product as J. so y= 17= Answrr
Structure Graph
Dependency Graph
Product
Furmose
Ingrodent
depeninary
Figure 5: An example with op 21 in iGSM-hard,,, used for training. Don't forget during testing evaluate models on op 32 which is even much harder.
REFERENCES
Zeyuan Allen-Zhu. Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers. SSRN Electronic Journal, May 2025. doi:
https:
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 1, Learning Hierarchical Lan- guage Structures. SSRN Electronic Journal, May 2023. https:: ssrn. com/ abstract=
5250639
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.1, Knowledge Storage and Extraction. In Proceedings of the 41st International Conference Machine Learning, ICML 2024, 2024. Full version available at wttttts://// //tp///s./.../////////. com/ abst 5200033353333333
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.2, Knowledge Manip- ulation. In Proceedings of the 13th International Conference on Learning Representatiinss, ICLR 2025, 2025a. Full version available at https:://ssstt http://cto/otttt///a//...

---

Published as a conference paper at ICLR 2025
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws. In Proceedings of the 13th International Conference on Learning Representattinss, ICLR 2025, 2025b. Full version available at httpp:: ssrn. com/ abstract=
Zeyuan Zhu and Xiaoli Xu. DOGE: Reforming AI Conferences and Towards a Future Civ-
ilization of Fairness and Justice. SSRN Electronic Journal, February 2025. doi: 10.2139/ssm.
5127931. hhttps://sst /ss.n. com ats/tbbtttttttt1tt
Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. convergence theory for deep learning via over- parameterization. In ICML, 2019, Full version available at
03962
Cem Anil, Yuhuai Wu, Anders Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Ramasesh, Am- brose Slone, Guy Gur-Ari, Ethan Dyer, and Behnam Neyshabur. Exploring length generalization in large language models. Advances in Neural Information Processing Systems, 35:38546-38555 2022.
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Ho- race He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. GPT- NeoX-20B: An open-source autoregressive language model. In Proceedings of the ACL Work- shop Challenges & Perspectives in Creating Large Language Models, 2022. URL https: //arxiv. org abs/ 2204 06745
Sébastien Bubeck, Varun Chandrasckaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Ka- mar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint 2rX1v:2303.12712, 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.44166 2021.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprini arXiv:2101 00000,0002220 2020.
Suriya Gunasekar, Yi Zhang. Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arXiv preprini arXiv:2306. / 1644, 2023.
John Hewitt and Christopher D. Manning. A structural probe for finding syntax word representa- tions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computationall Linguistics: Human Language Technologies, Volume I (Long and Short Pa- Minncapolis, Minnesota, June 2019. Association for Computational Lin- guistics. doi: (0.18653/v1/NI 1419. URt.https: / / ac antholoohyyl orw.gggyoorrooooo /Nrg/NT /N1 9-14-1111
pers), pp.
Samy Jelassi, Stéphane d' d'Acco,,, Carles Comingo-EEnichh Yuhuai Wu, Yuanzhi Li, and François Charton. Length generalization in arithmetic transformers. arXiv preprint arXiv:2306. 15400, 2023.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint (rXiv:2001.08361 2020.
Nayoung Lee, Kartik Sreenivasan, Jason D Lee, Kangwook Lee,
Lee, and Dimitris Papailiopoulos. Teaching arithmetic to small transformers. arXiv preprint arXiv:2307.03388/ 2023.
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 1.5 technical report. arXiv preprint XXiv:23090054633 2023.
Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. TinyGSM: achieving > 80% GSM8K with small language models. arXiv preprint arXiv:2312.09221, 2023.
OpenAl. Gpt-4 technical report, 2023,
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9 2019.
Lance Rips. The psychology of proof: Deductive reasoning in human thinking. Mit Press, 1994.
10

---

Published as a conference paper at ICLR 2025
Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. 2021.
Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physics of Language Models: Part 2.2, How to Learn From Mistakes on Brade-School Math Problems. In Proceedings of the 13th International Conference Learning Representations, ICLR 2025, 2025. Full version available at huhhtps:/ ssrn. com/ abst ract=
Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Dylan Slack, Qin Lyu, et al. A careful examination of large language model performance on grade school arithmetic. arXiv preprint arXiv:2405.0033: 2024.
Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, and Preetum Nakkiran. What algorithms can transformers learn? study length generalization. arXiv preprint arXiv:2310. /6028, 2023.


Reviews:
[{"review_id": "BMK4W0S5eR", "reviewer": "Reviewer_yuz7", "summary": "This paper attempts to measure the reasoning ability of models by training on a synthetic dataset based on GSM8k. Among other things, they show that training from scratch on this synthetic dataset of grade school math has some signs of generalization to harder problems.\n\n\n\nEdited: score to 6 after reviewer rebuttal.", "questions": "Q: My big question is whether this is a proper use of generalization, especially with LLMs, which are useful primarily because they generalize to very different kinds of data. \"Generalization\" from a synthetic dataset to a similar synthetic dataset is not nearly as ideal as generalization to a very different kind of dataset.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Measuring reasoning via training an LLM from scratch is an interesting direction. There are lots of interesting results and experiments in the paper.", "weaknesses": "I think the writing can be improved somewhat. Additionally, I think there are some questions about whether the findings are broadly applicable given a single synthetic eval on GSM8k. In some sense, LLMs are useful because they generalize widely beyond a small set of very specific problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9g3XAiyfWg", "reviewer": "Reviewer_UB35", "summary": "This paper dives deep into how language models solve grade-school math problems. The authors first design an elaborated data generation pipeline to synthesize a large amount of grade-school math problems with a specific form of Chain-of-Thoughts. Then the authors design a probing method to analyse how the language models solve the problems, and acquire some interesting conclusions.", "questions": "1. **Lack of analysis on data quantity**: See the second point in Weaknesses.\n2. **About the backward thinking process**: In line 447-453 of this paper, the authors define the “backward thinking process” as “because I want to compute X, but X depends on Y and Y depends on Z, so let me compute Z first”. Moreover, they conclude that this backward thinking process can be autonomously learned through language modeling with abundant data. **I can't understand how the author came to this conclusion**. I cannot understand how the authors arrive at this conclusion.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper carries out a substantial experiment (including large-scale data synthesis, fine-tuning, and probing) to systematically research the reasoning ability of LLMs in grade-school math problems.\n* To understand how the metal process of language models, the paper proposes a few **probing tasks** which align with human problem-solving strategies. Such probing tasks might be insightful to other works.\n* Multiple interesting conclusions are found in this paper.", "weaknesses": "* **Limitation of specific CoT form**: This paper trains the GPT-2 model using synthetic data with a specific form of CoT. The fune-tuning and probing experimental conclusions are based on such CoT. However, the CoT that is now used to train the LLMs on grade school math problems deviates considerably from this form. This limits the generalizability of the paper's conclusions.\n* **Lack of analysis on data quantity**: Although this paper synthesizes a large amount of data to train a language model, some studies have shown that more training data is not always better for language models. I expect to see a curve or table of [model performance - the quantity of training data].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZlbwGwggF4", "reviewer": "Reviewer_GM4V", "summary": "This work conducts controlled experiments to probe several fundamental questions about math reasoning capability of language models, the experimental results show that language models can solve math problems like humans and conduct backward thinking process.", "questions": "Since IGSM only has four categorizations, which may result in poor diversity in the description of quantitative relationships. Could this affect the generalization of the experimental conclusions of this paper?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- This work introduces a framework to generate a set of diverse GSM problems that focus on “logical reasoning” aspect.\n- This work finds that “the model has learned to plan ahead, identifying necessary\nparameters before starting to generate the solution”.", "weaknesses": "- The small model size of GPT2-small (100M-200M) may effect the generalization of the experimental conclusion. It would be appreciated if conducted experiments on LLMs with billion parameters.\n- The conclusions drawn from experiments on synthetic data may not be applicable to real-world scenarios with large data volumes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6VuPUJooaq", "reviewer": "Reviewer_Arcv", "summary": "The paper investigates the capability of language models (the authors used the GPT-2 model for their experiments), to solve grade-school math problems to explore whether models genuinely develop reasoning skills beyond template memorization. They address several key questions: Can LMs generalize reasoning skills? Do they replicate human-like problem-solving methods? Are their errors predictable, and does model architecture (depth versus width) influence reasoning abilities?\n\nIt is important to control the dataset to explore these questions. For this, the authors generate a synthetic dataset with high template diversity, reducing contamination and ensuring complex, logical dependencies in problems. Results indicate that the model can indeed generalize reasoning processes and even learns dependencies and plans solutions. The paper explores the models internal states through probing to understand how the model is solving the problem. \n\nOverall, the paper is highly experimental with the goal to identify the factors that enable a model to develop reasoning skills through systematically controlled experiments.", "questions": "It would be interesting to see some results on real-world datasets like GSM8K to see how much the models can generalize to real-world problems when trained on synthetic datasets.\n\nIs it possible to test the models on similar synthetic datasets like mathworld (https://arxiv.org/abs/2306.04347)?\n\nModel errors and how LLMs can fix them (Result 6) needs to be discussed in the paper and not left saying its there in another paper.\n\nThe results of the backward thinking process are missing from the paper and saying that the models learned it without any discussion is not correct. It needs to be added to the paper. \n\nFinally, it would be interesting to see an analysis of when a model learned a particular reasoning skill during training. And can it be learned in a continuous training fashion?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The biggest strength of the paper is the controlled experiments that the authors conducted. Training a GPT-2 model from scratch with their own synthetic data can give them full control over the experiments. Also, generating their own data can prevent contamination of the training test data, which might be one of the problems in the other benchmark data. \n\nThe authors also introduce the probing tasks to explore the internal states of the model, which gives some idea of the reasoning process of the model. In particular, the V-probing approach is interesting because it can tell whether pre-training or fine-tuning was responsible for the probing signal. \n\nFinally, the paper presents many interesting results, including the depth vs. width of the models, the generalizability of the model to unknown tasks of greater depth, models learning backward reasoning processes like humans, and so on. These results (if they can be scaled) may allow better design of language models in the future and may help to understand the role of synthetic data in learning a concept.", "weaknesses": "The weakness of the paper also lies in the main choice of experiments: the creation of the dataset and the choice of the model. \n- Regarding the dataset, the use of synthetic data may not fully capture the complexity and nuances of real-world mathematical problems. This limitation could affect the generalizability of the results to practical applications. It would be important to mix and match the training dataset with real-world data or test the methodology on some real-world data such as GSM8K.\n- Regarding the models, although it is difficult to test the methodology on multiple models, the model performance scales well with depth is kind of a known fact and with more layers, the performance improves creating a question that if scaling leads to memorization?\n\nThe use of math word problems is a good choice to control the experiments, but the question is if the results are limited to a structured reasoning problem like math word problems? This is important to know in order to understand the limitations of the approach. \n\nThe analysis of the model errors was omitted from the paper, saying that the authors are writing another paper because it is not good for the current paper. It would be good to see the error analysis. At least a summarized version needs to be presented. \n\nFinally, since the training of the model was done by the authors, it would be interesting to see in which training iterations the model learned certain skills. This can easily be discussed in the paper, but I found it missing. \n\n[MINOR] The use of words like AGI while experimenting with synthetic datasets and the GPT-2 model is not correct and can be avoided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tW0lwbC54S", "reviewer": "Reviewer_35Ln", "summary": "This work focuses on evaluating the grade-school level mathematical reasoning of neural language models (GPT models).\nThe authors set out to explore several critical questions surrounding the actual abilities of LMs to reason mathematically.\nTo make clear and controllable experiments, the authors introduces a synthetic dataset iGSM.\nUsing iGSM to train and test GPT-2, the authors argue that the model can truly understand and reason through problems rather than just memorizing instances.\nThe examination of the causes behind reasoning mistakes made by LMs is another highlight, shedding light on the limitations and potential areas for improvement.", "questions": "1. Can you provide some evidence to support the generalizability of your conclusions, even some of them?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Overall, the exploration of the model's hidden reasoning process provides valuable insights into the black-box nature of neural language models.\n2. This work introduces a new synthetic dataset iGSM which could facilitate the training and evaluation of contemporary large language models.\n3. This work provides many interesting findings. It provides a valuable addition to better understanding the reasoning capabilities emerged in large language models.", "weaknesses": "My major concern is about the generalizability of the conclusions drawn in this work.\nIt is unclear whether the experimental findings on formal language reasoning can be generalized to natural language reasoning.\nThe formal language patterns in iGSM are too much limited and inflexible compared with natural language expressions, although it may contain a large number of templates if omitting the primitives in the expressions.\n\nFor instance, for the math operation *multiplication*, there are many different ways to express using natural language, e.g., \"there are three children in the house and each child eats two eggs for breakfast\".\nBut in iGSM, it seems that *multiplication* is always mapped from \"times\" in the input expression.\nI think for grade-school level mathematical reasoning, one major capability is to map the flexible natural expressions to calculations.\nTherefore, I think reasoning on iGSM could be much easier than reasoning on natural language questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)