Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Authors: Roman Abramov
Felix Steinbauer
Gjergji Kasneci
Venue: ICML 2025 poster (2025)
Keywords: multi-hop reasoning
transformers
grokking
knowledge graphs
data synthesis
generalization circuits
factual reasoning
NLP
synthetic data augmentation.
Primary area: deep_learning->large_language_models
TL;DR: We extend grokking to real-world factual reasoning by augmenting knowledge graphs with synthetic data, enabling near-perfect multi-hop reasoning performance.

Abstract: Transformers have achieved great success in numerous NLP tasks but continue to exhibit notable gaps in multi-step factual reasoning, especially when real-world knowledge is sparse. Recent advances in grokking have demonstrated that neural networks can transition from memorizing to perfectly generalizing once they detect underlying logical patterns -- yet these studies have primarily used small, synthetic tasks. In this paper, for the first time, we extend grokking to real-world factual data and address the challenge of dataset sparsity by augmenting existing knowledge graphs with carefully designed synthetic data to raise the ratio $\phi_r$ of inferred facts to atomic facts above the threshold required for grokking. Surprisingly, we find that even factually incorrect synthetic data can strengthen emergent reasoning circuits rather than degrade accuracy, as it forces the model to rely on relational structure rather than memorization. When evaluated on multi-hop reasoning benchmarks, our approach achieves up to 95--100\% accuracy on 2WikiMultiHopQA -- substantially improving over strong baselines and matching or exceeding current state-of-the-art results. We further provide an in-depth analysis of how increasing $\phi_r$ drives the formation of generalizing circuits inside Transformers. Our findings suggest that grokking-based data augmentation can unlock implicit multi-hop reasoning capabilities, opening the door to more robust and interpretable factual reasoning in large-scale language models.

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Roman Abramov
Felix Steinbauer!
Gjergji Kasneci
12
Abstract
1 Introduction and Related Work
Transformers have demonstrated remarkable success across wide range of natural language processing (NLP) tasks, such text classification, summarization, and machine translation. Nevertheless, they still face significant chal- lenges when asked to perform multi-step or multi-hop fac- tual reasoning, particularly real-world scenarios where knowledge both vast and sparsely distributed. key reason for this difficulty lies in the model's tendency to memorize rather than generalize problem that becomes acute in -eteeeiiiiiievve tasks with insufficiently rich data distributions.
Transformers have achieved great success in nu- merous NLP tasks but continue to exhibit notable gaps in multi-step factual reasoning, especially when real-world knowledge is sparse. Recent ad- vances in grokking have demonstrated that neural networks can transition from memorizing to per- fectly generalizing once they detect underlying logical patterns- - yet these studies have primarily used small, synthetic tasks. In this paper, for the first time, extend grokking real-world fac- tual data and address the challenge of dataset spar- sity by augmenting existing knowledge graphs with carefully designed synthetic data to raise the ratio 6, of inferred facts to atomic facts above the threshold required for grokking. Surprisingly, we find that even factually incorrect synthetic data can strengthen emergent reasoning circuits rather than degrade accuracy, as it forces the model to rely on relational structure rather than mem- orization. When evaluated on multi-hop reason- ing benchmarks, our approach achieves up to 95- 100% accuracy on 2WikiMultHHoOQA substan- tially improving over strong baselines and match- ing exceeding current state-of-the-art results. We further provide in-depth analysis of how increasing o, drives the formation of generalizing circuits inside Transformers. Our findings sug- gest that grokking based data augmentation can unlock implicit multi-hop reasoning capabilities, opening the door more robust and interpretable factual reasoning in large-scale language models.
96.0%
$9.0%.
80-
70
59.0%
randem
grokked GPT2-small
GPT2-small
GPT-40
01-mini
Figure 1. Average accuracy on 2WikiMulilHopQA for comparison task. Despite GPT2-small being a model with 124 million parame- ters, grokked version achieves almost 100% accuracy, beating the most recent gpt-4o and ol ol-mini models.
From Toy Grokking to World Data. Recent work on grokking (Power et al., 2022) has shown that, under certain conditions, overparameterized neural networks sud- denly transition from pure memorization to near-perfect generalization after long training. Early studies have typi- cally focused highly controlled, synthetic tasks such as modular arithmetic simplified algorithmic datasets. In these toy settings, the number of inferred facts (i.e., multi- step or composed patterns) can be systematically increased until a threshold ratio & is reached, at which point a "gener- alizing circuit" emerges in the model (Belkin et al., 2019; Nakkiran et al., 2020; Thilak et al., 2022; Humayun et al., 2023; Nanda al., 2023).
'School of Computation, Information and Technology, Technical University of Munich, Munich, Germany "School of Social Sciences and Technology. Technical University of Munich, Munich, Germany. Correspondence to: Roman Abramov <roman. abrmmnn. @tum.de>, Felix Steinbauer <fc- ww.steinbaacrrrum.de:
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
However, real-world datasets present a stark contrast: fac-

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
tual knowledge is extremely sparse and often scattered across incomplete noisy knowledge graphs. Thus, the core challenge is to ensure that there are enough higher- order inferred facts in relation to the atomic facts (direct statements) to enable the internal circuit-formation process that grokking requires. Put differently, in real-world sce- narios, one cannot trivially guarantee a sufficiently large ratio & between multi-step (inferred) facts and single-hop (atomic) facts. Our work addresses precisely this obstacle by proposing data-synthesis strategy that augments and re-balances real-world knowledge bases.
intermediate reasoning steps must be spelled out in ex- ternal text (Plaat al., 2024), rather than learned as an implicit circuit.
Use standard graph-completion architectures: e.g., GNN-based solutions to augment partial knowledge graphs (Liu et al., 2022), which do not necessarily yield late-phase internal circuit formation or sudden generaliza- tion.
To the best of our knowledge, no existing work has used a grokking- baased approach demonstrate how Transformer can implicitly discover multi-hop reasoning skills on large- scale factual data.
Multi-hop Question Answering (QA). 2WikiMulti- HopQA (Yang et al., 2018; Ho et al., 2020; Trivedi al., 2022) is particularly well-suited to assess multi-step factual reasoning. This dataset contains Wikipedia-based queries that require retrieving and combining multiple pieces of evi- dence (spread across different pages or paragraphs) before producing an answer. This property aligns closely with our focus on multi-hop reasoning and underscores the need for implicit reasoning capability: in many cases, a system must link and traverse several factual nodes e.g., "Michelle is
Contributions.
In this paper, we bridge that gap through the following contributions:
We incorporate a targeted data synthesis procedure to ensure sufficiently large @ for each relation, thereby un- locking the potential for internal generalization circuits to form in real-world Wikipedia-basee tasks.
the wife of Obama" and "Michelle was born in 1964" - to
We show that even factually incorrect synthetic data can boost the ratio of inferred to alomic fucts, often strength- ening rather than harming logical consistency.
arrive at final conclusion (e.g., about Michelle's birth year or other derived facts). Moreover, they reflect large, com- plex knowledge graph with real-world entities, ambiguous language, and long-tail relations- - all of which make them an archetypal testbed for factual reasoning at scale.
Our experiments on 2WikiMultHHopQA confirm that once the ratio 6 surpasses a certain threshold, grokking emerges - enabling Transformers to perform complex multi-step reasoning without explicit chain-of-thought prompts or elaborate external scaffolding.
Grokking and Its Role in Transformer Generalization. The concept of grokking, introduced in Power et al. (2022). demonstrated that neural networks could learn not just su- perficial patterns but also deeper, generalizable reasoning mechanisms under prolonged training with suitable induc- tive biases. Subsequent work has linked grokking to double descent (Belkin et al., 2019; Nakkiran et al., 2020), the geometry of deep-network loss landscapes (Davies et al., 2023), and weight decay (Pezeshki et al., 2022: Nanda et al., 2023), suggesting that the right regularization can encour- age the emergence of generalizing circuits. These circuits once formed- enable out-of-distribution reasoning that surpasses naive memorization (Varma et al., 2023; Liu et al., 2023).
In the following sections, we detail the mathematical basis of our data augmentation strategy, the empirical setups on 2 2WikiMultiHopAA, and the new insights gained about im- plicit multi- hop reasoning. Our findings show that grokking is not an artifact confined to contrived toy datasets but a powerful mechanism that, with suitable data distribution ad- justments, can be harnessed for real-world factual reasoning at scale.
2 Problem Description
The concept of multi-hop reasoning presupposes knowl- edge graph (KG) whose nodes (entities) and edges (rela- tions) can be traversed via chain of inference steps. In our setting, this KG is encoded in textual form, but structurally, we are still dealing with multi-hop question answering over KG (Multi-hop KGQA). Prior work (Liu et al., 2022) has already observed that knowledge graph completion can be critical for multi-hop KGQA; for grokking- based Trans- former generalization circuits to form, it becomes impera- tive to extend the original KG such that sufficiently many multi-step (inferred) facts exist. This section formalizes our problem of augmenting a KG to enable Transformer grokking.
Gap in the Literature. Despite extensive research on knowledge graph completion (Liu et al., 2022) and multi- hop question answering via retrieval-based methods (Yang al., 2018; Ho et al., 2020), very few studies have examined whether the internal grokking phenomenon can be harnessed for implicit multi-hop reasoning in a real-world textual setting. Most prior approaches either:
Focus on toy tasks: e.g., modular addition or synthetic math problems (Power et al., 2022; Nanda et al., 2023; Wang et al., 2024).
Rely on explicit prompting or chain-of-thougle where
2

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
2.1
Definitions and Basies
wife of
sister of
We begin by introducing key notations for clarity. Readers can refer to Table at any time for a concise summary of the main symbols.
Obama
Michelle
Bob
mother of
born in
"Mary Poppins
aired in
1964
Alice
Knowledge Graph.
Definition 2.1 (Knowledge Graph), We define a knowledge graph as a tuple KG (V. FA), where
Figure 2. Exemplary Knowledge Graph with Synthesized Data. Four original nodes (black) and three relations (blue) result two inferred facts. Two additional synthetic nodes and relations (red) extend the amount inferred facts by four. Consequently, 1.2.
is a finite set of entities (nodes or vertices),
is finite set of relation types (edges/predicates), & FA V R V is finite set of atomic facts (triplets) of the form (h, t), where h E V is head (subject), E V is tail (object), and r E R is the relation.
ΦI = Φ2 increases from
aan ~ 0.66
successor node (no branching ambiguity):
Vr; E Pn 30 1, E such that 7 (0;-1, = Vuz E V: (1(0)-1,))) = = = Uj v.).
In natural language, entities are typically connected by rela- tional statements. Although some relations (e.g., "son of") are directed, one can also treat the KG as undirected for graph traversal since a statement is often queryable in both directions (subject/object)
Definition 2.7 (N-th Order Deductions). For an inference path Ps = (ri,...,'') rrx) of length n connecting to (head) to Un (tail), the n-th order deductions are:
Fn S (E.. un) vs V.r.ER).
Definition 2.2 (Norm over edges). For counting strictly directed edges in KG as if they were undirected, we use the (t.....t)E 1.
Example 2 (2-Hop Deductions), second-order (2-hop) fact arises from concalenating two atomic facts via one bridge entity. For instance:
trivial count: |
Definition 2.3 (Average branching factor). The average branching factor of knowledge graph is: -
(h, r1, b, r2,t) with (h, (b. r2,t) E
In Example 1, we might ask: "Which year was Obama's wife born in?" 1("Obama", "wife of "Michelle" and I( I("Michelle", "born in "1964".
Definition 2.4 (Atomic Facts). Following prior work, we use FA (or equivalently Fi) to denote the set of atomic facts, i.e., all first-order triplets explicitly stored the knowledge graph.
Inferred vs. Atomic Facts.
Example (Running Example: Basic KG). Consider the KG in Figure 2 with
Definition 2.8 (Inferred Facts), All n-th order facts with > 1 constitute the inferred facts,
V ["Obama,, "Michelle", "1964", "Mary Poppins"),
20
R = { (wwiee of", "born in", "aired in").
Fi
U Fr
The atomic facts FA include:
Clearly, FANFI 0.
("Michelle", "wife of", "Obama").
Example (3-Hop Deduction). Continuing the example of Obama, consider the path Ps ("wife of"" "born in", "aired in"). By chaining three steps, we deduce:
("Michelle", "born in", "196""))
("Mary Poppins", "aired in", "I964").
Hence, Fi = FA. The average branching factor here is b 0.75.
("Obama", "wife of",, "born in", "aired in", "Mary Poppins")
which answers "Which movie aired in the same year Obama's wife was born?"
Inference Steps and Paths.
Definition 2.5 (Inference Step). An inference step is func- tion / V V that traverses the graph from one entity to neighboring entity via one relation:
2.2 Generalization Over Inference Paths
Definition 2.9 (Implicit Reasoning). We define implicir reasoning as reasoning without the need for explicit inter- mediate prompts developer-introducee structure.
T(h, r) = where rh, FA.
Definition 2.6 (Inference Path). An inference path P., of
Unlike explicit reasoning approaches (e.g.. chain-of-thoon -ftthouhht prompts (Plaat et al., 2024)), implicit reasoning relies on the model forming internal circuits during training (Nanda
length n is a sequence relations P.. = (r,,..., Tw) E R".
The path is simple if each relation leads to exactly one
3

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
al., 2023). Grokking studies (Power et al., 2022; Wang al., 2024) have shown that Transformers can, under certain conditions, shift from memorizing to perfectly generalizing by learning these internal circuits.
@G Ri 3.6. Hence, the KB remains not fully generalizable, preventing Transformer from forming robust multi-hop circuits for all relations.
2.4 Setup and Querying
Relation-Speeiffe Ratios. As noted by Wang et al. (2024), critical factor for circuit formation is the ratio of inferred facts atomic facts for each relation r. Let
In practice, we test a model's multi-hop reasoning by asking queries whose unique answer resides at the end of a simple (acyclic) inference path. For instance:
FA,- S
FA and
Fr.- S Fi
Example 5 (Chaining 3 Steps). The textual query "Which movie aired in the same year as Obama's wife was born?"
denote the sets of atomic and inferred facts that involve relation r. Then:
corresponds 3-hop deduction:
Definition 2.10 (Generalization Ratio), For each relation
("Obama", "wife of",, "born in", "aired in ,t).
r E R, we define o
Ft.r
Here, the model must internally infer:
FA.r
T(" Obama", "wife of") - "Michelle", T I("Michelle", "born in " "1964", I("1964", "aired in") t "Mary Poppins".
When dr crosses a certain threshold-empiriees found to be around 3.6 for slow generalization and up to 18 for faster circuits in GPT-2 style Transformers- the model tends to form generalizing circuit for that relation et al., 2022). These thresholds are approximate and architecture- dependent (Nanda et al., 2023; Wang et al., 2024) but illus- trate the core principle: without sufficiently many multi-hop facts, the model never "grokks" the underlying relation.
Successful implicit reasoning requires the model to both memorize the atomic facts and chain them together via a generalizing circuit.
AF
AF
AF
AF
AF
AF
Partial vs. Full Generalizability. Even if a knowledge base is only partially rich in multi-hop data (i.e., some relations meet the threshold da. while others do not), it can still benefit certain reasoning tasks. However, to achieve full generalizability, all relations must surpass the threshold:
IF
ID
IF
OOD
Definition 2.11 (Generalizable Knowledge Base). A knowl- edge base ICG is generalizable if
Figure 3. Conceptual difference between ID and OOD: In- distribution (ID, 3orange) and Oft-s--distiibution (OOD, red) inferred facts are shown. All green components are seen during training, including all atomic facts (AF) and some inferred facts (IF).
Vr E R. or 2 éG.
where de is the minimal generalization ratio required by given model setup. If this condition holds for only subset of relations, we call KG partially generalizable.
Definition 2.12 (In-distribution vs. Out-of-distribution). In-distribution (ID): An inferred fact derived from com- binations of atomic facts that were present in the training data but never appeared together in this specific combination. The model has seen all knowledge components separately and different reasoning combinations involving them, but not in this particular test arrangement.
2.3 Bounds for Generalizable Knowledge Bases
Because each relation r has its own ratio é,, a knowledge base (KB) may fail to support grokking if any or remains too low. Analytic bounds (see Appendix A.3) reveal that:
The ratio & can be increased by adding new nodes or by augmenting edges related to r, but this effect limited by how the graph is connected and how often the relation r appears.
-usto--iisttibution (OOD): An inferred fact derived from atomic facts that were present in the training data but never used in any train reasoning paths. The model has seen the individual knowledge components, but not how they should be applied in the context being tested.
Typical real-world KGs tend to be sparse, which is why data synthesis (described in later sections) is crucial to boost ó,.
Figure 3 shows trained facts (atomic and inferred) related to ID and OOD testing facts.
Example 4 (Insufficient Branching Factor). Suppose a family KG has [father, ffather,mother, child]....] with edges like "parent of." "sibling of." "owned by." Although the graph is well-connected at a glance, its relation relation-specific branch- ing factors might still be too small to pass the threshold
As we show in our experiments. OOD queries can be espe- cially challenging, requiring truly structural generalization rather than partial memorization. This is precisely where sufficiently high o, ratios become critical for enabling the

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Transformers' grokking-driver jump from local memoriza- tion robust multi-hop reasoning.
Table I. Key Notation Table. Frequently used symbols through- out this paper.
2.5 Lemmas and Bounds
Symbol
Meaning
The first lemma we introduce makes clear that (1) picking nodes vs. arranging them in paths vs. the existence of edges leads to asymptotic upper bound, and (2) even large KBs do not grow on.r beyond roughly 6a-1 without additional data augmentation.
V
Set of entities (nodes) in the KG
R
Set of relation types (edges)
FA
Atomic facts (1-hop) S V x R x V
Fr
Inferred facts (multi-hup) U..2 Fr
b
Average branching factor |> !FAl
Fraell (ratio of inferred to atomic facts)
dr
Lemma 1 (Asymptotic Bound on the Number of n-hop Paths). Consider a knowledge base (KB) with [V] entities, average branching factor b, and an (approximate) random- graph assumption that each potential directed edge between two distinct nodes is present with probability M Then, for large (V), the expected number of valid n-hop paths satisfies:
FA.yl
dG Minimal generalization ratio for successful grokking
I(b,r)
Inference step: from entity h via relation neighbor
grokking (Power et al., 2022; Wang et al., 2024).
3.1 Dataset
2WikiMultiHoPQA Overview. We use the 2WikiMulti- HopQA dataset (Ho et al., 2020), a well-known benchmark for serieval-auggennt generation (RAG) and multi-hop QA. 2iikMuliiHoaA consists of Wikipedia paragraphs, supporting facts (triplets), and reasoning queries that often require chaining multiple pieces of evidence. Despite its breadth, the initial ratio á = 0.5 that is, each two atomic fact spawn only one inferred fact making it insufficient for grokking emerge naturally.
| (IV) 12 n+
(n + 1)! (
FF
Moreover. in the limit [V| - oc, the relation speciffc ratio &nr (i.e,, the ratio of n-hop to 1-hop facts for relation r) remains bounded above by ba-1
For Proof of Lemma I see Appendix A.I. For lower gen- eralization bound on the number of nodes [V], see Appendix A.3. The second lemma makes clear that each relation's 's branching factor b, must be sufficiently large to surpass the empirical threshold ΦG. If one or more relations fall below that threshold, full generalization across all relations will not occur even though partial generalization might emerge for the higher- b, relations.
Structured vs. Unstructured. We divide 2WikiMulti- HopQA into two subsets:
Structured: supporting facts are simplified into short triplets (e.g.. Paris -- country -- France). Unstructured: supporting facts are embedded in full Wikipedia paragraphs, offering richer context but also more noise and complexity.
Lemma 2 (Necessary Condition for Full Generalizabil- ity). Let &G be the minimal ratio required to trigger grokking-baat baasen generalization for given model architec- ture. A KB is fully generalizable over n-hop facts only if
Within these subsets, we focus on two of the four main multi-hop tasks:
1. Comparison, which compares attributes of different en- tities (e.g., same location or release year).
V
Φa MI - 1)" (141 (n 1)!
2. Composition, which chains multiple relations derive answers (e.g., who directed the sequel of certain film?).
onr 2 OG 7 b,
Our objective is to raise & for both comparison and com- position queries so that Transformers can develop internal reasoning circuits.
where by -1Aml is the relation-specific branching factor. [V]
Equivalently, if any by falls below this threshold, the KB cannot be fully generalizable.
3.2 Data Augmentation
For a Sketch of Proof of Lemma 2 see Appendix A.2.
To increase the coverage of multi-hop questions, we sys- tematically add both atomic and inferred facts via LLM- based generation. In each case, we ensure consistency with 2 2WikiMultiHoQAA style, maintain balanced class distribu- tions, and closely track Φ so it exceeds known thresholds for grokking (Wang et al., 2024).
3 Method
Our method follows a two-stage pipeline designed to en- able grokking in real-world multi-hop reasoning tasks. First, we augment the original Wiki2Hop dataset with synthetic knowledge, increasing the ratio of multi-hop (inferred) to single-hop (atomic) facts. Second, we train a Trans- former model for hundreds of thousands of steps, leveraging the late-phase generalization phenomenon characteristic of
3.2.1 COMPARISON TASK
In the comparison task, atomic facts (e.g., City country X) are paired to form questions about
5

---

Grokking in the Wild: Data Augmentation for Real- Heal-World Multi-Hop Reasoning with Transformers
Algorithm 1 Augmentation algorithm for comparison
Algorithm 2 Augmentation algorithm for composition
Require: ioc_examples // sample atomic facts
Require: text original atomic facts textual form
Require; detailed.examples // extended paragraphs
1: graph + parse graph(text)
I: atomic + gemerate _enerrat peaattioossoocccaaplllele
2: atomic + trrahhaagment_saae
2: if task type== == "full text" then
3: inferred < traphhanggment inferred()
3: atomic detalize locations(atomic, detailed examples)
4: inferred diivessfyyinferree
4: end
5: return atomic, inferred
5: inferred 6-
6: return atomic, inferred
Example of Augmented Facts. Table 2 provides a sam- ple of how an atomic fact, detailed version, and in- ferred question look after augmentation. Note how the final question elegantly ties both locations to a yes/no query on whether they share the same country.
shared attributes. Initially, we select 120 atomic facts and 60 inferred facts centered on geographic locations (India, France, the U.S., Canada, Russia). Using our augmenta- tion strategy. we expand this 1,000 atomic facts and 8,000 inferred facts, yielding ΦG 8 well above the bare-minimum ratio of 3.6 reported by Wang al. (2024) for slow grokking.
Table 2. Examples of augmented facts for the 2WikiMultiHopOA comparison task
Type
Example
Generating New Locations. We first produce atomic facts describing additional cities or regions not present in the original set. For the structured version, these remain in the (City, country, Country) format. For unstruc- tured data, we prompt I. Large Language Model (LLM) to generate concise, Wikipedia- paragraphs (e.g.. "Paris Louvre Museum: The Louvre is a world-famous art mu- seum...").
Atomic Fact
Louvre Museum -- country
France
Detailed
Paris Louvre Museum: The Louvre Museum is a world-famous art museum...
Fact
Inferred Question
Are Avignon Rocher des Doms and Paris Louvre Museum both located in the same country? (An- swer: Yes)
After these augmentation steps, the resulting dataset attains sufficiently high Φ. In the next section, we detail how we train GPT-2 style Transformer on this enriched corpus, and how prolonged optimization reveals the hallmark late- phase jump in multi-hop reasoning accuracy.
Generating Inferred Examples. Next, we create comparison-based queries by selecting two distinct location facts and prompting, e.g., "Are Avignon Rocher des Doms and Paris Louvre Museum both located in the same coun- try?." Algorithm shows the pseudo-code for this procedure, where generate innferredd systematically merges atomic facts into comparison questions:
4 Experiments
3.2.2 COMPOSITIONAL TASK
We evaluate our grokking based approach the 2WikiMul- tiHopQA dataset (Ho et al., 2020), augmented to ensure a sufficiently large ratio ø. We report results on both struc- tured (4.3) and unstructured (4.4) subsets, as well as the original (unaugmented) data (4.2). Finally, we compare performance across all settings (4.5) and offer qualitative insights (4.6).
Compositional tasks require linking multiple relations in sequence (e.g., Person spouse of -- X born in -- Year). We begin with 200 atomic facts and 100 inferred facts, ensuring no direct mention of dates as an answer, and expand to 800 atomic facts plus 5,000 inferred facts. This boosts &G to 6.25 enough to trigger grokking dynamics in practice.
4.1 Setup
Graph Processing. We transform the textual facts into graph representation, extracting nodes (entities) and edges (relations) via an ALM-based parser. We then enrich this graph by adding new atomic edges (avoiding cycles) and sampling multi-hop paths that yield additional inferred facts.
Model and Training. We train an 8-layer GPT-2-style Transformer (768 hidden units, 12 attention heads) from scratch using AdamW (?) with a learning rate of 10-5, batch size of 512, and weight decay 1. We rely on Trainer! with default scheduling, for speed. Train-
the BuggingFaad bf16 precision, and torchhcompil ing runs up to 300k steps or a late-phase jump in out-of- distribution (OOD) accuracy emerges. We use three random seeds on single A100 GPU (48 hours each), reporting the best run (variability +1-2%).
Inferred Question Pentads. Each multi-hop path of length two or three corresponds to pentad robji,rel, objz, obja). An LLM then converts these pentads into natural-language questions (c.g., "Why did Ran- dal Plunkett, 19th baron of Dunsany's 's father die?"), approx- imating real 2WikiMultiHoPQA style. Algorithm 2 details this approach:
'ntspa://github" wtto:oo///////// hog..///gg//gggiggaaeeaea transforrmes
6

---

Grokking in the Wild: Data Augmentation for Real-Worid Multi-Hop Reasoning with Transformers
ID vs. OOD Evaluation. In-Distribution (ID) queries reuse entity/relation combinations observed in training, whereas Out-of- Distributionn (OOD) queries involve en- tirely new combinations. The hallmark of "grokking" is a delayed surge in OOD accuracy after prolonged training (Power et al., 2022).
Table 3. Structured 2WikiMultiHopQA results. *It is unclear how provide clear ID/OOD distinction since models have seen extensive Wikipedia text during training.
Comparison
Composition
Avg
Method
ID
OOD
ID
OOD
ID
OOD
4.2 Original Dataset (No Augmentation)
GPT2-Small
0.70
0.59
0.03
0.70
0.31
GPT-4o*
0,87
0.25
0.56
Training solely the original structured comparison data produces 100% training accuracy with no late-phase OOD jump (Figure 4 (a)).
ol-mini*
0.88
0.32
0.60
Grokked
1.00
0.96
0.93
0.07
0.97
0.52
GPT2-Small
This plateau aligns with earlier findings that Φ = 0.5 is insufficient to induce grokking (Wang et al., 2024).
4.6 Qualitative Analysis
4.3 Structured Compositional and Comparison Tasks
Success Cases. When synthetic augmentation covers multi-hop chains (2-3 hops), the model handles queries like "Which painter was born in the same city as the founder of Company X?" or "Do City A and City share the same country?" tasks that otherwise fail lower &.
Structured Comparison Tasks. Figure 4 (b) demon- strates a clear late-phase jump in OOD accuracy when queries ask if two entities share property (c.g.. City A country -- France vs. City B -- country -- France). This simpler relational structure more readily triggers the formation of generalizing circuit.
Failure Cases. Rare relations or ambiguous entity names still present challenges. If multiple entities share a label but the dataset lacks proper disambiguation, the model may conflate them, hindering OOD performance. Appendix A.4 highlights some examples.
Structured Compositional Tasks. Figure 4 (c) shows training curves for compositional tasks using triplet-based facts, where chains of the form X spouse of Z needed.
These results confirm that increasing & via carefully designed synthetic data is key for grokking- based multi-hop QA. While composition tasks and unstruc- tured text may need larger or more targeted augmentation, the cammooriatioo-toggeeeraaiiiii tional queries demonstrates that grokking can significantly boost OOD performance in real-world factual reasoning.
Overall Findings.
Y nationality
ID Accuracy rises to near perfection, indicating strong memorization of seen patterns.
jump for simpler rela-
OOD Accuracy remains low, showing no late- late-phase im- provement. Complex multi-hop relations appear harder to internalize even with augmentation (Nanda et al., 2023).
5 Limitations and Future Work
4.4 Unstructured Tasks
There are several avenues for improving and extending our grokking-based approach to multi-hop reasoning. While our proof-of- experiments on 2WikiMultiHopQA provide promising evidence, there remain important open questions regarding data complexity, interpretability, and resource feasibility.
Moving to full Wikipedia paragraphs (vs. triplets) adds noise and length (see Figure 4 (d)):
Slower Convergence (ID): Parsing longer text delays training progress.
Modest OOD Gains: Even with data augmentation, text distractors and ambiguous references limit improvement.
Datasets and Benchmarks. Our results demonstrate that Transformer grokking be induced real-world datasets such as 2 WikiMuuliiHopAA. Nevertheless, a broader spec- trum of challenging reasoming benchmarks could illuminate the true scope and boundaries our method. For instance, tasks requiring longer reasoning chains, specialized domain knowledge (e-g., biomedical), or temporal reasoning may reveal nuanced constraints that do not emerge in standard Wikipedia-basse based QA.
For easier comparison queries, the model still attains decent ID accuracy but struggles to generalize OOD, reflecting the sparser effective o in unstructured text.
4.5 Comparison Table
Table 3 compares Grokked GPT-2-small against GPT- 4o and ol -iii on the structured dataset. Our method out- performs others, especially in the comparison task where we reach nearly 100% even in OOD settings. Pretrained models like GPT-4o may not provide a clear ID/OOD distinction since they have seen extensive Wikipedia text.
Analysis and Explainability. Although we observe emer- gent generalization circuits, the precise mechanics of how these circuits form remains only partially understood. Fu-
7

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
1.0
0.8
Accuracy
Training ID-test
0.6
00D-test
0.4
Training
Training tast
Original GPT-2-sma
Grokked GFT-2-small
a00-test
0OD-tast
0.0
10a
16a
10'
10P
10a
10a
10*
10a
10>
10*
16
10'
10f
10
Training Steps (log scale)
Training Steps (log scale)
Training Steps (log scale)
Training Steps (log scale)
(b)
(c)
(d)
Figure 4. (a) Accuracy on the comparison task for the original and grokked GPT-2-small. While both models eventually reach
perfect training accuracy (green curves), only the grokked GPT-2-small exhibits significant late-phase improvement in OOD accuracy
(blue curves). This indicates that the grokked model continues to learn generalizable structure beyond the training data. (b) Training curves for the structured comparison task. ID and OOD behave similarly. (c) The structured compositional task. We see near-perfect ID accuracy but no lale-phase jump in OOD test accuracy. (d) Training curves for the unstructured (full paragraph Wikipedia) comparison setting. Complexity slows convergence and limits OOD gains, although ID accuracy still improves significantly.
ture work can:
where the inference rules are less formally grounded.
Quantify factual drift: Investigate how adding synthetic (hallucinated) facts impacts the model's factual consis- tency and other downstream metrics.
Feasibility. Finally, we note that training large Trans- former architectures for extended periods, as required by grokking, can be prohibitively expensive. Techniques to reduce this overhead, such as those described by Lee et al. (2024), are essential. Concretely, future work might explore:
Mechanistic interpretability: Extend the logit-lens or attention-probing analyses of Wang et al. (2024) to more complex, real-world tasks. Doing so may reveal how sub-networks handle shifting knowledge distributions particularly if separate memory module is involved.
Determining how model size, dataset size, and ratio e collectively influence training cost.
Scaling Laws:
Accelerated Convergence: Applying curriculum learn- ing or specialized optimizers that shorten the "memoriza- tion phase" and expedite the onset of generalization. Pre-training: Pre-trained models can facilitate grokking by leveraging prior knowledge, improving performance, and accelerating the transition from memorization to gen- eralization. Since they already encode fundamental pat- terns (e.g., linguistic or mathematical rules), they might require less training time to achieve generalization.
Factuality, A key question is how synthetic data (some of it intentionally or accidentally hallucinated) affects the
model's ' factual accuracy. While our experiments show that moderate amounts of non-factual data can bolster general- ization, we acknowledge potential risks:
Distortion of real-world knowledge: Without careful fil- tering, hallucinations might overwrite obscure genuine facts.
Factual fragility: Certain tasks (e.g., medical or legal rea- soning) demand rigorous correctness, making any factual drift untenable.
In summary, while we establish the efficacy of grokking for multi-hop factual QA, there is ample room to refine, extend, and better explain these emergent capabilities. We hope our work can serve as a foundation for future explorations into more powerful, transparent, and efficient forms of implicit reasoning in large language models.
As partial solution, we envision more sophisticated constraint-bused data augmentation that preserves core fac- tuality while still boosting the inferred-to-atome ratio Φ. Investigating such hybrid strategies is intriguing direc- tion for future work.
6 Conclusion
We have demonstrated that carefully crafted data synthesis can reshape the distribution of factual language corpora in way that unlocks grokking- based generalization. Even moderately sized GPT-2 model can achieve substantial gains in multi-hop reasoning by leveraging the late-phase formation of internal circuits outperforming more power- ful models that do not receive synthetic data augmentation. Moreover, our empirical results indicate that factuality is not significantly compromised; on average, the model's 's answers become more accurate when given well-balanced mixture
Scope. We expanded the scope from contrived toy prob- lems to large-scale, factual datasets derived from Wikipedia. However, there remain many open questions:
Non- Non-Wikipedia Domains: Would the same grokking dynamics hold in domain-specific corpora (e.g., arXiv papers, biomedical literature, or news articles)?
Other Reasoning Paradigms: Beyond factual QA, it is unknown whether a generalizing circuit would also yield improvements on commonsense or moral reasoning tasks,
8

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
of real and synthesized facts. The main message of this work is that boosting the inferred-to-atomic ratio or via synthetic data remains the most direct route to emergent reasoning circuits.
Ho, X., Duong Nguyen, A.-K., Sugawara, S., and Aizawa, A. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 6609-6625, Barcelona, Spain (Online), 2020. International Committee on Computational Linguistics. doi: 10. 18653/v 1/2020. Being-main.5586 URL https:/ /twp::///www./// aclweb. org/ hrgg/anthologyy22000 wttp://.ooooggmmaa.a 5800
Nevertheless, our approach also highlights several limita- tions. Full generalization across all relations requires each relation's s atomic facts to be sufficiently augmented, which can be challenging for rare or fow-frequency relations. In many real-world corpora, knowledge graphs are not only sparse but also disconnected or partially non-injective, lim- iting the number of multi-hop paths the model can learn from.
Humayun, A. I., Balestriero, R., and Baraniuk, R. G. Training dynamics of deep network linear regions. CoRR, 0bs/2310.12977 2023. doi: 10.48550/ARXIV. 2310.12977. URL https: oow....o./....5550000 a arXiv.2310.12
Finally, natural language challenges persist in practical contexts. Real-world text often contains ambiguous ref- erences, unevenly distributed relations, and disjoint sub- graphs, making high-quality data augmentation non-trivial Nonetheless, our work illustrates the promise of implicit reasoning - once the ratio of inferred facts surpasses a crit- ical threshold, the model internalizes robust logic circuits that can tackle complex multi-hop queries. We hope these findings encourage further research on efficient synthesis methods, broader domain applications, and deeper analysis of the mechanics behind grokking in large language models.
Lee, J., Kang, B. G., Kim, K., and Lee, K. M. Grokfast: Accelerated Grokking by Amplifying Slow Gradients, June 2024. URL http: /waxivivvvv..///aa http://aaaxx...aaaaaaa/e/ 20233. arXiv:2405.20233 [cs].
Liu, L., Du, B., Xu, J., Xia, Y., and Tong, H. Joint Knowl- edge Graph Completion and Question Answering. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22, pp. 1098-1108, New York, NY, USA, August 2022. Associa- tion for Computing Machinery. ISBN 978-1-4503-9385- 0. doi: 10.1145/35546678.5339988 URL https: /dl. acm. owg//oii// /10
Impact Statement
By demonstrating how targeted data augmentation can un- lock emergent multi-hop reasoning capabilities, this work paves the way for more robust, interpretable, and efficient knowledge-intenssee NLP.
B. 3539289.
Liu, Z., Michaud, E. J., and Tegmark, M. Omnigrok: Grokking beyond algorithmic data. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenRe- view.net, 2023. URL https: //pp:://///eeeeeerr wwt.eeeeeweeeeeeeeee.e.e.....e.e net/ pdffid=zDi i Ho I 1aa
The ability to induce "grokking" on real-world factual datasets promises broader applications, ranging from high- stakes domains (e.g., medical, legal, educational) to every- day question answering.
At the same time, this work underscores the importance of careful curation of synthesized facts to prevent misin- formation. This balance between enhanced reasoning and factual accuracy marks a crucial step toward trustworthy, generalizable language models.
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., and Sutskever, I. Deep double descent: Where bigger models and more data hurt. In 8th International Con- ference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https:/ htLthtss://oppeeeeee /oppnreviewweeeeeeeoeee/ee net/ forum? i id=Blg5sAAtwr.
References
Belkin, M., Hsu, D., Ma. S., and Mandal, S. Rec- onciling modern machine-learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849- 15854, 2019. doi: 1.10733/pns.19000001111 URL https:/ /wwp://wwww/a// pnas. 19030001007
Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. Progress measures for grokking via mechanistic inter- pretability. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https: //openrvvi ew net / wtt d= 9XF SbDW.
Pezeshki, M., Mitra, A., Bengio, Y., and Lajoie, G. Multi- scale feature learning dynamics: Insights for double de- scent. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), International Confer- ence on Machine Learning, ICML. 2022, 17-23 July 2022,
Davies, X., Langosco, L., and Krueger, D. Unifying grokking and double descent. CoRR, abs/2303.06173,, 2023. doi: 10.48550/ARXX122200..066
URL https: //tbb.oogg/1..4855////rraa.// wttp://.....00///....000.0 /05.00//0.2.22203.00.....

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 17669-17690. PMIR, 2022. http::: httpp//tooeee....eeee/e/ mlr. www..p.r..es///..mmmmmmmmmmm...e v1 w162/pezeshki22a. htmm..
Plaat, A., Wong, A., Verberne, S., Broekens, J., Stein, N. v., and Back, T. Reasoning with Large Language Models, Survey, July 2024. URL http: /t///aaaa aarxivii... orgg/.gggaoo..... abs/ 1407111511, arXiv:2407.15511 [es].
Power, A., Burda, Y., Edwards, H., Babuschkin, and Misra, V. Grokking: Generalization beyond overfitting small algorithmic datasets. CoRR, /ttp//22...217,,... 2022. URL https::p///. //t:///arra /twvv..iivvvv.////v///e.//aaaava org/ abs /201 021.7.
Thilak, V., Littwin, E., Zhai, S., Saremi, O., Paiss, R., and Susskind, M. The slingshot mechanism: An empiri- cal study of adaptive optimizers and the grokking phe- nomenon. CoRR, abs/2206.048177 2022. doi: 10.48550/ ARXIV.2206.048. 048177. 48550/arXiv. 2206. 04817.
URL https: http://ooi.ooo/ org/10.
Trivedi, H., Balasubramanian, N., Khot, T., and Sabhar- wal, A. MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics, 10:539-554, May 2022. URL
ISSN 2307-387X. doi: 10.1 10.11162/acl https: / //o.//i..o//// .or. / 11 62 1et1111etaclla tttp//aclll__aaa
Varma, V., Shah, R., Kenton, Z., Kramár, J., and Kumar, R. Explaining grokking through circuit efficiency, Septem- ber 2023. URL http: / arxiv. /ttpaa..www..////22222./a ortp./////bb//...... 02390. arXiv:2309.02380 [cs].
Wang, B., Yue, X., Su. Y., and Sun, H. Grokked Trans- formers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization. June 2024. URL https: //openreview net / forumm i 8 IH5Sn5y.
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhut- dinov, R., and Manning, C. D. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answer- ing. Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp- 2369-2380 Brussels, Belgium, October 2018. Associa- tion for Computational Linguistics. doi: 10. D18-1255 1259. URL https: /tpos://aco /aa//claantt htap:::///elanthhoooo..ooo D1 018-1259/.
10

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Appendix
A.1 Sketch of Proof of Lemma 1
Proof. We break the argument into three parts.
1. Counting all potential (n + nn-d sequences. A simple n-hop path is determined by choosing ordered tuple of (n + 1) distinct entities. The number of ways to choose distinct nodes uo, U1, Un out of [V] is:
[V] n + (n + 1)!.
Here, (1141 is the number of ways to pick n + 1 distinct nodes (unordered), and (n + 1)! the number ways to order those nodes into possible directed path of length n.
2. Probability of each directed path being valid. Under the random-graph assumption, the probability that there is directed edge from 1;-1 to (for i = I,..,, nt) is [V Assuming independence across edges, the probability that all edges are present simultaneously is
-T-
Expected number of valid n-hop paths. node tuples), the expected value of Fr is:
By linearity of expectation (applied to each of the (M + 1)! ordered
| n2
B F
(n + 1)! (p--T)²
For large [ one the binomial approximation
n² for (1 such
P
D
(for fixed n as |V] + o0
n++1)
(n 1)!
Computing Bound By definition 2.3 and 2.10,
Fn.r and FArr - by.
on.r =
FArl
Together with the result from the third step being E1[FF|11 [P)) (121) 1)! 1)1( (M>(T)))) "
we obtain,
|Frrl
E3 [
DV- by
(1tt)) + 1)! (1))TT)
12
(n+1y). (rc + 1)16
1ilb 1)m
(DV) -1)T
|
(ID) - 1)
12
67

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Which gives the asymtotic upper bound.
lim M-
Elon.! lim
6--11
6r
[V]
[
Hence Onr (and therfore o, overall) remains boudned above by bm completing the proof.
A.2
Proof of Lemma? 2
Sketch of Proof. By definition 2.3 and 2.10,
[F and |Frrl Frr
Ourr
|V by.
From A.I, we have
[ n (144) 1
by " 1
Far S
(n + 1)!
because
br M
is the approximate probability that randomly chosen edge belongs to relation r.
Thus,
Fn.rl
FFarl
Our
FA,
[V] by
[V]
(n + 1) 6,--1 [V] (1V) 1)m
12 + 1/
Full generalization requires onr 2 da for all yielding the required constraint
ΦG S &n.r
| (n 1)16,"
ΦG
n 11
DV ([V) - 1)m
ΦG [V] (IV) 1)"
br
(111) (n + 1)!
In other words, if any relation has by (its branching factor) that fails to exceed this threshold, then 6n. cannot reach ΦG. Henee, that particular relation will not "grok," the KB is not fully generalizable over n-hop facts (although might still be partially generalizable for other relations).
A.3 Formal Derivation of the Node-Count Bound
Similar to A.2, for full generalization we can derive a bound for the node count VV. From the requirement Vr E R, dn. 2 Φg. we derive,
Vr E R, on,r 2 0G
(1)) | (n + 1)16--1 (n 11 matheal
+ min
2 OG
1)m
in other words, the worst generalizable relation r still needs to fulfill Φn. 2 &G for KB to be fully generalizable. Resolving after [ without the use of the binomial approximation yields,
12

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
[
(n + 1)16,-1
min
DI[V - 1)" 2 0G
n1 + /
|
(n + 1)16.
< min
2 00
(n + 1).((V) - n. - 1)! | -1
|
bア-1
min (|V) - n2 - 1)! [V([V 1)a
2 0G
(IV) - 1)!
min b-- 2 0g
(I1) - n - 1)(([2) - 1)a
(11 1)!
(I1) - n 1)!([/ 1)" 2 max
6
I(VI)
@G
max
T([ n)([] 1)
6,
Thus have:
M 2 minfeEN N: T(v (v 1)a
T(v)
ΦG max
For empirical example (i.e., using randomly generated graph), Figure 5.
www mm
3.b22
2-
formula
empirical
10
20
30
40
50
60
M
M (formula values). The green line empirical values obtained by randomly generating graph with the same amount of nodes (V)) and edges (DV),,) Due randomness, our generated/empiirial graph can have locally higher branching factors, resulting in overall more inferred facts. Our formula for Φa.- therefore effectively underestimates the true value of a Nevertheless, the formula correctly approximates the shape and order magnitude. This holds also for other combinations of b, and n.
Figure 5. Growth of øa.r (y-axis) with [V (x-axis) for b, The red line øa.r (47)
A.4 Qualitative Examples
In the following section, we present qualitative examples of QA pairs ("Question", "Ground Truth") ") from the composition and comparison tasks that the model failed to classify correctly. Some errors stem from inconsistencies in the 2WikiMul- tiHopQA dataset, while others arise due to our augmentation strategy. Overall, we believe that the model is capable of achieving 100% accuracy, as demonstrated previous studies.
Composition There are several grammatical inconsistencies the 2WikiMultiHopQA dataset that prevent our model from reaching 100% accuracy.
Nationality questions may have inconsistent ground truth formats For nationality related questions, the ground truth can be either an adjective or the name of a country, even when the latter grammatically incorrect.
13

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
Question:
What nationality is William Seymour, 3rd Duke of Somerset's father? Ground Truth: English
Question: What nationality is Amadeus VIII, Duke of Savoy's mother? Ground Truth: France
Adjective instead of noun in country-related answers In cases, the ground truth is an adjective instead of noun referring to country.
Question: Which country is the trumpeter of Paper Bird from? Ground Truth: American
Comparison Comparison emors are primarily due to limitations in the data augmentation algorithm, which failed to generate a sufficient number of inferred facts for some relational queries. Most of them are related to lakes, rivers, or airports, which we attribute to the sparsity of the base data.
Question:
Are both Wainwrighr/aiivrighh Field 21 Airport and Roberval Air Saguenay Water Aerodrome located in the same country?
Ground Truth: Yes
Question: Are Long Lake, East Ferris, Ontario, and Montreal Lake, Saskatchewan, both located in the same country?
Ground Truth: Yes
Question: Are Deer Creek, Osage River, and Big Prairie Dog Creek both located in the same country? Ground Truth: Yes
Question: Are Chicoutini/Sainn-HHonee Aerodrome and Rtishchevo Air Base both located in the same country? Ground Truth: No
A.5
Data Synthesis
Here, we present the prompts used throughout our data augmentation process. During our experiments, we leveraged both GPT-4o and ol-mini to generate diverse and high-quality augmented data.
A.5.1
COMPOSITION
Graph parsing
You are graph gpt. You build graph based the provided text. Find all objects, their relations and types.
Pick one of the following types: Person Location Object (include everything that was not above)
Return the following format with numbering: 1. <Avatar; #llmxdireetor>xilaes Cameron; Object>
Person> 2. <James Cameren; Person<dd
Question formatting
You are a question formatting assistant. Your task is to create questions based on the given relations and objects. Use the provided examples remains unchanged and enclosed in <a> tags. You may rephrase one question, given the example format. Strictly follow the logic of given examples. Connect it in the following logic: <obj1>> -> <rell> -> <rel2> -> <ebj3>
guide for the question style. Ensure that the
Return numbered respenses in format: 1. What is the director of the film that James
Cameron produced?<a>Stevee Spielberg</a> Cruiss7<e-christoohhr Nelan</a>
Who directed the movie starring Tom
14

---

Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers
A.5.2 COMPARISON
Atomic fact generation
You helpful assistant that generates geographical facts. Generate new unique locations and their ccuntries the following format: Fol low the style of the examples, but da not use the same lecations. Rules: 1. Use real locations and countries 2. Each location should be uni 3. DO NOT REUSE PROVIDED EXAMPLES 4. Do not answer the question - only provide locations 5. Do not formatting except for numbering 6. Generate equal anount of NEW!!! Locations for Following countries:
Detailed atomic fact generation
You are helpful assistant that generates geographical facta. Based the provided examples, generate paragraph for each cocation-countan pair. Strictly follow the style and lenght of the provided examples be not answer the question only provi de the paragraph with numbering. DO not return empty lines. One by one. Return the number according to the given data. Here are the exarples:
15


Reviews:
[{"review_id": "NvhjIy9JHY", "reviewer": "Reviewer_hZJa", "summary": "This paper explores extending the phenomenon of \"grokking\"—where neural networks transition from memorization to generalization after prolonged training—from synthetic tasks to real-world factual reasoning. The authors address the challenge of dataset sparsity in real-world knowledge graphs by proposing a data augmentation strategy to increase the ratio ($\\phi$) of inferred facts to atomic facts beyond the threshold required for grokking to emerge. This augmentation process includes the addition of both factually correct and, in some cases, incorrect synthetic data, with the latter aiming to encourage reliance on relational structures rather than memorization.\nThe paper evaluates its approach on the 2WikiMultiHopQA benchmark, focusing on multi-hop reasoning tasks. Results show that a GPT2-small model achieves up to 95-100% accuracy on comparison tasks after grokking occurs. \nThe paper also provides a formal theoretical framework to define necessary conditions for knowledge graphs to be \"generalizable,\" focusing on the branching factor of relations. These findings suggest that enhancing the inferred-to-atomic fact ratio is a critical factor for enabling robust generalization circuits in transformers. The authors acknowledge that while the benchmarks demonstrate improvements, further investigation is needed to evaluate the applicability of these methods to other domains and real-world scientific discovery tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful and constructive feedback.\n    \n**A. On model size and generalization:**\n We appreciate the suggestion regarding scaling effects. We are currently conducting additional experiments across GPT2 model sizes (124M to 1.5B parameters). Preliminary results indicate that the grokking behavior is qualitatively consistent across sizes, especially on structured data. However, larger models tend to reach generalization more quickly. We will include these findings, along with supporting graphs and analysis, in the final version. Notably, the critical ratio $\\phi$ remains stable across model sizes.\n    \n**B. On applicability to other domains:**\n        We agree that extending this work to broader domains is an important next step. We are actively exploring applications beyond fact-based reasoning, including logic-intensive tasks such as mathematical problem solving, and real-world scientific domains. These follow-up studies aim to evaluate the generality of $\\phi$-driven generalization. We believe this line of inquiry could inform the design of data-efficient training strategies in broader AI applications.\n    \n**C. On theoretical rigor:**\n        Thank you for the feedback on the formal results. We are currently refining the theoretical section to provide complete, rigorous proofs with clearly stated assumptions. The final version will replace the current sketches with detailed derivations, and we will revise the terminology (e.g., replacing “Lemma” if more appropriate) to ensure precision. We will also discuss the limitations and scope of the theoretical claims to better contextualize the results.", "claims_and_evidence": "The primary claims -- that grokking can be extended from synthetic to real-world factual reasoning, that the $\\phi$ ratio threshold provides a figure of merit for grokking, and the theoretical bounds necessary for generalization -- are well supported by the work.  The late-phase out-of-distribution jump demonstrated in Figure 3-b illustrates the general premise, and considerable attention is given to the formalism and bounds.", "methods_and_evaluation_criteria": "The paper uses data augmentation to increase the ratio of inferred-to-atomic facts in knowledge graphs, testing on 2WikiMultiHopQA across structured/unstructured formats. In-distribution and out-of-distribution evaluations are used to quantitatively assess generalization capabilities, and experimental evidence aligns with theoretical results.", "theoretical_claims": "The paper presents two key theoretical claims: 1) an asymptotic bound on the number of n-hop paths in knowledge graphs as a function of entities, branching factor, and edge probability; and 2) necessary conditions for a knowledge graph to be \"generalizable\" based on relation-specific branching factors.  The sketched proofs seem like they could be made more rigorous with more clearly articulated assumptions, but they provide sufficiently compelling intuition for the paper -- I'm not sure whether \"Lemma\" is the appropriate word for the results.", "experimental_designs_or_analyses": "The experimental design effectively isolates the impact of the $\\phi$ ratio on grokking and provides valuable comparisons across structured/unstructured formats. Minor weaknesses include testing only on GPT2-small and the lack statistical significance.", "relation_to_broader_scientific_literature": "I see no missing material on the relation of the papers results to broader scientific literature.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None that are fair to ask, as I think they are follow-on work to this paper -- there are a lot of interesting questions about the $\\phi$ scaling with architecture and model size that I hope to see the authors pull the thread on in the future.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7vXzKt5k7F", "reviewer": "Reviewer_vnAC", "summary": "The paper investigates the application of grokking—a phenomenon where neural networks transition from memorization to generalization after prolonged training—to real-world multi-hop reasoning tasks. The authors propose augmenting sparse knowledge graphs (KGs) with synthetic data to increase the ratio of inferred-to-atomic facts ($\\phi_r$), enabling Transformers to form internal reasoning circuits. Experiments on the 2WikiMultiHopQA benchmark show that their approach achieves near-perfect accuracy (95–100%) on comparison tasks, outperforming baselines and matching state-of-the-art models. The paper also provides mechanistic insights into how increasing $\\phi_r$ drives circuit formation in Transformers.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed and constructive feedback.\n    \n**A. On transparency and reproducibility:**  \n        We appreciate the emphasis on methodological clarity. In the final version, we will provide full details of the training procedure, including the exact prompts used for structured and unstructured data synthesis, temperature and sampling parameters, and other relevant hyperparameters. We will also include the target responses used for comparison and make code and data available to support reproducibility.\n    \n**B. On comparisons with large pre-trained models:** \n        We agree that comparing against models such as GPT-4o presents challenges due to potential training data leakage, especially with benchmarks like 2WikiMultiHopQA, which overlap with common pretraining corpora such as Wikipedia. We note that this limitation affects most publicly available models and motivates our decision not to report IID/OOD splits for those baselines. Nonetheless, the fact that our grokked GPT-2-small model surpasses these models in structured comparison tasks underscores the potential of $\\phi_r$-driven grokking as a viable, lightweight alternative for reasoning over factual knowledge.\n    \n**C. On model scaling and $\\phi_r$ thresholds:**\n        We are currently conducting experiments across a range of GPT2 model sizes (124M–1.5B parameters) to evaluate the effect of scaling on generalization dynamics. Preliminary results suggest that while grokking behavior remains consistent, larger models tend to reach generalization more rapidly. Notably, the critical $\\phi_r$ threshold required for generalization appears stable across model sizes. Please also refer to our response to Reviewer hZJa (Section A) for more details.\n    \n**D. On theoretical rigor:**  \n        We are in the process of finalizing complete versions of the theoretical results, including clearly stated assumptions and formal proofs to replace the current sketches in Appendix A.2–A.4. These revisions will be included in the final version.", "claims_and_evidence": "1. Synthetic data augmentation raises $\\phi_r$ above a threshold, enabling grokking in real-world KGs.  Supported by experiments showing that increasing $\\phi_r$ via synthetic multi-hop paths correlates with late-phase generalization (Figure 3).\n2. Even factually incorrect synthetic data can improve reasoning by forcing reliance on relational structure.  Partially supported by qualitative analysis showing improved OOD accuracy despite synthetic noise. However, no explicit ablation studies compare factual vs. non-factual synthetic data.  \n3. Transformers trained with this method achieve SOTA results on 2WikiMultiHopQA.  Validated by results in Table 3, where their GPT-2-small model outperforms GPT-4o and o1-mini on structured tasks.", "methods_and_evaluation_criteria": "**Methods**:  \n- **Data Synthesis**: Augment KGs with LLM-generated atomic/inferred facts to boost \\(\\phi_r\\).  \n- **Training**: Train GPT-2-style Transformers with prolonged optimization (300k steps) to induce grokking.  \n\n**Evaluation**:  \n- **Benchmark**: 2WikiMultiHopQA (structured/unstructured subsets).  \n- **Metrics**: In-distribution (ID) vs. out-of-distribution (OOD) accuracy.  \n\n**Strengths**:  \n- The focus on $\\phi_r$ as a key lever for grokking is novel and grounded in prior theoretical work.  \n- Evaluation across both structured (triplet-based) and unstructured (paragraph-based) settings adds robustness.  \n\n**Weaknesses**:\n- No comparison to retrieval-augmented methods, which are common in multi-hop QA.", "theoretical_claims": "The paper builds on grokking theory[1] but does not introduce new theoretical proofs. Instead, it adapts existing concepts (e.g., generalization thresholds for $\\phi_r$) to real-world KGs. The lemmas on asymptotic bounds for multi-hop paths (Appendix A.2–A.4) are intuitive but lack formal proofs. \n\n[1] Grokking: Generalization beyond overfitting on small algorithmic datasets", "experimental_designs_or_analyses": "**Strengths**:  \n- Clear ID/OOD splits to measure generalization.  \n- Multiple seeds and architecture details ensure reproducibility.  \n\n**Weaknesses**:  \n- **Synthetic Data Generation**: The process relies on LLM prompting but lacks transparency (e.g., prompts not fully disclosed).  \n- **Baselines**: Comparisons to GPT-4o are unclear due to potential data leakage (GPT-4 may have seen 2WikiMultiHopQA during pretraining).", "relation_to_broader_scientific_literature": "The work bridges grokking[1] and multi-hop QA [2,3]. It extends grokking beyond synthetic tasks, aligning with efforts to study emergent reasoning in Transformers[4]. However, it does not discuss connections to neuro-symbolic methods (e.g., logical reasoning modules).  \n\n\n- [1] Grokking: Generalization beyond overfitting on small algorithmic datasets\n- [2] HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering\n- [3] Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps.\n- [4] Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization,", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "**Strengths**:  \n- First to apply grokking to real-world KGs.  \n- Mechanistic analysis of circuit formation.  \n\n**Weaknesses**:  \n- Limited scalability analysis (e.g., larger KGs or models).  \n- Risks of synthetic hallucinations are acknowledged but not quantified.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "How does the required $\\phi_r$ threshold vary with model size?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3QHOzVH7Ud", "reviewer": "Reviewer_TzCM", "summary": "This paper examines the problem of learning multi-hop reasoning over knowledge graph facts. Prior works have shown that this is a challenging problem, particularly without chain-of-thought or externalized reasoning. However, recently, synthetic experiments have revealed that such multi-hop reasoning can be learned by models in a grokking regime -- provided that there exists a sufficient concentration of inferred facts in the dataset. This paper examines whether such a grokking phenomenon occurs in a real fact-learning setting. They first show theoretically that there exists a lower bound of the knowledge graph branching factor which determines whether  there are \"naturally\" a sufficient number of inferred facts to enable reliably learning multi-hop reasoning. In real world settings, based on the WikiData, the authors demonstrate that there do not exist a sufficient concentration of inferred facts and they describe possibilities of generating additional data to boost the formation of multi-hop reasoning. Empirically, they show that training on this augmented data does in fact induce grokking dynamics in real transformers. Impressively, they can compare with very small models and achieve comparable performance to the largest models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful and detailed feedback, which has helped us refine both the presentation and scope of the paper. Below, we address each of the raised concerns.\n    \n**A. On theoretical presentation and unused formalisms:**\n        We appreciate your observation regarding the mathematical complexity and notation density in the theory section. In response, we have carefully reviewed all definitions and formal constructs. A dependency analysis shows that each definition contributes to Lemma 1 or 2, either directly or through intermediate derivations. While we agree that the presentation could be simplified, we found that deferring key definitions to the appendix would compromise the readability and cohesion of the main theoretical argument. We will, however, streamline redundant notation (e.g., the unused edge norm) and improve clarity in the final version. Thank you also for catching the typo on line 192; we have corrected it.\n    \n**B. On in-distribution vs. training data distinction:**  \n        Thank you for highlighting the confusion here. We will clarify this distinction in the final version. In short: the training data consists of atomic facts and associated inferred facts used in specific combinations. In-distribution (ID) evaluation involves novel combinations of *seen atomic facts*, not seen together during training. For example, if the model saw the atomic facts for \"Avignon Rocher des Doms\" and \"Paris Louvre Museum\" in training, an ID question might involve \"Avignon Rocher des Doms\" and the \"Eiffel Tower\", a new combination of familiar elements. This contrasts with out-of-distribution (OOD) examples, which include entirely new atomic facts.\n    \n**C. On structured vs. unstructured settings and realism:**  \n We agree that structured settings simplify linguistic complexity. Our core claim is not that we have solved the problem for all real-world unstructured data, but that the grokking phenomenon (previously limited to toy datasets) can occur with real-world entities and linguistic structure, provided key conditions are met (notably a sufficiently high $\\phi_r$). The structured format allowed us to isolate this effect. In the unstructured setting, generalization circuits are harder to induce due to syntactic variability and ambiguity. We view extending grokking to unstructured data as an important avenue for future work. Our current contribution lies in demonstrating that grokking is not limited to synthetic data, and that it can emerge from real-world facts when structured and augmented appropriately.\n    \n**D. On relation-specific generalization ratios:** \n        The assumption that $\\phi_r$ can be modeled as relation-specific follows from prior work [1], which demonstrates that transformer-based generalization circuits rely primarily on relation-specific atomic facts. This behavior has been empirically confirmed in our own replications of their experiments. While entity frequency affects the number of available paths, it is the *relation* that governs the type and reusability of inference patterns, making $\\phi_r$ effectively relation-dependent. We clarify this distinction in Lemma 1 and Appendix A.1, where we show that while the number of unique paths grows with the number of entities $|\\mathcal{V}|$, the asymptotic behavior is dominated by the branching factor $b^{n-1}$, not by entity coverage.\n    \n**E. On model size effects:**\n        We are currently conducting experiments with varying GPT-2 sizes (124M–1.5B) to investigate the impact of scale on generalization dynamics. Early results suggest that larger models reach generalization faster, but the critical $\\phi_r$ threshold remains stable. For additional details, please see our response to Reviewer hZJa (Section A).\n    \n**F. On prompt transparency and training details:**\n        Thank you for raising this point. We will include full prompt templates, sampling parameters, and training details for both structured and unstructured settings in the final version. Briefly, we trained two separate GPT-2 small models from scratch—one for structured data (triple format) and one for unstructured data (natural language questions). Neither model used pretraining; both were trained solely on task-specific data.\n    \n**G. On circuit flexibility and multi-hop reasoning:**\n        The generalization circuits observed in our setting are relatively rigid and rely on deterministic retrieval and composition of atomic facts. For 2-hop reasoning, this involves a single inference operation; for $n$-hop paths, multiple sequential inference steps are required. Based on our theoretical framework and ongoing experiments, we are confident that $n$-hop grokking is feasible without fundamental changes to the architecture -- provided the augmented dataset supports sufficient multi-hop path coverage.\n    \nWe again thank the reviewer for the constructive feedback and will incorporate these improvements in the final submission.", "claims_and_evidence": "The main claim of this paper is that grokking can emerge in real settings (esp. for multi-hop fact reasoning settings). This is tested via a set of experiments on Wikipedia-based knowledge graph settings. They have two settings: a structured knowledge setting in which facts are represented as triples. In particular, this is where they observe the Grokking phenomena primarily. However, I believe that the structured setting eliminates much of the complexity and noise present in real fine-tuning datasets. On the other hand, they also consider an unstructured setting. In this setup, however, the results do not appear to confirm grokking as far as I can tell. Thus, I am not sure that this claim is necessarily supported.", "methods_and_evaluation_criteria": "The 2-Wiki-Multi-Hop dataset appears to be a reasonable setting to test and the model size is reasonable -- albeit maybe on a smaller end for real tasks. It would be interesting if the authors had some predictions about the impact of model size on their results. I also found that the discussion of in-distribution/out-of-distribution were a bit confusing. The authors state that in-distribution performance measures entity pairs seen during training. If that is the case, how does this differ from the training loss?", "theoretical_claims": "I have briefly reviewed the theoretical sections, but not the proofs. The theory appears to be predicated on the necessity of observing a given number of multi-hop paths. The main theoretical result proves a lower bound on the average branching factor of the knowledge graph necessary to achieve generalization. I think this analysis is generally ok, but it does assume the existence of some \"known\" generalization ratio for each relation. It would be better if the authors can justify why it is ok to model this generalization ratio as only depending on the relation. Could it not also depend on the coverage of the entities in the seen multi-hop examples?", "experimental_designs_or_analyses": "I did not have specific concerns about the validity of the experimental design, with the exception of my question about in-distribution/out-of-distribution accuracy above. Also for clarity, how exactly was Grokked GPT-2–small trained. Was it trained on the unstructured data or the structured data? Additional details of the prompts tried for the comparison to GPT-4/o1 would be quite helpful to better contextualize the performance of the proprietary models..", "relation_to_broader_scientific_literature": "This paper appears to borrow heavily from the formulation/problem setup etc of [1]. However, they extend the simulated/toy settings addressed by [1] and instead consider real knowledge graph data -- sourced from WikiData. Their claimed key contribution is that they show the phenomena introduced by [1] can in fact happen in practice. I will note that in the structured setting (the place where they see the  most direct example of grokking) the problem is actually not so different from the toy settings examined by [1].\n\n\n\n\n\n[1] Wang, B., Yue, X., Su, Y., and Sun, H. Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to\nthe Edge of Generalization, May 2024. URL http://\narxiv.org/abs/2405.15071. arXiv:2405.15071\n[cs].", "essential_references_not_discussed": "There is not one that I am aware of.", "other_strengths_and_weaknesses": "I believe that this paper could benefit significantly from better presentation. Overall, the problem studied, experiments, and theoretical arguments are interesting in my opinion but they are hard to parse. In the theory section, the authors introduce a large amount of formalism and notation. I find that this overwhelms me with many details (for example they introduce a norm over edges but this norm isn't used later in the text as far as I can tell). I  think the authors could make this paper significantly easier to follow by culling the notation and excessive detail in the theory section and deferring some notions to the supplementary material.", "other_comments_or_suggestions": "Line 192: \"you sure you wanted to change that to a ”not exists” because\" -- Is this a typo?", "questions_for_authors": "1) Please answer the questions in \"Experimental Design or Analysis\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents some interesting insights on how generalization may happen in language models' training. It argues that inferred-facts ratio should be higher and present a data augmentation method to achieve such a training set. Reviewers appreciated the insights presented in the paper, both from a theoretical and practical aspect. However, reviewers also noted that theory section could have had more details (beyond proof sketches) and the writing could have been clearer. 
On balance, I think the paper makes a good contribution and applies to the real-world usecase of reasoning over facts.
Decision: Accept (poster)