Unified View of Grokking, Double Descent and Emergent Abilities: A Comprehensive Study on Algorithm Task

Authors: yufei  huang
shending  hu
xu  han
zhiyuan  liu
maosong  sun
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 06:16:06

Abstract: Recent studies have uncovered intriguing phenomena in deep learning, such as *grokking*, *double descent*, and *emergent abilities* in large language models, which challenge human intuition and are crucial for a deeper understanding of neural models. In this paper, we present a comprehensive study on algorithm task to provide a unified view of these three phenomena, with a focus on the interplay between memorization and generalization. Through extensive experiments spanning a wide range of model sizes and training data quantities, we uncover four distinct training dynamics, each arising from unique combinations of model size and training data quantity, formulating a theoretical framework for further analysis. Utilizing this framework, we establish connections between *double descent* and *grokking* and propose two verifiable predictions regarding the occurrence of *double descent*, both substantiated by our experimental results. Moreover, we expand our experiments to the multi-task learning paradigm, demonstrating how algorithm tasks can be turned into emergent abilities by mixing some pure memorization data. This offers a novel perspective to understand *emergent abilities* in Large Language Models.

Main paper: 
 introduction: 
 There are several interesting phenomena in Deep Learning, among which grokking (Power et al., 2022) , double descent (Belkin et al., 2019; Nakkiran et al., 2020) and emergent abilities (Brown et al., 2020; Wei et al., 2022a; Ganguli et al., 2022; Srivastava et al., 2023) in current Large Language Models attract a lot of attention. Understanding these phenomena is important for us to reveal the mechanism of deep learning. Plenty of works (Liu et al., 2022; 2023; Thilak et al., 2022; Varma et al., 2023; Schaeffer et al., 2023; Michaud et al., 2023) have been done to explain these phenomenons from different perspectives. However, these works all concentrate on a single phenomenon and explain them separately. In this work, we provide a preliminary study to give a unified view of these three phenomena from the perspective of competition between memorization and generalization.Our work is based on Varma et al. (2023) 's explanation for grokking. They attribute grokking to the competition between two distinct types of circuits in the model: one responsible for memorization, which achieves high training accuracy but poor validation accuracy, and another for generalization, capable of high performance in both training and validation. The latter, although slower to develop, proves more efficient in terms of parameter norms, leading to the model finally transferring from memorization to generalization to achieve higher efficiency. Intriguingly, the efficiency of the memorization circuit is inversely related to the volume of training data, indicating that larger datasets reduce its efficiency. In contrast, the efficiency of the generalization circuit remains consistently stable, regardless of the size of the training data. As a result, Varma et al. (2023) demonstrate similar levels of efficiency and the model will show part of generalization ability. Crossing this threshold significantly enhances the probability of grokking.Building upon their findings, our study expands the investigation to various model sizes.We firstly observe that smaller models require a larger critical dataset size for grokking, implying an inverse relationship between model size and the necessary amount of training data for grokking. Conversely, a model's memorization capacity is directly proportional to its size, indicating that smaller models have a reduced capacity for memorization. We confirm these relationships by experiments in § 2.The reverse relationships with model size inevitably lead to an intersection point of the two curves (yellow star in Figure 1 This behaviour was first identified by Varma et al. (2023) . (d) Grokking (Power et al., 2022) :When the number of training data is beyond critical dataset size, the generalization circuits become more efficiency than memorization, leading the model transfer from memorization to generalization long after training performance become perfect.Analyzing Figure 1 , we can easily link double descent with grokking and figure out when double descent will happen. Specifically, for training data volumes below the intersection point, the model undergoes progression, memorization, semi-grokking and then grokking with model size increasing, leading the validation performance first increases, then decreases, and ultimately increases again, which is exactly the double descent phenomenon (Belkin et al., 2019; Nakkiran et al., 2020) . In contrast, when the quantity of training data surpasses the intersection point, an increase in model size results in the model through progression to grokking, resulting in a consistently positive correlation between model size and final validation performance. We can see that the occurrence of double descent is highly related to the position of the intersection point. Therefore, to further verify our illustration, we conduct experiments to move the position of the intersection point and successfully transform a validation accuracy function without clear double descent, into one with obvious double descent. This transformation is achieved by shifting the critical dataset curve upward, thereby also shifting the intersection point towards the upper right.Further, we extend our experiments to the multitask learning paradigm where an algorithm task and a pure memorization task are mixed to train the model. Interestingly, adding a pure memorization task largely hinders model from formalising the generalization circuits for the algorithm task. With model size increasing, model always achieves near zero validation performance until a relatively large model size, which is about 1570 times larger than training solely on the algorithm task. This phenomenon reminds us of the emergent abilities in Large Language Models (Brown et al., 2020; Wei et al., 2022a; Ganguli et al., 2022; Srivastava et al., 2023) . The pretraining stage can also be seen as a multi-task learning process, where model has to remember numerous world knowledge while developing some general rules and abilities, such as reasoning. Our study suggests that this multi-task learning feature can be one important reason for the emergent abilities in LLM.Overall, we make three key contributions in this study, which are outlined as follows:• We introduce an innovative framework designed for analyzing the performance and training dynamics in consideration of both the size of the model and the quantity of training data.• Utilizing this framework, we provide a nuanced illustration for the double descent phenomenon and establish a predictive method for identifying instances of double descent occurrence.• By extending to multi-task learning which consists of algorithm and pure memorization tasks, we convert algorithm tasks into an emergent ability. This offers a novel angle for understanding emergent abilities in Large Language Models.Key Takeaways. To better focus our readers' attention, we highlight the key takeaways from our analysis:1. Critical dataset size for generalization decreases with model size increasing.2. Memorization capacity increases with model size increasing.3. The training dynamics can be categorized into four types.4. Progression and grokking differ in parameter norm variations despite both showing delayed generalization ability during training.5. Double descent is caused by training dynamics moving from progression to memorization, then to grokking.6. We can make double descent more prominent by increasing the generalization difficulty.7. Emergent ability can be introduced by mixture of memorization tasks and generalization tasks.8. Separating memorization and generalization in parameter space leads to faster emergence. 
preliminary study about grokking: 
 In this section, we first recall the preliminary setup of Grokking experiments in Varma et al. (2023) (a + b) mod P; for a, b ∈ (0, ..., P -1) and P = 113 By using different tuples of a and b, this task can be easily split into non-overlap train and validation set. Therefore, the memorization behaviour and generalization behaviour can also be distinguished by the validation performance. Although Nanda et al. (2023) propose a method to identify the generalization circuit of modular addition, we directly use validation performance to distinguish for simplicity. We use D train add and D val add to represent the data number of training set and validation set on this modular addition task.Model Following previous work (Nanda et al., 2023; Varma et al., 2023) , we train a 1-layer simplified decoder-only transformer (Vaswani et al., 2017) model with 4 attention heads. We see the tasks above as classification tasks, where the label number is P. During training, a cross entropy loss L CE with AdamW (Loshchilov & Hutter, 2019) optimizer is utilized. We mainly vary model size by adjusting its hidden size dimension d h .In this study, we explore the impact of varying training dataset sizes, denoted as D train add , which range from 2, 000 to 4, 000 in increments of 200. Additionally, we investigate models with different hidden sizes, specifically d h ∈ {32, 36, 40, 44, 48, 56, 64}. For each unique combination of dataset size and model hidden size, we perform experiments using 11 distinct random seeds, and their average performance is reported.We first verify that modular addition task indeed show grokking in our experimental setup which is shown in Figure 1 (d) . Model quickly achieves perfect training accuracy while validation accuracy takes more training steps to become perfect. By comparing Figure 1 (c ) and (d), we can also find that model needs a specific amount of training data to show perfect generalization ability, which is the critical dataset size stated by Varma et al. (2023) . Then, we present the final validation performance as a function of the training dataset size for different models in Figure 2 to analyse the impact of model size on critical dataset size.TAKEAWAY-1 From the analysis presented in Figure 2 , it is evident that models with larger hidden sizes tend to exhibit grokking (perfect validation performance) with smaller datasets. For instance, a model with a hidden size d h of 64 achieves near-perfect validation accuracy with just 3, 000 training samples. In contrast, a model with a smaller hidden size of 32 requires an increase in training data size to 4, 000 to achieve similar validation accuracy. This trend is consistent across models with intermediate hidden sizes ranging between 32 and 64.In addition, our research also verifies the phenomenon termed semi-grokking (Figure 1 (c)), as proposed by Varma et al. (2023) . This phenomenon is very similar to grokking except for its partial generalization ability, which means model cannot show perfect validation performance in the end of training. Occasionally, semi-grokking can show several increasing stages of validation accuracy. Based on Varma et al. (2023) 's experiments, semi-grokking can be observed when the number of training data points, D train add , closely matches the critical dataset size for the current model, denoted as D M crit . Under these conditions, the model's memorization and generalization circuits demonstrate comparable efficiencies. This balance prevents the model from fully transitioning from memorization to generalization, resulting in semi-grokking.As the results shown in Figure 2 , a model with a hidden size d h of 64 shows middling validation accuracy and exhibits semi-grokking when the training dataset size, D train add , ranges between 2, 000 and 3, 000, where the critical dataset size, D M crit , for this model is approximately 3, 000. We observe that as the gap between D train add and D M crit narrows, there is a general improvement in the model's generalization ability, as reflected in its final validation accuracy.Since grokking requires the model to memorize all of the training data firstly, we designed experiments that specifically gauge the model's ability to memorize. For this purpose, we assign random labels to the training data (Zhang et al., 2021; Doshi et al., 2024) , compelling the model to focus solely on memorization. Given the modular addition task's adherence to the commutative law, which could affect the model's memorization capability on this task 1 , we assign identical random labels to (a + b) mod P and (b + a) mod P. This approach ensures a more accurate assessment of the model's true memorization ability for modular addition task. Our experiments encompass all 12, 769 pairs as training data, and we evaluate the model's accuracy on this entire set to represent its memorization capacity.TAKEAWAY-2 Unsurprisingly, the findings, illustrated in Figure 3 , reveal a distinctly positive correlation between the model size and its memorization capacity; larger models are capable of memorizing more training data. 
proposed framework: 
 In this section, we introduce our framework designed to analyze both the training dynamics and final validation performance. This approach is grounded on two key assumptions, which are substantiated by the experiments detailed in § 2. TAKEAWAY-3 Drawing from Assumption 3.1 and Assumption 3.2, we can construct a graphical representation for a specific task, as illustrated in Figure 1 . This graph delineates the relationship between memorization capacity and critical dataset size, highlighting their intersection point (marked by a yellow star in Figure 1 ). Consequently, this demarcates four distinct zones, each representing a unique training dynamic. These dynamics will be discussed detailly in the subsequent discussion (right side of Figure 1 ).In scenarios where the training dataset D train surpasses the memorization capacity D M mem of a model M, the model is incapable of memorizing the entire dataset. This limitation leads to a two-stage learning dynamic. Initially, the model memorizes a portion of the training data, which does not translate into improved validation performance. Subsequently, the model begins to generalize, improving both its training accuracy and validation performance, as shown in Figure 1 (a) .Memorization When the training data D train falls below the memorization capacity D M mem of the model, it can memorize the entire dataset. However, if D train is also significantly less than the critical dataset size D M crit , memorization circuits outperform generalization circuits in efficiency. This leads the model to opt for pure memorization, resulting in negligible validation performance, as illustrated in Figure 1 (b) .In cases where the training data D train approximates the critical dataset size D M crit , the efficiency of memorization and generalization circuits becomes comparable. This parity causes the model to struggle with transitioning entirely to generalization circuits. Consequently, the final model is a combination of both memorization and generalization circuits, yielding moderate validation accuracy. Additionally, multiple stages of increase in validation performance may be observed, indicating repeated shifts between memorization and generalization circuits, as shown in Figure 1 (c) .In the situation where the training data D train is between the memorization capacity D M mem and the critical dataset size D M crit , the model is able to memorize the entire training dataset. However, the circuits dedicated to memorization are less efficient compared to those for generalization. Initially, the model achieves perfect training accuracy through memorization. It then transitions to generalization circuits for higher efficiency, leading to a training dynamic demonstrated in Figure 1 (d) , where validation performance reaches near perfection well after the model has overfitted to the training set.Discussion Despite progression and grokking show similar validation performance dynamic during training, it is crucial to note the distinction between them in two fundamental aspects. Firstly, the generalization circuits in progression show up a moderate level of training accuracy. In contrast, grokking is characterized by the formation of generalization circuits only after the model has achieved perfect training accuracy. Secondly, the underlying mechanisms driving the formation of generalization circuits differ significantly. In the case of progression, generalization circuits are induced by the constraints of cross-entropy loss, as the model is unable to completely memorize all training data, preventing it from reaching near-zero training loss. Consequently, the generalization circuits in progression are developed to minimize training loss while concurrently enhancing validation performance. However, the emergence of generalization circuits in grokking is attributed to the model's preference towards more efficient circuits, which is minimizing the model's parameter norm. This leads to a notable trend for generalization where, during grokking, the model's parameter norm tends to decrease, whereas in progression, an increase in the parameter norm is typically observed, which is shown in Figure 7 . (TAKEAWAY-4) 
illustrate double descent: 
 The phenomenon of double descent, as observed by Nakkiran et al. (2020) , reveals an intriguing pattern wherein the increase in model size firstly detrimentally impacts validation performance before finally contribute positively to the validation performance. In this section, we provide a detailed exploration of the double descent phenomenon, coupled with predictions regarding its occurrence given our framework in § 3. Subsequently, we undertake a series of experiments designed to validate our theoretical illustration. TAKEAWAY-5 Our framework elucidates that the phenomenon of double descent is likely to manifest when the number of training data, D train , is substantially lower than the intersection point of two curves in Figure 1 . In such scenarios, the model undergoes a series of stages: progression, memorization, semi-grokking, and finally grokking. Initially, as the model size increases during the progression phase, there is an enhancement in its generalization ability. However, this ability declines, potentially to zero, upon transitioning into the memorization stage. This decline is in alignment with the critical interval of model size postulated by Nakkiran et al. (2020) . As the model continues to grow, it enters the semi-grokking and grokking phases, where its generalization ability is revived, leading to a secondary increase. This relationship results in the double descent curve observed in validation performance2 . Conversely, in instances where the training data exceeds the intersection point, the model predominantly experiences progression and grokking, resulting in a consistent upsurge in generalization ability as the model size increases. Under these conditions, the double descent phenomenon does not occur.To validate our illustration, we carried out a series of experiments centered on the modular addition task, as detailed in § 2. We set the training data sizes to {2600, 3000, 4000, 5000} and varied the model's hidden size from 4 to 100, following the sequence {4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 56, 64, 100} . Each configuration was tested across 11 experiments with distinct random seeds, and the average performance was reported, accompanied by a 95% confidence interval. These results are depicted in Figure 4 .Observations from Figure 4 (a) reveal that with a training dataset of 2600 samples, the models transition through the stages of progression, memorization, semi-grokking3 . In the progression phases, larger models generally demonstrate improved validation performance. However, the memorization phase, evident when the model's hidden size ranges between 24 and 40, leads to zero validation performance, resulting in the double descent pattern for this dataset size.Increasing the training data slightly to 3000 samples eliminates the memorization stage. The models then progress through progression, semi-grokking, and grokking, resulting in a less pronounced double descent curve. Still, a dip in validation performance is noticeable for model sizes between 28 and 36, as shown in Figure 4 (b). Outside this range, there is a general trend of increasing performance with larger model sizes.With the training data further increased to 4000 and 5000, models bypass the memorization and semi-grokking stages, transitioning directly from progression to grokking. This leads to a consistent improvement in validation performance as model size increases, without any occurrence of double descent, as illustrated in Figure 4 (c) and (d). we need is to increase the difficulty of generalization. To validate this hypothesis, we designed experiments centered around a task more complex than the modular addition which is defined as:(a + b) 2 mod P; for a, b ∈ (0, ..., P -1) and P = 113Given that this task also adheres to the commutative law, it does not affect the memorization capacity curve. For a direct comparison, we use 3000 training data, the same size that did not exhibit a very clear double descent in Figure 4 (b). We keep all other experimental setup the same as modular addition task. The outcomes are depicted in Figure 5 .The results from this more complex task reveal a more pronounced double descent phenomenon in the validation performance compared to the one observed in Figure 4 (b), which utilized an identical quantity of training data. Notably, in this task, the model exhibits memorization stage with zero validation performance when the hidden size of the model is 24. This memorization stage is not observed in the modular addition task with the same training data size of 3000. These behaviours corroborate our previously stated hypothesis and verify our illustration about double descent. 
multi-task learning leads to emergent ability: 
 In this section, we expand our research to the multi-task learning paradigm, which combines modular addition task, with a task focused solely on memorization. This approach reveals that the model's generalization ability on the algorithm task remains negligible until the model reaches a substantially larger size-specifically, 1570 times larger than training on a single task. As a result, the validation performance on algorithm task show an emergent phenomenon relative to model size.Our experiments utilize the modular addition task as the generalization component. For the memorization task, we assign random labels to a subtraction task, compelling the model to memorize these associations. By incorporating different calculation symbols (+, -) into the input tokens, we ensure that the memorization and algorithm task have no overlapping inputs. Our experiments involve 3000 data points from the modular addition task and a varying number of memorization data points, ranging from 3000 to 6000. We adjust the model size from a 1-layer transformer with a hidden size of 64 to an 8-layer transformer with a hidden size of 1024. Each experiment is conducted three times, and the highest validation accuracy is reported to showcase each model's optimal generalization ability. The results are presented in Figure 6 . TAKEAWAY-7 We observe that incorporating pure memorization data significantly impedes smaller models in developing generalization circuits. The emergence of generalization ability in the modular addition task is notable, with models typically displaying substantial validation performance at relatively larger sizes, approximately 1570 times larger than those trained solely on the modular addition task. Additionally, the volume of memorization data appears to have minimal impact on the emergent model size.Discussion From the perspective of the competition between memorization and generalization, the presence of a pure memorization task prevents the model from totally transitioning from memorization to generalization after memorizing all training data, as there are no generalization methods for the pure memorization task. However, once a sufficiently large model size is reached, the model's memorization capacity significantly exceeds the training data volume, allowing it to show functional differentiation for different abilities including memorization and generalization. The experiment in Appendix E, which reduces the emergent model size by allocating separate parameters for generalization and memorization data, further supports this hypothesis. This phenomenon echoes the emergent abilities observed in current Large Language Models, since the pretraining stage can also be seen as a multi-task learning scenario, where the model must retain a vast array of world knowledge while acquiring general rules and capabilities, such as in context learning (Brown et al., 2020) and multi-step reasoning (Wei et al., 2022b) . This observation also elucidates the hypothesis proposed by Hu et al. (2024) , where it is hypothesized that emergent abilities are formed through the competition of different neural circuits. Although the emergence of abilities in current LLMs is driven by a multitude of complex factors, we believe our experiments will offer fresh insights into understanding emergent abilities in LLMs and stimulate further research in this area. 
related work: 
 Grokking The phenomenon of grokking, where models demonstrate exceptional generalization capabilities well beyond the point of overfitting to training data, was first identified by Power et al. (2022) in various algorithm tasks. Thilak et al. (2022) demonstrate that grokking often comes with "Slingshot Effects," which may play a pivotal role in its emergence. Delving into the underlying mechanisms, Liu et al. (2022) approached grokking from a representation learning standpoint, uncovering its association with structured representation development. Merrill et al. (2023) analyse grokking through the competition between subnetworks. Beyond algorithm tasks, recent work (Liu et al., 2023; Murty et al., 2023 ) also discovered that grokking happens in a broader spectrum of realistic tasks. Additionally, Junior et al. (2024) explored predictive markers of grokking, highlighting "oscillations" within the loss landscape as potential indicators. A novel perspective by Varma et al. (2023) suggests that grokking can be interpreted through the competition between memorization and generalization circuits, influenced by the efficiency of these circuits. This demonstration can be included in our framework by a vertical line on the right of the intersection point in Figure 1 .The concept of double descent, as introduced by  
conclusion: 
 In this paper, we conduct a comprehensive study on algorithm task to show varying training dynamics across different model sizes and training dataset sizes and build a framework to analyse different phenomena. Based on this, we offer a comprehensive illustration of double descent and establish its connection with grokking. By integrating memorization and generalization tasks, we successfully induce emergent behaviour into generalization tasks, shedding new light on the understanding of emergent abilities. Future work on more realistic tasks and models will be crucial for a deeper and more comprehensive understanding of deep learning mechanisms. 
 
 Appendix: Our experiments and analysis are all conducted on algorithm tasks like modular addition due to their simplicity. Since there is no noise data and no overlap between training and validation set, we can easily distinguish model's memorization and generalization behaviours, and therefore fully focus on model's generalization ability in this task. However, we believe our framework can also make some sense for more realistic setting and real tasks since the concept of memorization and generalization still hold under these circumstances. Further experiments on realistic setting will be an important future direction.Besides, we only consider pure memorization and generalization in this work. However, neural networks can encompass different circuit types that may also affect the model's ability to memorize and generalize. For instance, the communicative law in algorithmic tasks can introduce heuristic circuits that enhance the model's capacity to memorize training data. Although we have taken steps to eliminate such heuristic bias in our training data, other biases may still exist that we have not accounted for.Data Split Since all the algorithm tasks in this paper follow communicative law, we split train and validation data based on tuples to avoid heuristic generalization method, which means if (a + b) mod P in training set, then (b + a) mod P is also in training set. This is also the reason that communicative law can influence model's memorization capacity since it only needs to memorize half of the training data.We utilize a 1-layer simplified decoder-only transformer with 4 attention heads for most of our experiments, which don't have bias in linear layer and layernorm. We utilize ReLU as the activation function in MLP and the intermediate size of MLP is 4 times model's hidden size (dimension of model's embedding). We mainly modify model's hidden size without changing attention heads number to change model size except for experiments in § 5 and Appendix E, where we also increase the layer number to support larger models.Training Hyper-Parameters We implement our experiments based on the trainer of Huggingface transformers (Wolf et al., 2020) . During training, we utilize a dropout of 0.1 and a constant learning rate of 0.001. AdamW with weight decay of 1.0 is utilized for optimization.For larger models experiments in § 5 and Appendix E, we find high learning rate of 0.001 will cause the model even cannot memorize the training data, so we adjust the learning rate to 0.0005.As we have discussed in § 3, the concepts of progression and grokking differ in two fundamental ways. The first difference lies in the training accuracy at the point of generalization, a distinction that is clearly observable in Figure 1 and Figure 7 . The second difference pertains to the underlying reasons for the emergence of generalization abilities. In the case of progression, generalization is triggered by non-zero training loss by pure memorization, leading to a reduction in both training and validation losses by generalization. Conversely, grokking occurs when generalization circuits prove to be more efficient than those used for memorization, a phenomenon that is verified by Varma et al. (2023) . This distinction can also be highlighted by the observed trends in parameter norms: during the shift from memorization to generalization, grokking exhibits a decreasing parameter norm, whereas progression demonstrates an increasing parameter norm, as depicted in Figure 7 .Different from modular addition task, which we can split into non-overlap training and validation set, real tasks always show some overlap or similar data between training andIt is easier for model to memorize training data which obeys commutative law, since it only needs to memorize half of the training data.The "descent" in double descent refers to validation error/loss, therefore in terms of validation performance, it's validated by the two "increase" stage.Due to the small number of training data, grokking doesn't happen in this model size range. 


Decision: The reviewers are largely positive, seeing the theoretical construction and the empirical results as highly valuable. The paper is interesting and should improve our understanding of several well-studied phenomena.

The paper would be improved by weakening its claims to account for the possibility of other factors in abrupt emergence. In particular, I find the consistency of a "memorized" vs "generalized" circuit to be overstated; there are many heuristics that can be used for generalization, which are not necessarily described as memorization, as seen in the recency/positional vs hierarchical heuristics in [1]. The empirical findings, other than the added scale factor, are not novel, but the framing of those findings is. 

[1] Kabir Ahuja, Vidhisha Balachandran, Madhur Panwar, Tianxing He, Noah A. Smith, Navin Goyal, Yulia Tsvetkov (2024). Learning Syntax Without Planting Trees: Understanding When and Why Transformers Generalize Hierarchically. https://arxiv.org/abs/2404.16367

[comment from PCs] Please carefully consider the AC comments and follow their suggestions.

Reviews: review: paper_summary: paper_summary: This paper conducts an investigation into algorithmic tasks with a focus on the balance between memorization and generalization within deep neural networks. With a proposed theoretical framework grounded in two main hypotheses concerning the dataset size's criticality and the model's memorization capacity, the authors offer a graphical depiction of four training dynamics: progression, memorization, semi-grokking, and grokking. This framework is employed to illustraet the double descent phenomenon and link it with grokking, while also exploring multi-task learning, highlighting how generalization capabilities in algorithm tasks are significantly enhanced as model size increases.
strength_weakness: strength_weakness: Major one: The empirical substantiation of the framework's applicability across varied model architectures and dataset complexities, as the current examination might be perceived as somewhat constrained, and its real-world applicability and practical implications remain a bit unclear. Empirical validation, especially in more realistic situations and model families (e.g., LLMs), could provide a more robust underpinning for the theoretical claims made. Can you try to empirically verify this framework on small-size language models such as GPT-2 (124M) or small-size diffusion models?
questions: questions: see weaknesses.
paper_summary: paper_summary: This paper proposes a hypothesis on the relationship between dataset size, the number of parameters in a neural model, and the grokking phenomenon. The authors propose an account of four distinct training dynamics based in this explanation, and show empirical evidence verifying each prediction. There is also an exploration of the connection between these results to double descent losses, as well as emergent abilities during pretraining.

I found the paper mostly easy to follow. While the set of experiments at first seems like Varma et al. (2023) with the added variable of model size, I think the proposal that double descent and emergent abilities can be explained by the proposed model is ambitious and interesting.
strength_weakness: strength_weakness: 1. Could engage with the relevant literature much more thoroughly. For example, [1,2,3] should be cited when discussing grokking and emergent abilities. [4] should be cited when discussing how larger models have greater memorization capacity/memorize more quickly. The results should be contextualized with [5].
2. The way Section 5 is written, it sounds like “implicit multi-task learning setups during pretraining are the cause of emergent abilities.” I think a much clearer framing would be something more like “this is actually a case of delayed grokking that occurs when the training corpus can only be explained by a mixture of both memorization *and* generalization circuits.” Also, this may just be one variant of emergent abilities; perhaps more complex latent linguistic phenomena simply require more parameters to reliably acquire.
3. One could view this as a simple extension of Varma et al. (2023); the main empirical contribution seems to be the addition of the model size variable. I personally don’t find this to be a significant downside, given the interesting perspectives presented and the decently thorough set of experiments.

References:

[1] Merrill et al. (2023). A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks. https://openreview.net/pdf?id=8GZxtu46Kx

[2] Murty et al. (2023). Grokking of Hierarchical Structure in Vanilla Transformers. https://aclanthology.org/2023.acl-short.38/

[3] Barak et al. (2022). Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit. https://openreview.net/pdf?id=8XWP2ewX-im

[4] Tirumala et al. (2022). Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. https://proceedings.neurips.cc/paper_files/paper/2022/hash/fa0509f4dab6807e2cb465715bf2d249-Abstract-Conference.html

[5] Gromov (2023). Grokking modular arithmetic. https://arxiv.org/abs/2301.02679
questions: questions: **Suggestions**:
1. Fig. 3 shows an interesting non-linear shape. This would be interesting to discuss; what kind of relationship (e.g., logistic, logarithmic) do we expect between hidden size and memorization capacity? Is this really a relationship between number of parameters and memorization, or more of a relationship with width specifically?
2. In Figure 4, the legend should go outside the figures.
3. In general, it would be nice to use fewer evaluative words/phrases when describing the proposed work. For example, “fresh insights”, “successfully”

**Typos**:
- p.5: Memorizatioin -> Memorization
- p.7: “into one does” -> “into one that does”
paper_summary: paper_summary: This paper aims to analyze observations regarding training dynamics in neural network-based models (grokking, double descent, emergent abilities) under a unified lens of memorization/generalization competition, following the Varma et al.'s explanation of grokking. Specifically, the claim is that the interaction between two factors (inverse correlation between scale and critical dataset size for grokking, correlation between scale and memorization capacity) can provide an explanation for all of the aforementioned phenomena. 

* Quality: The hypothesis being tested is clearly stated and the experiments conducted to test it seem sound. 
* Clarity: The exposition and writing are generally easy to follow, barring a few minor suggestions to improve (see question to authors section for more details).
* Originality: The attempt to tie together the various observations about training and scaling dynamics in recent literature is novel. However, this is not my direct area of expertise so I haven't been following the relevant literature fully---I am basing my assessment based on the claimed contribution. 
* Significance: The work makes a "connecting the dots" type of contribution that can potentially generate a wide range of concrete predictions about training and scaling dynamics of Transformer models. Generalizability of the findings outside the exact setup (architecture, task) tested is limited, but the hypothesis and predictions laid out are concrete enough that their generalizability can be easily tested under more diverse settings.
strength_weakness: strength_weakness: Limited testing environments (architectural and task variation), but I don't think this is a strong reason to reject.
questions: questions: - I think the patterns visualized in Figure 4 doesn't itself serve as full evidence that the patterns observed during the course of training is exactly "grokking" or other phases---for example, the part of the graphs labeled "grokking", because in the graph itself just shows the endpoint of training (I assume), could equally be derived from a set of underlying models with training dynamics showing a fully overlapping train/val graphs from the beginning to end, with endpoints converging to full accuracy. I'm not saying this is the case for the results reported and I do believe that if we zoomed in to the individual models' training dynamics, they are probably adhering to the claimed patterns. I'm just saying that the visualizations from the Figure 4 themselves don't inherent imply/serve as evidence that the labels they are being assigned unless one actually looks at the full training dynamics for the individual cases, which might mean that a different set of metrics that can show signatures of the phases need to be devised in addition to simply visualizing the final training/val performance across varying model size.
- What did the performance on the memorization examples in the multitask experiment look like?
- I would also be interested in knowing what the lower bound of the memorization dataset size for observing the effect in Section 5 - the smallest size (3000) is still quite substantial since it matches the size of the modular addition task dataset. It would be very interesting if we observed such an effect with only a very small number of memorization examples.

More minor presentation suggestions:
- Currently the right panel of Figure 1 is serving a dual purpose (illustrative and also showing experimental results), and they seem to contain insufficient information to actually show the points of the experiments. For example, the text says "We first verify that modular addition task indeed show grokking in our experimental setup which is shown in Figure 1 (d).", but it is unclear what subset of the experiments that Figure (1d) is pointing to (especially in contrast to 1c---the details are only vaguely suggested: "model needs specific amount of training data")
- Typo: Memorizatioin -> Memorization
- (1c) doesn't seem to show "multiple plateaus" (at least in a way that significantly differs from plateaus in 1d)
paper_summary: paper_summary: This paper considers some of the most intriguing phenomena in the field (grokking, emergence, double descent), which have proven to be interesting yet poorly understood. The most direct contributions are some interesting findings that arise from systematic exploration of different model and data sizes. 

In short: 
1. Methodologically, I do not find the paper to make meaningful contributions. The core ideas has been identified in prior work, and the extension of sweeping systematically over model/data sizes is extremely natural. This is not to say the paper needs to provide methodological insight to be valuable, just that I don't see it as providing methodological insight.
2. With respect to rigor, I generally find the decisions and execution reasonably, though I did not interrogate them too deeply.
3. With respect to findings, I see this as the area where the paper shines. The figures, and the discussion of them, seem clearly of interest and make progress in advancing our understanding of these fascinating topics. 

So, on balance, I generally am positive on the paper, find that it certainly befits COLM, and think it will gain traction as an interesting paper that makes people think and galvanizes further research.
strength_weakness: strength_weakness: Given the emphasis placed on the framework, I am generally left dissatisfied: how do we arrive at this framework, why is it good, and how would it connect to a more theoretical account? I am not even sure it makes sense to call it a framework: it seems more reasonable to say it is a way of assigning terms to some phenomena, which perhaps helps build common terminology, but I am not sure this really advanced my conceptual understanding.

While the paper is clear in a certain structural sentence, I found the writing more confusing than helpful in most places, perhaps because it is meandering. Since this is a general comment, it is hard to isolate specific paragraphs that could benefit from re-writing, so I would say the key principle I feel is not upheld is each sentence does not have a clear rhetorical purpose, nor is the ordering of the sentences particularly deliberate as far as I can tell. As a result, while I don't object to the writing, I feel it will not only inhibit my understanding but that of other readers.
questions: questions: No specific questions - I will monitor the responses to other reviewers to see if I want to adjust my score.

If you want to best attempt to raise my score:
1. Convince me the framework is the right conceptual model
2. Tell me how you will rewrite parts to make the writing sharper

There are a lot of terms/concepts/ideas floating around in the paper. I would encourage adding some clear takeaways after each section or so on. If you want the reader to reminder some findings, make them more salient (in intro, in the actual content sections), because I would worry readers will sort of leave this paper with a muddled conceptual model of what it says a few days after reading it.

Comments: title: Reviewer response to rebuttal
Thank you
comment: Thanks for the response, I have read the discussion with other reviewers as well and will keep my score as-is and recommend acceptance to the AC.
Thanks for the response and the additional experiments! My rating was positive to begin with and I didn't find glaring issues from other reviews, so I'll maintain my original score.
Thanks for the response. It sounds like some of the more straightforward writing concerns (R1 and R2) will be addressed. I also appreciate the inclusion of these new results; these make Fig. 3 even more interesting! That said, R3 still holds.

Overall, I believe that this paper makes an interesting contribution and will generate interesting discussion at the conference if accepted. Given this and the responses to other reviewers, I am raising my score to an 8.