Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks

## Abstract

Abstract Grokking is the intriguing phenomenon of delayed generalization: initially, a network achieves a memorization solution with perfect training accuracy and limited generalization solution; however, through further training, it eventually attains a generalization solution.
This paper counters previous notions that weight norm reduction explains grokking, by demonstrating through experiments that the identification of optimal subnetworks plays a crucial role in achieving generalization. It leverages the lottery ticket hypothesis to argue that finding these ‘lottery tickets’ is key to transitioning from memorization to generalization. Our research presents empirical evidence, showing that (1) with the proper subnetworks, the delayed generalization does not occur, (2) with the similar weight norm, the dense networks still require substantially longer training to achieve full generalization, (3) with only structure optimization (without updating the value of weights), we can convert the memorization solution to the generalization solution. These results emphasize the importance of subnetwork identification over traditional weight norm reduction theories in explaining grokking’s delayed generalization phenomenon.

## 1 Intoroduction

Understanding the mechanism of generalization is a central question in understanding the efficacy of neural networks.
Recently, Power et al. () unveiled the intriguing phenomenon of *delayed generalization (grokking)*; neural networks initially attain a *memorization network* $C_{mem}$ with the perfect training accuracy but poor generalization, yet further training transitions the solution to a *generalized network* $C_{gen}$.
While the phenomenon is initially observed in the modular addition task ($(a+b)\bmod p$ for $a,b\in(0,\cdots,p-1)$), subsequent observations of the same phenomenon have been noted in more complex datasets, encompassing modular arithmetic (Gromov, ; Davies et al., ; Rubin et al., ; Stander et al., ), MNIST (Liu et al., ), semantic analysis (Liu et al., ), n-k parity (Merrill et al., ), polynomial regression (Kumar et al., ) and hierarchical task (Murty et al., ).

*What underlies the transition between memorization and generalization? *
To answer the question, several studies examine the relationship between representation learning and the aforementioned delayed generalization (Power et al., ; Liu et al., ; Nanda et al., ; Gromov, ).
For example, Power et al. (); Liu et al. () explained that the topology of the ideal embeddings tends to be circles or cylinders within the context of modular addition tasks.
Nanda et al. () identified the trigonometric algorithm by which the networks solve modular addition after grokking and showed that it grows smoothly over training. Gromov () showed an analytic solution for the representations when learning modular addition with MLP.
These studies support the quality of representation as key to distinguishing memorizing and generalizing networks; however, representation learning does not explain what is happening within the network.
In other words, representations are the result of updating the network, but to understand the mechanism of grokking, it is necessary to comprehend how the network is being updated and how its learning dynamics are changing.

Regarding the connection between the property of the network and the process of grokking, simplicity (the weight norm of network parameters) is a dominant explanation for the moment.
For example, the original paper (Power et al., ) posited that weight decay plays a pivotal role in grokking, i.e., test accuracy will not increase without weight decay.
Liu et al. () analyzed the loss landscapes of train and test dataset, verifying that grokking occurs even in general deep learning tasks by entering the generalization zone defined by L2 norm, with models having large initial values $\bm{\theta}_{0}$.
More recently, Varma et al. () hypothesized the existence of generalization circuits and memorization circuits and demonstrated that the generalization circuits could produce higher logits with smaller weight norms.

This paper makes several contributions to further understanding the mechanism of grokking.
Firstly, we provide the empirical results that can not be well explained by the explanation sorely based on the weight norm.
To do so, we introduce the notion of the *grokking ticket*, which bridges the *lottery ticket hypothesis* (Frankle & Carbin, ) and grokking.
Specifically, we show that when using the lottery ticket obtained using the generalization solution, the training accuracy and test accuracy increase almost similarly, i.e., the delay of the generalization, which happens when training dense networks, disappears when using the proper subnetworks.
In contrast, the dense networks that have the same initial weight norm (either L2 or L1) with the growing ticket still induce delayed generalization while it slightly reduces it.
Additionally, we conduct experiments comparing the grokking ticket with the same sparsity, which are identified by various pruning methods at initialization (random pruning, Grasp (Wang et al., ), SNIP (Lee et al., ) and Synflow (Tanaka et al., )). The results reveal that poor selection of subnetworks *increases* the delay of generalization. These results suggest that the explanation based on the simplicity of the network (smaller weight norm) is insufficient to explain the process of the grokking; rather, we hypothesize that the transition phase corresponds to the discovery of the good subnetworks.

It is worth noting that the discovery of good subnetworks is tightly aligned with the representation learning perspective of the grokking (Power et al., ; Liu et al., ; Nanda et al., ; Gromov, ).
To connect the subnetwork perspective and representation learning, we investigate the merits of the grokking ticket from a representation learning perspective following Nanda et al. ().
Specifically, we demonstrate that the grokking tickets help to learn good representation faster by introducing the metric to measure the goodness of the representation on the arithmetic dataset.
This result indicates that the grokking ticket corresponds to good representation beyond mere sparsity.

We conducted three experiments to verify our good subnetwork hypothesis further. Our main findings can be summarized as follows.
(1) The test accuracy and the similarity of the subnetworks correlate. We use the Jaccard similarity as a metric of structural similarity and show that the structure of the subnetwork is rapidly getting similar during the transition between memorization and generalization. (2) When using the grokking ticket and proper pruning ratio, weight decay is *not* necessary to induce the generalization. The result also indicates that weight decay is essential for uncovering lottery tickets but becomes redundant after their discovery.
(3) The memorization solution can be transferred into a generalization solution only via structure optimization, i.e., without optimizing the weights.
Specifically, we employ the edge-popup algorithm (Ramanujan et al., ), which finds the good structure via gradient decent while keeping the weights unchanged, and demonstrates that the model can reach a generalization solution from a memorization solution through pruning without weight updates.

In summary, our contributions are below:

- •
We link two distinct phenomena: grokking and the lottery ticket. Our results show that the use of the lottery ticket eliminates the occurrence of delayed generalization, which was observed in the original network.
- •
We also present experimental results that cannot be adequately explained by the common weight norm explanation used in prior studies, which challenge the current understanding of the grokking and suggest further investigation.
- •
We show that the memorization solution can be transformed into a generalization solution with only structure optimization (without updating the value of weights).

## 2 Preliminary

Grokking
This paper focuses on grokking on the classification tasks commonly used in grokking phenomenon studies (Power et al., ; Nanda et al., ; Liu et al., ; ).
Let us assume we have training datasets $\mathbb{S}_{train}$ and test datasets $\mathbb{S}_{test}$, and train a neural network $f(\bm{x};\bm{\theta})$ where $\bm{x}$ is an input and $\bm{\theta}$ represents weight parameters of the networks.
Specifically, the network is trained using stochastic gradient descent over a classification loss and weight decay (L2 norm of weights $\|\bm{\theta}\|_{2}$):

$$ $\underset{{\bm{\theta}}}{\operatorname{argmin}}~{}\mathbb{E}_{(\bm{x},y)\sim \bm{S}}\left[\mathcal{L}(f(\bm{x};\bm{\theta}),y)+\frac{\alpha}{2}\|\bm{\theta }\|_{2}\right],$ $$

where $y\in\{1,...,k\}$ represents a class labels corresponding to the inputs $\bm{x}$, and $\alpha$ denotes a weighting parameters.

Power et al. () demonstrated that the optimization process finds a memorization solution $C_{mem}$ at the earlier training but abruptly transitions to the generalization solution $C_{gen}$ in certain types of tasks and network architectures.
Here, $C_{mem}$ refers to the solution with nearly perfect training performance but poor test performance, whereas $C_{gen}$ represents the generalization solution.
Specifically, the training accuracy reaches $g\%$ after $t_{mem}$ iterations ($g$ is the 99% in the original paper), but test accuracy achieves it after $t_{gen}>>t_{mem}$ iterations (e.g., $t_{mem}$ is $10^{2}\sim 10^{3}$ and $t_{gen}$ is $10^{6}$ in the original paper).
This is a surprising finding since the loss of the network should be already saturated at the $t_{mem}$, and thus, the underlying mechanism during the transition between $C_{mem}$ and $C_{gen}$ (denoted as *transition phase*) is not trivial.
Throughout the paper, we denote the weights at iteration $t$ as $\bm{\theta}_{t}$, and $\bm{\theta}_{gen}$ and $\bm{\theta}_{mem}$ for the weights at the $\bm{\theta}_{t_{gen}}$ and $\bm{\theta}_{t_{mem}}$ respectively, with slightly abuse the notion.

Lottery ticket hypothesis
In this paper, intertwine the grokking phenomenon with other intriguing observations in training over-parameterized networks, *lottery ticket hypothesis* (Frankle & Carbin, ).
Informally, the lottery ticket hypothesis states that randomly initialized over-parameterized networks include sparse subnetworks that reach good performance after train, and the existence of the subnetworks is key to achieving good generalization in deep neural networks.
More formally, et $f(\bm{x};\bm{\theta}_{0})$ be a dense network whose weights are randomly initialized ($\bm{\theta}_{0}\sim D_{\bm{\theta}}$).
After training $f(\bm{x};\bm{\theta_{0}})$ on a training dataset for $j$ iterations, it reaches a minimum validation loss with a test accuracy $a$.
Then, there exists a mask $\bm{m}\in\{0,1\}^{|\bm{\theta}_{0}|}$ such that $f(\bm{x};{\bm{\theta}_{0}}\odot\bm{m})$ reaches a test accuracy $a^{\prime}$ after training for $j^{\prime}$ iterations on the same setting, where $a^{\prime}\geq a,~{}j^{\prime}\leq j$, and $~{}\|\bm{m}\|_{0}\ll|{\bm{\theta}}|$.
Here, $\|\bm{m}\|_{0}$ is the number of non-zero elements of the mask $\bm{m}$.
This paper refers to the mask itself or the sparse subnetworks $f(\bm{x};\bm{m}\odot\bm{\theta}_{0})$ as lottery ticket.
Specifically, after training neural networks for a certain number of iterations and obtaining $\bm{\theta_{T}}$, we select the top $(1-k)$ weights with the largest absolute values (we refer to $k$ as pruning rate).
This process is called the *magnitude pruning*.
Furthermore, to discuss the importance of pruning in grokking, we use the edge-popup algorithm (Ramanujan et al., ), which learns the method of pruning without weight updates, which selects subnetworks based on a score $s~{}(s\in\mathbb{R},s\leq 0)$. In other words, when pruning a certain proportion $k$ of weights from the given weights $\bm{\theta}_{0}$, the model predicts using edges with the top $(1-k)$ scores in a forward pass. For a detailed description of edge-popup, refer to G.1.

## 3 Bridging Grokking and Lottery Ticket

In this section, we explain the experimental settings we deal with and introduce the concept of *Grokking ticket*, connecting the lottery ticket and grokking. We also demonstrate the validity of the grokking ticket as an explanation for delayed generalization.

### 3.1 Experiment Setup

Modular Addition
Following the approach in the original paper (Power et al., ) and other grokking literatures (Nakkiran et al., ; Liu et al., ; Gromov, ; Liu et al., ), we constructed a dataset of equations of the form:
$(a+b)\%p=c$.
The task involves predicting $c$ given a pair of $a$ and $b$.
We utilize the following detailed configurations in our setup: $p=67$, $0\leq a,b,c<p$.
Considering all possible pairs of $a$ and $b$, the dataset size is 4489. We split the dataset into training (40%) and test (60%).

#### MLP

Following the approach in Liu et al. (), we design the MLP as follows. Firstly, we map the one-hot encoding of $\bm{a}$, $\bm{b}$ with the embedding weights $W_{emb}$: $E_{a}=W_{emb}\bm{a},E_{b}=W_{emb}\bm{b}$.
We then feed the embeddings $E_{a}$ and $E_{b}$ into the MLP as follows:

$$ $\mathrm{softmax}(\sigma((E_{a}+E_{b})W_{in})W_{out}W_{unemb}),$ (1) $$

where $W_{emb}$, $W_{in}$, $W_{out}$, and $W_{unemb}$ are the trainable parameters, and $\sigma$ is an activation function (ReLU Nair & Hinton ()).
The dimension of the embedding space is 500, and $W_{in}$ projects into 48-dimensional neurons. We refer to this model as the base model.

#### Optimization

Following (Nanda et al., ), we used the AdamW optimizer (Loshchilov & Hutter, ) with a learning rate $10^{-3}$, the weighting of weight decay $\alpha=1.0$, $\beta_{1}=0.9$, and $\beta_{2}=0.98$.
We initialize weights as follows:

$$ $\bm{\theta}_{0}\sim\mathcal{N}(0,\kappa/\sqrt{d_{in}})$ $$

where $d_{in}$ represents the dimensionality of the layer preceding each weight. If nothing is specified, assume $\kappa=1$.

### 3.2 Grokking Ticket

Figure: Figure 1: Test accuracy of grokking ticket and base model. The grokking ticket shows an improvement in test accuracy at nearly the same time as the improvement in training accuracy of the base model.
Refer to caption: x1.png

In this section, we introduce the concept of *grokking ticket*, and demonstrate their validity as an explanation for delayed generalization.
We find a grokking ticket as follows:

- 1.
Randomly initialize a neural network $f(.;\bm{\theta}_{0})$.
- 2.
Train the network using section 2 until full generalization, arriving at parameters $\bm{\theta}_{gen}$.
- 3.
Create a mask vector $\bm{m}$ by pruning $k$% of the parameters in $\bm{\theta}_{gen}$ through magnitude pruning.
- 4.
Reset the remaining parameters to their values in $\bm{\theta}_{0}$, and obtain the *grokking ticket* $f(\bm{x};\bm{m}\odot\bm{\theta}_{0})$.

Without mentioning otherwise, a prune rate of $k=0.6$ is used throughout experiments.

Figure: Figure 2: (Left) Test accuracy dynamics of the base model, grokking ticket, and controlled dense model (L1 norm and L2 norm). The grokking ticket reaches generalization much faster than other models. (Center) L2 norm dynamics of the base model, grokking ticket, and controlled dense model. (Right) L1 norm dynamics of the base model, grokking ticket, and controlled dense models. From the perspective of L2 norm, all models appear to converge to a similar solution (Generalized zone). However, from the perspective of L1 norm, they converge to different values.
Refer to caption: x2.png

We refer to the subnetwork that is found before reaching generalization as a ‘non-grokking ticket.’
Figure 1 shows, using MLP and modular addition, the test accuracy of the grokking ticket and the base model. The base model refers to a dense model trained from random initial values. The grokking ticket shows an improvement in test accuracy at nearly the same time as the improvement in training accuracy of the Base model. Furthermore, the results indicate that non-grokking tickets do not lead to generalization, regardless of the epoch in which they were acquired.
This result shows that if a good subnetwork is discovered, delay generalization does not occur, suggesting that the lack of a good structure is an important factor in causing delay generalization.
We demonstrated that the grokking ticket accelerates generalization in various configurations, including Transformer, MNIST, and other modular arithmetic in Appendix A. Additionally, the effect of the pruning rate on the grokking ticket can also be found in Appendix C.

## 4 Is Weight Norm Sufficient to Explain Grokking?

As described in the introduction, the weight norm is regarded as a key ingredient to explain the mechanism of grokking Liu et al. (); Varma et al. ().
In this section, we first provide experimental results to counter the explanation. Firstly, we conduct a series of controlled experiments using the aforementioned grokking ticket and show that even when we control the initial network’s weight norm, the test accuracy transition is significantly varied.
Additionally, compared to networks with the same sparsity, we show that the grokking ticket possesses good properties beyond mere sparsity.
Finally, we also connect the grokking ticket with the representation learning perspective, which is another well-investigated perspective of the grokking.

### 4.1 Controlling Weight Norm of Initial Network

Liu et al. (); Varma et al. () stated that the norm of a network’s weights is a key factor for grokking. Liu et al. () assert that grokking occurs norm of the weights enter the ‘Generalized zone’ (Fort & Scherlis, ) through regularization.
In this section, we show that during training, the model does not search for a ‘Generalized zone’ defined by L2 norms but searches for a Grokking ticket that generalizes.
In order to investigate whether the L2 norm or the grokking ticket are more plausible explanations of the grokking mechanism, we prepared two dense models with equal L2 and L1 norms, named ‘controlled dense model’.
Such dense models are obtained through the following process:

- 1.
Obtain grokking ticket in the approach described in subsection 3.2.
- 2.
Get weight $L_{p}$ norm ratio $r_{p}=\frac{\|\bm{\theta}_{0}\odot\bm{m}\|_{p}}{\|\bm{\theta}_{0}\|_{p}}$
- 3.
Create weights $\bm{\theta}_{0}\cdot r_{p}$ with the same $L_{p}$ norm as the grokking ticket.

Figure 2 (left) shows the test accuracy dynamics of the base model, grokking ticket, and controlled dense model. Despite having the same initial weight norms, the grokking ticket arrives at generalization faster than controlled dense models (L1 norm and L2 norm).
Specifically, the grokking ticket generalizes almost simultaneously with the training accuracy of the base model. These results show that the delay increase in test accuracy is attributable not to the weight norms but to the discovery of good subnetwork.
The middle of the Figure 2 shows the dynamics of the L2 norms for each model. Similar to Liu et al. (), it is evident that the L2 norms decrease in correspondence with the rise in test accuracy, converging towards a ‘Generalized zone’. However, as shown on the right side of the Figure 2, the final convergence points of the L1 norms vary for each model.
This phenomenon of having similar L2 norms but smaller L1 norms suggests that subnetworks (grokking ticket) weights become stronger, as indicated in Miller et al. (). Similar results have been observed in Transformer as well (Figure 11).
These results demonstrate that weight norms are insufficient as an explanation for grokking.We have also included similar results for transformers in Appendix B.

### 4.2 Controlling Sparsity

In subsection 4.1, we show the discovery of the sparse network is a critical factor to explain the transition of the memorization to generalization.
However, it raises questions about whether sparse models are crucial for grokking or grokking tickets possess good properties beyond mere sparsity.

In this section, by using various pruning methods, we show that mere sparsity is insufficient.
We compared the grokking ticket with the subnetworks identified differently.
Specifically, we tested three well-known pruning at initialization (PaI) methods (Grasp (Wang et al., ), SNIP (Lee et al., ), and Synflow (Tanaka et al., )) and random pruning as baseline methods.
For details on each of the pruning methods, refer to the section Appendix F.
Figure 3 compare the transition of the test accuracy of these PaI methods and the grokking ticket.
The results show that all PaI methods perform worse than the base model or, in some cases, perform worse than the random pruning.
The results show that poor selection of the subnetworks hurts the generalization, suggesting that the grokking ticket hold good properties beyond just a sparsity.

Figure: Figure 3: Comparing test accuracy of the different pruning methods. All PaI methods perform worse than the base model or, in some cases, perform worse than the random pruning.
Refer to caption: x3.png

### 4.3 Linking Grokking Ticket and Representation

Figure: (a) Base model, 0k
Refer to caption: x4.png

In subsection 4.2, we demonstrate that grokking tickets possess good properties beyond mere sparsity.
In this section, we clarify the relationship between the acquisition of good representations and the discovery of good subnetworks (grokking ticket), both qualitatively and quantitatively.
First, following Pearce et al. (), we investigated the periodicity of each neuron by plotting the neuron direction of the weight matrix ($W_{in}$) obtained by multiplying $W_{emb}$ and $W_{inproj}$ (defined in subsection 3.1). This two-dimensional weight matrix represents the input dimensions (67) on one axis and the number of neurons (48) on the other. Figure 4 shows the transition of the input-side weights for each neuron on each phase (0k: initialization, 2k:memorization, 30k:generalization). While in the initialization and memorization phase, no specific frequency is observed in each neuron, after grokking (generalization phase), each neuron’s weight of the base model shows a specific frequency. On the other hand, a grokking ticket has neurons with periodic weights similar to that observed after grokking appears at an early stage (2k epochs). This result shows finding a good subnetwork generalizes faster since it finds better representation faster.

Subsequently, to examine the frequency characteristics of each neuron, we performed frequency decomposition of weight values for each neuron using a discrete Fourier transform.
The discrete Fourier transform of $f(x)$ is as follows:

$$ $F(\omega)=\sum_{x=0}^{N-1}{f(x)\exp(-i\frac{2\pi\omega x}{N})}$ $$

The number of sample points ($N$) is 67, the same as the input dimension.
We conduct this discrete Fourier transform for both the input-side weights.

Figure: Figure 5: Frequency entropy and test accuracy of the base model and the grokking ticket. The grokking ticket converges to a smaller frequency entropy faster than the base model, indicating the acquisition of better features.
Refer to caption: x10.png

For the results of this discrete Fourier transform refer to Appendix E.
Finally, to quantitatively evaluate the representation learning of the base model and grokking ticket, we compared the *frequency entropy* of the weights for each neuron.
The frequency entropy is as follows:

$$ $H(p)=-\frac{1}{K}\sum_{i=0}^{K}{\sum_{\omega=0}^{[\frac{p}{2}]}{F_{i}(\omega) \log{F_{i}(\omega)}}}$ $$

The number of neurons ($K$) is 48, the number of frequency points ($[\frac{p}{2}]$) is 33 and $p_{i}(t)$ is the normalized value at the $t^{th}$ sample point for the $i^{th}$ neuron.
Figure 5 show the *frequency entropy* of the base model (green) and grokking ticket (blue). In the base model, corresponding to the increase in test accuracy, the frequency entropy decreases (i.e., it converges to specific frequencies).
On the other hand, the grokking ticket exhibits a rapid decrease in entropy compared to the base model. This indicates that the complexity of the frequency characteristics decreases in the grokking ticket, leading to the acquisition of a simpler representation.

## 5 A More Plausible Hypothesis: The Discovery of Good Subnetwork

In section 4, we demonstrate that weight norms are insufficient as an explanation for delayed generalization. *So, what is the true cause of delayed generalization?*
Our hypothesis posits that the cause is the exploration of structure.
If this hypothesis is correct, then the following outcomes should be observed.
(1) Discovery of the grokking ticket corresponds to an improvement in test accuracy. (2) With precise pruning ratios, the grokking ticket does not require weight decay to generalize. (3) Memorization solution ($C_{mem}$) can be transformed into generalization solution ($C_{gen}$) with pruning only without weight update.

### 5.1 Correlation Between the Discovery of Grokking Ticket and Test Accuracy

Figure: Figure 6: The Jaccard similarity between each on each layer and test accuracy.
Refer to caption: x11.png

In this section, we conduct a more rigorous analysis of how the grokking ticket is acquired, and we show that the discovery of the grokking ticket corresponds to an improvement in test accuracy.
Firstly, we propose a metric of structural changes in the network, named *Jaccard Simirality* (JS) using approach Paganini & Forde (); Jaccard ().
We measured the similarity between the masks of two masked networks $(\bm{m_{1}},\bm{m_{2}})$ using the following metric:

$$ $\mathrm{JS}(\bm{m_{1}},\bm{m_{2}})=\frac{|\bm{m_{1}}\cap\bm{m_{2}}|}{|\bm{m_{1 }}\cup\bm{m_{2}}|}$ $$

If this metric is close to 1, it indicates that the structures are similar to each other and vice versa.
In Figure 6, the red line shows the JS between the masks at varying epochs and the grokking ticket’s mask for each layer. The result shows that as test accuracy (green) increases, the JS (red) increases on every layer.
In other words, this result suggests that the discovery of grokking tickets is a plausible explanation for the increase in test accuracy.

### 5.2 Weight Decay work as Structure Explorer

In this section, we show that with precise pruning ratios, the grokking ticket does not require weight decay to generalize, indicating that weight decay is essential for uncovering good subnetworks but becomes redundant after their discovery.
We first explore the *critical pruning ratio*, which is the maximum pruning rate that can induce grokking (Figure 7-a).
In this case (Figure 7-a), we recognize that the critical pruning rate is between 0.8 and 0.9 because if the pruning rate increases to 0.9, the test accuracy dramatically decreases.
Thus, we gradually increased the pruning rate in increments of 0.01 from 0.8 and found that the $k=0.81$ is the critical pruning ratio.
We then compare the behavior of the grokking ticket without weight decay ($\alpha=0.0$) and the base model.
Figure 7-b show the results of the experiments. As shown in the figure, the test accuracy reaches perfect generalization without weight decay.
The results show that the grokking ticket with the critical pruning ratio does not require any weight decay during the optimization.

Figure: (a) Test accuracy with different pruning rate
Refer to caption: x12.png

### 5.3 Transforming θ m ⁢ e ⁢ m subscript 𝜃 𝑚 𝑒 𝑚 \theta_{mem} italic_θ start_POSTSUBSCRIPT italic_m italic_e italic_m end_POSTSUBSCRIPT into θ g ⁢ e ⁢ n subscript 𝜃 𝑔 𝑒 𝑛 \theta_{gen} italic_θ start_POSTSUBSCRIPT italic_g italic_e italic_n end_POSTSUBSCRIPT without Updating Weights

In this section, we demonstrate that the memorization solution ($C_{mem}$) can reach the generalization solution ($C_{gen}$) with pruning only without weight updates.
To verify this, we introduce edge-popup Ramanujan et al. (), a method that learns how to prune weights without weight updates.
We validate our claim by optimizing using three different methods:

- 1.
Training the model from random initial values ($\bm{\theta}_{0}$) with weight update (same as base model).
- 2.
Training the model from random initial values ($\bm{\theta}_{0}$) by edge-popup without weight update.
- 3.
Training the model from the memorization solutions ($\bm{\theta}_{mem}$) by edge-popup without weight update.

Malach et al. (); Pensia et al. () suggested that any model can be approximated by merely pruning a wider random model. Therefore, we prepare a model with its width expanded by a factor of $\gamma$ and proceed to validate this assertion. In this experiment, $\gamma$ values are 1, 4, 8, and 16, and the pruning rate of edge-popup is set at 0.6.
Figure 8 shows test and train accuracy with various model widths with weight update (green), edge-popup from $\bm{\theta}_{0}$ (blue), and edge-popup from $\bm{\theta}_{mem}$(red).
At the 400 epoch (the dashed vertical line), all models learned with weight update exhibit low generalization performance and high training accuracy. Therefore, we designate the weights at 400 epochs as the $\bm{\theta}_{mem}$.

In edge-popup from $\bm{\theta}_{0}$ (blue), the results show, particularly in models with smaller widths, that generalization solutions could not be achieved.
However, in the edge popup from the $\bm{\theta}_{mem}$ (red), it is observed that all models can reach the generalization solution without any change in weights (merely by pruning weights). Moreover, they are able to generalize faster than with weight updates (green).
This result demonstrates that it is possible to transition from a memorization solution to a generalization solution solely through pruning, without updating the weights. This indicates that the discovery of a good subnetwork is occurring between memorization and generalization.

Figure: Figure 8: Test accuracy and Train accuracy with various model widths through weight update (green), edge-popup from random initialization (blue), and edge-popup from memorization (red). We designate the memorization solution at the 400 epoch (indicated by the dashed vertical line). The model can reach a generalization from a memorization solution without weight update.
Refer to caption: x14.png

## 6 Related Works and Discussion

In this paper, we conducted a set of experiments to understand the mechanism of grokking (delayed generalization). Below is a summary of observations.
(1) Good subnetworks are a more important factor to explain grokking than the weight norm by comparing it with the same level of weight norm.
(2) With the same level of sparsity, the grokking ticket shortens the transition between memorization and generalization solution compared to random pruning and well-known pruning at initialization methods.
(3) Both qualitatively and quantitatively, finding a grokking ticket corresponds to the acquisition of representations.
(4) Grokking ticket is gradually discovered, corresponding to test accuracy.
(5) Weight decay is not needed as long as the grokking ticket is present.
(6) Simply pruning without updating weights from a memorization solution leads to an increase in test accuracy.
In this section, we connect these results with a prior explanation of the mechanism of grokking and relevant related works.

### 6.1 Weight norm reduction

Liu et al. (; ); Varma et al. () showed weight decay plays a pivotal role in grokking, suggesting that the reduction of weight norms is crucial for generalization.
Our work is well aligned with these studies in the sense that these studies assume the simplicity of the solution is key to achieving the generalization solution. However, our studies further suggest that the weight norms are insufficient as an explanation for grokking.

### 6.2 Representation learning

Liu et al. (); Nanda et al. (); Liu et al. (); Gromov () showed the quality of representation as key to distinguishing memorizing and generalizing networks.
In Figure 4, we demonstrate that the grokking ticket contributes to the acquisition of a good representation. Not only that, but our results also demonstrate how the network is updated during the process of grokking, connecting the lottery ticket hypothesis.

### 6.3 Sparsity and Efficiency

Merrill et al. () argued that the grokking phase transition corresponds to the emergence of a sparse subnetwork that dominates model predictions.
They empirically studied the internal structure of networks undergoing grokking on sparse parity tasks.
On the other hand, we are conducting tasks (modular arithmetic, MNIST) commonly used in grokking research and architecture (MLP, Transformer), and we are pruning weights in connection with the lottery ticket hypothesis. Furthermore, we quantitatively demonstrate how these subnetworks are acquired, as well as their relationship to representation learning.

### 6.4 Lottery ticket Hypothesis

It is worth noting that it is commonly observed that the lottery ticket (good subnetworks) helps to accelerate that generalization .Frankle & Carbin (), which is closely related to the observations in our experiments in section 3. However, this is the first study that connects the lottery ticket with grokking, i.e., explaining the delayed generalization via the lottery ticket. None of the prior studies connect lottery tickets with grokking.
Furthermore, we also analyze the merits of lottery tickets from a representation perspective by conducting the analysis introduced in the context of grokking.

## Appendix A Different configurations of the task and the architecture.

### A.1 MLP for MNIST

We use 4-layer MLP for the MNIST classification.
The difference from regular classification is that we are using Mean Squared Error (MSE) for the loss. We adopted this setting following prior research (Liu et al., ). In (Liu et al., ), it was confirmed in the Appendix that grokking occurred without any problems even when trying with cross-entropy.
Figure 9 show the test and train accuracy on various configurations. It is evident that grokking ticket accelerate generalization in all configurations, and the exploration of grokking ticket contributes to generalization.

### A.2 Transformer for modular addition

Similar to Nanda et al. (), we use 1-layer transformer in all experiments. We use single-head attention and omit layer norm.

We denote our hyperparameters as follows: $d_{vocab}=67$ is the size of the input and output spaces, the same as $p$, $d_{emb}=500$ is the embedding size, $d_{mlp}=128$ is width of the mlp layer.

We denote the parameters as follows: $W_{E}$ (embedding layer), $W_{pos}$ (positional embedding), $W_{Q}$ (query) ,$W_{K}$ (keys), $W_{V}$ (values), $W_{O}$ (attention output), $W_{in}$ and $b_{in}$ (the first layer weights and bias of the MLP ), $W_{out}$ and $b_{out}$ (the second layer weights of the MLP) and $W_{U}$ (unembedding layer).

We describe the process of obtaining the logits for a single-layer model. Note that loss is only calculated from the logits on the final token. We denote the token in position $i$ at $l$ layer as $x_{i}^{(l)}$. Note that $i$ is 0 0 or 1, because the number of input tokens is 2 and $x_{i}^{(0)}$ is one-hot vector. We denote the attention scores as $A$. We denote the triangular matrix with negative infinite elements as $M$, which is used for causal attention.
The logits are calculated via the following equation:

$$ $\displaystyle\begin{split}x_{i}^{(1)}&=W_{E}x_{i}^{0}+W_{pos}x_{i}^{(0)}\\ A&=\mathrm{softmax(x^{(1)T}W_{K}^{T}W_{Q}x^{(1)}-M)}+x^{(1)}\\ x^{(2)}&=W_{0}W_{V}(x^{(1)}A)+x^{(1)}\\ x^{(3)}&=W_{out}\mathrm{ReLU}(W_{in}x^{(2)}+b_{in})+b_{out}+x^{(2)}\\ \mathrm{logits}&=\mathrm{softmax}(W_{U}x^{(3)})\\ \end{split}$ $$

Figure: Figure 9: Comparison of base model (green) and Grokking ticket (blue). Each column corresponds to the different configurations of the task (Modular Addition and MNIST) and the architecture (MLP and Transformer). The dashed line represents the results of the training data.
Refer to caption: x15.png

### A.3 Modular Arithmetic

Following (Power et al., ), we tried several other operations (for a prime number $p=67$).

$$ $\displaystyle\begin{split}a+b\quad(mod\>p)&\quad for\;0<a,b<p\\ a-b\quad(mod\>p)&\quad for\;0<b<a<p\\ ab\quad(mod\>p)&\quad for\;0<a,b<p\\ a^{2}+b^{2}\quad(mod\>p)&\quad for\;0<a,b<p\\ \end{split}$ $$

Figure 10 shows comparison of base model (solid line) and grokking ticket (dashed line) with various operations. It is evident that Grokking ticket reach generalization faster than the base model on various task , indicating that the analysis is independent of the task. We also tested random label and of course, the accuracy is random.

Figure: Figure 10: Test accuracy of base model and Grokking ticket on various datasets.
Refer to caption: x16.png

## Appendix B Is Weight Norm Sufficient to Explain
Grokking on Transformer?

### B.1 Controlling Weight Norm OF Initial Network

Figure 11 show the accuracy of base model, Grokking ticket and Controlled dense (L1 and L2) on Transformer. The results show Grokking ticket generalize faster than any other model.
The results suggests that even in the case of Transformer, the discovery of Grokking ticket is more important than weight norms.

Figure: Figure 11: Accuracy of base model, Grokking ticket and Controlled dense on Transformer.
Refer to caption: x17.png

## Appendix C Quantifying Pruning: An In-Depth Analysis of Model Simplification

In Figure 1, we only tested when the pruning rate was $0.6$.
The left panel of Figure 12 depicts the test accuracy across various pruning rates, while the right panel demonstrates the correlation between pruning rate and the epoch until generalization.
A pruning rate of $0.0$ represents the base model because no weights are pruned and the model is rewound to its initial weights.
We can make the following observations:
(1) Most pruning rato (0.1, 0.3, 0.5 and 0.7) accelerate the generalization, indicating that the above observation does not heavily depend on the selection of the pruning rato. When the pruning rate is 0.5, the speed of grokking is the fastest, according to the right figure.
(2) At the same time, with a higher pruning rate, such as 0.9, the grokking phenomenon is not observed, which might be due to the lack of capacity.
Even with a low pruning rate, grokking occurs, but it doesn’t happen when the rate is too high. This result implies that the model still has redundant weights, which are likely removed by weight decay.

Figure: (a) Test accuracy with different pruning rate
Refer to caption: x18.png

## Appendix D The effect of Dead neuron on Groking ticket

We investigated the periodicity of weights during grokking and the relationship with grokking ticket. During this process, neurons appeared whose input-side weights became all zeros. We refer to these neurons as ”Dead neurons“. To investigate the impact of these neurons on grokking ticket, we conducted the following two experiments:

- 1
Only the weights attached to the dead neurons were pruned. (Kill only the dead neurons)
- 2
The masks on the weights attached to the dead neurons were set to 1. (Revive the dead neurons)

In this experiment, the edges connected to dead neurons constituted 18.75% of the total.
In Figure 13, when only the dead neurons were killed, grokking did not accelerate (Kill only dead neurons).
This result implies that the acceleration of generalization is not simply due to a reduction in the parameters to be optimized, as would be the case if considering the impact of dead neurons.
When the masks of dead neurons were set to 1 while keeping the other masks intact, the speed was almost the same as that of grokking ticket in Figure 13 (Revive dead neurons).
This suggests that it’s not the elimination of neurons that matters, but rather the significance lies in the other structures. It indicates that a kind of functional bias towards the task is being acquired as a structure.

Figure: Figure 13: Test accuracy of base model and Grokking ticket on various datasets.
Refer to caption: x20.png

## Appendix E Discrete fourier transform

Figure 14 shows that Fourier transform ($F(\omega)$ of the inside-weights ($W_{in}$) of each neurons.
After generalization, the frequency characteristics of most neurons in the base model are prominent for a specific frequency.
In Grokking ticket, neurons responsive to specific frequencies emerge at an early stage. (2k epochs).

Figure: (a) base model, 0k
Refer to caption: x21.png

## Appendix F Pruning at initialization methods

Currently, the methodologies of pruning neural networks (NN) at initialization (such as SNIP, GraSP, SynFlow) still exhibit a gap when compared to methods that use post-training information for pruning (like Lottery Ticket). Nonetheless, this area is experiencing a surge in research activity.

The basic flow of the pruning at initialization is as follows:

- 1.
Randomly initialize a neural network $f(\bm{x};\bm{\theta}_{0})$.
- 2.
Prune p% of the parameters in $\bm{\theta}_{0}$ according to the scores $S(\bm{\theta})$, creating a mask m .
- 3.
Train the network from $\bm{\theta}_{0}\odot\bm{m}$.

According to Tanaka et al. (), research on pruning at initialization boils down to the methodology of determining the score in the above process 2, which can be uniformly described as follows:

$$ $S(\bm{\theta})=\frac{\partial R}{\partial\bm{\theta}}\odot\bm{\theta}$ $$

When the $R$ is the training loss $L$, the resulting synaptic saliency metric is equivalent to $|\frac{\partial L}{\partial\bm{\theta}}\odot\bm{\theta}|$ used in SNIP (Lee et al., ).
$-(H\frac{\partial L}{\partial\bm{\theta}})\odot\bm{\theta}$ use in Grasp (Wang et al., ).Tanaka et al. () proposed synflow algorithm $R_{SF}=1^{T}(\prod_{l=1}^{L}|\bm{\theta}^{|l|}|)1$. In section 4.2, all initial values were experimented with using the same weights and the same pruning rate.

## Appendix G Edge-popup algorithm

In the edge-popup algorithm, the way of pruning is learned by updating the scores of each weight without changing their values.

###### Definition G.1 .

(edge-popup Algorithm)
If $w_{uv}\mathcal{Z}_{u}$ denotes the weighted output of neuron $u$, $\mathcal{I}_{v}$ denotes the input of neurons $v$, and then the score $s_{uv}$ is update as

$$ $s_{uv}\leftarrow s_{uv}-\alpha\frac{\partial\mathcal{L}}{\partial\mathcal{I}_{ v}}\mathcal{Z}_{u}w_{uv}$ $$

###### Definition G.1 .

## Appendix H Comparing Models

Table 1 compares the speed of generalization for each model. The results show that Grokking ticket are significantly faster than the same weight norm (Controlled dense) and that Edge-pop from memorization is also faster than Controlled dense). In this case, we calculate $t_{gen}$ with a generalization threshold of 90%.

**Table 1: Comparison of the generalization speed for each model.**
|  | Base model | Grokking ticket | Controlled L1 | Controlled L2 | PaI ticket | Edge-pop from $\bm{\theta}_{mem}$ |
| --- | --- | --- | --- | --- | --- | --- |
| $t_{gen}$ | 21497 | 566 | 15364 | 9659 | 35297 | 6611 |

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. It would be great if you provide more theoretical insight into why it is enough for generalization to have the presence of certain subnetworks (grokking tickets)? It would be interesting to know if there is an undercurrent mechanism, besides the empirical evidence, that could be given to support the idea that it was really the subnetworks that drive the transition from memorization to generalization and not the weight norm. Lastly, would it be possible to learn other relevant theories on generalization, for example, simplicity bias or double descent, which could place these findings into a wider context?

2. Although the paper does very nicely on modular arithmetic and MNIST, how would grokking tickets generalize to a few domains, perhaps such as NLP, like using a model such as BERT? Or image recognition, such as applying a CNN to CIFAR-10? This would help in generalizing the results. Can the authors comment on whether grokking tickets applies to other forms of tasks or known limitations for those other domains?

3. Since the results show generalization with the appropriate subnetworks instead of depending on weight decay, do the authors have any insights on practical takeaways? For instance, how do these grokking tickets that were discovered alter regimes of training, pruning of models, and the choice of architecture? Including more specific recommendations on how to leverage grokking tickets in practice better opens up findings for practitioners to put into action.

4. The paper introduces Jaccard similarity to measure structural similarity between subnetworks, yet does not delve enough into the implications of this metric. Could the authors go into additional detail about which variations in structural similarity correlate with stages of grokking? Specifically, does some degree of Jaccard similarity correspond to a "critical" sub-network structure predictive of successful generalization? Further research on the nature of structural similarity resulting from training might shed further light on grokking tickets.

5. The authors introduce a critical pruning ratio, such as 0.81, needed to achieve generalization without weight decay. Would the authors comment upon how that ratio could vary by architectures and datasets? A more in-depth examination of how the result is sensitive to this pruning rate and others can help to explain how reliably grokking tickets can be identified across different setups.

6. While the authors have employed Jaccard similarity and frequency entropy to compute quality metrics of subnetworks, do they also explore other metrics for validation of soundness of their results? For example, capturing the degree of similarity in activation of neurons or any such sparsity-based metrics might prove the significance of certain structures of subnetworks as well. This will likely reiterate that in fact subnetworks are prime essentials for generalization.

7. The authors show that typical pruning-at-initialization methods, such as SNIP and Synflow, cannot efficiently produce grokking tickets. Are the authors willing to provide more analysis into why such classical PaI methods do not elicit generalization? For example, is something inherently different between the identified subnetworks by the former methods compared to grokking tickets? This comparison would help clarify the unique properties of grokking tickets and guide future improvements in these techniques.

8. The experimental results seem to suggest that the structures of subnetworks change over time. Can the authors provide some illustrations-for example, weight heatmaps or connectivity graphs of subnetworks-probing the evolution through multiple training phases? Such illustrations would make the process more intelligible, where grokking tickets form and generalize, and serve to additionally demonstrate their role in delayed generalization.

9. The paper records that grokking tickets facilitate the generalisation when compared with dense networks. Is it possible for the authors to probe for a predictable point during the training regime where these sub-networks first arise? Investigating whether there is some measurable "onset" of generalization with grokking tickets, maybe utilising Jaccard similarity or other measures, might reveal important transition points in the training regime.

10. Some phenomena of generalization, like the double descent phenomenon, have been studied to some large extent. Would it be interesting if the authors were able to delineate how grokking tickets might relate to these other phenomena? For example, is an appearance of grokking tickets associated with first descent in a curve describing a situation of double descent? If one is able to draw these relations, it may give grokking tickets a context of being part of a bigger picture of generalization dynamics of neural networks.
soundness: 2
strengths: Originality is demonstrated because the author tries to bridge two concepts that were seen independently before: the lottery ticket hypothesis and grokking. To put it in words, this can be shown through the proposition that the primary driving force behind grokking is not weight norm reduction but identification of a special subnetwork of "grokking tickets". Fresh angle of attack, challenging explanations of the form: what's really critical here is not just the simplicity of the model but the actual discovery of the subnetworks. One might expect this to have the effect of making a contribution that is more novel in its theoretical framing and prompts further exploration into the roles of sparsity and subnetwork structure more generally in generalization.

Quality-wise, the paper is sound because the experimental setup is robust, and careful comparisons across various models, including MLPs and transformers as well as tasks like modular arithmetic and MNIST, assure to critically analyze the idea. The methodology of the experiment controls very well, isolating the effects of subnetworks from confounding factors like weight norm. Indeed, that work can be labeled as comprehensive because it shows the use of multiple pruning techniques, establishment of a critical pruning ratio, and the use of experiments with edge-popup. All these conclusions strengthen the final conclusion drawn by the paper and reflect what should have been the actual efforts of the authors in being extremely cautious while testing their hypothesis.

The paper is clearly written in general. Authors described and explained the main concepts, including grokking tickets and their corresponding different pruning techniques used to find those subnetworks. Different experimental results of figures and graphs added clarity between grokking, weight norms, and subnetworks. Some parts of the theoretical sections, especially on the aspects that involve metrics, such as Jaccard similarity and frequency entropy, are highly supported by visual aids that enlighten the sub-network relationships and generalization.

The paper carries great value in terms of possibilities for reshaping discussions regarding delayed generalization and generalization mechanisms for neural networks. Treating the subnetworks as the focal point for grokking thus opens up avenues for research into model efficiency and interpretability, and maybe even efficiency in pruning. Moreover, if validated further, this approach is bound to have a considerable impact on thinking about training over-parameterized networks among practitioners and researchers and especially in identifying the optimal subnetwork. This approach for grokking could then be applied further to the realms of reinforcement learning and others, all pointing towards a wide-ranging impact of the discovery.
summary: This paper examines the phenomenon of "grokking," in which neural networks first achieve high training accuracy but poor generalization, then later switch to a generalization solution with more training. Earlier explanations explaining grokking along the lines of reducing weight norm are challenged, and it is argued that the identification of the "grokking tickets" specific subnetworks aligned with the lottery ticket hypothesis-play a key role in enabling generalization.

The authors provide empirical evidence that (1) grokking tickets can be used to overcome the phenomenon of delayed generalization observed in dense networks; (2) similar weight norms fail to overcome the need for long training if the subnetwork proper is missing, and finally, structure optimization alone--without weight updating--can transform memorization solutions into generalization solutions. These suggest that good sub-network search is more important to grokking than weight norm reduction, but it does offer a different perspective in regards to how generalization within neural networks happens.
weaknesses: While extremely compelling and a new perspective on grokking, there are still some weaknesses of the paper, especially regarding theoretical justification, experimental scope, and clarity of interpretation.

One weakness lies in the theoretical motivation for why subnetworks (grokking tickets) are enough to explain grokking. Although experimentally subnetworks are shown to generalize faster than their dense counterparts, why it is that the subnetwork is generalizing and not in cases where weight norm fails remains slightly implicit. A discussion of some of the theoretical underpinnings of why some subnetworks well-perform, perhaps leveraging the insights from some recent sparsity studies on neural networks (e.g., Varma et al., 2023; Merrill et al., 2023) would significantly strengthen the conceptual framework and move it closer in line with prior work. Making such results more explicitly connectable to theoretical explanations of neural network generalization, such as in terms of double descent or simplicity bias, would go a long way in contextualizing the findings within a broader landscape of generalization theory.

The experimental setup is comprehensive but could be further extended to strengthen the generality of the results. For example, the main experiments are with simple modular arithmetic tasks and MNIST, which are suitable but quite simplistic and not necessarily representative of more complex distributions or task structures, as in natural language processing or computer vision. Further experiments would expand the complexity of the datasets and architectures used-for instance, BERT on NLP tasks or ResNet on CIFAR-10, which might be even leading towards more inference on whether the grokking tickets are consistently beneficial across a wide range of tasks and architectures. In this sense, an expanded effort would go to show how solid the results are and could further generalize their applicability.

Finally, even though the authors do account for the role of structural similarity in terms of Jaccard similarity in their grokking explanation, one might further explore what the implications of such a metric would be. Indeed, this could again relate to how variations in structural similarity over training align with the grokking process or consider how it would relate to properties of generalization. The other aspect where there is room for improvement includes the fact that the significance of subnetwork structures can be further ascertained through the use of other metrics for measuring the similarity of networks, based on weight sparsity patterns or neuron activations. Lastly, the paper could do much better in establishing the connection between experimental findings and practical implications.

For example, even though the study proves that a subnetwork can be learned without weight decay, it may be more helpful to know how this might impact the practices in the real world in terms of training or pruning neural networks. It would also provide more concrete recommendations or at least some hint on possible applications of grokking tickets to better connect the theoretical insights to practice. This will not only make the contribution much more vivid, but also emphasize how important the findings are for the more general community of practitioners of machine learning. In other words, the paper does provide a good foundation, yet it could further be improved with a more detailed theoretical justification, broader experimental validation across complex tasks, a deeper exploration of structural metrics, and clearer practical implications. Such refinements would make the work more comprehensive and more impactful in advancing our understanding of grokking and neural network generalization.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. How are grokking tickets fundamentally different from typical winning tickets found in lottery ticket hypothesis studies? Are there specific characteristics or properties that distinguish grokking tickets, or could they potentially overlap?
2. Do the authors have insights into the specific structures of the subnetworks (grokking tickets) that facilitate the transition from memorization to generalization? For example, do different types of tasks have different structures in their grokking tickets?
3. How does the finding generalize to other neural network structures (e.g., Transformer) and language-based tasks?
4. How does pruning affect the abilities of pretrained models in terms of grokking?
5. Why does periodicity in representations correlate with good performance?
soundness: 3
strengths: The authors establish an innovative link between two significant phenomena in deep learning: grokking and the lottery ticket hypothesis, which provides new insights into understanding the generalization of neural networks. The experiments span multiple tasks, and the authors have also conducted thorough ablation studies to verify their hypothesis.
summary: This work explores the phenomenon of grokking in neural networks, where models initially memorize training data without generalizing well, but after extended training, they suddenly begin to generalize effectively. Specifically, it investigates the relationship between grokking and the lottery ticket hypothesis---within a large neural network, there exist smaller, trainable subnetworks (or "winning tickets") that can achieve comparable performance to the original network. The authors introduce the concept of "grokking tickets", which are subnetworks identified during the generalization phase of training. They further show that the change of inner structure (subnetworks obtained by the magnitude pruning) highly correlates with the change in test accuracy.
weaknesses: While this work presents an interesting investigation connecting grokking with the lottery ticket hypothesis, it is still unclear how the finding can be of practical usage, and some claims in this work still need more justifications (e.g., "the acquired good structure is linked to good representations") In particular, although the authors introduce Fourier Entropy to quantify the periodicity in the learned representations, it is still unclear how specific/different "periodicity" can be regarded as good representations. Also, the paper mainly explores grokking within fully connected networks and lacks a broad examination across various network architectures. For example, adding experiments across a wider range of architectures (e.g., Transformers) would increase the generalizability and appeal of the findings.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Which figure is line 486 trying to refer to? Right now it seems that there is a missing reference.
soundness: 2
strengths: 1. The paper has a clear presentation of the experimental setup, which follows the previous works on grokking. This experimental design choice increases the effectiveness of their results.

2. In Section 3, the paper presented an interesting relationship between the pruning ratio and the duration of the grokking period.

3. In Section 5, the paper explored the connection between the test accuracy during training and the change of the masks.
summary: This paper studies the grokking phenomena of the subnetworks identified by the pruning methods in the Lottery Ticket Hypothesis. The findings of the paper are mainly three-fold. First, they show that, compared with the whole network, the subnetworks exhibits a shorter "grokking period", leading to a better generalization result faster. Second, they provide evidence to support that neither the weight norm nor the sparsity of the subnetwork is the crucial factor for the shorter "grokking period". Lastly, they show that the difference between the pruned subnetworks is related to the test accuracy at the point they are pruned, and that the subnetworks learns better representations from data.
weaknesses: 1. The paper does not have a clear and unified argument. It seems that, instead of using the LTH as a tool to understand the grokking behavior of neural network training, the paper focus more on the grokking behavior of the pruned subnetworks. From this perspective, though they consist of interesting findings, the results of the paper lacks practical implications: we are not sure how the grokking behavior of the subnetworks is going to shed light on the grokking behavior of neural network in general from an "inner structure" perspective.

2. The paper does not decouple the behavior of "shorter grokking period" from the "easier training" property of the winning tickets. It was observed in the line of the LTH works that subnetworks that are winning tickets are easier to train, which include the behavior of achieving a better generalization result from a smaller number of epochs. It is not sure what is the de-facto distinction between this "easy-to-training" behavior and the result in this paper.

3. Some experimental design in Section 4 is weird. In particular, in Section 4.1 when constructing the neural network with the same weight norm as the pruned network, the paper directly shrinks each weight by a factor. There is no evidence that this method of reducing the weight norm will preserve the network performance or lead to better generalization performance, as is the case in the weight-decay training. Therefore, the paper could be comparing the lottery tickets with some arbitrarily bad neural network. Moreover, in Section 4.2, the paper compared the lottery tickets with PAI method, but it is known that PAI method usually sacrifices performance for better computation efficiency. Based on these experimental design, what Section 4 is doing is simply comparing lottery tickets with not-as-good neural network that has the same weight norms or sparsity, which makes the argument that the paper is trying to make trivial.

4. In Section 5.3, the paper tries to argue that the subnetworks has shorter grokking period because they learn better representations. Given that grokking is determined by generalization performance, it seems that, for the specific task design, the only way for the neural network to achieve good generalization is to learn the good representation (in other words, no benign overfitting is possible). This may not be the case for more complicated tasks.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - Why the Jaccard distance can be used as a progress measure? The dynamics of Jaccard distance and that of test accuracy change show similar trend.

- Is it possible to obtain a good lottery ticket during memorization phase? It would be great if a mask can be found before the phase transition, so that it can be applied to the model and accelerates grokking.
soundness: 2
strengths: - The paper is well-written and easy to follow. It studies the hidden mechanism of grokking, which is an important question. 

- The idea that a small proportion of generalized network is sufficiently for neural networks to generalize much faster and accelerate grokking is interesting.
summary: The paper found that given an appropriate lottery ticket of a specific task, the model generalizes much faster than the base model. They also found that the structure of the subnetwork changes rapidly during phase transition and pruning during training can accelerate generalization.
weaknesses: - The hyperparameters (e.g. learning rate and weight decay) are not tuned. For example, I think for the modular addition task, the base model can generalize much faster than 25k steps given proper learning rate and weight decay. It is thus not fair to say the lottery ticket can accelerate 65 times in modular addition. Also the results in Table 1 would be more convincing if the hyperparameters are tuned.

- Why the network structure can be measured by Jaccard distance? It seems intuitive that when the model undergoes a phase transition (from memorization to generalization), its weight norm will change rapidly, causing the Jaccard distance to also increase rapidly.

Meta-review: This paper studies the connection between lottery tickets and grokking. The authors make the observation that using lottery tickets chosen at the time of test generalization can mitigate grokking challenges. Here, grokking refers to the phenomena that, for certain tasks such as modular arithmetic, test accuracy is observed to reach high accuracy much later than training accuracy. Overall, the reviewers and the AC believe that this is a nice connection and authors make various observations around this.

On the other hand, the paper has some fundamental issues which results in the reject recommendation. One issue is the concept of "grokking tickets" and how it is different from LTH (this concern is shared by some reviewers as well). The definition of "grokking tickets" is solely based on choosing a lottery ticket based on its test accuracy. This definition is not really specific to grokking and this sounds like an obvious thing to do. I feel like naming is misleading essentially. I understand that this choice can particularly benefit tasks where grokking (train-test discrepancy) is observed but the definition itself is not grokking specific. Additionally, the LTH has a rich literature. I would be surprised if nobody has studied related criteria (based on test error) to decide how to pick the lottery tickets. The related work section (in Page 10) has only a short paragraph and a single citation on LTH which makes me concerned about potential missing related work here. A second issue is that, given this is an empirical work, the current set of experiments don't meet the bar in terms of how comprehensive they are (also shared by some reviewers). A final issue is lack of theoretical depth and limited methodological contributions.

Decision: Reject