Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies the properties of grokking from the loss landscapes. The phenomenon of "LU" mechanism of the training and test loss is adopted to understand grokking from its dependence on data size, weight decay, and representations. In particular, the experiment tailor-designed to reveal the connection between grokking and deep representation learning is interesting and impressive. 
strength_and_weaknesses: Strength:

(1) It is interesting and novel to understand grokking from the lens of neural loss landscapes. More importantly, the connection between grokking and data size, weight decay, and representation learning are demonstrated with sufficient empirical analysis. 

(2) It conducts sufficient and intuitive experiments to reveal the dependence of grokking on representation learning, which is helpful for the community to understand grokking in the learning process with complex datasets. Meanwhile, it contributes a useful new tool for characterizing data-model interaction and representation learning. 

Weaknesses:

(1) Some terms should be explained in the paper for self-consistency, such as algorithmic datasets.

(2) The color bar of Fig.6 (b) is in [0.4, 2.0], which is different from other figures. And as claimed in the paper, the error = 1 - acc. This is inconsistent. 
clarity,_quality,_novelty_and_reproducibility: Clarity: This paper is well-written and easy to follow. 

Quality: High, all claims in this paper are well supported by thorough analysis and empirical experiments.

Novelty: It is novel to understand grokking from the loss landscapes. The experiment results are impressive and interesting. 

Reproducibility: Hard. As claimed in the paper, grokking itself is hard to observe and depends on many factors. 

summary_of_the_review: Overall, it is a decent paper.  it proposes to interpret the grokking phenomenon from the perspective of loss landscapes. And sufficient empirical analysis is provided to support the claims of the paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes the LU mechanism for the phenomenon of Grokking observed by Power et. al. 2020 - which states that the train and test curves follow an L and U shaped curve with weight norms respectively. They verify this observation in a student teacher setup, and show that it can arise in non-algorithmic datasets if initialized in a certain weight regime for appropriate sample size. Further, they try to differentiate the algorihtmic and non-algorithmic datasets in terms of their dependence on representations. 
strength_and_weaknesses: Strengths:
1. The LU mechanism hypothesis is clearly stated, and the proposal to construct the landscape with fixed weight norm solutions is a nice idea to verify it (barring the possible optimization concerns for the constrained optimization problem). 
2. The experiments on MNIST, IMDB support the LU mechanism hypothesis - I particularly liked Figure 3 as it illustrates the idea clearly.   
3. The authors are able to demonstrate grokking for non-algorithmic datasets by changing the sample size and weight initialization - this is an important finding.  
4. The proposed difference between algorithmic and non-algorithmic datasets is plausible - in particular that the weight norm increases and then drops as in Figure 9. The representation messiness argument also supports this conclusion, but I found this to be a little weak (see comments below). 

Weaknesses:  
1. Representation messiness:   
- The experimental setup is hard to understand in Section 5.1 - I would suggest the authors include an appendix section describing the exact architecture, and the trainable/frozen components of this architecture. Does R_random depend on the input at all? If not, how is this a good proxy for representation learning?  
- This part of the paper makes a reduction to a different architecture (from Liu et. al. 2022) and assumes that the representations are a convex combination of linearly separable and random gaussian representations. It is not clear to me why this is general and if the conclusions transfer to a setting where the representations are learnable (as is the case in grokking generally). 
2. Minor complaints:  
- The authors claim that changing the x-axis to weight norm removes double descent - this is only true for some limited cases and does not hold generally for neural networks as is also stated in the reference cited by authors. I would recommend that the authors add this clarification.  
- Figure 6: Please state which experimental setup these loss landscapes have been plotted for. 

clarity,_quality,_novelty_and_reproducibility: Clarity: The paper is written fairly clearly and the figures convey the main points. The experimental setup is unclear in Section 5.  
Quality: The hypothesis is well-formed and the experiments support the hypothesis.  
Novelty: To my knowledge, this paper is the first to show grokking for non-algorithmic datasets. While there have been some papers that explore some aspects of grokking, this paper adds to that literature.  

summary_of_the_review: The authors state three conclusions in their paper - the LU mechanism, the occurrence of grokking for wide range of datasets under certain conditions and the dependence of grokking on learning representations. The first two are well-supported by their experiments as the setup is general enough that the conclusions are believable. However, the third conclusion is made in an artificially reduced setup. While this setup supports the hypothesis, it is unclear to what extent this is related to the general phenomenon of grokking where representations are learnt and not fixed. Nevertheless, the contribution of the first two conclusions and weak evidence for the third conclusion seems enough for acceptance to the conference. I would recommend the authors include some discussion for their choice of setup in Section 5.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper analyses the phenomenon of grokking (discovering solutions that generalise well after the model has overfit the data) from the perspective of loss landscapes, specifically the disparity between the training and the generalisation landscapes. The authors show that grokking can be induced on standard ML benchmarks such as MNIST classification by (1) increasing the weight initialisation range, (2) drastically reducing the number of data points in the training dataset. Further, the authors conduct experiments to show that grokking is linked to the correlation between the discovery of a good representation and generalisation. For loss landscapes analysis, low-dimensional projections are used, where the weights are represented as a weight norm. The reduced 1D loss landscape visualisation seems to be a powerful tool.
strength_and_weaknesses: Strengths:

1) The loss landscape perspective is refreshing and very tangible, I think this is the right way to think about the problem of grokking.
2) Reduced 1D loss landscapes are a powerful visualisation tool, and may find further uses in the NN analysis and understanding.
3) Empirical results are convincing.

Weaknesses:
1) The authors refer to “standard initialisation”, and then explain that it stands for default initialisation in PyTorch. While this can be easily looked up at the time of writing, I am not in favour of such loose definitions. What if the next version of PyTorch picks a different default? Surely that should not lead to non-reproducible results. As such, I would like to request the authors to rather define the initialisation scheme explicitly.
2) Even though the authors have managed to induce “grokking” for MNIST and other tasks, the exercise seemed very artificial: yes, we can induce grokking by chopping off most of the dataset and raising the initialisation range. However, grokking does not seem to be something that we necessarily want to induce: looking at the results obtained, avoiding overfitting in the first place seems to still be the better option. Is the significance of this phenomenon perhaps blown out of proportion? I wish the relevance of grokking to standard ML tasks in light of the new experiments was discussed a bit more thoroughly.
3) The paper has minor language mistakes:
Page 5: “For large data size say the full dataset” -> e.g. the full dataset
Page 6: “by constrast” -> contrast 
Page 9: “dramaticness” -> not a valid word, rather use “severity”
clarity,_quality,_novelty_and_reproducibility: The paper is mostly clear and well-written. To me, the reduced 1D loss is the strongest contribution of the paper, since it is a general tool for loss landscape analysis, and we do not have enough of those. The link to representation learning is interesting, although, just like with grokking in general, I am left with a “so what?” question. Nonetheless, this is most definitely an original, interesting work.
summary_of_the_review: Overall, this is a good paper with brilliant loss landscape visualisations. Please refer to the least of strengths and weaknesses for the justification of my recommendation.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes an explanation for when grokking happens---that it occurs when the initialization norm is too large so the model takes longer to get to the correct norm magnitude—and illustrates their thesis with results on synthetic and natural data. Defining grokking as a delay in generalization until after training set interpolation, they propose that grokking is caused by a mismatch between training and test loss curves, ie, overfitting, and described this pattern as the “LU mechanism”. They elicit grokking phenomena on natural data (IMDB and MNIST), at least if we define grokking as a delayed generalization curve. 
strength_and_weaknesses: Strengths:
- This paper is well written and presented, a pleasure to read.
- The experiments make the claims clear.
- They successfully induce delayed generalization in natural datasets, suggesting that their claims might apply in realistic settings. These results happen outside of the teacher students set in, so they are more convincing than the algorithmic experiments.
- The observation that larger dataset sizes expands the goldilocks zone is a very tidy way of thinking about generalization.
- I found myself fairly convinced by the experiments that initialization magnitude is at least one factor that can induce grokking.
- It’s good that they discuss the impact of weight decay repeatedly, as it is clearly an important factor.

Weaknesses:
- I do not generally consider it necessary to have a theoretical component of every empirical study, and many empirical studies make strong and intriguing claims that layout groundwork for a principled understanding. However, I think that the particular claims made in this paper would benefit enormously from a theoretical analysis. My issue is that they make out the initialization norm to be of particular significance, when other hyperparameters might (in fact, empirically do) have an equivalent “goldilocks zone”. Additional experiments about other hyperparameters might help, but I think that the insights that we can glean from purely empirical study of this phenomenon are limited. 
- I feel that the connection between overfitting and grokking is fairly obvious, as their own definition of grokking makes clear. Given the way that they implicitly define grokking, I don’t believe that they can describe the LU mechanism as a “cause” rather than as a rephrasing of the same definition.
- These settings used to illicit grokking are somewhat artificial: the initial norm, the constrained search space, the weight decay requirement. There is no satisfactory explanation given for why weight decay is so crucial; this is where a theoretical analysis might have been helpful.
- Teacher student setups can fundamentally change training dynamics, so I’m not confident that the results from those experiments generalize to other settings. If the claims made from these experiments do not apply outside of a teacher student setting, I’m not sure I buy them. Especially because the particular representations learned by the teacher could easily require its own particular magnitude that is not necessary for all representations.
- It is not clear to me what differentiates the LU mechanism from the existing concept of overfitting. Although they described their setup as different from ‘related phenomena’  (Schoenholz et al., 2016; Yang and Schoenholz, 2017; Nakkiran et al., 2021), it’s not clear to me how these citations are not directly describing the LU mechanism.
- The effect that this paper attributes to the importance of representation learning (section 5) might be more precisely described as the effect of the distribution shift at inference time. 
- “We treat the converging point after training as the global minimum on the spherical surface” — why? Is this actually a convex a surface somehow?
- It appears to me that the interpolations are between the final learned model and a random gaussian. It seems to me these experiments and plots would be more informative if the random model was the initialization weights, so that we could see a planar view of the path from initialization and not just any random direction. I personally would find that more convincing it as an explanation of the training process, compared with a random direction.
- On importance of representation learning: it seems to me that this might best be described more precisely as the significance of distribution shift between train and test time. Otherwise, I would expect you to formalize what it would mean to have representation learning be important.
- The discussion section: “(ii) Pre-training avoids learning representations from scratch, hence helps reduce possible grokking.” But why wouldn’t we see grokking in the pretraining stage itself?
- Is it definitely the case that grokking is not observed in language models if you use larger initializations? Otherwise, you should probably remove the discussion section on language models entirely or present evidence that grokking does not happen at any stage. I don't think there's anything backed up in this section, and it doesn't really relate to the rest of the results. 

Minor/references:
- You refer to default initialization settings in pytorch, but I think you need to give details about what those defaults are, as they might change between versions.
- Footnote 2: I’m not sure this footnote should be there. It is well known that increased capacity allows tighter interpolation of the training set, but justifying this statement by presenting an artificial scenario in which a larger model is equivalent to the smaller model is not a reflection of real practices, where we would not initialize zero-vectors when expanding an architecture.
- There is an existing literature on distinctions between multi epoch vs ideal infinite data sampling in practice, which might be worth digging into and adding to your citations, eg Nakkiran (2019) https://arxiv.org/abs/1912.07242
- “We run experiments with two initializations α = 0.5 (small) and α = 2.0 (large)” — Above, it said that alpha was the constant weight norm throughout training, but here it says that it is only the initialization. I can see now that this is supposed to just be the weight constraint, but it’s a little confusing in the wording.
- A number of these results, especially in the section on LU, seem to boil down to demonstrating that overfitting does exist. I’m not sure that this is necessary to make such a focus.
- Formula 4 should have a citation to existing work on sharpness or linear interpolation.
- I’m a little confused about the meaning of v′ = vcosθ, given that θ was previously defined as a thresholding value for the regression model, and I don’t see how that definition connects to the use the variable in section 5.1. 
- This paper uses natbib. When you make an inline citation, e.g., “Foo et al. (2009) claim…” you should use \citet{} instead of \cite{} or  \citep{} . An example of this issue is the entire first paragraph of section 6.
clarity,_quality,_novelty_and_reproducibility: The paper is extremely clear and seems to mostly be very reproducible, though I would like to see the footnote about pytorch modified to include details of the default initialization. The paper is of very high quality in general. 

The novelty is middling, with some new ideas and some old ideas presented as new ideas (LU mechanism).
summary_of_the_review: I found many of the analyses here intriguing, and ultimately I would very much like to see this paper published. I was particularly impressed by their ability to induce grokking in a realistic setting. However, they seem to be making the claim that grokking is in general caused by initializing with a large norm. The experiments do not fully support that this is the case in the main setting in which grokking has been observed, which is the algorithmic setting. I would welcome more experiments around a wider set of hyperparameters, to demonstrate that delayed generalization does not result from other settings. Without these experiments or an added theoretical analysis, I don't feel confident recommending acceptance, but I'm ready to change my mind in response to a small number of additional experiments: evidence that the effects observed are particular to the norm magnitude and illustrations of the messiness plots but with the initialization weights instead of random weights, so that we can see something closer to the actual trajectory of training. Furthermore, I think that the paper should be somewhat modified in its definitions and reasoning, as the LU mechanism appears to fall almost trivially from their definition of grokking, rather than being an attributed cause.

**After discussion:**

I'm raising my score from 6 to 8 to reflect expanded experiments that lend some additional credence to the link between behavior in the algorithmic and natural settings, and the removal and finetuning of some unsupported speculation and imprecise connections.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies the grokking phenomenon and provide an explanation related to the norm of the weights. In particular, the hypothesis that when the weights are initialized to have higher norm than the final solution, then there will be an initial slow period when the norm of the weights increases followed up by a fast period where the performance improves significantly. Authors also show that using their explanation they can produce grokking in variety of settings. Perhaps the main weakness of the work is that main experiments are done with MSE loss. Given the sensitivity of MSE loss to the norm of the weights, it is not clear if similar results can be shown for cross-entropy. I think this paper shows that grokking can happen because of the norm of the weights but not that all observed grokking phenomena are because of the norm of the weights.
justification_for_why_not_higher_score: Because they use MSE loss in their experiment and it is not clear if the results extend to using cross-entropy
justification_for_why_not_lower_score: Because the paper does a nice job of explaining a mechanism that leads to grokking
Decision: Accept: notable-top-25%