reviews: 
Comments: This work aims to ground the phenomenon of grokking in the mathematical framework of dynamical systems theory. The authors use the dynamical systems framing to develop a new progress measure that is independent of which prime is used for modular arithmetic. They use this insight to reduce grokking by choosing a specific embedding of the data.
Strengths: The study of grokking is very relevant and the idea of using dynamical systems theory to study it is interesting. The reduction of the grokking (Figure 7) is promising, and the new progress measure could be of interest to the field.
Weaknesses: There are several major weaknesses of this paper: 

1. It was unclear to me what the embedding described in Section 3 is. Because this was unclear, much of the rest of the work was unclear. I would strongly recommend spending time making this section as clear as possible. In particular, 
     a. Figure 2 says that $p$ symbols are used to represent the range $0$ to $p - 1$. What does this mean? Are these the embeddings, $\textbf{p}$. Is $p$ the same as the prime $p$ used in modular arthimetic?
      b. What is $c$ and $\gamma$? From context, I would assume that $c$ is the target output and $\gamma$ is the error, but this was never defined. What size is $c$?
      c. When is logit used? 

2. The establishment of the model was also confusing. In particular, explaining what the sums in Eqs. 4-5 are over (and why there are sums) would be helpful in understanding what the authors did. 

3. The authors make several assumptions that are not justified or motivated. For instance, Assumption 1 and 2 come out of nowhere. Similarly, saying $\alphda_d$ is "guessed to be approximately a linear function in specific situations" is very vague. 

4. The main result (Theorem 4) is not stated mathematically and is vague. It was not clear what was really being gained from the theorem, nor was it clear how this was proven.

5. The progress measure (Eq. 12) is defined to be a function of $n$ but $n$ is not present in any of the equations. This makes it unclear what Pro 1 is saying. 

6. "According to our theory, another factor that influences the rate of grokking is the size of d_model". It is not clear to me where the size of d_model comes in to the results presented in the main text. 

7. The authors claim that increasing the size of the dataset affects the rate of grokking. But then they choose to increase $p$ and not $\beta$. The rationale for this was not clear. Additionally, given the bounds given in Sec, 5.2.2. $\beta$ and $p$ are in the numerator. Therefore, increasing either leads to greater norm, which I thought meant longer time to achive high test accuracy. 

8. The mitigation of grokking (Figure 7) is interesting, but it is not clear how the authors use their previous results to choose a Toeplitz matrix. If anything, it feels like cheating slightly since the task is periodic and they choose a circulant matrix (or maybe not cheating, but rather they use features of the task instead of the dynamical systems picture they have developed). 

9. There are a number of major typos, including sentences that do not make grammatical sense, sentences that end abruptly, and mis-spelling that makes reading this manuscript challenging.
Recommendation: 1
Reviewer confidence: 3
Soundness/correctness: 1
Clarity: 1