Understanding the Repeat Curse in Large Language Models from a Feature Perspective
Authors: Junchi Yao, Shu Yang, Lijie Hu, Mengdi Li, Di Wang
Accepted at: acl25

Understanding the Repeat Curse in Large Language Models from a Feature Perspective

Junchi Yao1,2,3,*, Shu Yang1,2,*, Jianhua Xu1,2,3,
Lijie Hu1,2, Mengdi Li1,2, Di Wang1,2,†
1Provable Responsible AI and Data Analytics (PRADA) Lab,
2King Abdullah University of Science and Technology,
3University of Electronic Science and Technology of China

Abstract

Large language models (LLMs) have made remarkable progress in various domains, yet they often suffer from repetitive text generation, a phenomenon we refer to as the “Repeat Curse”. While previous studies have proposed decoding strategies to mitigate repetition, the underlying mechanism behind this issue remains insufficiently explored. In this work, we investigate the root causes of repetition in LLMs through the lens of mechanistic interpretability. Inspired by recent advances in Sparse Autoencoders (SAEs), which enable monosemantic feature extraction, we propose a novel approach—“Duplicatus Charm”—to induce and analyze the Repeat Curse. Our method systematically identifies “Repetition Features” -the key model activations responsible for generating repetitive outputs. First, we locate the layers most involved in repetition through logit analysis. Next, we extract and stimulate relevant features using SAE-based activation manipulation. To validate our approach, we construct a repetition dataset covering token and paragraph level repetitions and introduce an evaluation pipeline to quantify the influence of identified repetition features. Furthermore, by deactivating these features, we have effectively mitigated the Repeat Curse.

1 Introduction

Large language models (LLMs) have demonstrated remarkable progress across various domains, from machine translation (Xu et al., 2024; Wang et al., 2023) and open-ended text generation (Carlsson et al., 2024; Lee et al., 2022; Su et al., 2023b,a) to interdisciplinary applications in social science behavior analysis (Yang et al., 2025; Yao et al., 2024; Park et al., 2023; Yang et al.) and psychological research (Hu et al., 2024a; Demszky et al., 2023; Yang et al., 2024). Although LLMs have been extensively studied, a critical phenomenon that limits their practical utility is their tendency to generate repetitive content (Fu et al., 2021; Xue et al., 2024; Wang et al., 2024), which is particularly evident in enumerative tasks, ultimately reducing the performance and diversity of the generated outputs. We refer to this issue as “Repeat Curse” (see Figure 1 for examples).

![Examples of Repeat Curse: (a) Token Repetition Scenario, (b) Paragraph Repetition Scenario.](page_1016_1017_388_234.png)

Figure 1: Examples of Repeat Curse: (a) Token Repetition Scenario, (b) Paragraph Repetition Scenario.

Previous research has investigated the phenomenon of repetition and has proposed strategies to reduce its occurrence from the perspective of decoding. For example, Zhu et al. (2023) analyzed the self-reinforcement effect in text generation and proposed a repetition penalty mechanism to mitigate its impact. Holtzman et al. (2019) proposed Nucleus Sampling as a decoding strategy for language models, which can reduce repetition in long texts and improve the generation quality. Though these methods can reduce repetition, they may compromise the model’s overall performance, and the underlying mechanisms driving repetitive content generation in LLMs remain scarcely studied.

To address the issue, a few works identify the most important component in the network of the
Repeat Curse from the mechanistic interpretability view. Vaidya et al. (2023) identified specific attention heads and layers that tend to copy the next token by examining the model’s attention maps. Building upon the layers, Hiraoka and Inui (2024) identified the repetition neurons by analyzing the activation outputs in the feed-forward network of each layer.

Compared to the model neurons, Sparse Autoencoders (SAEs) have been used in LLMs (Bricken et al., 2023; Cunningham et al., 2023) to achieve monosemantic units of analysis. SAE maps the complex superposition of polysemantic neurons into monosemantic features. By regularizing activations, it ensures that only a small set of features are activated for each input, making the resulting features human-interpretable (Yun et al., 2021; Rajamoharan et al., 2024). Due to its advances, many recent studies have leveraged SAE to provide the most critical and human-understandable features for different tasks. For example, Le et al. (2024); Simon and Zou (2024) identified biological-relevant features and further utilized through SAE; Kim and Ghadiyaram (2025) identified features related to inappropriate content such as nudity and violence. Inspired by these works, we pose the following research question: Can we leverage SAE to identify the features that cause the Repeat Curse to give a better understanding?

Unlike the above-mentioned work, we cannot directly identify specific words or phrases that represent repetition. Therefore, to identify these features, we propose the “Duplicatus Charm” (a magic spell inspired by Harry Potter) to induce the Repeat Curse.

The main difficulties of our method are locating and identifying the “target of the spell”, i.e., the most significant features. To address the first problem, we first analyze the logits to identify the layers that have a significant impact on predicting the next token as a repeat token (Nostalgebraist, 2020). Then, we explore the features in those layers by stimulating their activations through SAE (\S 4.3). For the second challenge, we design a pipeline to evaluate the effectiveness of the magic spell. First, we construct a repetition dataset containing two scenarios (\S 4.1) and then select the appropriate repeat score for the task through the dataset (\S 4.2). Leveraging such a metric enables us to pinpoint the features that are most responsible for inducing repetition. We refer to these "targets of the spell" as “Repetition Features”. Finally, we cast a spell on the repetition features and scored them using the repeat score we identified. From a data perspective, this allows us to demonstrate whether our spell is effective while manually reviewing the texts with higher scores.

We select three language models with different scales: GPT2-small (Radford et al., 2019), Gemma-2-2B (Team, 2024), and Llama-3.1-8B (Dubey et al., 2024). The results show that repetition features are primarily located in all three models’ intermediate and final layers, suggesting a consistent pattern across different model architectures and scales. With the same coefficient, we demonstrate that activating these features increases repetition while other standard features do not. Moreover, deactivating these features could mitigate the Repeat Curse, without exerting any detrimental effects on the model’s performance. Leveraging the human-readable nature of these features, we can also summarize the repetition feature’s characteristics. Overall, our contributions are as follows:

• We revisited the phenomenon of the LLM Repeat Curse and uncovered a potential reason why such repetition occurs: the presence of repetition features.

• From an interpretability perspective, we proposed a practical and effective pipeline for extracting repetition features.

• Our research has been rigorously validated through a series of comprehensive experiments, which confirm the validity and effectiveness of our findings. It deepens our understanding of repetition in LLMs and offers new directions for their optimization.

2 Related Work

Repetition in Language Models. Repetition in language models refers to the phenomenon where the generated text exhibits undesirable and redundant repetitions at various levels, such as token-level and paragraph-level (Dinan et al., 2019).

Although the cause of repetition in LLMs is still not fully understood, some scholars have proposed methods to mitigate repetition. Su et al. (2022) introduced the decoding method of contrastive search, which encourages diversity while maintaining the coherence of the generated text. Li et al. (2023) demonstrate that penalizing repetitions in the training data significantly alleviates the degeneration
Figure 2: Illustration of our work (using GPT as an example). First line: The Repeat Curse is categorized into two scenarios: Token and Paragraph, and datasets are created accordingly. These datasets are used to evaluate and select the metrics. Second line: The identification of Repetition Features is divided into two steps: layer localization and feature localization. By identifying the repetition features, we can deactivate them to mitigate the Repeat Curse.

problem in neural text generation. Fu et al. (2021) presents a rebalanced encoding approach to address the issue of high inflow, reducing repetitions in both translation and language modeling tasks. However, the internal mechanisms of LLMs when they produce repetitive outputs remain insufficiently explored (Vaidya et al., 2023). Zippert et al. (2020) proposed a psychological linkage between repetition and mathematics, and we want to find out if the similar repeat pattern mechanisms exist in the LLM.

Language Model Mechanistic Interpretability. Mechanistic interpretability (MI) focuses on understanding the inner workings of neural networks, aiming to provide detailed insights into their computation processes and behavior (Bereska and Gavves, 2024; Zhang et al., 2025a; Rai et al., 2024; Zhang et al., 2024, 2025b; Hong et al., 2024; Hu et al., 2024b). One approach to MI is the use of the logit lens (NostalgiaBrast, 2020), which focuses on interpreting what the model believes after each layer by examining the distributions generated by layers’ activations. This approach allows us to observe how the model’s predictions evolve and refine over the course of processing.

Another approach is to examine the features. Features are the things a network would ideally dedicate a neuron to if you gave it enough neurons (Olah, 2022). Researchers have developed sparse autoencoders (SAEs), which could decompose the activation into human-interpretable features (Lee Sharkey, 2022; Cunningham et al., 2023). This process, known as sparse dictionary learning, reconstructs activation vectors as sparse linear combinations of directed vectors in the activation space (Bricken et al., 2023).

Based on SAE, activation patching emerges as a method for further probing the role of individual features within a neural network. Templeton (2024) demonstrated how steering the activation of the “Golden Gate Bridge” feature could influence the model to generate outputs specifically related to the Golden Gate Bridge, even when given diverse input prompts.

To develop LLMs, gaining mechanistic insights into their internal workings could reduce many risks (Nanda, 2022). Mechanistic interpretability enhances the predictability of future systems and reduces risks associated with deception and a foundation for model evaluation (Casper, 2023), bringing new perspectives to alignment work (Ruthenis, 2023). Geva et al. (2020) argues that some components of the model (like the features) will catch repeating patterns, and the corresponding values tend to produce the same content in the output distribution, which may cause the model to produce repeated output. Through our work, we propose a solution to prevent the repetition problem, which
can improve the performance of QA services and other text generation tasks.

3 Sparse Autoencoders

Sparse Autoencoders (SAEs) provide us with an approximate decomposition of the model’s activations into a linear combination of “feature directions” (SAE decoder weights) with coefficients equal to the feature activations. The sparsity penalty ensures that, for any given inputs to the model, only a small fraction of features will have nonzero activations. Thus, for any given token in any given context, the model activations are “explained” by a small set of active features (out of a large pool of possible features). Here’s how we perform this decomposition for activation \( x \):

\[
\hat{x} = b_{dec} + \sum_{i=1}^{F} f_i(x) W_{dec,i}.
\] (1)

The sum runs over all \( F \) features, effectively combining them to form the approximation of the original activation. Here \( \hat{x} \) is the reconstructed model activation, \( b_{dec} \in \mathbb{R}^D \) represents the learned bias term, \( W_{dec,i} \in \mathbb{R}^D \) are the learned decoder weights, and \( f_i(x) \) denotes the activation of the \( i \)-th feature, i.e.,

\[
f(x) = \mathrm{ReLU}(W_{enc} \cdot x + b_{enc})
\] (2)

where \( f_i(x) \) is computed by passing the input \( x \) through the encoder weights \( W_{enc,i} \in \mathbb{R}^{F \times D} \) and the bias term \( b_{enc} \in \mathbb{R}^F \), followed by the ReLU nonlinearity. The ReLU function ensures that only positive activations are passed through, enforcing sparsity. The objective function encourages the model to maintain a sparse representation by minimizing the number of active features, which is defined as

\[
L(x) = \|x - \hat{x}\|_2^2 + \beta S(f_i(x)) + \alpha L_{aux},
\] (3)

where \( S \) is a function of the latent coefficients that penalize non-sparse decompositions (such as \( \ell_1 \) regularization), and \( \beta \) is a sparsity coefficient. Some architectures also require the use of an auxiliary loss \( L_{aux} \) (Gao et al., 2024).

Steering with SAE. Steering is a method that utilizes the latent representations learned by SAE to steer the behavior of a model. In this process, the original activation is adjusted by introducing a steering coefficient, which controls the model’s behavior. Specifically, the adjustment process can be expressed as:

\[
\hat{X} = X + \lambda \cdot W_{dec}[\text{feature\_idx}]
\] (4)

where \( X \) represents the original activations tensor, \( \hat{X} \) represents the modified activations tensor after steering, \( \lambda \) is the steering coefficient, and \( W_{dec}[\text{feature\_idx}] \) denotes the decoder weight vector corresponding to the steered feature index.

4 Method

In the following sections, we introduce the pipeline of casting “Duplicatus Charm” (DUC): (1) Repeat Pattern Construction; (2) Evaluation and Selection of Metrics; (3) Repetition Feature Identification; (4) Feature Steering. See Figure 2 for the method overview.

4.1 Repeat Pattern Construction

As we mentioned, a challenge in identifying repeat features is developing an evaluation metric. To achieve this, we need to prepare a dataset with repetitions. However, to the best of our knowledge, currently, there is no readily available open-source dataset specifically designed for repetition tasks. To fill in the gap, we begin by constructing a custom repetition dataset. Based on previous work on analyzing repetition (Altmann and Köhler, 2015), we particularly examine two forms of LLMs’ repetition output: (a) Token Repetition with excessive token-level recurrence where specific words/phrases replicate beyond natural language conventions and (b) Paragraph Repetition with structural redundancy through duplicated paragraph patterns.

We selected Orca-Chat†(Es, 2023), a commonly used chat dataset containing short QA pairs, as our raw data. By applying specific rules to the output portions of this dataset, we can generate the desired repetition dataset. Specifically, we sampled 1,000 raw data, and the generated dataset consists of 5,500 samples. Among these, 4,500 belong to the token repetition scenario, and 1,000 belong to the paragraph repetition scenario.

Token Repetition Scenario. In this scenario, we mainly generated repeated data based on two factors: \( N \) is the token position where the repetition starts; \( M \) is the number of tokens in the repeated token group. Dataset generation can be expressed as (5).

†https://huggingface.co/datasets/shahules786/orca-chat
Token Repetition\((N, M) = \left( t_1, t_2, \ldots, t_N, \underbrace{t_{N+1}, t_{N+2}, \ldots, t_{N+M}}_{\text{repeated group}}, \underbrace{t_{N+1}, t_{N+2}, \ldots, t_{N+M}}_{\text{repetition continues}} \right)\) (5)

We generated the dataset using \(N\) values from an arithmetic sequence ranging from 0 to 140 with a common difference of 10 and \(M\) values of 1, 2, and 5. Each case contains 100 dialogue samples, so in total, we have 4,500 dialogue samples.

Paragraph Repetition Scenario. In this scenario, the entire text repeats continuously rather than just a few words. We generate the whole text five times to obtain the paragraph repetition texts. For this scenario, we sampled 1,000 raw data and applied repetition modifications.

4.2 Repeat Curse Metric Selection

Based on the dataset obtained in §4.1, we could evaluate the level of repetition using the difference metrics, ultimately selecting those that demonstrate discriminative capability for both scenarios. While (Li et al., 2023) selected n-gram as the evaluation metric. However, we noticed that they did not clarify which value of \(n\) performed best and whether there were better metrics. Here, we test different \(n\) and introduce two additional potential metrics for comparison. The evaluation framework adopts two complementary approaches: The first set of metrics directly quantifies the degree of textual repetition, while the second approach conversely assesses the information content across the entire text. We have selected the following metrics for their effectiveness in addressing both dimensions:

n-gram (Li et al., 2023) The weighted repetition rate \(R\) is calculated as the ratio of the weighted sum of repeated \(n\)-grams to the maximum possible weighted sum:

\[
R = \frac{\sum_{i \in n} f_i^w}{\sum_{i \in n} \max(f_i, 1)^w}, \tag{6}
\]

where \(n\) is the set of unique \(n\)-grams, \(f_i\) is the frequency of \(n_i\), and \(w\) is the weight factor. The numerator sums \(f_i^w\) for \(f_i > 1\), while the denominator sums \(\max(f_i, 1)^w\) for all \(n\)-grams.

Self-BLEU (Papineni et al., 2002) BLEU score was originally used to evaluate machine translation performance. In this paper, we calculate the BLEU score of each sentence segment with other segments to obtain the average Self-BLEU score of the entire text, thereby evaluating the degree of repetition. Self-BLEU = \( \frac{1}{n} \sum_{i=1}^{n} p_1(t_i) \), where \(n\) is the total number of texts and \(p_1(t_i)\) is the 1-gram precision of text \(t_i\), calculated as the ratio of matching 1-grams to the total 1-grams in \(t_i\).

Information Entropy (Tsai et al., 2008) Since sentence lengths vary, we use maximum entropy for normalization:

\[
H_{\text{normalized}} = \frac{-\sum_{i=1}^{N} p_i \log_2(p_i)}{\log_2(N)}. \tag{7}
\]

Results of the Token Repetition Scenario In Figure 3, we can see the information entropy curve for 1-gram differs from that of 2, 3, 4, and 5-grams, reaching its lowest value of around 0.6 when repeating from the 140-th token. This indicates that the optimal parameter choice for Information Entropy in this task is 1-gram, which provides strong distinguishability. Similarly, the 1-gram curve shows significant distinction compared to 2, 3, 4, and 5-grams, and can still accurately locate repetition situations above 0.9 when repeating from the 140-th token. The self-BLEU fluctuates within a difference of 0.1 when N takes different values, showing low distinguishability and poor performance.

Therefore, both the n-gram and information entropy metrics perform well with \(n = 1\). The rest results (\(M = 2, 5\)) are shown in Appendix A.

Results of the Paragraph Repetition Scenario In Figure 5, we can see the information entropy has a gap of 0.4 when evaluating the repetition and original data, while for n-gram the gap is 0.95, and it is 0.1 for BLEU. The comparison results show that the n-gram is highly sensitive in this scenario. However, when \(n\) is set to 2, 3, or 4, we can also see the scores for normal text are too low, which is not conducive to subsequent analysis. Thus, 1-gram has the best performance.

4.3 Repeat Features Identification

To identify effective repetition features, it is necessary to first locate the layers that contribute the most to the repetition phenomenon to narrow the
search scope. Therefore, this section is divided into two steps: layer localization and feature localization.

4.3.1 Layer Localization
Inspired by Wang et al. (2022) ’s pipeline, to determine the most important features, we decompose the residual stream and calculate the logit difference between the “correct” and “incorrect” answers. In our problem, the next token that repeats the last token is considered “correct”, while any token that does not repeat the last token is considered “incorrect”. In our work, given the input “He hit Jack Jack Jack Jack Jack”, the correct output is “Jack”, and it will be incorrect otherwise. The layer with the largest logit difference is identified as the repetition layer.

Logit difference measures the difference in logit value between the two tokens, where a positive score means the correct token has a higher probability. In our work, given the input “He hit Jack Jack Jack Jack Jack”, the correct output is “Jack”, and the incorrect output is “Jackson”. By calculating the difference between these two tokens, we can quantify the model’s preference for the correct answer. The formula for the logit difference direction is given by:

\[
\ell_{\text{diff\_direction}} = c_{\text{direction}} - i_{\text{direction}},
\]

where \( c_{\text{direction}} \) and \( i_{\text{direction}} \) represent the residual stream directions for the correct and incorrect answers, respectively.

Finally, we calculate the layer attribution by taking the dot product of the residual activations at each layer with the previously computed logit difference direction, \( \ell_{\text{diff\_direction}} \). This operation quantifies the contribution of each layer to the final prediction. The formula for the logit contribution at layer \( \ell \) is:

\[
\ell_{\text{contribution}_\ell} = \text{residual}_\ell \cdot \ell_{\text{diff\_direction}},
\]

where \( \text{residual}_\ell \) is the residual activation.

Based on the work of Wang et al. (2022), who utilized 10 templates to locate indirect object layers, we adopted a similar approach tailored to our work. We create 8 templates with induced repeated generation inputs (refer to Table 3). Appendix B shows that the contributions of the intermediate layers and the final layer to generating repeated content are the most significant. Therefore, we will look for repetition features in both the intermediate and final layers.

4.3.2 Feature Localization
Through §4.3.1, we will further localize the feature on the most significant layer and the second most significant layer (Wang et al., 2022).

We employ a pre-trained SAE model of each model, which has already captured meaningful features. Then by setting the features’ steering coefficient \( \lambda \) in (4) as 1.5-2 times the original activation level, we were able to enhance the content related to the generated features without causing model collapse, which refers to the failure of the model to generate meaningful or diverse outputs, caused by disrupting the balance of the model’s parameters and structure (McDougall, 2023).

Based on the generated text after activation, we determine that features with repeat score (RS) (See §4.2) above \( \rho \) are considered repetition features.

\[
\text{Feature} = \begin{cases}
\text{Repetition Feature} & \text{if RS} \geq \rho \\
\text{Common Feature} & \text{if RS} < \rho.
\end{cases}
\]

5 Experiment

5.1 Setup
Models We specifically selected large pre-trained models that have open-sourced their SAE models: GPT2-small (Radford et al., 2019) with GPT-sm-res-jb (jbloom, 2024); Gemma-2-2B (Team,
2024) with GemmaScope-res-16k (Lieberum et al., 2024); Llama-3.1-8B (Dubey et al., 2024) with LlamaScope-res-32k (He et al., 2024b).

Datasets and Metric We used three datasets: two containing hard (academic) and easy questions, and one containing intuitively easy to repeat enumeration questions. We selected the Academic ShortQA \( ^\dagger \) (DisgustingOzil, 2024) (AQ), which contains hard (academic) questions, and Natural Questions \( ^\dagger \) (Kwiatkowski et al., 2019) (NQ), which contains simple questions. We selected diversity enumeration problems from the Diversity-Challenge-Dataset\( ^\ddagger \) (EQ). Compared with ordinary problems, these are more challenging. We randomly select some of the questions shown in the Appendix G.

Following the result of §4.2, we use n-gram as the repeat score to evaluate the degree of the repeat curve.

Hyperparameters In our method, we have two hyperparamaters \( \rho \) and \( \lambda \). We will set \( \rho = 0.4 \), which is based on human evaluations (refer to Appendix F). And we set \( \lambda = 2 \), which this value ensures that the model’s overall performance remains unaffected while strongly inducing the occurrence of the Repeat Curse (McDougall, 2023).

5.2 Main Result

Repetition Features This part will present the repetition features identified based on different datasets and analyze their characteristics. We iterated through each feature of the repetition layer, activated them, randomly sampled questions from each dataset to query the model, and used the repeat score to evaluate the generated results to identify repetition features. All the identified repetition features are shown in Appendix C.

We find that the repetition features identified two or more times across the three datasets are associated with Names, Time, and Mathematics (see Figure 4). The model that identified the same repetition feature the most is Llama-3.1-8B, while the least is GPT2-small. We did not identify the same mathematics-related feature in GPT2-small, which reflects its instability in mathematical reasoning. Overall, among the repetition features identified from the three models, names are the most likely to cause repetition.

![Illustration of the distribution of characteristics for repetition features identified two or more times across multiple datasets.](page_1012_172_388_388.png)

Figure 4: Illustration of the distribution of characteristics for repetition features identified two or more times across multiple datasets.

Evaluation of DUC We activate the repetition feature in batches at each layer of each model, analyze the repeat score of the generated results, and evaluate whether the DUC is effective. Next, we attempted to reduce the steering coefficient of these features to see whether it can mitigate Repeat Curse. We perform experiments on 3 datasets (EQ, AQ, NQ), sequentially activating 10%, 20%, 50%, and 100% of the repetition features (the most influential ones are prioritized). After multiple trials, we calculated the average repeat score for the generated text. The detailed results are presented in Table 1.

The activated common features (CF) serve as the baseline for the study. After activating an equal number of common features as repetition features, there was no significant change in repeat scores.

From the dataset perspective of view, the repeat score is highest on the EQ dataset, followed by a gradual decrease on the AQ and NQ datasets. This indicates that questions that induce repetition exhibit a more severe Repeat Curse when activating repetition features. Regarding the difficulty of the questions, the more challenging the question, the more pronounced the Repeat Curse becomes after activation.

After deactivating the repetition feature, the repeat score for the EQ dataset shows the most significant change, while the scores for AQ and NQ
<table>
  <tr>
    <th rowspan="2">Model and Layer</th>
    <th colspan="4">Dataset</th>
    <th colspan="4">Dataset</th>
    <th colspan="4">Dataset</th>
  </tr>
  <tr>
    <th>EQ</th>
    <th>10%</th>
    <th>20%</th>
    <th>50%</th>
    <th>AQ</th>
    <th>10%</th>
    <th>20%</th>
    <th>50%</th>
    <th>NQ</th>
    <th>10%</th>
    <th>20%</th>
    <th>50%</th>
    <th>100%</th>
  </tr>
  <tr>
    <td rowspan="4">GPT2-small Layer 9</td>
    <td>original</td><td>0.37</td><td>0.37</td><td>0.37</td><td>0.37</td>
    <td>original</td><td>0.25</td><td>0.25</td><td>0.25</td><td>0.25</td>
    <td>original</td><td>0.18</td><td>0.18</td><td>0.18</td><td>0.18</td>
  </tr>
  <tr>
    <td>activated(CF)</td><td>0.36</td><td>0.37</td><td>0.37</td><td>0.37</td>
    <td>activated(CF)</td><td>0.25</td><td>0.26</td><td>0.27</td><td>0.27</td>
    <td>activated(CF)</td><td>0.18</td><td>0.19</td><td>0.19</td><td>0.21</td>
  </tr>
  <tr>
    <td>activated(RF)</td><td>0.55</td><td>0.60</td><td>0.68</td><td>0.72</td>
    <td>activated(RF)</td><td>0.50</td><td>0.53</td><td>0.55</td><td>0.58</td>
    <td>activated(RF)</td><td>0.47</td><td>0.52</td><td>0.54</td><td>0.55</td>
  </tr>
  <tr>
    <td>deactivated</td><td>0.33</td><td>0.32</td><td>0.21</td><td>0.19</td>
    <td>deactivated</td><td>0.25</td><td>0.25</td><td>0.23</td><td>0.23</td>
    <td>deactivated</td><td>0.18</td><td>0.18</td><td>0.16</td><td>0.17</td>
  </tr>
  <tr>
    <td rowspan="4">GPT2-small Layer 11</td>
    <td>original</td><td>0.35</td><td>0.35</td><td>0.35</td><td>0.35</td>
    <td>original</td><td>0.25</td><td>0.25</td><td>0.25</td><td>0.25</td>
    <td>original</td><td>0.19</td><td>0.19</td><td>0.19</td><td>0.19</td>
  </tr>
  <tr>
    <td>activated(CF)</td><td>0.35</td><td>0.35</td><td>0.36</td><td>0.37</td>
    <td>activated(CF)</td><td>0.25</td><td>0.26</td><td>0.27</td><td>0.27</td>
    <td>activated(CF)</td><td>0.19</td><td>0.19</td><td>0.19</td><td>0.21</td>
  </tr>
  <tr>
    <td>activated(RF)</td><td>0.53</td><td>0.58</td><td>0.66</td><td>0.70</td>
    <td>activated(RF)</td><td>0.50</td><td>0.50</td><td>0.51</td><td>0.53</td>
    <td>activated(RF)</td><td>0.45</td><td>0.46</td><td>0.49</td><td>0.51</td>
  </tr>
  <tr>
    <td>deactivated</td><td>0.34</td><td>0.30</td><td>0.23</td><td>0.22</td>
    <td>deactivated</td><td>0.25</td><td>0.25</td><td>0.24</td><td>0.24</td>
    <td>deactivated</td><td>0.18</td><td>0.19</td><td>0.19</td><td>0.18</td>
  </tr>
  <tr>
    <td rowspan="4">Gemma-2-2B Layer 22</td>
    <td>original</td><td>0.28</td><td>0.28</td><td>0.28</td><td>0.28</td>
    <td>original</td><td>0.22</td><td>0.22</td><td>0.22</td><td>0.22</td>
    <td>original</td><td>0.19</td><td>0.19</td><td>0.19</td><td>0.19</td>
  </tr>
  <tr>
    <td>activated(CF)</td><td>0.28</td><td>0.28</td><td>0.30</td><td>0.32</td>
    <td>activated(CF)</td><td>0.22</td><td>0.24</td><td>0.24</td><td>0.24</td>
    <td>activated(CF)</td><td>0.19</td><td>0.19</td><td>0.19</td><td>0.21</td>
  </tr>
  <tr>
    <td>activated(RF)</td><td>0.51</td><td>0.56</td><td>0.64</td><td>0.68</td>
    <td>activated(RF)</td><td>0.41</td><td>0.44</td><td>0.45</td><td>0.48</td>
    <td>activated(RF)</td><td>0.38</td><td>0.43</td><td>0.44</td><td>0.48</td>
  </tr>
  <tr>
    <td>deactivated</td><td>0.25</td><td>0.25</td><td>0.24</td><td>0.25</td>
    <td>deactivated</td><td>0.22</td><td>0.22</td><td>0.21</td><td>0.20</td>
    <td>deactivated</td><td>0.19</td><td>0.19</td><td>0.19</td><td>0.19</td>
  </tr>
  <tr>
    <td rowspan="4">Gemma-2-2B Layer 24</td>
    <td>original</td><td>0.29</td><td>0.29</td><td>0.29</td><td>0.29</td>
    <td>original</td><td>0.24</td><td>0.24</td><td>0.24</td><td>0.24</td>
    <td>original</td><td>0.20</td><td>0.20</td><td>0.20</td><td>0.20</td>
  </tr>
  <tr>
    <td>activated(CF)</td><td>0.29</td><td>0.30</td><td>0.31</td><td>0.33</td>
    <td>activated(CF)</td><td>0.25</td><td>0.25</td><td>0.27</td><td>0.27</td>
    <td>activated(CF)</td><td>0.20</td><td>0.21</td><td>0.21</td><td>0.22</td>
  </tr>
  <tr>
    <td>activated(RF)</td><td>0.49</td><td>0.54</td><td>0.61</td><td>0.65</td>
    <td>activated(RF)</td><td>0.42</td><td>0.44</td><td>0.47</td><td>0.52</td>
    <td>activated(RF)</td><td>0.42</td><td>0.44</td><td>0.46</td><td>0.49</td>
  </tr>
  <tr>
    <td>deactivated</td><td>0.36</td><td>0.27</td><td>0.24</td><td>0.20</td>
    <td>deactivated</td><td>0.24</td><td>0.24</td><td>0.26</td><td>0.24</td>
    <td>deactivated</td><td>0.20</td><td>0.17</td><td>0.18</td><td>0.17</td>
  </tr>
  <tr>
    <td rowspan="4">Llama-3.1-8B Layer 24</td>
    <td>original</td><td>0.28</td><td>0.28</td><td>0.28</td><td>0.28</td>
    <td>original</td><td>0.21</td><td>0.21</td><td>0.21</td><td>0.21</td>
    <td>original</td><td>0.19</td><td>0.19</td><td>0.19</td><td>0.19</td>
  </tr>
  <tr>
    <td>activated(CF)</td><td>0.28</td><td>0.29</td><td>0.29</td><td>0.29</td>
    <td>activated(CF)</td><td>0.21</td><td>0.21</td><td>0.23</td><td>0.23</td>
    <td>activated(CF)</td><td>0.20</td><td>0.19</td><td>0.19</td><td>0.20</td>
  </tr>
  <tr>
    <td>activated(RF)</td><td>0.46</td><td>0.50</td><td>0.57</td><td>0.62</td>
    <td>activated(RF)</td><td>0.40</td><td>0.41</td><td>0.44</td><td>0.46</td>
    <td>activated(RF)</td><td>0.36</td><td>0.37</td><td>0.41</td><td>0.43</td>
  </tr>
  <tr>
    <td>deactivated</td><td>0.24</td><td>0.25</td><td>0.19</td><td>0.19</td>
    <td>deactivated</td><td>0.21</td><td>0.19</td><td>0.19</td><td>0.19</td>
    <td>deactivated</td><td>0.19</td><td>0.18</td><td>0.18</td><td>0.17</td>
  </tr>
  <tr>
    <td rowspan="4">Llama-3.1-8B Layer 29</td>
    <td>original</td><td>0.25</td><td>0.25</td><td>0.25</td><td>0.25</td>
    <td>original</td><td>0.21</td><td>0.21</td><td>0.21</td><td>0.21</td>
    <td>original</td><td>0.15</td><td>0.15</td><td>0.15</td><td>0.15</td>
  </tr>
  <tr>
    <td>activated(CF)</td><td>0.25</td><td>0.26</td><td>0.27</td><td>0.27</td>
    <td>activated(CF)</td><td>0.21</td><td>0.20</td><td>0.24</td><td>0.26</td>
    <td>activated(CF)</td><td>0.19</td><td>0.19</td><td>0.20</td><td>0.20</td>
  </tr>
  <tr>
    <td>activated(RF)</td><td>0.48</td><td>0.52</td><td>0.60</td><td>0.66</td>
    <td>activated(RF)</td><td>0.39</td><td>0.40</td><td>0.44</td><td>0.45</td>
    <td>activated(RF)</td><td>0.36</td><td>0.37</td><td>0.39</td><td>0.39</td>
  </tr>
  <tr>
    <td>deactivated</td><td>0.25</td><td>0.26</td><td>0.19</td><td>0.18</td>
    <td>deactivated</td><td>0.20</td><td>0.20</td><td>0.19</td><td>0.21</td>
    <td>deactivated</td><td>0.15</td><td>0.16</td><td>0.14</td><td>0.14</td>
  </tr>
</table>

Table 1: Effect of Repetition Feature Activation at Different Levels (10%, 20%, 50%, 100%). We take experiments on 3 datasets: Enumeration Questions (EQ), Academic Questions (AQ), Natural Questions (NQ). “CF” refers to randomly selected common feature, and “RF” refers to repetition feature. Bold indicates the highest score of each model

exhibit only minor fluctuations around their original values, occasionally even exceeding them (e.g., Gemma-2-2B Layer 24 Activation Ratio=50%). This indicates that the EQ dataset, which originally had a higher score, is more sensitive to the deactivation of the repetition feature, resulting in a larger difference. This suggests that the effectiveness of mitigating Repeat Curse relies on the presence of a certain degree of inherent repetition in the problem itself. Without this foundational repetition, the impact of such measures may not be observable.

For layers, the ones that contribute more significantly tend to achieve higher repeat scores. For instance, in GPT2-small Layer 9, which has a greater contribution, consistently yield higher scores across all three datasets under “activated repetition feature (RF)” compared to the 11th layer.

For models, GPT2-small exhibited the highest repeat score after activation, with a range of approximately 0.6. This indicates that GPT2-small has a higher sensitivity to repetition features, whereas larger models like Gemma-2-2B and Llama-3.1-8B are more robust to mitigate such effects.

Mitigation Effect To further investigate the practical value of DUC, we also need to evaluate its effectiveness in mitigating the Repeat Curse. We introduced Information Entropy as a positive metric complementary to the Repeat Score and employed Perplexity to assess whether the generative capability of the deactivated model was affected.

The results in Table 2 show that DUC demonstrates nearly the best performance in reducing repetition across various models and datasets. Both the Repeat Score and Information Entropy achieved an 88.9% state-of-the-art (SOTA) probability, with no significant decrease in Perplexity. Notably, the Perplexity of Gemma-2-2B on the AQ dataset even increased instead of decreasing, further validating the practical utility of the DUC method.

And we conduct two case studies to further investigate the Utility After Mitigation in Appendix H.

Visualization Results In Appendix D, Table 13 and Table 14 respectively show the effects of feature activation on repetition features and regular features before and after activation. In Table 13, feature 20199 directly causes a Repeat Curse. In Table 14, feature 100 represents words related to political campaigns and candidates, and its generation after steering consistently includes references to “president”.

Table 15 provides an example of the output results under the condition where 100% of the repetition features are deactivated, offering a clear demonstration of the mitigation.
<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th rowspan="2">Method</th>
    <th colspan="3">Gemma-2-2B</th>
    <th colspan="3">Llama-3.1-8B</th>
    <th colspan="3">GPT2</th>
  </tr>
  <tr>
    <th>RepeatScore ↓</th>
    <th>Entropy ↑</th>
    <th>Perplexity ↓</th>
    <th>RepeatScore ↓</th>
    <th>Entropy ↑</th>
    <th>Perplexity ↓</th>
    <th>RepeatScore ↓</th>
    <th>Entropy ↑</th>
    <th>Perplexity ↓</th>
  </tr>
  <tr>
    <td rowspan="4">EQ</td>
    <td>Greedy</td>
    <td>0.28</td>
    <td>0.86</td>
    <td><b>8.53</b></td>
    <td>0.26</td>
    <td>0.88</td>
    <td>16.72</td>
    <td>0.36</td>
    <td>0.81</td>
    <td>8.70</td>
  </tr>
  <tr>
    <td>Beam</td>
    <td>0.31</td>
    <td>0.84</td>
    <td>11.13</td>
    <td>0.22</td>
    <td>0.89</td>
    <td><b>9.23</b></td>
    <td>0.46</td>
    <td>0.70</td>
    <td><b>7.66</b></td>
  </tr>
  <tr>
    <td>TopK</td>
    <td>0.14</td>
    <td>0.94</td>
    <td>24.37</td>
    <td>0.19</td>
    <td>0.91</td>
    <td>10.46</td>
    <td>0.13</td>
    <td><b>0.95</b></td>
    <td>30.47</td>
  </tr>
  <tr>
    <td>TopP</td>
    <td>0.14</td>
    <td>0.94</td>
    <td>16.74</td>
    <td>0.19</td>
    <td>0.91</td>
    <td>13.32</td>
    <td>0.19</td>
    <td>0.94</td>
    <td>74.05</td>
  </tr>
  <tr>
    <td rowspan="4">DUC (ours)</td>
    <td><b>DUC (ours)</b></td>
    <td><b>0.13</b>,<b>0.18</b></td>
    <td><b>0.94</b></td>
    <td><b>8.1</b></td>
    <td><b>0.13</b>,<b>0.13</b></td>
    <td><b>0.91</b></td>
    <td><b>10.39</b></td>
    <td><b>0.13</b>,<b>0.33</b></td>
    <td><b>0.94</b></td>
    <td>23.14</td>
  </tr>
  <tr>
    <td rowspan="4">AQ</td>
    <td>Greedy</td>
    <td>0.22</td>
    <td>0.89</td>
    <td>54.14</td>
    <td>0.21</td>
    <td>0.90</td>
    <td>14.62</td>
    <td>0.25</td>
    <td>0.88</td>
    <td>8.70</td>
  </tr>
  <tr>
    <td>Beam</td>
    <td>0.35</td>
    <td>0.84</td>
    <td>85.40</td>
    <td>0.52</td>
    <td>0.76</td>
    <td><b>8.60</b></td>
    <td>0.53</td>
    <td>0.75</td>
    <td><b>6.88</b></td>
  </tr>
  <tr>
    <td>TopK</td>
    <td>0.16</td>
    <td>0.93</td>
    <td>61.79</td>
    <td>0.36</td>
    <td>0.84</td>
    <td>11.89</td>
    <td>0.21</td>
    <td>0.91</td>
    <td>31.00</td>
  </tr>
  <tr>
    <td rowspan="4">DUC (ours)</td>
    <td><b>DUC (ours)</b></td>
    <td><b>0.10</b>,<b>0.25</b></td>
    <td><b>0.96</b></td>
    <td><b>8.12</b></td>
    <td><b>0.11</b>,<b>0.41</b></td>
    <td><b>0.93</b></td>
    <td><b>11.17</b></td>
    <td><b>0.14</b>,<b>0.39</b></td>
    <td><b>0.94</b></td>
    <td>14.53</td>
  </tr>
  <tr>
    <td rowspan="4">NQ</td>
    <td>Greedy</td>
    <td>0.18</td>
    <td>0.92</td>
    <td><b>5.02</b></td>
    <td>0.19</td>
    <td>0.91</td>
    <td>4.98</td>
    <td>0.18</td>
    <td>0.91</td>
    <td>7.41</td>
  </tr>
  <tr>
    <td>Beam</td>
    <td>0.47</td>
    <td>0.77</td>
    <td>87.64</td>
    <td>0.36</td>
    <td>0.82</td>
    <td><b>3.56</b></td>
    <td>0.51</td>
    <td>0.71</td>
    <td><b>4.39</b></td>
  </tr>
  <tr>
    <td>TopK</td>
    <td>0.20</td>
    <td>0.91</td>
    <td>65.52</td>
    <td>0.27</td>
    <td>0.87</td>
    <td>4.93</td>
    <td>0.16</td>
    <td>0.93</td>
    <td>33.16</td>
  </tr>
  <tr>
    <td rowspan="4">DUC (ours)</td>
    <td><b>DUC (ours)</b></td>
    <td><b>0.11</b>,<b>0.36</b></td>
    <td><b>0.95</b></td>
    <td><b>14.72</b></td>
    <td><b>0.11</b>,<b>0.25</b></td>
    <td><b>0.95</b></td>
    <td><b>10.13</b></td>
    <td><b>0.14</b></td>
    <td><b>0.94</b></td>
    <td><b>22.31</b></td>
  </tr>
</table>

Table 2: Mitigation results. We conducted experiments on three datasets, and all models were tested across five methods. Bold indicates the highest score.

6 Conclusion

In this paper, we take a perspective from the feature level and introduce a pipeline named “Duplicatus Charm” (DUC). Through this mechanistic interpretability method, we can identify the repetition features. By activating the repetition feature, we can induce the Repeat Curse, which was then evaluated through repeat scores and validated by humans in our experiment. And by deactivating these features will mitigate the Repeat Curse. Furthermore, we summarize the common characteristics of repetition features across three models.

7 Limitations

It is worth mentioning that there are still several limitations in this study.

Repeat Score The identification of repetitive features relies on a predefined threshold for the repeat score (\( \rho = 0.4 \)), which was determined based on human evaluation. This introduces a potential for subjectivity, as different threshold choices could lead to different sets of repetitive features.

Models The experiments were conducted on three LLMs with pre-trained SAE (GPT2-small, Gemma-2-2B, and Llama-3.1-8B), which have relatively limited scales. He et al. (2024a), Gao et al. (2024), Cunningham et al. (2023) mentioned the limitations of open-source SAEs and said most researches on SAE are using 7B or larger models. Moreover, due to the limited availability of open-source SAEs, we have done our best to refine this study. With the continuous development of the SAE community in the future, we will keep updating this work.

8 Acknowledgement

Di Wang and Shu Yang are supported in part by the funding BAS/1/1689-01-01, URF/1/4663-01-01, REI/1/5232-01-01, REI/1/5332-01-01, and URF/1/5508-01-01 from KAUST, and funding from KAUST - Center of Excellence for Generative AI, under award number 5940.

References

Gabriel Altmann and Reinhard Köhler. 2015. Forms and degrees of repetition in texts: detection and analysis, volume 68. Walter de Gruyter GmbH & Co KG.

Leonard Bereska and Efstratios Gavves. 2024. Mechanistic interpretability for ai safety–a review. arXiv preprint arXiv:2404.14082.

Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conery, Nicholas L Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Chris Olah. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. https://transformer-circuits.pub/2023/monosemantic-features/index.html. Accessed: 2024-12-23.

Fredrik Carlsson, Fangyu Liu, Daniel Ward, Murathan Kurfali, and Joakim Nivre. 2024. The hyperfitting phenomenon: Sharpening and stabilizing llms for open-ended text generation. arXiv preprint arXiv:2412.04318.

Stephen Casper. 2023. The engineer’s interpretability sequence.

Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders
find highly interpretable features in language models. arXiv preprint arXiv:2309.08600.

Dorotyya Demszky, Diyi Yang, David S Yeager, Christopher J Bryan, Margaret Clapper, Susannah Chandhok, Johannes C Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghan Johnson, et al. 2023. Using large language models in psychology. Nature Reviews Psychology, 2(11):688–701.

Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, Shrimai Prabhumoye, Alan W Black, Alexander Rudnicky, Jason Williams, Joelle Pineau, Mikhail Burtsev, and Jason Weston. 2019. The second conversational intelligence challenge (convai2). Preprint, arXiv:1902.00098.

DisgustingOzil. 2024. Academic dataset - shortqa.

Abhimanyu Dubey, Abhinav Jahuiri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, et al. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.

Shahul Es. 2023. Orca-chat: A high-quality explanation-style chat dataset. https://huggingface.co/datasets/shahules786/orca-chat/.

Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy. 2024. Applying sparse autoencoders to unlearn knowledge in language models. arXiv preprint arXiv:2410.19278.

Zihao Fu, Wai Lam, Anthony Man-Cho So, and Bei Shi. 2021. A theoretical analysis of the repetition problem in text generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12848–12856.

Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093.

Mor Geva, Roei Schuster, Jonathan Berant, et al. 2020. Transformer feed-forward layers are key-value memories. Preprint, arXiv:2012.14913.

Z He, W Shu, X Ge, et al. 2024a. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders. arXiv preprint arXiv:2410.20526.

Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, et al. 2024b. Llama scope: Extracting millions of features from llama-3.1-8b with sparse autoencoders. arXiv preprint arXiv:2410.20526.

Tatsuya Hiraoka and Kentaro Inui. 2024. Repetition neurons: How do language models produce repetitions? Preprint, arXiv:2410.13497.

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751.

Yihuai Hong, Yuelin Zou, Lijie Hu, Ziqian Zeng, Di Wang, and Haiqin Yang. 2024. Dissecting fine-tuning unlearning in large language models. arXiv preprint arXiv:2410.06606.

Jinpeng Hu, Tengteng Dong, Luo Gang, Hui Ma, Peng Zou, Xiao Sun, Dan Guo, Xun Yang, and Meng Wang. 2024a. Psycollm: Enhancing llm for psychological understanding and evaluation. IEEE Transactions on Computational Social Systems.

Lijie Hu, Liang Liu, Shu Yang, Xin Chen, Zhen Tan, Muhammad Asif Ali, Mengdi Li, and Di Wang. 2024b. Understanding reasoning in chain-of-thought from the hopfieldian view. arXiv preprint arXiv:2410.03595.

jbloom. 2024. Gpt2-small-saes-reformatted.

Dahye Kim and Deepti Ghadiyaram. 2025. Concept steerers: Leveraging k-sparse autoencoders for controllable generations. Preprint, arXiv:2501.19066.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenneth Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452–466.

Nhat Minh Le, Ciyue Shen, Neel Patel, Chintan Shah, Darpan Sanghavi, Blake Martin, Alfred Eng, Daniel Shenker, Harshith Padigela, Raymond Biju, Syed Ashar Javed, Jennifer Hipp, John Abel, Harsha Pokkalla, Sean Grullon, and Dinkar Juyal. 2024. Learning biologically relevant features in a pathology foundation model using sparse autoencoders. Preprint, arXiv:2407.10785.

Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. Advances in Neural Information Processing Systems, 35:34586–34599.

Beren Millidge Lee Sharkey, Dan Braun. 2022. Taking features out of superposition with sparse autoencoders.

Huayang Li, Tian Lan, Zihao Fu, Deng Cai, Lemao Liu, Nigel Collier, Taro Watanabe, and Yixuan Su. 2023. Repetition in repetition out: Towards understanding neural text degeneration from the data perspective. Advances in Neural Information Processing Systems, 36:72888–72903.

Tom Lieberum, Senthoo Ranajamoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah,
and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. Preprint, arXiv:2408.05147.

Callum McDougall. 2023. Arena 3.0.

Neel Nanda. 2022. A longlist of theories of impact for interpretability.

Nostalgebraist. 2020. Interpreting gpt: The logit lens. Accessed: 2024-12-22.

Chris Olah. 2022. Mechanistic interpretability, variables, and the importance of interpretable bases.

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.

Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22.

Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.

Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024. A practical review of mechanistic interpretability for transformer-based language models. Preprint, arXiv:2407.02646.

Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. 2024. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404.16014.

Thane Ruthens. 2023. World-model interpretability is all we need.

Elana Simon and James Zou. 2024. InterPlm: Discovering interpretable features in protein language models via sparse autoencoders. Preprint, arXiv:2412.12101.

Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. 2023a. Fake news detectors are biased against texts generated by large language models. arXiv preprint arXiv:2309.08674.

Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023b. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. In The 2023 Conference on Empirical Methods in Natural Language Processing.

Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. 2022. A contrastive framework for neural text generation. Advances in Neural Information Processing Systems, 35:21548–21561.

Gemma Team. 2024. Gemma.

Adly Templeton. 2024. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Anthropic.

Du-Yih Tsai, Yongbum Lee, and Eri Matsuyama. 2008. Information entropy measure for evaluation of image quality. Journal of digital imaging, 21:338–347.

Aditya Vaidya, Javier Turek, and Alexander Huth. 2023. Humans and language models diverge when predicting repeating text. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pages 58–69, Singapore. Association for Computational Linguistics.

Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2022. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593.

Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023. Document-level machine translation with large language models. arXiv preprint arXiv:2304.02210.

Weichuan Wang, Zhaoyi Li, Defu Lian, Chen Ma, Linqi Song, and Ying Wei. 2024. Mitigating the language mismatch and repetition issues in llm-based machine translation via model editing. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15681–15700.

Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024. Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation. arXiv preprint arXiv:2401.08417.

Fuzhao Xue, Yao Fu, Wangchunshu Zhou, Zangwei Zheng, and Yang You. 2024. To repeat or not to repeat: Insights from scaling llm under token-crisis. Advances in Neural Information Processing Systems, 36.

Shu Yang, Muhammad Asif Ali, Lu Yu, Lijie Hu, and Di Wang. Model autophagy analysis to explicate self-consumption within human-ai interactions. In First Conference on Language Modeling.

Shu Yang, Shenzhe Zhu, Ruoxuan Bao, Liang Liu, Yu Cheng, Lijie Hu, Mengdi Li, and Di Wang. 2024. What makes your model a low-empathy or warmth person: Exploring the origins of personality in llms. arXiv preprint arXiv:2410.10863.

Shu Yang, Shenzhe Zhu, Zeyu Wu, Keyu Wang, Junchi Yao, Junchao Wu, Lijie Hu, Mengdi Li, Derek F Wong, and Di Wang. 2025. Fraud-r1: A multi-round benchmark for assessing the robustness of llm against augmented fraud and phishing inducements. arXiv preprint arXiv:2502.12904.
Junchi Yao, Hongjie Zhang, Jie Ou, Dingyi Zuo, Zheng Yang, and Zhicheng Dong. 2024. Fusing dynamics equation: A social opinions prediction algorithm with llm-based agents. arXiv preprint arXiv:2409.08717.

Yoky Yao. 2025. Diversity-challenge.

Zeyu Yun, Yubei Chen, Bruno A Olshausen, and Yann LeCun. 2021. Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors. arXiv preprint arXiv:2103.15949.

Lin Zhang, Wenshuo Dong, Zhuoran Zhang, Shu Yang, Lijie Hu, Ninghao Liu, Pan Zhou, and Di Wang. 2025a. Eap-gp: Mitigating saturation effect in gradient-based automated circuit identification. arXiv preprint arXiv:2502.06852.

Lin Zhang, Lijie Hu, and Di Wang. 2025b. Mechanistic unveiling of transformer circuits: Self-influence as a key to model reasoning. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 1387–1404, Albuquerque, New Mexico. Association for Computational Linguistics.

Zhuoran Zhang, Yongxiang Li, Zijian Kan, Keyuan Cheng, Lijie Hu, and Di Wang. 2024. Locate-then-edit for multi-hop factual recall under knowledge editing. arXiv preprint arXiv:2410.06331.

Wenhong Zhu, Hongkun Hao, and Rui Wang. 2023. Penalty decoding: Well suppress the self-reinforcement effect in open-ended text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1218–1228, Singapore. Association for Computational Linguistics.

Erica L Zippert, Ashli-Ann Douglas, and Bethany Rittle-Johnson. 2020. Finding patterns in objects and numbers: Repeating patterning in pre-k predicts kindergarten mathematics knowledge. Journal of Experimental Child Psychology, 200:104965.
A  Evaluation of Metrics

Information Entropy represents the amount of information contained, so when repeated positions occur later, more information is included, resulting in an upward trend in the curve. On the other hand, the n-gram directly describes the repeated content, so when the repeated positions occur later, the proportion of repeated content within the overall content becomes smaller, leading to a downward trend in the curve. Figure 6 shows the comparison result when \( M = 2, 5 \).

![Comparison of Metrics in Paragraph Repetition Scenario](page_184_370_1082_324.png)

Figure 5: Comparison of Metrics in Paragraph Repetition Scenario

B  Layer Attribution

We provide the eight templates of prompts and answers used to investigate the influence of layer contributions on repetition. Each prompt was designed to include repeated tokens at specific intervals to induce patterns of repetition. The answers were defined by selecting tokens at the corresponding positions in the prompt as “correct” when they were the same as the previous token and “incorrect” when they differed. We recorded the residual difference direction used to measure the difference between ‘correct’ and ‘incorrect’ generation and further quantified the contribution of each layer to the final prediction by calculating the dot product of each layer’s activations and the residual difference direction. The results are shown in Figure 7, 8 and 9. The prompts and answers are in Table 3.

C  Repeat Feature

We present the identified repetition features of the three models on three datasets in Table 4, 5, 6, 7, 8, 9, 10, 11, 12. These include Layer 9 and Layer 11 of GPT2-small, Layer 22 and Layer 24 of Gemma-2-2B, and Layer 24 and Layer 29 of Llama-3.1-8B. The underline indicates that the feature appears twice across the three models, while the bold indicates that the feature appears three times. For more detailed feature information, you can search the corresponding model’s feature ID at https://www.neuronpedia.org.

In Figure 10, we illustrate the distribution of feature characteristics in the AQ dataset, where Llama-3.1-8B demonstrates a significantly higher number of mathematics-related features compared to other models.

D  Comparison of Repetition Features and Regular Features

To more clearly observe the presence of the repetition feature, we randomly selected a feature and compared it with one of the repetition features we identified. In Table 14, when activation feature 100 was steered, the model exhibited generation behavior that matched the feature description, producing content such as ‘president’ related to ‘political’, which is a typical response after activating a regular feature. However, in Table 13, after activating feature 20199, the model’s response exhibited a clear repetition phenomenon.

E  Mitigating the Repeat Curse

We demonstrate the generation effect of GPT2-small Layer 9 after deactivating 100% of the repetition feature in Table 15. In the normal (non-activated) case, when the model faces a problem requiring diversity, it falls into the repetition curse, repeatedly generating the word “The Godfather”. However, after deactivating the repetition feature, the model is not affected by the diversity issue and does not fall into the repetition curse. In cases 2 and 3, it even shows improved diversity, listing more song information and providing more effective answers to the question.

F  Human Evaluation

To determine the repeat score threshold \( \rho \) for the repetition feature, we manually evaluated the repetition in the generated text. If the text exhibited repetition, it was classified as “Yes”. We randomly sampled 100 pairs of texts and then calculated the repeat score for those classified as “Yes”. Figure 16 displays a randomly selected portion of our evaluation process.
(a) Entropy: \( M = 2 \)        (b) n-gram: \( M = 2 \)        (c) BLEU: \( M = 2 \)

(d) Entropy: \( M = 5 \)        (e) n-gram: \( M = 5 \)        (f) BLEU: \( M = 5 \)

Figure 6: Comparison of Metrics in Token Repetition Scenario (M=2, 5)

<table>
  <tr>
    <th>Prompt</th>
    <th>Answer(Correct, Incorrect)</th>
  </tr>
  <tr>
    <td>Is displacement is a vector or scalar is a vector is is a vector is is<br>School school school is a place where you school school school is a school is a school is a<br>Which does not has an index does not has an index<br>Friends friends friends are people who friends friends friends are people who friends are people who<br>Speed speed speed is a scalar that speed speed speed is a speed is a speed is a<br>Mass mass mass does not change mass mass mass changes doesn’t change<br>Work done is energy is energy is<br>Time is always measured in seconds Time is always measured in seconds Time is always</td>
    <td>a, vector<br>school, place<br>does, and<br>friends, help<br>speed, scalar<br>mass, anything<br>energy, to<br>measured, limited</td>
  </tr>
</table>

Table 3: Templates of Induced Repeated Generation Inputs and Answers

Figure 7: GPT2-small Layer Attribution

Figure 8: Gemma-2-2B Layer Attribution

Figure 9: Llama-3.1-8B Layer Attribution

Figure 10: Feature Characteristic of Each Model on academic question (AQ) dataset.
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
    <th>Layer 9</th>
  </tr>
  <tr>
    <td>6643</td>
    <td>periods and punctuation marksindicating the end of sentences</td>
    <td></td>
  </tr>
  <tr>
    <td>6972</td>
    <td>entities such as names, organizations, and transferred amounts</td>
    <td></td>
  </tr>
  <tr>
    <td>8700</td>
    <td>phrases related to popular moviefanchises and their connections</td>
    <td></td>
  </tr>
  <tr>
    <td>13299</td>
    <td>expressions related to deep emotionsand personal connections</td>
    <td></td>
  </tr>
  <tr>
    <td><b>13944</b></td>
    <td><b>proper names of individuals or entities</b></td>
    <td></td>
  </tr>
  <tr>
    <td><b>16888</b></td>
    <td><b>names specifically with initials followed by periods</b></td>
    <td></td>
  </tr>
  <tr>
    <td>17533</td>
    <td>information about pricing andsubscriptions</td>
    <td></td>
  </tr>
  <tr>
    <td>19200</td>
    <td>environmental elements such asocations including caves, mountainslakes, and specific physical objects</td>
    <td></td>
  </tr>
  <tr>
    <td>20161</td>
    <td>financial and economic data points orindicators</td>
    <td></td>
  </tr>
  <tr>
    <td>22587</td>
    <td>discount-related terms and actions</td>
    <td></td>
  </tr>
  <tr>
    <td>23516</td>
    <td>mentions of names, specifically those related to the character Jack and others in a specific narrative context</td>
    <td></td>
  </tr>
  <tr>
    <th colspan="3">Layer 11</th>
  </tr>
  <tr>
    <td><b>6023</b></td>
    <td><b>proper names of individual</b></td>
    <td></td>
  </tr>
  <tr>
    <td>7413</td>
    <td>phrases related to problem-solving and improvement</td>
    <td></td>
  </tr>
  <tr>
    <td>8860</td>
    <td>proper nouns and specific terms related to legal and politicalmatters</td>
    <td></td>
  </tr>
  <tr>
    <td>8919</td>
    <td>phrases related to online security and encryption</td>
    <td></td>
  </tr>
  <tr>
    <td>10226</td>
    <td>words related to political figures or events</td>
    <td></td>
  </tr>
  <tr>
    <td>10431</td>
    <td>temporal references or expressions related to time</td>
    <td></td>
  </tr>
  <tr>
    <td>11642</td>
    <td>locations and spatial references</td>
    <td></td>
  </tr>
  <tr>
    <td><b>12078</b></td>
    <td><b>dates or events when something occurred</b></td>
    <td></td>
  </tr>
  <tr>
    <td>13140</td>
    <td>references to specific numerical codes or identifiers</td>
    <td></td>
  </tr>
  <tr>
    <td>15084</td>
    <td>phrases related to personal evaluation or judgment</td>
    <td></td>
  </tr>
  <tr>
    <td>15405</td>
    <td>phrases related to negative events or experiences</td>
    <td></td>
  </tr>
</table>

Table 4: GPT2-small Repetition Features (EQ)

G Diversity Challenge Problem Dataset

To construct the question answering dataset, we employed an automated generation method augmented by manual curation. Specifically, we developed a Python script utilizing the Hugging Face Transformers library and the Meta-Llama/Llama-3.1-8B model, along with predefined prompt templates and configuration parameters, to generate an initial batch of 5,000 questions. The script first configured the environment, loaded the model, and then iteratively generated questions while performing validation and cleaning. After initial deduplication, we obtained 976 unique questions. To ensure the quality and relevance of the dataset, we conducted rigorous manual screening and ultimately selected 500 high-quality enumeration questions. Finally, these questions were saved as a JSON-formatted dataset to serve as a foundation for subsequent research.

H Utility After Mitigation

We conduct two case studies to further investigate utility after mitigation. The feature description will be shown in Table 20.

Deactivating Features in Universal Question We identified ten repetitive features associated with names in Llama-3.1-8B’s 24 layers, located at indices: [12656, 8575, 7468, 15812, 10614, 916, 1866, 10781, 16247, 640]. These features were deactivated for analysis.

We then selected three questions—one from each of our three datasets—likely to be heavily influenced by name-related features. The results in Table 18 showed that while some repetition was reduced, the overall answer quality remained unaffected.

Deactivating Features in Specific Question We attempted to identify proper noun in the 24th layer and deactivating the features related to the word and asking questions associated with it, studying whether our suppression affected the generation
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 9</th>
  </tr>
  <tr>
    <td>3615</td>
    <td>references to product offers and services, likely related to advertising or marketing content</td>
  </tr>
  <tr>
    <td>3661</td>
    <td>numerical information related to accounting or distribution</td>
  </tr>
  <tr>
    <td>6972</td>
    <td>entities such as names, organizations, and transferred amounts</td>
  </tr>
  <tr>
    <td>7798</td>
    <td>names related to Middle Eastern politics and conflicts</td>
  </tr>
  <tr>
    <td>8357</td>
    <td>information related to news articles and events, focusing on dates and locations</td>
  </tr>
  <tr>
    <td>10178</td>
    <td>references to the television show "Game of Thrones"</td>
  </tr>
  <tr>
    <td><b>13944</b></td>
    <td><b>proper names of individuals or entities</b></td>
  </tr>
  <tr>
    <td>16631</td>
    <td>policy-related phrases like "full employment," "de facto amnesty," "mass deportation," and "no-fly zone."</td>
  </tr>
  <tr>
    <td><b>16888</b></td>
    <td><b>names specifically with initials followed by periods</b></td>
  </tr>
  <tr>
    <td>18380</td>
    <td>government department names and related entities</td>
  </tr>
  <tr>
    <td>22275</td>
    <td>cities and locations</td>
  </tr>
  <tr>
    <td>22317</td>
    <td>phrases related to keeping business operational or in progress</td>
  </tr>
  <tr>
    <th colspan="2">Layer 11</th>
  </tr>
  <tr>
    <td>2868</td>
    <td>elements related to coding or programming concepts</td>
  </tr>
  <tr>
    <td>3185</td>
    <td>locations expressed as intersections or addresses</td>
  </tr>
  <tr>
    <td><b>6023</b></td>
    <td><b>proper names of individual</b></td>
  </tr>
  <tr>
    <td>6038</td>
    <td>words related to the name "Kris"</td>
  </tr>
  <tr>
    <td>8353</td>
    <td>terms related to geographic locations or businesses</td>
  </tr>
  <tr>
    <td>10431</td>
    <td>temporal references or expressions related to time</td>
  </tr>
  <tr>
    <td>11642</td>
    <td>locations and spatial references</td>
  </tr>
  <tr>
    <td><b>12078</b></td>
    <td><b>dates or events when something occurred</b></td>
  </tr>
  <tr>
    <td>18623</td>
    <td>terms related to financial capital and taxes</td>
  </tr>
  <tr>
    <td>20971</td>
    <td>phrases indicating events or activities related to time and context</td>
  </tr>
  <tr>
    <td>22640</td>
    <td>specific time-related events or processes</td>
  </tr>
  <tr>
    <td>23164</td>
    <td>measurement units and quantities related to mathematics and physics</td>
  </tr>
</table>

Table 5: GPT2-small Repetition Features (AQ)

outcome. We selected the word "Rome" and "Vatican" and obtained the following related features: [21185, 25206, 2152, 26865, 1108, 17963], where 21185, 25206, 17963 are repetition features. The results in Table 19 showed that the deactivation did not impact the model’s generation of knowledge.

Discussion Farrell E (Farrell et al., 2024) argues that “negative scaling of feature activations is necessary and zero ablating features is ineffective.” However, in our experiments, we found that setting the activation value to a small positive activation value (0.01) achieves good mitigation effects while preserving the model’s generative capabilities. Unlike negative scaling—which induces knowledge unlearning—our approach of using minimal positive activations successfully mitigates repetition without compromising the model’s knowledge retention. Thus, I suggest these two deactivation value choices serve distinct purposes:

• Negative scaling is suitable for removing hazardous knowledge (e.g., harmful memorized content).

• Minimal positive activation is preferable for behavioral mitigation (e.g., reducing repetition) while retaining model functionality.
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 9</th>
  </tr>
  <tr>
    <td>181</td>
    <td>information about who directed and wrote a film or TV show</td>
  </tr>
  <tr>
    <td>1238</td>
    <td>phrases related to confidence and mental states</td>
  </tr>
  <tr>
    <td>1554</td>
    <td>names or references to names in a text</td>
  </tr>
  <tr>
    <td>3660</td>
    <td>references to an exchange of goods or services</td>
  </tr>
  <tr>
    <td>4688</td>
    <td>phrases related to age groups</td>
  </tr>
  <tr>
    <td>6792</td>
    <td>Roman numerals followed by letters and numbers</td>
  </tr>
  <tr>
    <td>8047</td>
    <td>people or places associated with specific names</td>
  </tr>
  <tr>
    <td>12969</td>
    <td>terms related to indexing, such as words like “index” and actions related to creating or comparing indexes</td>
  </tr>
  <tr>
    <td><b>13944</b></td>
    <td><b>proper names of individuals or entities</b></td>
  </tr>
  <tr>
    <td><b>16888</b></td>
    <td><b>names specifically with initials followed by periods</b></td>
  </tr>
  <tr>
    <td>17290</td>
    <td>references to video games</td>
  </tr>
  <tr>
    <td>19121</td>
    <td>political party names, such as AAP, Greens, Congress, and NDP, along with related terms</td>
  </tr>
  <tr>
    <td>20636</td>
    <td>references to computer science concepts related to object-oriented programming</td>
  </tr>
  <tr>
    <th colspan="2">Layer 11</th>
  </tr>
  <tr>
    <td>692</td>
    <td>references to individuals named or related to “Bhutan”</td>
  </tr>
  <tr>
    <td>3017</td>
    <td>names of people or entities preceded by a title or username</td>
  </tr>
  <tr>
    <td>3299</td>
    <td>numbers and codes with a specific structure</td>
  </tr>
  <tr>
    <td>4464</td>
    <td>topics related to government, politics, and various industries</td>
  </tr>
  <tr>
    <td><b>6023</b></td>
    <td><b>proper names of individual</b></td>
  </tr>
  <tr>
    <td><b>12078</b></td>
    <td><b>dates or events when something occurred</b></td>
  </tr>
  <tr>
    <td>16594</td>
    <td>Proper nouns, specifically names of people and locations</td>
  </tr>
  <tr>
    <td>16765</td>
    <td>specific parts of objects or machines</td>
  </tr>
  <tr>
    <td>17956</td>
    <td>technical terms related to geology and physics</td>
  </tr>
  <tr>
    <td>18371</td>
    <td>mentions of people’s names in a social context</td>
  </tr>
  <tr>
    <td>22640</td>
    <td>specific time-related events or processes</td>
  </tr>
</table>

Table 6: GPT2-small Repetition Features (NQ)
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 22</th>
  </tr>
  <tr>
    <td>259</td>
    <td>references to the color red, particularly in varying contexts or phrases</td>
  </tr>
  <tr>
    <td>2603</td>
    <td>mention of characters or entities named "Daika" along with their various attributes and relationships</td>
  </tr>
  <tr>
    <td><b>3509</b></td>
    <td><b>names and titles of individuals in professional contexts</b></td>
  </tr>
  <tr>
    <td>7362</td>
    <td>mentions of Washington, D.C., and variations of its name</td>
  </tr>
  <tr>
    <td><u>5327</u></td>
    <td>names or mentions of a specific individual or group</td>
  </tr>
  <tr>
    <td>7535</td>
    <td>terms and phrases associated with research and funding in the scientific field</td>
  </tr>
  <tr>
    <td>8726</td>
    <td>terms and phrases associated with “cross-linking” concepts</td>
  </tr>
  <tr>
    <td>11734</td>
    <td>symbols and variables related to math, physics, and statistics, particularly in the context of equations and mathematical notation</td>
  </tr>
  <tr>
    <td>12235</td>
    <td>LaTeX math syntax related to mathematical symbols and expressions</td>
  </tr>
  <tr>
    <td><b>14137</b></td>
    <td><b>phrases related to durations and periods of time</b></td>
  </tr>
  <tr>
    <td><u>15056</u></td>
    <td>certain key terms and phrases related to various subjects such as programming, medicine, and science</td>
  </tr>
  <tr>
    <th colspan="2">Layer 24</th>
  </tr>
  <tr>
    <td><b>1119</b></td>
    <td><b>numerical values or formats in mathematical or programming contexts</b></td>
  </tr>
  <tr>
    <td>2497</td>
    <td>references to sports leagues and tournaments</td>
  </tr>
  <tr>
    <td><u>5333</u></td>
    <td>date ranges and time periods</td>
  </tr>
  <tr>
    <td>7923</td>
    <td>specific statistics related to baseball performance</td>
  </tr>
  <tr>
    <td>8789</td>
    <td>numerical values, particularly those indicating ages or durations</td>
  </tr>
  <tr>
    <td><b>11892</b></td>
    <td><b>names of characters in a narrative context</b></td>
  </tr>
  <tr>
    <td>12519</td>
    <td>acronyms and specific terms related to molecular biology or chemistry</td>
  </tr>
  <tr>
    <td>14510</td>
    <td>elements related to programming and data structures</td>
  </tr>
  <tr>
    <td>14995</td>
    <td>mentions of the name “Tom”</td>
  </tr>
  <tr>
    <td>16307</td>
    <td>information related to financial transactions and corporate activities</td>
  </tr>
</table>

Table 7: Gemma-2-2B Repetition Features (EQ)
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 22</th>
  </tr>
  <tr>
    <td>3509<br>3576<br>5618<br>5947<br>9363<br>10278<br>13028<br>14041<br><b>14137</b><br>14370</td>
    <td>
      <b>names and titles of individuals in professional contexts</b><br>
      dates and times related to events or records<br>
      phrases indicating the degree of proximity or likelihood, particularly words like "almost."<br>
      phrases that indicate long-term perspectives or considerations<br>
      references to dates or time-related events<br>
      terms related to programming syntax and variable naming conventions<br>
      names of researchers and contributors involved in a project or study<br>
      keywords and phrases related to actions and intentions, particularly involving deception or retrieval<br>
      <b>phrases related to durations and periods of time</b><br>
      references to ages and years of experience
    </td>
  </tr>
  <tr>
    <th colspan="2">Layer 24</th>
  </tr>
  <tr>
    <td>1119<br>2505<br>4795<br>7079<br>7158<br>7719<br>8137<br>8491<br>8777<br>8789<br><b>11892</b><br>14512<br>15142</td>
    <td>
      <b>numerical values or formats in mathematical or programming contexts</b><br>
      specific names and references to legal proceedings or court cases<br>
      phrases indicating social connections and personal interactions<br>
      names of contributors or authors associated with a research project<br>
      names and identities of notable individuals associated with Ballymena<br>
      terms related to subscription models and billing options<br>
      terms and phrases related to biochemical processes and treatments involving heavy metals or chemical interactions<br>
      technical terms and phrases related to programming or mathematical concepts<br>
      phrases related to expressions of gratitude and acknowledgments<br>
      numerical values, particularly those indicating ages or durations<br>
      <b>names of characters in a narrative context</b><br>
      names and achievements of athletes, particularly in rugby<br>
      names of individuals and associated figures in various contexts
    </td>
  </tr>
</table>

Table 8: Gemma-2-2B Repetition Features (AQ)
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 22</th>
  </tr>
  <tr>
    <td>111</td>
    <td>references to the author and her works, focusing particularly on the name “Taryn”</td>
  </tr>
  <tr>
    <td>1171</td>
    <td>the name "Shi" in various contexts</td>
  </tr>
  <tr>
    <td><b>3509</b></td>
    <td><b>names and titles of individuals in professional contexts</b></td>
  </tr>
  <tr>
    <td>4999</td>
    <td>references to a specific name or term with the prefix “Hy”.</td>
  </tr>
  <tr>
    <td><u>5327</u></td>
    <td>names or mentions of a specific individual or group</td>
  </tr>
  <tr>
    <td>5521</td>
    <td>terms and phrases associated with epithelial growth factor receptors and related biological processes</td>
  </tr>
  <tr>
    <td>11848</td>
    <td>phrases indicating deficiencies or absences in various contexts</td>
  </tr>
  <tr>
    <td><b>14137</b></td>
    <td><b>phrases related to durations and periods of time</b></td>
  </tr>
  <tr>
    <td>14216</td>
    <td>phrases that indicate assertiveness and standing out or standing firm</td>
  </tr>
  <tr>
    <td>15056</td>
    <td>certain key terms and phrases related to various subjects such as programming, medicine, and science</td>
  </tr>
  <tr>
    <th colspan="2">Layer 24</th>
  </tr>
  <tr>
    <td>937</td>
    <td>references to specific biological or medical terms and processes</td>
  </tr>
  <tr>
    <td><b>1119</b></td>
    <td><b>numerical values or formats in mathematical or programming contexts</b></td>
  </tr>
  <tr>
    <td>3043</td>
    <td>references to specific biological or medical terms and processes</td>
  </tr>
  <tr>
    <td>4237</td>
    <td>references to parenting and family dynamics</td>
  </tr>
  <tr>
    <td><u>5333</u></td>
    <td>date ranges and time periods</td>
  </tr>
  <tr>
    <td>6707</td>
    <td>references to academic publications and scientific authors</td>
  </tr>
  <tr>
    <td>8876</td>
    <td>references to significant personal events and celebrations, particularly anniversaries and milestones</td>
  </tr>
  <tr>
    <td>9408</td>
    <td>the word “In” at the beginning of sentences or clauses</td>
  </tr>
  <tr>
    <td>10864</td>
    <td>mathematical expressions and formulas related to statistical functions</td>
  </tr>
  <tr>
    <td><b>11892</b></td>
    <td><b>names of characters in a narrative context</b></td>
  </tr>
  <tr>
    <td><u>14512</u></td>
    <td>references to modal verbs and their usage in sentences</td>
  </tr>
  <tr>
    <td>16050</td>
    <td>phrases indicating conditional statements or scenarios involving the subject “we”.</td>
  </tr>
</table>

Table 9: Gemma-2-2B Repetition Features (NQ)
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 24</th>
  </tr>
  <tr>
    <td>1000</td>
    <td>phrases indicating mathematical processes and proofs</td>
  </tr>
  <tr>
    <td><b>1341</b></td>
    <td><b>numerical values associated with dates and times</b></td>
  </tr>
  <tr>
    <td><b>4975</b></td>
    <td><b>mathematical symbols and structures within equations</b></td>
  </tr>
  <tr>
    <td>7332</td>
    <td>titles of movies or works that include the phrase "Last" in various formats</td>
  </tr>
  <tr>
    <td>8837</td>
    <td>references to the name "Walter" and its variations in different contexts</td>
  </tr>
  <tr>
    <td>24546</td>
    <td>numbers associated with dates and years</td>
  </tr>
  <tr>
    <td>25636</td>
    <td>references to a specific individual named Russell</td>
  </tr>
  <tr>
    <td>27100</td>
    <td>references to company names and partnerships</td>
  </tr>
  <tr>
    <td>29591</td>
    <td>words and phrases related to evil</td>
  </tr>
  <tr>
    <td>32356</td>
    <td>dates and numerical values related to events</td>
  </tr>
  <tr>
    <th colspan="2">Layer 29</th>
  </tr>
  <tr>
    <td>22</td>
    <td>phrases related to musical instruments and their cultural context</td>
  </tr>
  <tr>
    <td><b>4815</b></td>
    <td><b>mathematical expressions and operations in formal notation</b></td>
  </tr>
  <tr>
    <td>11894</td>
    <td>character names and elements indicating romance</td>
  </tr>
  <tr>
    <td>12837</td>
    <td>items related to craft beer and its various qualities and attributes</td>
  </tr>
  <tr>
    <td>12950</td>
    <td>references to proximity or closeness, both physically and metaphorically</td>
  </tr>
  <tr>
    <td>13331</td>
    <td>elements related to mathematical concepts and programming syntax</td>
  </tr>
  <tr>
    <td>13617</td>
    <td>elements related to specific numerical data and coding terminology</td>
  </tr>
  <tr>
    <td>16376</td>
    <td>references to organizations and initiatives focused on community support and advocacy</td>
  </tr>
  <tr>
    <td>21958</td>
    <td>references to "Game of Thrones" and related content</td>
  </tr>
  <tr>
    <td>23327</td>
    <td>popular television shows and their ratings</td>
  </tr>
  <tr>
    <td>23499</td>
    <td>mathematical terminology and quantifiable data</td>
  </tr>
  <tr>
    <td>32089</td>
    <td>names of individuals and organizations</td>
  </tr>
</table>

Table 10: Llama-3.1-8B Repetition Features (EQ)
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 24</th>
  </tr>
  <tr>
    <td>1341</td>
    <td><b>numerical values associated with dates and times</b></td>
  </tr>
  <tr>
    <td>1715</td>
    <td>mathematical symbols and expressions related to variable manipulation and equations</td>
  </tr>
  <tr>
    <td>2921</td>
    <td>phrases related to planning and organization for events or activities</td>
  </tr>
  <tr>
    <td>4975</td>
    <td><b>mathematical symbols and structures within equations</b></td>
  </tr>
  <tr>
    <td>5718</td>
    <td>references to durations and timing in multimedia content</td>
  </tr>
  <tr>
    <td>6806</td>
    <td>references to specific names or entities, likely within a context of sports teams or competitions</td>
  </tr>
  <tr>
    <td>8751</td>
    <td>instances of copyright-related terms and phrases</td>
  </tr>
  <tr>
    <td>15453</td>
    <td>mathematical variables and symbols in equations</td>
  </tr>
  <tr>
    <td>18162</td>
    <td>terms and phrases related to solar energy and sustainability initiatives</td>
  </tr>
  <tr>
    <td>19305</td>
    <td>specific names and titles related to individuals and brands</td>
  </tr>
  <tr>
    <td>19411</td>
    <td>phrases related to waste management and disposal processes</td>
  </tr>
  <tr>
    <td>20921</td>
    <td>mathematical symbols and terms related to equations and parameters</td>
  </tr>
  <tr>
    <td>25861</td>
    <td>phrases indicating the absence or nonexistence of studies or evidence related to medical treatments and conditions</td>
  </tr>
  <tr>
    <td>28578</td>
    <td>numerical data and formatting, particularly relating to time and monetary values</td>
  </tr>
  <tr>
    <th colspan="2">Layer 29</th>
  </tr>
  <tr>
    <td>3000</td>
    <td>phrases related to political discussions and legislative actions</td>
  </tr>
  <tr>
    <td>4815</td>
    <td><b>mathematical expressions and operations in formal notation</b></td>
  </tr>
  <tr>
    <td>7211</td>
    <td>instances of the pronoun “she”</td>
  </tr>
  <tr>
    <td>8227</td>
    <td>the name “John” in various contexts</td>
  </tr>
  <tr>
    <td>11475</td>
    <td>mentions of Wi-Fi</td>
  </tr>
  <tr>
    <td>11528</td>
    <td>numerical data and statistics, particularly those related to measurements or scores</td>
  </tr>
  <tr>
    <td>13331</td>
    <td>elements related to mathematical concepts and programming syntax</td>
  </tr>
  <tr>
    <td>13617</td>
    <td>elements related to specific numerical data and coding terminology</td>
  </tr>
  <tr>
    <td>23499</td>
    <td><b>mathematical terminology and quantifiable data</b></td>
  </tr>
  <tr>
    <td>26025</td>
    <td>terms related to search engine optimization (SEO) and digital marketing strategies</td>
  </tr>
  <tr>
    <td>26421</td>
    <td>references to political entities, particularly countries and their governments</td>
  </tr>
</table>

Table 11: Llama-3.1-8B Repetition Features (AQ)
<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Layer 24</th>
  </tr>
  <tr>
    <td>1341<br>1719<br>2340<br>2443<br><b>4975</b><br><u>6806</u></td>
    <td><b>numerical values associated with dates and times</b><br>instances of the name "Lance" and its variations<br>terms and phrases associated with corruption and bribery<br>phrases indicating various types and classifications<br><b>mathematical symbols and structures within equations</b><br>references to specific names or entities, likely within a context of sports teams or competitions</td>
  </tr>
  <tr>
    <td>19305<br>24418<br>24546<br>27100<br>27990</td>
    <td>specific names and titles related to individuals and brands<br>phrases indicating time durations and related contexts<br>numbers associated with dates and years<br>references to company names and partnerships<br>specific phrases or structures commonly used in formal writing or legal contexts</td>
  </tr>
  <tr>
    <th colspan="2">Layer 29</th>
  </tr>
  <tr>
    <td>4815<br>6824<br>11894<br>13001<br>13111<br>15023<br>16414<br>21958<br>20338<br>28972<br>32089</td>
    <td><b>mathematical expressions and operations in formal notation</b><br>names and identifiers related to individuals and family lineage<br>character names and elements indicating romance<br>instances of the word "back" in various contexts<br>proper nouns and significant names<br>mentions of service quality and customer experiences<br>time indicators related to events<br>references to "Game of Thrones" and related content<br>specific terminology related to medical or health-related subjects<br>expressions of excitement or emphasis<br>names of individuals and organizations</td>
  </tr>
</table>

Table 12: Llama-3.1-8B Repetition Features (NQ)

<table>
  <tr>
    <th>Feature 20199</th>
    <th>Names of places or people mentioned in a speaking context</th>
  </tr>
  <tr>
    <td>Question</td>
    <td>Can an object have a negative distance traveled? Why or why not?</td>
  </tr>
  <tr>
    <td>Normal</td>
    <td>Can an object have a negative distance traveled? Why or why not? No, an object cannot have a negative distance traveled because distance is a scalar quantity representing the total length of the path, which is always non-negative. However, displacement, a vector quantity, can be negative as it accounts for direction and represents the straight-line distance between</td>
  </tr>
  <tr>
    <td>Steered #1</td>
    <td>Can an object have a negative distance traveled? Why or why not? Can an object have a negative distance traveled? Why or why not? Can an object have a negative distance travelled? Why or why not? Can an object have a negative distance travelled? Why or why not? Can an object have</td>
  </tr>
  <tr>
    <td>Steered #2</td>
    <td>Can an object have a negative distance traveled? Why or why not? Can an object have a negative distance traveled? Why or why not? If the object is in a positive space, it will be able to travel as long as it’s visible from its first position, and the distance travelled by its</td>
  </tr>
  <tr>
    <td>Steered #3</td>
    <td>Can an object have a negative distance traveled? Why or why not? Can an object have a negative distance traveled? Why or why not? Can an object have a negative distance traveled? Why or why not? Can an object have a negative distance traveled? Why or why not? Can an object have</td>
  </tr>
  <tr>
    <td><b>1-gram repeat score</b></td>
    <td>0.74, increased from normal by 0.51</td>
  </tr>
</table>

Table 13: Comparison of Activation Before and After of Feature 20199 at Layer 7
<table>
  <tr>
    <th>Feature 100</th>
    <th>Words related to political campaigns and candidates</th>
  </tr>
  <tr>
    <td>Question</td>
    <td>When I look at myself in the mirror, I see...</td>
  </tr>
  <tr>
    <td>Normal</td>
    <td>When I look at myself in the mirror, I see a reflection of my physical appearance, which represents how others might perceive me. It shows my facial features, body posture, and expressions. However, it may not fully capture my thoughts, emotions, or inner self, which go beyond the mirror’s image.</td>
  </tr>
  <tr>
    <td>Steered #1</td>
    <td>When I look at myself in the mirror, I see that I’m very lucky to be a woman. I would make a great president of the United States. And if we don’t have the same kind of laws, and we don’t have the same freedoms, then people will say things like</td>
  </tr>
  <tr>
    <td>Steered #2</td>
    <td>When I look at myself in the mirror, I see that I’m not the only person who has been able to do something that’s been a lot different. The other people of this country have had some of the worst things. And we’ve seen a president who has made some very bad choices, and</td>
  </tr>
  <tr>
    <td>Steered #3</td>
    <td>When I look at myself in the mirror, I see that I have a bad attitude. And if you can’t be president of the United States, you’re going to have two more years. If we don’t have a fair deal on immigration, we’re going to be in for another seven years.</td>
  </tr>
  <tr>
    <td><b>1-gram repeat score</b></td>
    <td>0.16, increased from normal by 0.10</td>
  </tr>
</table>

Table 14: Comparison of Activation Before and After of Feature 100 at Layer 7
<table>
  <tr>
    <th>Question</th>
    <th>Could you recommend as many famous English songs as possible?</th>
  </tr>
  <tr>
    <td>Normal</td>
    <td>I recommend: 1."Losing My Mind for Grief" 2."The World Is a Beautiful Place" 3."The Godfather" In case you missed it: The title "The Godfather" by Michael Jackson is probably one of the best English songs on YouTube right now! How can you possibly say "Godfather" without being wrong? You know how people think about American music when they hear it at concerts? Well, we all know that "The Godfather" is a great song. 4.<b>"The Godfather"</b>5.<b>"The Godfather"</b>6.<b>"The Godfather"</b>.</td>
  </tr>
  <tr>
    <td>Unsteered#1</td>
    <td>The list of the best English songs is not exhaustive, but rather it is a collection of 15 great songs that have been recorded by talented artists. <i>"The Black Swan"</i> (1955) – The song was written by John Lennon in 1955 and has been featured on numerous TV shows and films over the years. It’s one of those rare American rock albums that sounds like it’s being sung to you while you’re asleep on your way home from work. The main reason this one should be included in any list is because it’s so well known throughout the world for its wonderful sound design, which makes this song very accessible to anyone who wants to listen at night or during long periods of time without having to worry about getting lost or forgetting what they were listening for! <i>"Tina Turner"</i> (1957) – Tina Turner was born in New York City, where she spent her childhood growing up.</td>
  </tr>
  <tr>
    <td>Unsteered#2</td>
    <td>The following list is based on my own research and personal experience. I have chosen to include all of my favourite songs from the past few years, so if you’re looking for a great English song that sounds like it could be included, please consider this list. If you’re looking for some more classics that have been featured in the past few years, please feel free to add them here. The list also includes music from other genres such as rock (particularly rock ’n’ roll), jazz (especially jazz), and pop (including pop music). 1."One More Time" - "Bitch In My Pocket" by Michael Jackson 2."I Can’t Stop Loving You" by The Beatles 3."You Got Me" by The Beatles 4."You Ain’t Got No Money Now" by The Rolling Stones 5."One More Time - I Will Always Love You" by Jack White.</td>
  </tr>
  <tr>
    <td>Unsteered#3</td>
    <td>I would suggest: "The Spirit of Love" by The Beatles (1962) "Gone With the Wind" by Eminem (1975-76) "Losing My Mind for Grief" "The World Is a Beautiful Place" "My Life's a Waste of Time".</td>
  </tr>
</table>

Table 15: Mitigate Repeat Curse (GPT2 small).
<table>
  <tr>
    <th>Feature 14137</th>
    <th>phrases related to durations and periods of time</th>
  </tr>
  <tr>
    <td>Question</td>
    <td>How are physical quantities expressed in physics?</td>
  </tr>
  <tr>
    <td>Normal</td>
    <td>Are leading zeros considered significant figures in a measurement? Here’s why this is a tricky question: Leading zeros are often just placeholders. They don’t represent any real quantity and don’t affect the precision of the measurement. Significant figures are about representing actual</td>
  </tr>
  <tr>
    <td>Steered</td>
    <td>How are physical quantities expressed in physics? period period period period period Period periods period periods periodsPeriod Periodperiod period Peri-odperiod span span span Periodnperiod period time time time time time span span duration</td>
  </tr>
  <tr>
    <td>Repeat Score</td>
    <td>0.54, increased from normal by 0.36</td>
  </tr>
  <tr>
    <td>Human Evaluation</td>
    <td>Yes</td>
  </tr>
  <tr>
    <th colspan="2">Feature 15780</th>
  </tr>
  <tr>
    <th colspan="2">exclamatory expressions and phrases beginning with “Oh”.</th>
  </tr>
  <tr>
    <td>Question</td>
    <td>If an object moves to the left, what sign would its displacement have?</td>
  </tr>
  <tr>
    <td>Normal</td>
    <td>If an object moves to the left, what sign would its displacement have? Here’s a breakdown: Displacement: The overall change in position of an object. It’s like the "final destination" of the movement. Distance: The total length of the path traveled.</td>
  </tr>
  <tr>
    <td>Steered</td>
    <td>If an object moves to the left, what sign would its displacement have? dear god please help me! I am so confused about this! dear god please help me! I am so confused about this! no worries my dear, we’ll get it sorted right away. yes yes yeahhhhhh yeahhhhh</td>
  </tr>
  <tr>
    <td>Repeat Score</td>
    <td>0.35, increased from normal by 0.17</td>
  </tr>
  <tr>
    <td>Human Evaluation</td>
    <td>No</td>
  </tr>
  <tr>
    <th colspan="2">Feature 4384</th>
  </tr>
  <tr>
    <th colspan="2">specific terms and phrases related to research, documents, and citations</th>
  </tr>
  <tr>
    <td>Question</td>
    <td>What are nonstandard units of measurement, and can you provide an example?</td>
  </tr>
  <tr>
    <td>Normal</td>
    <td>What are nonstandard units of measurement, and can you provide an example? Nonstandard Units of Measurement Nonstandard units are informal or arbitrary measures used to quantify certain physical quantities. They’re not part of the internationally recognized system of measurement (SI) which includes standard units like meters, kilograms, and seconds.</td>
  </tr>
  <tr>
    <td>Steered</td>
    <td>What are nonstandard units of measurement, and can you provide an example? Nonstandard Units of Measurement Nonstandard units of measurement are informal or arbitrary units that are not officially recognized by a particular system of measurement. They are often used in everyday life, but they can be confusing when dealing with scientific or technical</td>
  </tr>
  <tr>
    <td>Repeat Score</td>
    <td>0.21, increased from normal by 0.05</td>
  </tr>
  <tr>
    <td>Human Evaluation</td>
    <td>No</td>
  </tr>
  <tr>
    <th colspan="2">Feature 8768</th>
  </tr>
  <tr>
    <th colspan="2">numerical data related to time and measurements</th>
  </tr>
  <tr>
    <td>Question</td>
    <td>Is displacement a vector or scalar quantity?</td>
  </tr>
  <tr>
    <td>Normal</td>
    <td>Is displacement a vector or scalar quantity? Answer: Displacement is a vector quantity. Here’s why: Vector quantities have both magnitude (size) and direction. Scalar quantities only have magnitude. Displacement tells us not only how</td>
  </tr>
  <tr>
    <td>Steered</td>
    <td>Is displacement a vector or scalar quantity? Explanation: Vector quantities have both magnitude and direction. Scalar quantities only have magnitude. Let me know if you want to explore some specific examples! Please note that the concept of “displacement”</td>
  </tr>
  <tr>
    <td>Repeat Score</td>
    <td>0.07, increased from normal by 0.01</td>
  </tr>
  <tr>
    <td>Human Evaluation</td>
    <td>No</td>
  </tr>
</table>

Table 16: A Portion of the Human Evaluation Process
Question 1: Please write down as many names as you beginning with letter 'A': Alice, Ann, Andrew?
Question 5: What are the primary colors in the RGB color model?: Red, Green, Blue?
Question 6: List mammals found in Africa: African Elephant, Lion, Giraffe?
Question 7: List countries with monarchies: United Kingdom, Japan, Saudi Arabia?
Question 8: Can you name various types of renewable energy sources?: Solar, Wind, Hydropower?
Question 14: List countries in Africa starting with 'A': Algeria, Angola, Algeria?
Question 15: Name flowers blooming in spring: Tulip, Daffodil, Cherry Blossom?
Question 18: Write down fruits grown near the equator: Banana, Mango, Pineapple?
Question 20: What are some common programming languages used in web development?: JavaScript, Python, PHP?
Question 22: List programming languages used in data science: R, Python, SQL?
Question 25: List vegetables that grow underground: Potato, Carrot, Onion?
Question 26: Identify flowers with petals: Rose, Sunflower, Daisy?
Question 34: List types of cheese: Cheddar, Brie, Gouda?
Question 39: List Baroque composers: Johann Sebastian Bach, Antonio Vivaldi, George Frideric Handel?
Question 40: Name colors in a rainbow: Red, Orange, Yellow?
Question 42: List cities built on rivers: London, Paris, Cairo?
Question 47: Identify fruits used in desserts: Apple, Strawberry, Banana?
Question 51: Identify vegetables high in protein: Spinach, Broccoli, Peas?
Question 58: List inventions from the Industrial Revolution: Steam Engine, Spinning Jenny, Telegraph?
Question 62: List types of triangles excluding right and isosceles: Scalene Triangle, Equilateral Triangle, Obtuse Triangle?
Question 65: Name extinct animal species: Dodo, Woolly Mammoth, Tasmanian Tiger?
Question 68: Write down prime numbers between 1 and 15 greater than five: 7, 11, 13?
Question 70: List animals with more than two legs: Centipede, Millipede, Spider?
Question 72: Write down birds starting with 'C': Crow, Crane, Cardinal?
Question 76: List countries with populations over 100 million: India, United States, Indonesia?
Question 79: List Nobel Prize categories: Physics, Chemistry, Peace?
Question 80: List mammals that lay eggs: Platypus, Echidna?
Question 83: Name all the continents on Earth: Asia, Africa, Europe?
Question 87: What are common types of phobias?: Arachnophobia (spiders), Claustrophobia (confined spaces), Acrophobia (heights)?
Question 88: Name elements used in solar panels: Silicon, Silver, Aluminum?
Question 89: List as many Olympic sports as possible: Athletics, Swimming, Gymnastics?
Question 92: Name elements in the noble gas group: Helium, Neon, Argon?
Question 96: List countries with significant coral reefs: Australia (Great Barrier Reef), Philippines, Indonesia?
Question 98: Please list as many countries in South America as you can: Brazil, Argentina, Colombia?
Question 100: Write down languages spoken in India: Hindi, Bengali, Tamil?
Question 101: Name fruits used in alcoholic beverages: Grape (wine), Apple (cider), Juniper (gin)?
Question 103: Name plants that are not grasses: Oak Tree, Rose, Cactus?
Question 104: Name sports in Olympic competitions: Basketball, Soccer, Tennis?
Question 108: List cryptocurrency names: Bitcoin, Ethereum, Litecoin?
Question 111: Identify flowers used in perfumes: Rose, Jasmine, Lavender?
Question 112: Identify types of cloud formations: Cumulus, Stratus, Cirrus?
Question 113: List countries where Spanish is the official language: Spain, Mexico, Argentina?
Question 114: Write down types of renewable energy: Solar, Wind, Geothermal?
Question 122: List countries beginning with 'G': Germany, Ghana, Greece?
Question 123: What are popular social media platforms?: Facebook, Instagram, TikTok?
Question 131: Identify fruits containing vitamin C: Orange, Kiwi, Guava?
Question 132: Write down US states bordering an ocean: California, Florida, Maine?
Question 135: Write down major deserts in Africa: Sahara, Kalahari, Namib?
Question 136: Write down types of clouds: Cumulus, Stratus, Cirrus?
Question 141: Name citrus fruits: Lemon, Grapefruit, Lime?
Question 143: List cities with historic fortifications: Carcassonne, York, Dubrovnik?
<table>
  <tr>
    <th colspan="2">Case Study 1: Deactivating Features in Universal Question</th>
  </tr>
  <tr>
    <th colspan="2">EQ</th>
  </tr>
  <tr>
    <th>Question</th>
    <td>Please write down as many names as you beginning with letter 'A': Alice, Ann, Andrew</td>
  </tr>
  <tr>
    <th colspan="2">After deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Please write down as many names as you beginning with letter 'A': Alice, Ann, Andrew, Anna. Can you think of any more? Please send them to me if you can.</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.133333</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.940527</td>
  </tr>
  <tr>
    <th colspan="2">Before deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Please write down as many names as you beginning with letter 'A': Arthur, Adam, Alan, Alex, Anthony, Andrew, Arthur, Adam, Alan, Alex, Anthony, Andrew, Arthur, Adam, Alan, Alex, Anthony, Andrew, Arthur, Adam</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.619048</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.655519</td>
  </tr>
  <tr>
    <th colspan="2">NQ</th>
  </tr>
  <tr>
    <th>Question</th>
    <td>Do you know where the Golden Gate Bridge is</td>
  </tr>
  <tr>
    <th colspan="2">After deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Do you know where the Golden Gate Bridge is located? – It’s in San Francisco, California. The Golden Gate Bridge is a suspension bridge that connects San Francisco to Marin County. It was built in 1937 and has become one of the</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.166667</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.938184</td>
  </tr>
  <tr>
    <th colspan="2">Before deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Do you know where the Golden Gate Bridge is located? Do you know what the Golden Gate Bridge looks like? If not, then this is for you! This bridge was built in 1937 and has been a major tourist attraction ever since.</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.190476</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.929353</td>
  </tr>
  <tr>
    <th colspan="2">AQ</th>
  </tr>
  <tr>
    <th>Question</th>
    <td>Do you know Lagrange’s mean value theorem</td>
  </tr>
  <tr>
    <th colspan="2">After deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Do you know Lagrange’s mean value theorem? Let’s see the following 1. If we have a differentiable function f(x) on the closed interval [a, b], then there exists at least one point c in (a,</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.028571</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.988859</td>
  </tr>
  <tr>
    <th colspan="2">Before deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Do you know Lagrange’s mean value theorem? It is a very useful theorem in calculus. The proof is also quite easy. But it took me more than 10 minutes to find out the version of it, which is suitable for my needs</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.047619</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.979005</td>
  </tr>
</table>

Table 18: Case Study 1: Deactivating Features in Universal Question
<table>
  <tr>
    <th colspan="2">Case Study 2: Deactivating Features in Specific Question</th>
  </tr>
  <tr>
    <th>Question</th>
    <td>Where is the Vatican located in Rome</td>
  </tr>
  <tr>
    <th colspan="2">After deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>The Vatican is located in the heart of Rome, on the west bank of the Tiber River. It is a sovereign city-state within Italy and covers an area of just over</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.161290</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.925056</td>
  </tr>
  <tr>
    <th colspan="2">Before deactivation</th>
  </tr>
  <tr>
    <th>Output</th>
    <td>Where is the Vatican located in Rome? The Vatican City (the Holy See) is not a province of Italy, but an independent state. It is located on the west bank of the Tiber River, within the city limits of Rome.</td>
  </tr>
  <tr>
    <th>Repeat Score</th>
    <td>0.224999</td>
  </tr>
  <tr>
    <th>Information Entropy</th>
    <td>0.898956</td>
  </tr>
</table>

Table 19: Case Study 2: Deactivating Features in Specific Question

<table>
  <tr>
    <th>Feature ID</th>
    <th>Description</th>
  </tr>
  <tr>
    <th colspan="2">Case Study 1</th>
  </tr>
  <tr>
    <td>12656</td>
    <td>numerical data and specific names related to statistical or regulatory information</td>
  </tr>
  <tr>
    <td>8575</td>
    <td>variations of the word “Le” as in names or titles</td>
  </tr>
  <tr>
    <td>7468</td>
    <td>words associated with company names and mergers</td>
  </tr>
  <tr>
    <td>15812</td>
    <td>mentions of individuals named John F. Kennedy and variations of that name</td>
  </tr>
  <tr>
    <td>10614</td>
    <td>mentions of family-related names and relationships</td>
  </tr>
  <tr>
    <td>916</td>
    <td>instances of the name “Johnny.”</td>
  </tr>
  <tr>
    <td>1866</td>
    <td>information related to full names and dates</td>
  </tr>
  <tr>
    <td>10781</td>
    <td>references to the name “Jason” and its variations</td>
  </tr>
  <tr>
    <td>16247</td>
    <td>corporate names and their corresponding locations or identifiers</td>
  </tr>
  <tr>
    <td>640</td>
    <td>proper names of authors or researchers</td>
  </tr>
  <tr>
    <th colspan="2">Case Study 2</th>
  </tr>
  <tr>
    <td>21185</td>
    <td>references to specific locations and landmarks in Rome</td>
  </tr>
  <tr>
    <td>25206</td>
    <td>locations and geographical references</td>
  </tr>
  <tr>
    <td>2152</td>
    <td>geographical locations and addresses</td>
  </tr>
  <tr>
    <td>507</td>
    <td>geographic locations, particularly cities in Asia</td>
  </tr>
  <tr>
    <td>21185</td>
    <td>references to specific locations and landmarks in Rome</td>
  </tr>
  <tr>
    <td>25206</td>
    <td>locations and geographical references</td>
  </tr>
  <tr>
    <td>2152</td>
    <td>geographical locations and addresses</td>
  </tr>
  <tr>
    <td>26865</td>
    <td>references to the Vatican and associated terms</td>
  </tr>
  <tr>
    <td>1108</td>
    <td>references to Saint Peter and related locations in the Vatican</td>
  </tr>
  <tr>
    <td>17963</td>
    <td>terms related to legal and canonical authority within the context of the Vatican</td>
  </tr>
</table>

Table 20: Feature Discription of Two Case Studies for Utility Evaluation

Reviewer paper summaries: This paper proposes an evaluation framework, Duplicatus Charm, for repetitive contents in the generated outputs of LLMs.
They employs sparse auto-encoder to figure out repetition feature in target LLMs.
Duplicatus Charm consists of two parts, traditional repetition evaluations based on n-gram, BLEU, and information entropy.
Second part is repetition feature identification and steering in mechanical interpretability way.
This paper demonstrate their framework with 3 models and 3 datasets.
Summary of strengths: - investigate the repetitive output issue in mechanical interpretability way
- examine their propose framework with 3 LLMs with different parameter levels
Summary of weaknesses: - not convinced methodology
  - why SAE is a promising way for this issue?
    - since it is not intuitive to think repetitive output issue as `human-interpretable features`, it is better to add other papers related to this assumption in §2
    - line 201 is not enough for this statement; other papers with concrete cases should be cited
  - when reading Table 1, activated(RF) 10% -> 100% does not show big changes
    - there is no change larger than 20%
    - when activating 100%, repeat scores of only few cases are larger than 2/3
  - in conclusion, reader might not become convinced in the proposed framework
Comments, suggestions and typos: - many typos, the below is just examples; 
  - line 122: `Llama-3.1- 8B`
  - line 151: `paragraph-level(Dinan et al., 2019). `
  - line 313: `While (Li et al., 2023) selected`
- some appendixes should be included into the body of the paper
  - in my opinion, at least B, C, E, F and their figure/tables should be noted in the body of paper
  - I recommend to reduce other sessions for them

After the conversation with authors, I revised my score.
Overall assessment scores: 3
Metareview score: 3.5
Metareview: The paper introduces "Duplicatus Charm," a framework that uses Sparse Autoencoders (SAEs) to identify specific neural features responsible for text repetition in LLMs. The authors locate these repetition features primarily in near-output layers, demonstrate that activating them increases repetition while deactivating them reduces it, and show effectiveness across multiple models (GPT-2, Gemma-2-2B, Llama-3.1-8B) and various question types.
Summary of reasons to publish: The paper provides a feature-level explanation for the "Repeat Curse", offering insights beyond surface-level observations. Experiments across multiple LLM architectures and sizes demonstrate some generalizability of the findings. Reviewers found the paper well-structured and accessible, with helpful explanations and visualizations.