Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
Abstract
While large-scale neural language models, such as GPT2 and BART, have achievedimpressive results on various text generation tasks, they tend to get stuck in undesirable sentence-level loops with maximization-based decoding algorithms ( e.g. ,greedy search).
This phenomenon is counter-intuitive since there are few consecutive sentence-level repetitions in human corpora (e.g., 0.02% in Wikitext-103).
Toinvestigate the underlying reasons for generating consecutive sentence-level repetitions, we study the relationship between the probabilities of the repetitive tokensand their previous repetitions in the context.
Through our quantitative experiments,we find that 1) Language models have a preference to repeat the previous sentence;2)
The sentence-level repetitions have a self-reinforcement effect : the more timesa sentence is repeated in the context, the higher the probability of continuing togenerate that sentence; 3) The sentences with higher initial probabilities usuallyhave a stronger self-reinforcement effect.
Motivated by our findings, we propose asimple and effective training method DITTO (PseuDo-RepetITion PenalizaTiOn),where the model learns to penalize probabilities of sentence-level repetitions frompseudo repetitive data.
Although our method is motivated by mitigating repetitions, experiments show that DITTO not only mitigates the repetition issue withoutsacrificing perplexity, but also achieves better generation quality.
Extensive experiments on open-ended text generation (Wikitext-103) and text summarization(CNN/DailyMail) demonstrate the generality and effectiveness of our method.
Introduction
Recently, large-scale Transformer-based [31] neural language models (e.g., GPT2 [26], BART [15]and OPT [35]) have shown remarkable capability to generate text and achieved better performancethan before, such as open-ended generation [26, 4] and summarization tasks [15, 35].
However,models with standard maximization-based decoding are known to get stuck in redundant consecutiverepetitions [11].
As shown in Figure 1, the model has a stronger preference for consecutive sentencelevel repetitions than the word- or phrase-level while human language has fewer consecutive sentencelevel repetitions, which shows a discrepancy between human language and the generated texts.
Existing approaches to mitigate repetitions can be categorized into decoding-based and training-basedmethods.
Decoding-based methods rectify these issues by soft or hard n -gram blocking [23, 14] andstochastic sampling [6, 12, 26, 11].
Training-based methods minimize the probabilities of tokens thatalready are generated in the previous context [32, 17].
Despite their effectiveness, the reasons whythe model prefers repetitions and how repetition occurs during decoding are still unclear.
Fu et al .
[8] is the first to analyze the repetition problems from a theoretical perspective by assumingthat the language models can be approximated by short-sighted first-order Markov models.
However,
Holtzman et al .
[11] observe the cases of a positive feedback loop of repetitions, which indicatesthat language models do look at the long-distance context and may not be simply viewed as firstorder Markov models.
The cases also reveal that the probabilities of repetitive tokens have certainrelationships with previous repetitions in context.
However, they do not analyze why the modelprefers consecutive repetitions.
In this paper, we further dig into the problem and conduct quantitativeexperiments to analyze the underlying issue of the repetition.
Prefix: The rim is slightly raised about the surrounding surfaceand it has an outer rampart that has been only lightly impacted .
No significant craters intersect the rim , and it is sloped about 1 @.@ 5 °toward the direction 50 – 90 °from the Earth .
Greedy Decoding: The age of the crater is about 3 @.@ 6 billion yearsand it has been in the proximity of the south lunar pole for at least 10 @,@ 000 years .
The South
Crater is located on the southern edge of the northern highlands . The South Crater is located on the southern edge of the northern highlands .
The South
Crater is located on the southern edge of the northern highlands . The South Crater is located on the southern edge of the northern highlands .
The South
Crater is located on the southern edge of the northern highlands . The South Crater is located on the southern edge of the northern highlands .
The South
Crater is located on the southern edge of the northern highlands .
The South Crater is located on the southern edge of the northern highlands . …
We train a Transformer model (750M parameters, similar to GPT-2 Large) on large-scale
Wikitext greedily
Compared to human language, the model has substantially more consecutive sentence-level repetition
For a quantitative investigation on consecutive repetitions, we compare the probabilities of the sametokens in repetitive sentences.
For example, given a sequence, ‘I love oranges .
I love oranges .’,we
compare the probability P θ ( ‘oranges’ | ‘I love oranges .
I love’ ) with P
θ ( ‘oranges’ | ‘I love’ ) .
Thedifference between them is that, for the second ‘oranges’, there is already a token ‘oranges’ that sharesthe same sentence-level context ‘I love’.
We manually repeat the sentence n times as the context sothat the next ‘oranges’ has n repetitions in context.
In this way, we can investigate the relationshipbetween the probability of the token and the number of repetitions in context.
For example, we canfirst construct the context by repeating the sentence ‘I love oranges .’
n times plus ‘I love’, and thenobtain the probability that the model outputs ‘orange’ at the current step.
Through our quantitative investigation across the different corpus, we find that 1)
The model tendsto raise the probability of repeating the previous sentence.
Specifically, even if there is only onesentence-level context repetition, the probability of repetition at the current step increases in mostcases.
The cause of the phenomenon may be that the model is so confident with its context, when thereis a previous token (i.e., ‘oranges’) sharing the same sentence-level context (i.e., ‘I love’), that themodel learns a shortcut to directly copy the token; 2) Self-reinforcement effect: the probability ofrepetition increases almost monotonically with the number of historical repetitions.
Finally, theprobability of repetition stabilizes around a certain ceiling value.
As shown in Figure 2, as the numberof repetitions increases, the probability of the word ‘rounds’ and ‘general’ both increase almostmonotonically and finally stabilize; 3) Sentences with higher initial probabilities usually have astronger self-reinforcement effect.
For example, in Figure 2, we can find that the sentence with ahigher initial probability (i.e., the red bar at ‘0’ of x-axis) grows faster and can reach an extremelyhigh value with a few repetitions.
Furthermore, the sentences with a higher initial likelihood (e.g.,sentences generated by the model itself with maximization-based decoding algorithms) may have astronger self-reinforcement effect.
According to our findings, the reasons why the model tends to repeat themselves are as follows:
Thesentence repetition occurs since the previous sentence generated by the maximization-based decodingalgorithms has a relatively high probability, and the model tends to further increase the probabilityof repeating that sentence.
Once the model generates one repetitive sentence, the probability of therepetitive sentence would be further enhanced since there are more repetitions sharing the samesentence-level context to support the choice of copying.
As a result, the model gets stuck in thesentence-level loops due to the self-reinforcement effect .
To mitigate the tendency to repeat previous sentences and overcome the self-reinforcement effect,we aim to make the model aware that the probabilities of generating redundant sentences shouldbe reduced.
In addition, the more sentence-level repetitions exist, the lower the probability of theredundant sentence should be.
In this paper, we propose a simple and effective training-based method,
PseuDo-RepetITion PenalizaTiOn ( DITTO ).
We first manually construct pseudo data by repeating asentence sampled from the training corpus.
Then, we design a repetition penalization loss function on these pseudo data for the model to learn to exponentially decay the repetition probability asthe number of sentence repetitions increases.
The experimental results show that DITTO not onlysignificantly reduces repetitions without sacrificing language model capability in terms of perplexity,but also achieves better generation quality.
In the Wikitext-103 [19] open-ended generation task, ourmethods can achieve lower perplexity and higher MAUVE scores than the strong competitors.
Withstochastic decoding strategies such as top- k [6] and nucleus sampling [11], generated texts by ourmethods are the closest to human-written texts, as measured by MAUVE score and human evaluation.
In the commonly-used CNN/DailyMail summarization benchmark [10, 20], our methods also achievethe best performance among the strong competitors.
Analyzing Repetition
In this section, we would like to quantitatively investigate the relationship between the probabilitiesof repetitive sentences and previous repetitions in context.
We first define several metrics to measurehow the probability changes as the number of repetitions increases.
Then, we summarize our findingsfrom the experimental results, and discuss how sentence-level repetitions occur during decoding.
2.1 Experiment Design
Formally, we have a sentence s from a corpus D and repeat it N times to construct a sample sequencex =
( s 0 , s 1 , s 2 , · · · , s N ) where s n = ( x n, 1 , · · · , x n,L s ) .
x n,l is the l -th token in the n -th repetition ofthe sentence s and L s is the number of tokens in sentence s .
Denote the sentence-level context of tokenx n,l as x n,<l =
( x n, 1 , · · · , x n,l − 1 ) .
We define that there is a sentence-level context repetition for thetoken x n,l if and only if there is another same token x i,l in context
such that x i,<l = x n,<l
and i < n .
For example, for the sequence ‘I love oranges .
I love oranges .’
aforementioned, the second token‘oranges’ has a sentence-level context repetition .
By feeding the sequence x to a pre-trained model
P θ , we can obtain the probability distribution P θ ( x n,l | x <n,l ) where x <n,l = ( s 0 , · · · , s n − 1 , x n,<l ) .
To study whether P θ ( x n,l | x <n,l ) increases as n increases, we define several metrics as follows:
• Average Token Probability: TP ( s n ) =
1 L s (cid:80) L s l =1 P θ ( x n,l | x <n,l ) , which is the average tokenprobability of n -th repetitive sentence s n .
TP ( s 0 ) is the initial probability of tokens in the firstsentence.• Rate of Increased Token Probability : IP ( s n ) =
1 L s (cid:80) L s l =1 1 ( P θ ( x n,l | x <n,l ) >
P θ ( x 0 ,l | x < 0 ,l ))where 1 is the indicator function.
We use IP ( s n ) to calculate that, how many probabilities of tokensincrease in s n compared to the initial ones in s 0 .
[Figure]
• Winner Rate: We say x n,l is a winner if P θ ( x n,l | x <n,l ) >
P θ ( x 0 ,l | x < 0 ,l ) and x n,l =arg max P ( ·| x <n,l ) .
Then, we define the winner rate as WR ( s n ) =
1 L s (cid:80) L s l =1 1 ( x n,l is a winner ) .
A higher winner rate means that the repetitions are
more likely to
be
generated
by
a
maximization-
based
decoding
algorithm
such
as greedy decoding.
Following the previous work [32, 17], the pre-trained model is a 16-layer Transformer decoder trainedon Wikitext-103 [19].
The details are introduced in Sec. 4.1.
Given the corpus D , we can calculatethe average values of TP, IP and WR of the n -th repetitive sentence s n as
[Equation]
by enumerating all sentences s in D .
To further study the effect of different corpus, we construct threecorpus 1) D random : the tokens of sentences are randomly sampled from the vocabulary of the model;2)
D book : the sentences are randomly sampled from BookCorpus [36], and 3) D wiki : the sentences arerandomly sampled from the dev set of Wikitext-103.
The size of all different corpus is 1,000.
2.2 Results and Analyses
In the analyses, we attempt to answer: 1) Why sentence repetitions occur?
2) Why the model getsstuck into the repetition loop? 3)
What kinds of sentences are more likely to be repeated?
Why sentence repetitions occur?
As shown in Figure 3 (b), IP 1 is higher than 90% across thevarious corpus, meaning that even if there is only one sentence-level context repetition, the probabilityof repetition increases in most cases.
That indicates the model has a strong preference to repeatthe previous sentence .
Note that the token repetition has not occurred at the current prediction step,and there is only the same sentence-level context.
For example, model assigns a higher probabilityto P θ ( ‘oranges’ | ‘I love oranges .
I love’ )
than P θ ( ‘oranges’ | ‘I love’ ) since the model has seen thepattern ‘I love oranges’ in the previous context.
Thus, the model may be too confident with itsprevious context repetition and learn a ‘cheap’ shortcut where it directly copies the next token,‘oranges’.
It is quite different from human language because human would try to avoid full sentencerepetition, as the repeated sentence has no new information in most cases.
Why the model gets stuck into the repetition loop?
Furthermore, as shown in Figure 3, as thenumber of repetitions increases, TP, IP and WR increase monotonically.
That means, the sentencelevel repetition has a self-reinforcement effect: the more times a sentence has been repeated in thecontext, the higher the probability of continuing to generate that sentence.
Finally, IP, WR, TPconverge around certain ceiling values.
What kinds of sentences are more likely to be repeated?
We can find that the sentences with highinitial sentence probability TP 0 , such as sentences sampled from D wiki and D book , have a strongerself-reinforcement effect: TP and WR grows faster and reach the high ceiling values with a fewrepetitions.
The higher TP and WR, the more likely the sentence to be repetitively generated bymaximization-based decoding algorithms.
Note that sentences generated by a maximization-baseddecoding algorithm have higher initial likelihoods.
Thus, the model prefers to repeat itself.
Analyses During decoding with maximization-based algorithms, the sentence-level repetitionoccurs because 1) Previous generated sentences have high likelihoods and thus have more potentialto be repeated; 2) Given previously generated sentences, the model is more likely to generate therepetitive sentence since the model is confident on the historically generated context and tries to findhints in context for the current generation; 3) Once the model repeats the sentence for several times, itwould get stuck in the sentence loop due to self-reinforcement effect .
The effectiveness of stochasticsampling approaches such as top- k [6] and nucleus sampling [11] may rely on 1) compared to themaximization-based decoding algorithms, the likelihood of previously generated sentences is lowerand thus the sentences have less risk of being repeated; 2) current tokens are generated stochasticallyrather than chosen with maximum probability which can avoid the self-reinforcement effect.
Pseudo-repetition Penalization Training
According to our analyses in Sec. 2, it can be clearly seen that the core issue of sentence-levelrepetition for the model is the tendency to repeat previous sentences and the self-reinforcement effect.
In this section, we propose a simple and effective method, named PseuDo-RepetITion PenalizaTiOn( DITTO ).
It first manually feeds repetitive sentences to the model and then explicitly teaches themodel to be averse to such repetitions.
Pseudo Repetitive Data Construction.
To construct a pseudo repetitive sample x , we first randomly pick a sentence s from the training corpus.
Then, the pseudo data x = ( s 0 , · · · , s N ) =( x 0 , 0 , · · · , x 1 , 0 , · · · , x N, 0 , · · · , x N,L s ) is constructed by repeating the sentence s by N + 1 times.
The sentences are repeated until they reaches the maximum input sequence length of the model (e.g.,1,536 tokens in the Transformer for open-end generation).
Sentence-level Repetition Penalization.
To define the per-step penalization loss for token x ∈{ x 1 , 0 , · · · , x N,L s } , we define the training objective for the l -th token in the n -th repetition of thesentence s as
[Equation]
DITTO
[Equation]
P
[Equation]
where P ∗ θ ( · ) means that the value is excluded for gradient backpropgation, which can implemented bytensor.detach in pytorch [22].
λ is the penalization factor.
When λ = 1 , the loss function is minimizedwhen the probability of token x n,l in n -th repetition is same as that in the ( n − 1) -th repetition to avoidthe self-reinforcement effect; when λ < 1 , the probability of token x n,l in n -th repetition should besmaller than that in the ( n − 1) -th repetition to make model averse to sentence-level repetition.
Inother words, the probability of tokens in repetitive sentence should decay exponentially with a factorof λ , where λ is a hyper-parameter.
In our experiments, we apply the sentence-level repetition penalization by fine-tuning a standard
MLE baseline.
Fine-tuning is done by equally mixing the sentence-level repetition penalizationupdate and normal MLE loss update.
We find that, appending another sentence sampled from thetraining corpus randomly as the prefix to the pseudo repetitive data x can achieve better and morestable performance.
We have tried other alternative loss functions, but they do not perform better.
See Appendix C for the details.
Discussion In human language, there are necessary token-level and phrase-level repetitions thatnaturally occurs.
For example, names of people, city and lemma, set phrases and proverbs appearmany times in long documents such as Wikipedia.
Thus, given the prefix, the model should have theability to copy these words or phrases from the prefix and increase their repetition probabilities.
Theseuseful repetitions can be viewed as positive samples , and our pseudo repetitive data can be viewed asnegative samples .
Combining them for training, the model should learn to distinguish between usefuland useless contexts.
Thus, although our method is motivated by overcoming sentence-level repetitionissues, it may improve the learning ability and generalizability of the model.
Unlike previous methods(e.g.
, token-level unlikehood training [32], n -gram blocking [23, 14] and contrastive search [30]) thatput hard constraints to penalize any repetitions (regardless of the necessity of repetitions), our trainingobjective explicitly encodes the intuition that the model should be inclined to avoid over-repetitions.
Therefore, it may alleviate the drawbacks of over-penalization.
Experiments
4.1 Setup
Dataset and Model Training We train models on the benchmark dataset Wikitext-103 [19] toevaluate the performance of open-ended generation.
The dataset contains over 100 million words.
Theexperiments are conducted at the word level.
The model architecture and training hyper-parametersexactly follow the implementations of Welleck et al .
Specifically, we use a 16-layer Transformerwith 8 attention heads, hidden size 1024 and fully-connected dimension 4096.
There are a total of750 million parameters, similar to GPT-2 Large [26].
We first train the baseline model with standardmaximum likelihood (MLE) for a maximum of 150k updates with a batch size of 3 samples.
Themaximum length of each training sample is 1,536 tokens.
Then, we fine-tune the model that has thebest validation perplexity with DITTO training for 10k steps.
Unless otherwise mentioned, λ is set as0.5.
Baseline models are trained following their official implementations.
All models are trained on
NVIDIA Tesla V100 GPUs.
The experiments are implemented based on fairseq codebase [21].
Evaluation Following common practices [32, 17], we evaluate the quality of open-ended sequencecompletions on the test set of Wikitext-103, where the prefix is 50 tokens, and models autoregressivelygenerate the next 100 tokens.
Models including various baselines are selected with the best perplexityon the validation set for a fair comparison.
The evaluation metrics are listed as follows:
• MAUVE : MAUVE [24] is a metric to measure how close model-generated text is to humanlanguage by approximately calculating the KL divergence between the distribution of modelgenerated sequences and human language.
The range of MAUVE is from 0 to 1.
A higher MAUVEscore indicates better generated sequences.
• Perplexity and Accuracy : Given the prefix and true next token, we use perplexity, and next-tokenprediction accuracy to measure the language modeling ability.
• Repetition :
Following previous work [32, 17], we calculate the portion of duplicate 4grams ( Repetition-4 ) in a
generated sequence to measure phrase-level repetition, definedas 1 . 0 − | unique 4-grams | / | 4-grams | , and average over completions.
Similarly, we use portion of duplicate sentences ( Repetition-Sen ) to measure sentence-level repetition, defined as1 . 0 − | unique sentences | / | sentences | .
Since there are natural repetitions in human language, theoptimal model should produce text whose repetition metrics are close to that of the gold text.
4.2 Results of Open-ended Generation
Method Comparison We compare with training-based methods since decoding-based methodscan be readily applied to models with our training method.
Following [32, 17], we first comparedifferent training-based algorithms based on greedy decoding.
The results are presented in Table 1.
Our DITTO can significantly reduce the phrase-level and sentence-level repetitions and achieve thebest MAUVE score of 0.77.
Although UL-token+seq can generate fewer repetitions, its improvementin the general quality of the model measured by MAUVE is limited since its generations are usuallynot relevant to the prefix (See Appendix E for examples).
Compared to the standard baseline MLE,other competitors reduce repetitions by sacrificing perplexity while DITTO achieves lower perplexityand higher accuracy.
These results show that DITTO can not only mitigate repetitions but also helpthe model improve the language modeling and generation quality.
Compatibility with Stochastic Decoding Strategies Sampling-based decoding algorithms such astop- k
[6] and top- p (nucleus sampling)
[11] are widely used in the various applications [1, 27] andlarge models [26, 4].
We also confirm that our DITTO is compatible with these popular stochasticdecoding strategies.
The results are presented in Table 2.
It can be seen that models trained with
DITTO can reduce repetitions compared to the MLE baseline and achieve the closest results to humanin repetition metrics.
For the quality of generated sequences, DITTO achieves the highest MAUVEof 0.96 with top- k or nucleus sampling.
[Table] [Table]
[Table]
Human Evaluation We conduct a pairwise crowdworkerevaluation to judge the quality of the generations of DITTOcompared to other baselines.
For each pair of methods, themodel generates texts based on the same 100 random prefixes from the test set of Wikitext-103.
Evaluators areasked to independently judge which generation is betterin terms of their grammaticality, relevance, coherence andinformativeness.
The evaluation interface and more detailsare in Appendix D. As shown in Table 3, DITTO consistently and significantly outperforms other competitorsacross different decoding strategies.
Self-reinforcement Effect As mentioned in Sec. 2, thesentence-level repetition has a self-reinforcement effect,which is the core issue of sentence-level repetition.
Wefurther study it when the model is trained with different methods.
The settings of experiments followthose in Sec.
As shown in Figure 4, we can find 1) TP, IP and WR increase almost monotonically in
UL and SG, which shows that the self-reinforcement effect has not been solved in these training-basedmethods; 2) When the model is trained with DITTO, these metrics drop rapidly as the number ofrepetitions grows, showing the effectiveness of DITTO in overcoming the self-reinforcement effect.
4.3 Results of Directed Generation
Setup We further conduct experiments on the directed abstractive summarization task
CNN/DailyMail [10, 20]: given an input document, the model generates several sentences as summarization.
We adopt the state-of-the-art model BART-large [15] as our baseline, which is large-scaleencoder-decoder Transformer architecture trained on 160Gb data.
For DITTO training, given adocument and its summarization, we construct the pseudo data by first random sampling a sentencefrom the summarization and then repeating it until reaching the maximum summarization length
[Figure]
[Table]
of the decoder model
while leaving the document unchanged.
We follow the official implementations [15] to train the BART-large model on the CNN/DailyMail and then fine-tune the model withthe best validation perplexity using DITTO training.
The λ is set as 0.9.
During inference, tri-gramblocking [29, 20] and beam search (beam size = 5) are used as in Lewis et al .
Results We evaluate the performance of the model with the standard F1-based ROUGE [28]scores (ROUGE-1, ROUGE-2, ROUGE-L).
As shown in Figure 4, our training method consistentlyoutperforms UL and SG with a large margin on ROUGE scores and outperforms other competitivebaselines.
The results also show that DITTO is compatible with the n-gram blocking, a decodingbased method for mitigating repetitions, demonstrating the generality of our approach.
4.4 Analyses
Auto-completion with different decoding lengths From a practice point of view, we analyze
DITTO in different decoding lengths.
As shown in Figure 5(a) and (b), the models trained with
DITTO consistently and significantly outperform those trained with MLE with the constraints ofdifferent decoding lengths, which shows the effectiveness and robustness of our method.
Hyper-parameter Study Towards better usage and understanding of DITTO, we study the twohyper-parameters: 1) mix ratio of pseudo data and actual data, denoted as γ , and 2) penalizationfactor λ .
As is shown in Figure 5(c), models achieve the best performance with γ = 0 .
5 on bothdatasets.
Thus, equally mixing the DITTO update and the MLE training loss update is recommendedin practice.
As for λ , we can observe that the optimal value varies in different tasks.
In open-endedgeneration task Wikitext-103, a stronger penalization with λ = 0 .
5 is preferred.
However, a mildpenalization with λ = 0 .
9 is better in the summarization task.
We conjecture that the repetition issueis more severe in generation tasks with more freedom, such as the open-ended generation.
Related Work
Repetition in Neural Text Generation.
Repetition has been a key problem in neural text generationfor various tasks including open-ended generation (e.g., text continuation [11, 32, 17]) and directgeneration (e.g., summarization [29, 18]).
Previous work [11, 32, 16, 13] observes that, with
[Figure]
maximization-based decoding strategies such as greedy search, language models prefer to generatebland and consecutive repetitions at the word-level, phrase-level, and sentence-level.
Consecutiveword-level and phrase-level repetition are lexical repetitions [9] where the sentence structure isincorrect, while sentence-level repetition is semantic repetition that indicates the expression is notinformative and has a discrepancy with human language [9].
Recently, large-scale pre-training with
Transformer architecture [31] such as GPT-2 [26] and BART [15], have greatly improved the abilityof language modeling and can generate fluent sentences similar to human language.
However, thegenerations still have a large number of unexpected consecutive sentence-level repetitions [26, 4, 8]in practice, which remains a severe problem in neural text generation.
Approaches for Mitigating Repetition.
Approaches for mitigating repetition in neural text generation can be categorized into training-based [32, 17] and decoding-based [29, 6, 11] approaches.
The representative training-based approaches are unlikelihood training (UL) [32] and straight togradient (SG) [17].
UL reduces the probability of negative candidate tokens, and SG improves theprobability of tokens that do not belong to negative tokens, where the negative tokens are a set oftokens that appear in the previous context.
However, our experiments in Sec.
4.2 show that theycannot solve self-reinforcement issue.
In contrast, our work analyzes why models prefer to repeatthemselves and quickly get stuck into the sentence-level loop, and mitigate the self-reinforcementissue.
Many decoding-based methods have been proposed to rectify these issues in various tasks.
In summarization, n -gram blocking [29, 20, 15] is often used to block repetitive n -grams fromsubsequent generations.
In open-ended tasks, top- k [6] and nucleus sampling [11] truncate unreliabletail and employ sampling according to token distribution for generating fluent and informative texts.
Analyses of Repetition Problem.
Although previous work has noticed the repetition issue andmany approaches have been proposed to mitigate it at the model training or decoding stages, little hasbeen discussed about the causes of the repetition.
Fu et al .
[8] theoretically analyzes the problemby assuming the language models can be approximated to first-order Markov models.
However, thecases of positive feedback loop of repetitions presented by Holtzman et al .
[11] indicate the repetitionprobability has complex relationships with a quite long-range context and language models maynot be simplified as first-order Markov models.
However, there is a lack of detailed numeric andstatistical results across various sentences and analyses in their work.
In contrast, we further dig intothe problem and conduct quantitative experiments to analyze underlying issues of repetition.
Conclusion and Future Work
We quantitatively investigate why language models with maximization-based decoding prefer consecutive sentence-level repetitions.
Our observations and analyses show that the issue of the repetition isthe tendency to repeat previous sentences and the self-reinforcement effect.
Guided by our analyses,we propose a simple and effective method named DITTO by constructing pseudo data and teachingmodel to learn to gradually decay the repetition probability as the number of sentence repetitionsgrows.
The experiments on the Wikitext-103 open-ended generation task and the CNN/DailyMailsummarization task demonstrate the superiority of our methods.
This work focuses on the sentence structure repetition issue.
However, how to overcome semanticrepetition while not hurting the modeling ability of the language model is still under exploration,which we leave for future work.
We hope our quantitative analyses and approaches to mitigatesentence-level repetitions can help the NLP community better understand, resolve the repetition issueand improve the fundamental language modeling ability of neural language models.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 6.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See Section 4.1.
The code is attached in supplemental materials.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.1.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See reported results in Table 1,2 and
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.1.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 4.1.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[Yes] See Appendix D.
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]