Relating Neural Text Degeneration to Exposure Bias (2021)

Abstract: This work focuses on relating two mysteries in neural-based text generation: exposure bias, and text degeneration. Despite the long time since exposure bias was mentioned and the numerous studies for its remedy, to our knowledge, its impact on text generation has not yet been verified. Text degeneration is a problem that the widely-used pre-trained language model GPT-2 was recently found to suffer from (Holtzman et al., 2020). Motivated by the unknown causation of the text degeneration, in this paper we attempt to relate these two mysteries. Specifically, we first qualitatively and quantitatively identify mistakes made before text degeneration occurs. Then we investigate the significance of the mistakes by inspecting the hidden states in GPT-2. Our results show that text degeneration is likely to be partly caused by exposure bias. We also study the self-reinforcing mechanism of text degeneration, explaining why the mistakes amplify. In sum, our study provides a more concrete foundation for further investigation on exposure bias and text degeneration problems.

## introduction one mythology in neural text generation is exposure bias @xcite @xcite . in the context of text generation, exposure bias refers to mistakes made by the model at the beginning of text generation, which may amplify, and lead the model to a state unseen in training time, and may thus cause misbehavior in the following generation. phenomena related to exposure bias were first observed in @xcite in the self-driving vehicles field. after that, exposure bias was mainly discussed in the context of imitation learning @xcite @xcite . in 2015, @xcite introduced it in the context of neural text generation. however, its impact on text generation is questionable from both the empirical and the theoretical perspectives. empirically, despite the number of studies for its remedy @xcite @xcite @xcite @xcite , phenomena resulted from exposure bias have not yet been explicitly identified. on the other hand, theories attained in the context of imitation learning may not be applicable to the above text generation tasks. for example, @xcite shows a o(t 2 ) trend of cost with respect to the number of steps t in an episode. it implies that the cost grows quadratically when t is large. however, most of natural language process tasks, e.g. machine translation and image captioning, do not generate very long text. the impact of exposure bias is thus not clear for text generation tasks. a younger mystery is the recently discussed enigma of text degeneration @xcite . it refers to the phenomenon in which bland or strange repetitive texts may be generated when the likelihood is the objective of generation, for example, when some commonly used strategies, such as greedy decoding and beam-search decoding, are used. especially, the prior work @xcite observed such problems in gpt-2 @xcite , a pre-trained language model that has been shown useful in many nlp tasks @xcite @xcite @xcite . despite many attempts proposed to address this issue @xcite @xcite , its root cause remains unknown. motivated by the unknown issues, we wonder whether text degeneration can be connected to the well-known exposure bias. if text degeneration is the misbehavior caused by exposure bias, it actually provides us a perfect opportunity to identify the existence of exposure bias. one of misbehavior of text degeneration is the occurrence of repetitive loops. it is a phenomenon that a model tends to graphql is an interesting technology originating at facebook. it is a query language that allows you to query a database and then query the database for the results.\n \n the query language is called queryql. it is a query language ... we first saw anki overdrive, the company's follow-up to the original game, in the early 2000s. it was a game that was a bit of a hit, and it was a game that was a bit of a hit that was a bit of a hit that was a hit that ... to the best of our knowledge, this paper is the first work that attempts to relate text degeneration to exposure bias. we first conclude two necessary conditions of its occurrence based on the intuition of exposure bias in literature in section 3.4. we then inspect the two necessary conditions qualitatively and quantitatively. in section 4.1, we find that before text repeating starts, gpt-2 generates unnatural text. in section 4.3, we show that the hidden states of gpt-2 deviate to an area less similar to the states generated by encoding real text. the above observations satisfy the intuition of exposure bias that mistakes are made in the early stage and are amplified afterward. according to the indications we discover, we conclude that exposure bias is likely to co-occur with repetitive loops. finally, we investigate how the mistakes are amplified after repetitive loops occur in section 5. we discover the self-reinforcing mechanism of text degeneration. the results provide a possible outline of how a model is trapped in repetitive loops. these findings should be helpful for future studies on exposure bias and remedies for text degeneration. ## background and notations to better elaborate the investigation of the above problems, background knowledge and notations are briefly introduced here. ## mechanisms after a repetitive loop starts while the above experiments show the indications of exposure bias, in this section we further investigate how the early stage mistakes cause the model to degenerate. figure 4 indicates some mistakes are made prior to time step ρ + λ. thus, in this section, we investigate the characteristics of the sequence generated prior to ρ + λ, the looping sequence ŷρ • • • ŷρ+λ (as defined in section 3.3). ## conclusion in conclusion, we provide a deeper insight into the relation between exposure bias and text degeneration. we qualitatively and quantitatively show that mistakes are indeed made in the early stage of generation. in particular, some significant mistakes are made prior to ρ + λ, the time step when the model starts repeating. we then show why the model fails to recover from the mistakes. the looping sequence, which is the sequence generated prior to ρ + λ, and repeated sequences are loopinginducing. that is how the model fails to recover from the mistakes, and how the mistakes amplify. our contributions are four-fold: 1) we explicitly formulate the necessary indications for the detection of exposure bias. 2) for each condition, we design the associated experiments for validation. 3) by the experiments, we show that text degeneration is likely to be partly caused by exposure bias. 4) finally, we provide a possible explanation how gpt-2 fails to recover from the mistake. our formulation and the conducted experiments build a solid foundation for future study on exposure bias.