Adversarial Text Generation via Sequence Contrast Discrimination (2020)

Abstract: In this paper, we propose a sequence contrast loss driven text generation framework, which learns the difference between real texts and generated texts and uses that difference. Specifically, our discriminator contains a discriminative sequence generator instead of a binary classifier, and measures the ‘relative realism’ of generated texts against real texts by making use of them simultaneously. Moreover, our generator uses discriminative sequences to directly improve itself, which not only replaces the gradient propagation process from the discriminator to the generator, but also avoids the time-consuming sampling process of estimating rewards in some previous methods. We conduct extensive experiments with various metrics, substantiating that our framework brings improvements in terms of training stability and the quality of generated texts.

## introduction generating human-like texts has always been a fundamental problem in the natural language processing field, which is essential to many applications such as machine translation @xcite , image captioning @xcite , and dialogue systems @xcite . currently, the dominant approaches are auto-regressive models, such as recurrent neural network @xcite , transformer @xcite , and convolutional seq2seq @xcite , which have achieved impressive performances for the task of language generation using the maximum likelihood estimation (mle) method. nevertheless, some studies reveal that such settings may have three main drawbacks: first, the mle method makes the generative model extremely sensitive to rare samples, which results in the learned distribution being too conservative @xcite . second, autoregressive generation models suffer from exposure bias @xcite due to the dependence on the previous sampled output during the inferring phase. third, they only consider the word-level objective and may fail to guarantee some sentencelevel goals, such as realism, preserving semantic consistencies, long-range semantic structure, and so on @xcite . recently, lots of recent studies @xcite @xcite @xcite @xcite @xcite try to apply generative adversarial networks (gan) @xcite ) in text generation, which uses discriminator networks as loss functions to ensure these higher-level objectives. however, the discreteness of texts makes it difficult for the gradient to pass from the discriminator to the generator. the current solution is mainly based on reinforcement learning @xcite or differentiable sampling functions @xcite . in addition, considering the complexity of the language, the generator is easily much weaker than the discriminator in practice, making it difficult to obtain a clear optimization direction from the discriminator and learn from scratch. in this paper, by borrowing techniques from contrastive learning @xcite @xcite , we propose a sequence contrast loss driven adversarial learning framework for text generation, slgan. in our framework, the discriminator d is not just a simple binary classifier, but a siamese network composed of a sequence generator g d , which can provide sequences with discriminative information. in other words, our discriminator d measures the gap between the generated texts and the real texts, rather than simply predicting the probability of the generated data (by generator g) being real. specifically, these discriminative sequences with wellformed textual structure information can be used to we conduct experiments on both synthetic and real datasets, and use various metrics (i.e., fluency, novelty, generalization, diversity, human evaluation, and learning curve) to show that our approach not only produce more realistic samples but also greatly stabilize the adversarial training process. ## method the architecture of our proposed model is depicted in figure 1 . the whole framework can be divided into two adversarial learning objectives: generator learning and discriminator learning. the goal of the discriminator d is to learn the difference ('relative realism') between fake texts (y, texts generated by generators) and real texts (x). while the goal of the generator g is to use this difference (discriminative sequences) to generate more realistic texts, which contains a word-level item (l mle ) and a sentencelevel item ( ladv ). to achieve the above goals, we start with two things. one is that discriminator d observes and uses both 'fake' and 'real' data at the same time, rather than considering them in an alternating fashion. the other is that the inside of the discriminator is not a binary classifier, but a sequence generator g d . g d aims to generate discriminative sequence ŷ, which can be considered as a sequence representation to be used for better measurement of 'relative realism'. to some extent, g d can be seen as an 'amplifier', and the closer the input text is to real texts, the less it changes. further, ŷ not only can be used to measure the 'relative realism' of generated texts against real texts, but also can be used to directly affect g through sequence contrast loss. therefore, by calculating the contrastive loss, the gradient back-propagation process from the discriminator to the generator is avoided, which is of significant importance in adversarial learning. discriminator learning: the contrastive loss of our discriminator takes the output of the discriminative sequence generator g d for a positive example (real texts x), and calculates its similarity to an example of the same class (x) and contrasts that with the distance to negative examples (y, texts generated by generators): we use the kl-divergence to measure how similar two word distributions of generated sequences are to each other, and the inter-class loss sim d is: in addition, we consider making ŷ meaningful, with the purpose that it can be used not only to discriminate but also to represent 'realism' features. we hence rewrite the intra-class loss sim s with a similar idea as: in practice, we add noise to x by randomly replacing input word with the noise word (< @xmath0. generator learning: the loss function of our generator includes two terms: one term (l mle ) is to concern word-level fitness and another term ( ladv ) is to ensure a higher level of 'realism' resembling qualities: furthermore, the other goal of generator g is to minimize sim d in eq 2, with the intuition that using a discriminator network to learn the loss function of sentence-level properties (e.g., long-range semantic structure, preserving semantic consistencies, etc.) over time, rather than explicitly formulating these properties. according to the discriminator's loss (eq 1), the closer as such, we resort to an approximation approach to define the generator's adversarial loss as: in this way, we can directly guide the generation of g by measuring the sequence contrast loss of the output between g and g d , which not only avoids the gradient back-propagation process from the discriminator to the generator, but also makes the generator use the discriminator's discriminative information more effectively. ## conclusion and future work in this study, we propose a sequence contrast loss for adversarial text generation, where the discriminator outputs discriminative sequences rather than binary classification probabilities. extensive experimental results demonstrate that our model brings improvements in training stability and the quality of generated texts. in future work, we will expand our method to have specific targets, to benefit more conditional text generation tasks (e.g., sentimental text generation, dialogue response generation). textgan a man riding a motorcycle . (very short) is to a bathroom with a sink . (unreadable) a man is on a motorcycle . a white toilet a sink . with a sink and a table . a motorcycle in a blue sky . a bathroom with a sink . a man is sitting on a motorcycle . (repeated) a bathroom with a sink . slgan a group of people sat in front of the house together . several people stood in front of the bicycle . a person is holding a monitor range in the kitchen . a woman is riding a motorcycle on the street . three adults sat in his car with hats . two people in a public parking lot . white bathtub , toilet and basin under the bathroom wall . an old brick building with a wooden manufacturer next to it . a motor scooter parked in the street with a crowd waiting for a parade.