Counter-Contrastive Learning for Language GAN s (2021)

Abstract: Generative Adversarial Networks (GANs) have achieved great success in image synthesis, but have proven to be difficult to generate natural language. Challenges arise from the uninformative learning signals passed from the discriminator. In other words, the poor learning signals limit the learning capacity for generating languages with rich structures and semantics. In this paper, we propose to adopt the counter-contrastive learning (CCL) method to support the generator’s training in language GANs. In contrast to standard GANs that adopt a simple binary classifier to discriminate whether a sample is real or fake, we employ a counter-contrastive learning signal that advances the training of language synthesizers by (1) pulling the language representations of generated and real samples together and (2) pushing apart representations of real samples to compete with the discriminator and thus prevent the discriminator from being overtrained. We evaluate our method on both synthetic and real benchmarks and yield competitive performance compared to previous GANs for adversarial sequence generation.

## introduction unsupervised text generation has achieved great success in plenty of applications, from dialogue generation to machine translation @xcite . common approaches to language models are maximizing the log-likelihood of tokens of discrete sequences given historical observations. nevertheless, language models trained with maximum likelihood estimation (mle) can result in exposure bias issues @xcite , a distributional shift between input sequences during training and inference stages. generative adversarial networks (gans) hold the promise of training language models, as an alternative method to mle. gans learn to sample * corresponding author. previous innovations adopt various approaches to enhance the learning signals for generators, such as leaking information from the discriminator to the generator @xcite , directly matching the fake data distribution to that of real data @xcite , learning to rank samples out of a collection of curated samples @xcite , leveraging more powerful generator architectures to learning representations @xcite , etc. however, the problem of language gans' training is far from being fully solved. inspired by the recent success in contrastive learning approaches @xcite in learning effective representations, we propose a countercontrastive learning objective to aid the adversarial learning of sequence generators in language gans. conventional contrastive learning methods aim at pulling positive samples together and pushing away positive samples from negative ones. in contrast, we propose counter-contrastive learning (ccl) method that (1) pulls the generated samples and real samples together (to generate real-looking data) and (2) pushes away the real samples (to hinder the training of the discriminator). empirical results on both synthetic and real datasets demonstrate competitive results compared with previous language gans and prove the effectiveness of our method. ## language gans language gans have attracted extensive interest due to their ability to mitigate the exposure bias issue. the objective of language gans is to train a language generator g(z; θ (g) ) that can output real-looking text samples that resemble those in the training data p data (x). from the game theory metaphor, language gans consist of a generator and a discriminator playing a two-player minimax game. the generator network decodes the randomly initialized starting token z into the language sequence g(z; θ (g) ), where the training signal is provided by the discriminator network d(x; φ (d) ) that is trained to distinguish between the samples drawn from the real data distribution p data and those produced by the generator. in this paper, we adopt the relativistic discriminator loss (jolicoeur-martineau, 2018) as the training objective: there have been a large variety of language gans that resorted to reinforcement learning (rl) heuristics with monte carlo search to gather the update rewards from the discriminator. the instability of rl training can further plague the reward sparsity problem. existing work @xcite demonstrated that gumbel-softmax relaxation @xcite is effective in language gans, thus we use the gumbel-softmax reparameterization instead of policy gradient in our experiments. ## contrastive learning contrastive learning aims at learning informative representations by pulling together positive neighbors and pushing way non-neighbors @xcite . assuming a set of paired examples @xmath0, x + i )} n @xmath1, where (x i , x + i ) are positive pairs. let the h i and h + i denote the representations of x i and x + i , the contrastive learning training objective is: ## related work a variety of language gans integrated the rl paradigm into gans. seqgan @xcite firstly takes the text generation as a markov decision-making process and trains the language generator with the policy gradient algorithm. rankgan @xcite and sal @xcite enrich the restrictive signals by ranking constructed pairs. leakgan @xcite leaks the hidden states of the generator to promote the generator training. another line of previous work either approximates the categorical sampling or optimizes on continuous representations, such as gumbel-softmax gan @xcite , textgan @xcite , fmgan @xcite and relgan @xcite . our work aims to integrate the prevalent contrastive learning approach in supporting the generator training, which lies in the line of methods using comparative signals or ranking classifiers, such as rankgan and sal. from the perspective of feature matching, the counter-contrastive learning objective can be considered as a contrastive signal to draw together the fake and real sample representations. ## conclusion in this paper, we introduce a counter-contrastive learning objective to advance the training of language gans. it pulls the representation of generated and real samples together to promote the generator training, and pushes apart real sample pairs to depress the discriminator training as a competitor. our future work will include extending the counter-contrastive learning method to other text generation tasks such as machine translation and dialogue generation.