Paraphrase Generation via Adversarial Penalizations (2020)

Abstract: Paraphrase generation is an important problem in Natural Language Processing that has been addressed with neural network-based approaches recently. This paper presents an adversarial framework to address the paraphrase generation problem in English. Unlike previous methods, we employ the discriminator output as penalization instead of using policy gradients, and we propose a global discriminator to avoid the Monte-Carlo search. In addition, this work use and compare different settings of input representation. We compare our methods to some baselines in the Quora question pairs dataset. The results show that our framework is competitive against the previous benchmarks.

## introduction paraphrase generation is a task in nlp which aims to transform a given sentence in another with the same meaning. this task is challenging because of the complexity of semantic and syntactic relationships in language. moreover, the capacity to generate paraphrases automatically is an opportunity to use data augmentation in nlp. however, one of the main problems of paraphrase generation is that the meaning of a sentence can be changed radically by modifying a word. paraphrase generation is a hot task and recent neural-approaches have addressed the problem. we organize previous works of paraphrase generation into two groups: task-support and task-based. the task-support paraphrase generation works create paraphrases to add training data or adversarialexamples. paraphrases have been created to augment data in question answering @xcite . some techniques are substitution of words @xcite , make syntactic changes to the original sentences @xcite @xcite , and back-translation @xcite . other works apply rule-based generative models to perform multiple changes @xcite . on the other hand, task-based paraphrase generation works aim to benchmark their results on specific paired datasets. in this paper, we focus on taskbased works. @xcite use a stacked residual long-short term memory (lstm) network that outperforms a vanilla lstm. @xcite apply a variational autoencoder (vae) to get better results than the stacked lstm. @xcite use a seq2seq-based model with a dictionary-based attention mechanism. the dictionary search guides the insertion or deletion of a word. @xcite generate paraphrases using a deep reinforcement learning framework. @xcite propose an attention network using word embeddings information. @xcite present an adversarial setup over latent space using a conditional vae as a generator. @xcite propose a vae to make syntactically and semantically changed paraphrases. @xcite propose a transformer @xcite with multiple encoders to process extra semantic information of the input. the work of @xcite shows a hybrid model between a transformer and a recurrent neural network (rnn). @xcite design a transformer-based model that can generate paraphrases at different levels of granularity. from the prior works on paraphrase generation, most of them learn by conditional maximum likelihood estimation (mle). however, @xcite highlight the exposure bias problem @xcite in paraphrase generation. some works address the problem by applying reinforce @xcite to generate text in their adversarial setups @xcite @xcite @xcite . however, simi-lar to prior works @xcite , we observe that reinforce has a high variance and is difficult to tune. at the same time, pre-trained language models (lms) @xcite @xcite have outperformed previous works in many nlp tasks. an important reason is that they provide contextual representations of words, which are more specific than static word embeddings @xcite @xcite . however, static word embeddings consume fewer resources and are faster than pretrained lms. @xcite shows that the static embeddings extracted from pre-trained lms outperform glove @xcite and fasttext @xcite in many word vector benchmarks. in this paper, we propose an adversarial model (generator-discriminator) to address the english paraphrase generation task. unlike previous approaches, we train our model using a weighted conditional maximum likelihood by a "penalization" score given by the discriminator. also, we test variations of our setup by changing the input representations and the monte-carlo search. overall, our contributions are as follows. ## preliminaries before presenting our model, we provide some preliminaries about the mle training and rein-force algorithm in sequential problems. ## methodology our adversarial setup is composed of two neural networks like previous works: a generator and a discriminator. we make some observations of prior approaches to propose our model. on the one hand, mle could suffer exposure bias when switching from train to inference. on the other hand, we observe that the reinforce algorithm is very fluctuating when training because it relies only on the discriminator. furthermore, it is slow to train due to the monte-carlo search. we propose a weighted conditional maximum likelihood objective to train our generator. in consequence, we take advantage of the conditional mle principle and, also, we can guide the training using a penalization function. the penalization function is similar to the reward in reinforce. the conditional mle part let us reduce the variance of training. in addition, we substitute the monte-carlo search by using a discriminator that outputs a score for each token. ## conclusions we propose an adversarial setup to address the paraphrase generation task. the automatic evaluation results show some improvements over previous baselines. the human evaluation suggests a tradeoff between the fluency and diversity of the generated paraphrases of the fine-tuned model. we conclude that our setup is a suitable option to re-inforce and mle training. in addition, the case study shows that our method helps to improve the quality of paraphrases in general.