D iv GAN : Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network (2020)

Abstract: Paraphrases refer to texts that convey the same meaning with different expression forms. Traditional seq2seq-based models on paraphrase generation mainly focus on the fidelity while ignoring the diversity of outputs. In this paper, we propose a deep generative model to generate diverse paraphrases. We build our model based on the conditional generative adversarial network, and propose to incorporate a simple yet effective diversity loss term into the model in order to improve the diversity of outputs. The proposed diversity loss maximizes the ratio of pairwise distance between the generated texts and their corresponding latent codes, forcing the generator to focus more on the latent codes and produce diverse samples. Experimental results on benchmarks of paraphrase generation show that our proposed model can generate more diverse paraphrases compared with baselines.

## introduction the task of paraphrase generation refers to rewriting a given sentence to a new paraphrase sentence, which requires that the generated sentence and input sentence are different in expression form, but have the same expressed meaning. paraphrase generation is a fundamental task of natural language processing (nlp). the technique of paraphrase generation has been widely used in many downstream applications, such as information retrieval, question answering, machine translation, and so on. early works on paraphrase generation mainly focus on rule-based @xcite , grammar-based @xcite , lexicon-based @xcite , and statistical machine translation (smt)-based methods @xcite . recently, with the release of large-scale paraphrase datasets, sequence-to-sequence (seq2seq) models @xcite @xcite @xcite @xcite ## methods given an input sentence @xmath0, x 2 , • • • , x n }, we seek to generate a set of k paraphrase sentences @xmath1, y (2) , • • • , y (k) }, that all @xmath2, but are different in expression form. ## experiments 4.1 dataset there are many datasets for paraphrase generation. we choose the two most widely used datasets, quora 1 and mscoco @xcite for experiments. quora quora dataset consists of over 400k candidate question paraphrase pairs, and each pair has a manually annotated label. the two questions are paraphrasing each other only when the question pair is annotated as 1. this dataset contains 155k paraphrase question pairs in total. mscoco mscoco is a benchmark for the task of image captioning. this dataset contains over 82k training and 42k validation images, and each image contains at most five human-labeled captions. similar to previous work on paraphrase generation, we consider different captions of the same image as paraphrases. following previous work, we reduce the sentences to the size of 15 words. ## case study first, using traditional beam search can produce different outputs, but the generated texts are of high similarity with minor modification (for example, replacing "can you" with "do you", or replacing "while awake" with "while you are awake"). secondly, using the sampling strategy during decoding sometimes produces unnatural output, especially at the beginning or end of the sentence (see the second and third sentences in ## conclusions in this paper, we propose a conditional generative adversarial network based model to tackle the task of diverse paraphrase generation. to solve the problem of the minor impacts of the latent codes and the mode collapse in the conditional gan, we propose to add a diversity loss term to the objective. the diversity loss term encourages the generator to explore more in the latent space and generate samples from some minor modes. experimental results demonstrate the effectiveness of the proposed diversity loss term. in the future, we will apply the diversity loss to more tasks and models.