Unsupervised Paraphrasing with Pretrained Language Models (2021)

Abstract: Paraphrase generation has benefited extensively from recent progress in the designing of training objectives and model architectures. However, previous explorations have largely focused on supervised methods, which require a large amount of labeled data that is costly to collect. To address this drawback, we adopt a transfer learning approach and propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking (DB). To enforce a surface form dissimilar from the input, whenever the language model emits a token contained in the source sequence, DB prevents the model from outputting the subsequent source token for the next generation step. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair (QQP) and the ParaNMT datasets and is robust to domain shift between the two datasets of distinct distributions. We also demonstrate that our model transfers to paraphrasing in other languages without any additional finetuning.

## introduction paraphrase generation restates text input in a different surface form while preserving its semantics. it has various applications on downstream nlp tasks including text summarization @xcite , semantic parsing @xcite , as well as diversifying text generation for user-facing systems such as chatbots. to evaluate model robustness, a paraphraser can be used to generate adversarial examples, which also serve as augmented data to train the target neural networks @xcite . besides, paraphrasing queries makes question answering systems more likely to match with keywords in a knowledge base @xcite . we first train a task-adapted model with a denoising objective so that it is able to reconstruct input text. we then use dynamic blocking (db) to generate pseudopairs of paraphrasing data. finally, the generated data is used to train the self-supervised model. however, it is expensive to annotate paraphrases, resulting in only a few human-labeled datasets. the existing ones are either small-scale like mrpc @xcite , or of closed domains like qqp foot_0 which consists entirely of questions. consequently, previous work explored automatically (hence noisily) annotated datasets such as @xcite , twitter url paraphrase corpus @xcite , paranmt @xcite , and para-bank @xcite , or re-purposed datasets including mscoco @xcite and wikianswers @xcite . the scarcity of highquality datasets motivates us to consider unsupervised alternatives. in this work, we explore a transfer learning approach, which leverages unsupervised large-scale pretrained models like t5 @xcite and bart @xcite . the effectiveness of bert-score @xcite in identifying text similarity hints that pretrained language models are equipped with extensive knowledge in paraphrasing. this knowledge may be attributed to the fact that text spans shar-ing similar context usually stay semantically close together -word embedding @xcite being a classic example. in other words, the paraphrasing capability of language models stems from the strong correlation between context and semantic similarity. in this work, we use pre-trained autoregressive lms to leverage such implicit knowledge for paraphrasing in an unsupervised setting. 2 for paraphrasing, decoder-only lms merely output a continuation of the input, while sequenceto-sequence models like bart tend to copy the input through even when paired with popular decoding algorithms such as greedy decoding, beam search or top-k/p sampling @xcite because the probabilities of the input tokens during generation are all peaked. to address this issue, we propose dynamic blocking (db), a decoding algorithm that effortlessly transforms pre-trained autoregressive language models into natural paraphrasers with the help of task-adaption and selfsupervision (figure 1 ). to obtain a surface form different from the input, whenever we emit a token that is present in the source sequence, this algorithm prevents the model from outputting its immediate successor for the next generation step. the algorithm is based on the intuition that during inference, although the top candidate at each generation step corresponds to a peaked probability, the rest of the distribution still contains rich linguistic knowledge suitable for paraphrasing. this is in similar spirit with using soft targets for model distillation @xcite . through automatic and human evaluations, we demonstrate that our approach outperforms previous models (including supervised, in-domain models and the ground-truth targets) on both qqp and paranmt datasets and incurs no performance loss under domain shifts (i.e., finetuned on qqp and evaluated on paranmt, and vice versa). for automatic evaluations, we propose a reference-independent automatic metric named bert-ibleu, which is a harmonic mean of bertscore and one minus self -bleu. we show that this new metric correlates significantly better with human evaluation than traditional metrics. on the qualitative side, we illustrate with concrete examples that our model generates paraphrases that exhibit diverse syntactic structures. finally, we observe that our model can generate paraphrases in other languages without any additional training. 2 we will release all codes. our contributions are: (1) a training pipeline that leads to a strong, unsupervised paraphrasing model; (2) a novel decoding algorithm that effectively diversifies paraphrase generation; (3) a new automatic metric that evaluates paraphrasing quality more accurately. ## model figure 1 shows the training pipeline of our paraphrasing model, which consists of three key components, namely task-adaptation, self-supervision and dynamic blocking. overall we decode the taskadapted model with dynamic blocking to generate self-supervision data, which is in turn used to train the final model. ## related work paraphrase generation has been a long-standing task that has several applications on downstream nlp tasks including text summarization @xcite , semantic parsing @xcite , and question answering @xcite . early works on paraphrase generation mostly rely on rule-based or statistical machine translation systems @xcite @xcite . supervised approaches neural sequence-tosequence (seq2seq) models have been used to address this task @xcite @xcite @xcite ; sometimes such models are also used to evaluate paraphrasing quality @xcite . round-trip translation between two languages (i.e., back-translation) with strong neural machine translation (nmt) models has also become a widely used approach for paraphrase generation @xcite . consequently, supervised models using datasets like paranmt obtain their performance mainly from sequence-level distillation @xcite , where the data comes from the underlying supervised translation models. there have been several previous works @xcite @xcite @xcite ) that make use of syntactic structures to produce more diverse paraphrases. more recently, @xcite employ distinct generators to produce diverse paraphrases. retrieval-augmented generation methods have also been investigated @xcite . however, most of these approaches require parallel data. unsupervised approaches unsupervised paraphrasing, on the other hand, is a rather less explored and more challenging problem in nlp. bowman et al. ( 2016 ) train a variational autoencoder (vae) to maximize the lower bounds for the reconstruction log-likelihood of the input sentence without requiring any parallel corpora. sampling from the trained vae's decoder leads to sentences that can practically be considered as paraphrases as the decoder aims to reconstruct the input sentence by its training objective. @xcite introduce a constrained sentence generation approach by using metropolis-hastings sampling, which allows for decoding with complicated discrete constraints such as the occurrence of multiple keywords, hence not requiring any parallel corpora. @xcite introduce a model that allows interpo-lation from continuous auto-encoders to vectorquantized auto-encoders. @xcite cast the paraphrasing as an optimization problem, where it searches the sentence space to find the optimal point for an objective function that takes semantic similarity, expression diversity, and language fluency into account. siddique et al. (2020) optimize a similar objective with deep reinforcement learning. transfer learning there have been few works leveraging pre-trained language models for paraphrasing, either in a supervised @xcite or an unsupervised @xcite setting. both works employ gpt-2 as their backbone generation model. similarly, we opt for more recent large-scale pre-trained models like bart and t5. ## conclusion we design an effective training pipeline that enables large-scale pre-trained models to generate high-quality paraphrases in an unsupervised setting through task-adaptation, self-supervision, and a novel decoding algorithm named dynamic blocking. we demonstrate with automatic and human evaluations that our model achieves state-of-theart results on benchmark datasets. we also show that our model generates paraphrases that exhibit syntactic diversity, as well as generalizes to other languages without any additional training. overall our work motivates a deeper investigation into self-supervised techniques for paraphrase generation as well as extensions such as context-aware paraphrasing, where the output conditions not only on the sentences to be paraphrased, but also on the context around them. we leave this as future work.