Learning to Selectively Learn for Weakly-supervised Paraphrase Generation (2021)

Abstract: Paraphrase generation is a longstanding NLP task that has diverse applications on downstream NLP tasks. However, the effectiveness of existing efforts predominantly relies on large amounts of golden labeled data. Though unsupervised endeavors have been proposed to alleviate this issue, they may fail to generate meaningful paraphrases due to the lack of supervision signals. In this work, we go beyond the existing paradigms and propose a novel approach to generate high-quality paraphrases with data of weak supervision. Specifically, we tackle the weakly-supervised paraphrase generation problem by: (1) obtaining abundant weakly-labeled parallel sentences via retrieval-based pseudo paraphrase expansion; and (2) developing a meta-learning framework to progressively select valuable samples for fine-tuning a pre-trained language model BART on the sentential paraphrasing task. We demonstrate that our approach achieves significant improvements over existing unsupervised approaches, and is even comparable in performance with supervised state-of-the-arts.

## introduction paraphrase generation is a fundamental nlp task that restates text input in a different surface form while preserving its semantic meaning. it serves as a cornerstone in a wide spectrum of nlp applications, such as question answering @xcite , machine translation @xcite , and semantic parsing @xcite . with the recent advances of neural sequence-tosequence (seq2seq) architecture in the field of language generation, a growing amount of literature has also applied seq2seq models to the sentential paraphrasing task. despite their promising results, collecting large amounts of parallel paraphrases is often timeconsuming and requires intensive domain knowl- * work was done as an intern at amazon alexa ai. as an understudied problem, weakly-supervised paraphrase generation is challenging mainly because of the following reasons: (i) although weak supervision has been applied in different lowresource nlp tasks @xcite , for paraphrase generation, it is unclear how to automatically acquire abundant weak supervision data that contains coherent, fluent and diverse paraphrases; (ii) weakly-labeled paraphrases tend to be noisy and are not equally informative for building the generation model @xcite @xcite . hence, selecting valuable parallel sentences from weakly-labeled data is vital for solving the studied problem; and (iii) the state-of-the-art paraphrasing methods are predominantly built upon traditional seq2seq models, while the necessity of learning from scratch largely magnifies the learning difficulty when dealing with scarce or noisy training data @xcite . thus it is imperative to seek a more robust and knowledge-intensive backbone for learning with weakly-labeled paraphrases. to address the aforementioned challenges, we present a novel approach for learning an effective paraphrasing model from weakly-supervised parallel data. by virtue of a simple yet effective pseudo paraphrase expansion module, for each input sentence, we are able to obtain multiple similar sentences without unbearable labeling cost and treat them as paraphrases. to mitigate the inaccurate supervision signals within the weakly-labeled parallel data and build an effective paraphrasing model, we further select valuable parallel instances by proposing a novel framework named learning-to-selectively-learn (ltsl). remarkably, ltsl leverages meta-learning to progressively exert the power of pre-trained language model, i.e., bert @xcite and bart @xcite , with weakly-labeled paraphrasing data. from a meta-learning perspective, the bertbased gold data selector is meta-learned to select valuable samples from each batch of weakly paired sentences, in order to fine-tune and maximize the performance of the bart-grounded paraphrase generator. afterwards, the paraphrase generation performance change on a small validation set will be used to perform meta-optimization on the data selection meta-policy. this way the two pre-trained components gold data selector and paraphrase generator in ltsl are able to reinforce each other by continuously learning on a pool of meta-selection tasks. to summarize, the major contribution of this work is three-fold: unsupervised paraphrase generation. due to the burdensome labeling cost of supervised counterparts, unsupervised paraphrasing methods have drawn increasing research attention in the community. methods based on variational autoencoders (vae) are first proposed to generate paraphrases by sampling sentences from the learned latent space @xcite @xcite , while the generated sentences are commonly less controllable. to tackle this issue, cgmh @xcite uses metropolis-hastings sampling to add constraints on the decoder at inference time. furthermore, researchers try to improve the generation performance in terms of semantic similarity, expression diversity, and language fluency by using simulated annealing @xcite , syntactic control @xcite , or dynamic blocking @xcite . in addition, pre-trained translation models have been explored to generate paraphrases via backtranslation @xcite . but still, those methods can hardly achieve comparable results with supervised approaches. learning with weak supervision. the profound success of machine learning systems largely benefits from abundant labeled data, however, their performance has been shown to degrade noticeably in the presence of inaccurate supervision signals @xcite , especially in an adversary environment @xcite . as one of the central problems in weak supervision, learning with noisy labels has received much research attention. existing directions mainly focus on: estimating the noise transition matrix @xcite , designing robust loss functions or using regularizations @xcite @xcite , correcting noisy labels @xcite and selecting or reweight- ## proposed approach figure 1 illustrates our method for solving weaklysupervised paraphrase generation. in essence, there are two sub-tasks: (1) how to obtain abundant weakly-labeled parallel data from the unlabeled corpus; and (2) how to build a powerful paraphrase generation model from noisy weak supervision data. formally, given a set of source sentences @xmath0, we first obtain a weakly-labeled parallel corpus d @xmath1, y i )} m @xmath2. in this work, we aim to denoise the weak supervision data by selecting a subset of valuable instances ## experiments for the evaluation, we briefly introduce the experimental settings and conduct extensive experiments to corroborate the effectiveness of our approach. the details of our experimental settings and implementations can be found in the appendix. ## conclusion in this work we investigate the problem of paraphrase generation under the low-resource setting and propose a weakly-supervised approach. from automatic and human evaluations, we demonstrate that our approach achieves the state-of-the-art results on benchmark datasets. an interesting direction is to improve the generation performance by leveraging weakly-labeled data from different sources. we leave this as future work.