P ara AMR : A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation (2023)

Abstract: Paraphrase generation is a long-standing task in natural language processing (NLP). Supervised paraphrase generation models, which rely on human-annotated paraphrase pairs, are cost-inefficient and hard to scale up. On the other hand, automatically annotated paraphrase pairs (e.g., by machine back-translation), usually suffer from the lack of syntactic diversity – the generated paraphrase sentences are very similar to the source sentences in terms of syntax. In this work, we present ParaAMR, a large-scale syntactically diverse paraphrase dataset created by abstract meaning representation back-translation. Our quantitative analysis, qualitative examples, and human evaluation demonstrate that the paraphrases of ParaAMR are syntactically more diverse compared to existing large-scale paraphrase datasets while preserving good semantic similarity. In addition, we show that ParaAMR can be used to improve on three NLP tasks: learning sentence embeddings, syntactically controlled paraphrase generation, and data augmentation for few-shot learning. Our results thus showcase the potential of ParaAMR for improving various NLP applications.

## introduction paraphrase generation is a long-standing task in natural language processing (nlp) @xcite @xcite . it has been applied to various downstream applications, such as question answering @xcite , chatbot engines @xcite , creative generation @xcite , and improving model robustness @xcite . most existing paraphrase generation models require a large amount of annotated paraphrase pairs @xcite @xcite . since human-labeled instances are expensive and hard to scale up @xcite @xcite , recent research has explored the possibility of generating paraphrase pairs automatically. one popular approach is back-translation @xcite , which generates paraphrases of a source sentence by translating it to another language and translating back to the original language. although backtranslation creates large-scale automatically annotated paraphrase pairs, the generated paraphrases usually suffer from the lack of syntactic diversity -they are very similar to the source sentences, especially in syntactic features. consequently, supervised paraphrase models trained with those datasets are also limited in their ability to generate syntactically diverse paraphrases. furthermore, not all words can be perfectly translated into another language. as we will show in section 4.3, this mismatch may produce subpar paraphrases. in this work, we leverage abstract meaning representation (amr) @xcite to generate syntactically diverse paraphrase pairs. we present paraamr, a large-scale syntactically diverse paraphrase dataset based on amr backtranslation. as illustrated by figure 1 , our approach works by encoding a source sentence to an amr graph, modifying the focus of the amr graph that represents the main assertion, linearizing the modified amr graph, and finally decoding the linearized graph back to a sentence. since the new sentence shares the same amr graph structure as the source sentence, it preserves similar semantics to the source sentence. at the same time, the change of focus makes the new main assertion different from that source sentence. when linearizing the amr graph, a different concept will be emphasized at the beginning of the string. therefore, the decoded sentence may have a much different syntax from the source sentence. our quantitative analysis (section 4.2) and qualitative examples (section 4.3) show that the paraphrases of paraamr are syntactically more diverse than existing datasets @xcite ), while at the same time preserving good semantic similarity between paraphrased sentences. in addition, our human evaluation results (section 4.4) confirm that paraamr is indeed more syntactically diverse than prior datasets. to showcase the benefits of syntactically diverse paraphrases, we conduct experiments on three downstream tasks: learning sentence embeddings (section 5.1), syntactically controlled paraphrase generation (section 5.2), and data augmentation for few-shot learning (section 5.3). we observe that models trained on paraamr achieve better performance on all three downstream tasks compared to other datasets, thus indicating its potential value for various nlp applications. foot_0 2 related work paraphrase generation and datasets. traditional paraphrase generation models are usually based on hand-crafted rules, including rule-based methods @xcite , thesaurus-based methods @xcite , and lattice matching methods @xcite . in recent years, different neural models have been proposed for paraphrase generation @xcite @xcite @xcite @xcite @xcite . some advanced techniques are proposed as well, such as multi-round generation @xcite , reinforcementlearning-based paraphrasing @xcite , and prompt-tuning @xcite . to properly train those neural models, however, we needs a large corpus of annotated paraphrase pairs. most existing paraphrase datasets and related resources, such as mrpc @xcite , pan @xcite , ppdb @xcite , and quora @xcite , have limited scale. therefore, researchers have focused on automatically generating large-scale paraphrase corpora. one notable example is paranmt @xcite , which is created by machine backtranslation -translating texts to another language and translating them back to the original language. ## paraamr we propose paraamr, a large-scale syntactically diverse paraphrase dataset. figure 1 illustrates the overall framework to construct paraamr by amr back-translation. in summary, we encode a source sentence to an amr graph, modify the focus of the amr graph (see section 3.3), linearize the modified amr graph, and finally decode the linearized graph to a syntactically diverse paraphrase. we describe the details in the following. ## comparison to prior datasets we compare paraamr with the following three datasets. (1) paranmt @xcite create paraphrase pairs by english-czech-english back-translation. (2) parabank1 @xcite adds lexical constraints during the decoding of back-translation to increase the lexical diversity of generated paraphrases. (3) para-bank2 @xcite proposes cluster-based constrained sampling to improve the syntactic diversity of generated paraphrases. ## applications we focus on three downstream applications of paraamr corpus: learning sentence embeddings (section 5.1), syntactically controlled paraphrase generation (section 5.2), and data augmentation for few-shot learning (section 5.3). we demonstrate the strength of paraamr and compare with prior datasets: paranmt @xcite , parabank1 @xcite , and parabank2 @xcite . ## conclusion in this work, we present paraamr, a large-scale syntactically diverse paraphrase dataset created by amr back-translation. our quantitative analysis, qualitative examples, and human evaluation demonstrate that the paraphrases of paraamr are more syntactically diverse than prior datasets while preserving semantic similarity. in addition, we conduct experiments on three downstream tasks, including learning sentence embeddings, syntactically controlled paraphrase generation, and data augmentation for few-shot learning, to demonstrate the advantage of syntactically diverse paraphrases.