Unsupervised Syntactically Controlled Paraphrase Generation with A bstract M eaning R epresentations (2022)

Abstract: Syntactically controlled paraphrase generation has become an emerging research direction in recent years. Most existing approaches require annotated paraphrase pairs for training and are thus costly to extend to new domains. Unsupervised approaches, on the other hand, do not need paraphrase pairs but suffer from relatively poor performance in terms of syntactic control and quality of generated paraphrases. In this paper, we demonstrate that leveraging Abstract Meaning Representations (AMR) can greatly improve the performance of unsupervised syntactically controlled paraphrase generation.Our proposed model, AMR-enhanced Paraphrase Generator (AMRPG), separately encodes the AMR graph and the constituency parse of the input sentence into two disentangled semantic and syntactic embeddings. A decoder is then learned to reconstruct the input sentence from the semantic and syntactic embeddings. Our experiments show that AMRPG generates more accurate syntactically controlled paraphrases, both quantitatively and qualitatively, compared to the existing unsupervised approaches. We also demonstrate that the paraphrases generated by AMRPG can be used for data augmentation to improve the robustness of NLP models.

## introduction syntactically controlled paraphrase generation approaches aim to control the format of generated paraphrases by taking into account additional parse specifications as the inputs, as illustrated by figure 1 . it has attracted increasing attention in recent years since it can diversify the generated paraphrases and benefit a wide range of nlp applications @xcite @xcite , including task-oriented dialog generation @xcite , creative generation @xcite , and model robustness @xcite . recent works have shown success in training syntactically controlled paraphrase generators @xcite @xcite . although their models can generate high-quality paraphrases and achieve good syntactic control ability, the training process needs a large amount of supervised data, e.g., parallel paraphrase pairs. annotating paraphrase pairs is usually expensive because it requires intensive domain knowledge and high-level semantic understanding. due to the difficulty in collecting parallel data, the ability of supervised approaches are limited, especially when adapting to new domains. to reduce the annotation demand, unsupervised approaches can train syntactically controlled paraphrase generators without the need for parallel pairs @xcite @xcite . most of them achieve syntactic control by learning disentangled embeddings for semantics and syntax separately @xcite . however, without parallel data, it is challenging to learn a good disentanglement and capture semantics well. as we will show later (section 4.1), unsupervised approaches can generate bad paraphrases by mistakenly swapping object and subject of a sentence. in this work, we propose to use abstract meaning representations (amr) @xcite to learn better disentangled semantic embeddings for unsupervised syntactically controlled paraphrase generation. amr is a semantic graph structure that covers the abstract meaning of a sentence. as shown in figure 2 , two sentences would have the same (or similar) amr graph as long as they carry the same abstract meaning, even they are expressed with different syntactic structures. this property makes amrs a good resource to capture sentence semantics. based on this, we design an amr-enhanced paraphrase generator (amrpg), which separately learns (1) semantic embeddings with the amr garphs extracted from the input sentence and (2) syntactic embeddings from the constituency parse of the input sentence. then, amrpg trains a decoder to reconstruct the input sentence from the semantic and syntactic embeddings. the reconstruction objective and the design of the disentanglement of semantics and the syntax makes amrpg learn to generate syntactically controlled paraphrases without using parallel pairs. our experiments show that amrpg performs better syntactic control than existing unsupervised approaches. additionally, we demonstrate that the generated paraphrases of amrpg can be used for data augmentation to improve the robustness of nlp models. ## conclusion we propose amrpg that utilizes amr to learn a better disentanglement of semantics and syntax without using any parallel data. this enables am-rpg to captures semantics better and generate more accurate syntactically controlled paraphrases than existing unsupervised approaches. we also demonstrate that how to apply amrpg to improve the robustness of nlp models.