GCPG : A General Framework for Controllable Paraphrase Generation (2022)

Abstract: Controllable paraphrase generation (CPG) incorporates various external conditions to obtain desirable paraphrases. However, existing works only highlight a special condition under two indispensable aspects of CPG (i.e., lexically and syntactically CPG) individually, lacking a unified circumstance to explore and analyze their effectiveness. In this paper, we propose a general controllable paraphrase generation framework (GCPG), which represents both lexical and syntactical conditions as text sequences and uniformly processes them in an encoder-decoder paradigm. Under GCPG, we reconstruct commonly adopted lexical condition (i.e., Keywords) and syntactical conditions (i.e., Part-Of-Speech sequence, Constituent Tree, Masked Template and Sentential Exemplar) and study the combination of the two types. In particular, for Sentential Exemplar condition, we propose a novel exemplar construction method — Syntax-Similarity based Exemplar (SSE). SSE retrieves a syntactically similar but lexically different sentence as the exemplar for each target sentence, avoiding exemplar-side words copying problem. Extensive experiments demonstrate that GCPG with SSE achieves state-of-the-art performance on two popular benchmarks. In addition, the combination of lexical and syntactical conditions shows the significant controllable ability of paraphrase generation, and these empirical results could provide novel insight to user-oriented paraphrasing.

## introduction paraphrase generation @xcite refers to restating a given sentence into an alternative surface form while keeping the semantics unchanged. 1 it is of long-standing interest @xcite , with various applications such as ques-there was a picture of the revolving earth that have emerged. there was a picture of the rotating earth that showed up. ## related work in this section, we summarize existing works on syntactically and lexically cpg. syntactically cpg generates a paraphrase constrained by a prespecified sentence of a certain syntax structure namely exemplar. however, the exemplar is only available during inference, resulting in a key challenge: obtaining manual exemplars for existing paraphrasing training datasets is prohibitively expensive. to address this, some of the previous works construct syntactical features from target sentences during training, such as pos tagging, constituent tree, mask template as illustrated in lct to reorder the source sentence then paraphrasing. aesop @xcite selects target lct adaptively while paraphrasing. different from using lct, sgcp @xcite introduces a graph encoder to encode the constituent tree of exemplar as the condition. masked template replaces several words of the exemplar with a special token to form a template as the condition. bcpg @xcite follows bert @xcite to randomly mask exemplar words, parafragpt @xcite further masks exemplar words with certain pos types. however, @xcite advocate to directly utilize the sentential exemplar (the sentence) as the condition, because they believe "any syntactically valid sentence is a valid exemplar". since exemplar is only available in testing, they construct exemplar by replacing words of the target sentence with others that have the same pos type. in addition, lexical constraints decoding is widely explored in text generation @xcite @xcite , such as neural machine translation @xcite ) and text summarizing. ctrlsum @xcite uses target entity words as keywords to hint model while summarizing. however, lexically cpg constraints paraphrasing with pre-specified keywords, which is rarely explored but undoubtedly indispensable in cpg. @xcite make the first attempt to integrate keywords with copy mechanism. despite their progress, existing works only focus on a special condition under either lexically or syntactically cpg. in comparison, gcpg jointly includes lexically and syntactically cpg. ## conclusions in this paper, we propose a general framework gcpg, enabling flexibly combine lexical and syn-tactical conditions and exploring their mutual effectiveness. under gcpg, we provide sse that allows gcpg to directly model syntax information from natural sentences and better utilize plms. as we tentatively give a successful implementation of leveraging two types of conditions in a unified circumstance, multilingual cpg and more types of conditions are barely being discussed. in the future, we will investigate to uniformly represent these conditions in a more superior way.