A Quality-based Syntactic Template Retriever for Syntactically-Controlled Paraphrase Generation (2023)

Abstract: Existing syntactically-controlled paraphrase generation (SPG) models perform promisingly with human-annotated or well-chosen syntactic templates. However, the difficulty of obtaining such templates actually hinders the practical application of SPG models. For one thing, the prohibitive cost makes it unfeasible to manually design decent templates for every source sentence. For another, the templates automatically retrieved by current heuristic methods are usually unreliable for SPG models to generate qualified paraphrases. To escape this dilemma, we propose a novel Quality-based Syntactic Template Retriever (QSTR) to retrieve templates based on the quality of the to-be-generated paraphrases. Furthermore, for situations requiring multiple paraphrases for each source sentence, we design a Diverse Templates Search (DTS) algorithm, which can enhance the diversity between paraphrases without sacrificing quality. Experiments demonstrate that QSTR can significantly surpass existing retrieval methods in generating high-quality paraphrases and even perform comparably with human-annotated templates in terms of reference-free metrics. Additionally, human evaluation and the performance on downstream tasks using our generated paraphrases for data augmentation showcase the potential of our QSTR and DTS algorithm in practical scenarios.

## introduction paraphrase generation (pg) @xcite is to rephrase a sentence into an alternative expression with the same semantics, which has been applied to many downstream tasks, such as question answering @xcite and dialogue systems @xcite @xcite @xcite . on this basis, to improve the syntactic diversity of paraphrases, syntactically-controlled paraphrase generation (spg) is proposed and attracts extensive attention in the research community @xcite @xcite @xcite @xcite @xcite . different from traditional pg models, spg models take syntactic templates as additional conditions to generate paraphrases conformed with the corresponding syntactic structures, whose forms generally include syntax parse trees foot_0 and sentence exemplars. after years of research, spg models can already generate syntax-conforming and highquality paraphrases with human-annotated or wellselected syntactic templates (refer to the first case in figure 1 ). however, such promising performance heavily relies on those satisfying templates, whose difficult acquisition in practice largely hinders the applica-tion of spg models. firstly, manually tailoring templates for every source sentence is practically unfeasible since it is time-consuming and laborious. alternatively, automatically retrieving decent templates is also difficult, and unsuitable templates would induce semantic deviation or syntactic errors in the generated paraphrases (refer to the second case in figure 1 ). on templates retrieval, current solutions are mostly heuristic methods and assume that suitable templates should satisfy certain conditions, e.g., high frequency in the corpus @xcite or high syntactic similarity on the source side @xcite . the only exception is @xcite , which utilizes contrastive learning to train a retriever. nonetheless, it also assumes that a suitable template should be wellaligned with the corresponding source sentence. albeit plausible, the retrieval standards in these methods have no guarantee of the quality of the generated paraphrases, which may lead to the unstable performance of spg models in practice. to address this limitation, we propose a novel quality-based syntactic template retriever (qstr) to retrieve templates that can directly improve the quality of the paraphrases generated by spg models. different from previous methods, given a source sentence and a candidate template, qstr scores the template by estimating the quality of the to-be-generated paraphrase beforehand. to achieve this, we train qstr by aligning its output score with the real quality score of the generated paraphrase based on a paraphrase-quality-based metric, i.e., parascore @xcite . with sufficient alignment, templates with higher scores from qstr would be more probable to produce high-quality paraphrases. moreover, when generating multiple paraphrases for each source sentence, we observe a common problem in qstr and previous methods that the top-retrieved templates tend to be similar, which may result in similar or even repeated paraphrases. aiming at this, we further design a diverse templates search (dts) algorithm to enhance the diversity between multiple paraphrases by restricting maximum syntactic similarity between candidate templates. experiments on two benchmarks demonstrate that qstr can retrieve better templates than previous methods, which help existing spg models generate paraphrases with higher quality. additionally, the automatic and human evaluation showcase that our dts algorithm can significantly improve current retrieval methods on the diversity between multiple paraphrases and meanwhile maintain their high quality. in the end, using templates from qstr to generate paraphrases for data augmentation achieves better results than previous retrieval methods in two text classification tasks, which further indicates the potential of qstr in downstream applications. in summary, the major contributions of this paper are as follows foot_2 : ## related work as the syntactically-controlled paraphrase generation task is proposed @xcite and has received increasing attention, previous work mainly focuses on improving the performance of the generated paraphrases conforming to the corresponding human-annotated templates. more specifically, most of them modify the model structures to better leverage the syntactic information of templates @xcite @xcite . furthermore, @xcite and @xcite generate paraphrases based on the pre-trained language models, e.g., bart @xcite , prophetnet @xcite , and yield better performance. compared to the concentration on spg models, only rare methods focus on how to obtain templates for these spg models in practice. among them, @xcite and @xcite directly use the most frequent templates in the corpus. @xcite select the syntax parse trees of the target sentences in the corpus whose source sentences are syntactically similar to the input source sentence. @xcite retrieve candidate templates based on distance in the embedding space. however, the retrieval standards in these heuristic methods cannot guarantee the quality of generated paraphrases. on the contrary, our qstr directly predicts the quality of the paraphrases to be generated with the template, and the retrieved templates have greater potential to generate high-quality paraphrases. ## methodology in this section, we first introduce our qstr ( §3.1), which includes the model architecture ( §3.1.1) and the training objective ( §3.1.2). then we design a diverse templates search algorithm to improve the mutual diversity between multiple paraphrases ( §3.2). ## analysis 6.1 ablation study to verify the effectiveness of the two training objectives for qstr, we further conduct an ablation study. specifically, we calculate the pearson correlation coefficient (pcc) between the predicted scores from qstr and the true quality scores from parascore ref . ## conclusion in this work, we propose a quality-based template retriever (qstr) to retrieve decent templates for high-quality spg. moreover, we develop a diverse templates search (dts) algorithm to reduce the repetitions in multiple paraphrases. experiments show that the spg models can generate better paraphrases with the templates retrieved by our qstr than other retrieval methods and our dts algorithm further increases the mutual diversity of the multiple paraphrases without any loss of quality. furthermore, the results of the human evaluation and the downstream task also demonstrate that our qstr and dts algorithm can retrieve better templates and help spg models perform more stably in practice.