Vector-Quantized Prompt Learning for Paraphrase Generation
Authors: Haotian Luo
Yixin Liu
Peidong Liu
Xianggen Liu
Venue: EMNLP 2023 Findings (2023)
Keywords: Natural Language Processing
Natural Language Generation
Text generation
Paraphrase
TL;DR: We present vector-quantized prompts as the cues to control the paraphrase generation of pre-trained models

Abstract: Deep generative modeling of natural languages has achieved many successes, such as
producing fluent sentences and translating from one language into another. However, the development of generative modeling techniques for paraphrase generation still lags behind largely due to the challenges in addressing the complex conflicts between expression diversity and semantic preservation. This paper proposes to generate diverse and high-quality paraphrases by exploiting the pre-trained models with instance-dependent prompts. To learn generalizable prompts, we assume that the number of abstract transforming patterns of paraphrase generation (governed by prompts) is finite and usually not large. Therefore, we present vector-quantized prompts as the cues to control the generation of pre-trained models.
Extensive experiments demonstrate that the proposed method achieves new state-of-art results on three benchmark datasets, including Quora, Wikianswers, and MSCOCO. We will release all the code upon acceptance.

Vector-Quantized Prompt Learning for Paraphrase Generation
Haotian Luo"
Yixin Liu"
Peidong Liu
Sichuan University
Sichuan University
Sichuan University
haotiaanlu228o2@mmall. com
liuyixin22@stu. scu. edu. cn
hugh@stu. scu, scu. edu.
Xianggen Liu
Sichuan University
liuxianggen@scu.co edw.ecccceedeeee. cn
Abstract
Zhao al., 2009; Lin and Pantel, 2001) and statis- tical machine translation (Quirk et al., 2004; Zhao et al., 2008) to generate paraphrases. These meth- ods are easy to interpret and analyze, but struggle to yield fluent and diverse sentences, Recently the accumulation of the paraphrase data provides an unprecedented opportunity to directly learn the paraphrasing transformations in end-to-end man- ner (Vaswani et al., 2017). For instance, Wang et al. (2019) formulate paraphrasing as a super- vised encoding-decoding problem and use stacked residual LSTM networks to generate paraphrases.
Deep generative modeling of natural languages has achieved many successes, such as produc- ing fluent sentences and translating from one language into another. However, the develop- ment of generative modeling techniques paraphrase generation still lags behind largely due to the challenges in addressing the com- plex conflicts between expression diversity and semantic preservation. This paper pro- poses to generate diverse and high-quality para- phrases by exploiting the pre-trained models with instance-depene dependent prompts. To learn gen- eralizable prompts, we assume that the num- ber of abstract transforming patterns of para- phrase generation (governed by prompts) is finite and usually not large. Therefore, we present vector-quantized prompts the to control the generation of pre-trained mod- els. Extensive experiments demonstrate that the proposed method achieves new state-of-art results on three benchmark datasets, including Quora, Wikianswers, and MSCOCO. We will release all the code upon acceptance.
A good paraphrase sentence that shares sim- ilar semanties but has noticeable syntactical lexi- differences from the original one (Lin and Wan, 2021). To improve the diversity of generated sen- tences, (Gupta et al., 2018) introduce the varia- tional auto-encoder (VAE) to perform paraphrase generation. (Li et al., 2018) propose multiple gen- erators with different granularity levels learn the mapping relationship between input and output spectively, and then combine them to complete the paraphrase generation task. But those generated paraphrases tend to only make trivial changes to original sentences, such as modifications of syn- onyms.
1 Introduction
Paraphrase generation aims produce sentences that have different expressions but convey the same semantic meaning given a particular sentence. Para- phrasing is a common phenomenon that reflects the diversity of human languages, serving as an important research topic in natural language pro- cessing. It has broad applications such as in ques- tion answering (Mckeown, 1983) and information retrieval (Knight and Marcu, 2000). However, automatically generating accurate and different- appearing paraphrases still very challenging, since requires the abilities of both understanding and generation.
Further, Hosking and Lapata (2021) leverage au- toencoder to encode the structure and semantics the sentence separately, and generate paraphrases by perturbing the structure encoding. Liu et al. integrate the word editing and rule-based transfor- mation operations into deep learning and achieve the previous SOTA performance in paraphrase gen- cration (Liu ct al., 2022, 2020). However, due the limitation of scales of the paraphrasing datasets, neural networks tend to generate the paraphrases with local changes to the inputs rather than global modifications on sentence structures.
Conventional methods draw on rule-based sys- tems (Mckeown, 1983; Barzilay and Lee, 2003;
In this work, we aim exploit the knowledge of the pre- trained language model balance expres-
"These authors contributed equally to this work. 'Corresponding authors.
sion diversity and semantic preservation. There-

---

Prompt A What is the main reason of global warming?
Why does the Earth's temperature rise?
Pretrained Language Model
Prompt B
What is the reason of global warming?
Why does the global warming happen?
Figure An example of the ideal prompt that induces the pre-trained language model generate particular paraphrases. The proposed VQPrompt model aims to learn such prompts for each given sentence.
fore, inspired by (Bhardwaj al., 2022) pro- pose vector-quantized prompt learning frame- work, called VQPrompt, to generate diverse and high-quality paraphrases. In particular, VQPrompt comprises a prompt encoder and a pre-trained gen- erative language model. The prompt encoder pro- duces discrete prompts and the generative language model accepts both the prompts and the input sen- tence to generate the corresponding paraphrases. To make the cector-quantization work, also in- troduce K-means training strategy dynamically update the codebook the prompt encoder.
as early the last century. Representative meth- ods include dictionary-based and template-based methods. Dictionary- based methods look up syn- onyms in the dictionaries such as HowNet (Dong and Dong, 2003) or WordNet (Miller, 1995) to replace words in the original sentence, thereby gen- crating corresponding paraphrases (Kauchak and Barzilay, 2006). The advantage of such a rule- based approach is that it is interpretable and con- trollable. Its shortcomings lie in the heavy work- load of manually writing rules, and the generated sentences are not smooth enough.
We evaluate the effectiveness of our model on four paraphrasing datasets, namely, Quora, Wikian- swers, and MSCOCO. Experimental results show that VQPrompt achieves a new state-of-the-art para- phrasing performance in terms of both automatic metries and human evaluation. In summary, our contributions are follows:
With the accumulation of paraphrase corpus, re- searchers then start to model the paraphrase gen- eration task as a single-language statistical transla- tion process, thereby improving the fluency of gen- erating paraphrase sentences (Quirk et al., 2004; Zhao al., 2009). The statistical translation model learns the transition probability from the original sentence to the paraphrases from a large amount training data. For example, Dolan et al. collected a large number of news texts from the Internet and built a paraphrase generation corpus, and then used statistical machine translation methods to generate paraphrase sentences (Dolan et al., 2004). How- ever, statistical paraphrasing methods still require heavy feature engineering.
We propose vector-quantized prompt learning to adapt large pre-trained language models for paraphrase generation.
We introduce a K-means training strategy to dynamically update the codebook in vector quantization (VQ), addressing the index col- lapse of VQ.
In recent years, deep neural network has become the mainstream paraphrase generation method due to its powerful fitting ability (Chowdhury et al., 2022; Hosking and Lapata, 2021). Similar to the statistical paraphrasing methods, the neural- based paraphrase generation method formulates the paraphrase generation lingle-lagguage transla- tion process, but adopts an secoding-decoding net- work structure and an end-to-end training method. The first deep paraphrasing method takes the long
The proposed method achieves new state- of-the-art performances in three benchmark datasets and presents modest interpretability.
2 Related Work
One of the characteristics of the paraphrase gener- ation task is that there exist several general trans- formation rules. Therefore, rule-based methods have been used for paraphrase generation tasks

---

short-term memory network LSTM (Hochreiter and Schmidhuber, 1997) as the encoder and de- coder. In order to solve the long-distance depen- dency problem in the encoding process, Wang et al. used the multi-head attention network Trans- former (Vaswani et al., 2017) as the encoder and de- coder, and achieved further performance improve- ment (Wang et al., 2019).
3.1.1 Prompt Encoder
The prompt encoder aims to generate prompts for the pre-trained language model to produce rea- sonable paraphrases. The proposal of the prompt encoder stems from the assumption that the pre- trained language model (PLM) is powerful to gen- erate sentences with arbitrary contents if given suit- able inputs. Therefore, for particular input tence, the corresponding prompt is all we need for paraphrase generation in this work.
An ideal paraphrase not only needs to have the same semantics but also should have a significant change in expression from the input sentence (i.c., expression difference) (Bhagat and Hovy, 2013). Aiming at the problem of expression differences in generated sentences, researchers have made lot of attempts in different dimensions (Lin and Wan, 2021; Li et al., 2019; Hosking and Lapata, 2021; Meng al., 2021). For example, Li et al. pro- posed multiple generators with different granularity levels to learn the mapping relationship between input and output respectively, and then combine them to complete the paraphrase generation task (Li et al., 2019). Lin and Wan utilized back-translation and multiple rounds of iterative generation meth- ods to produce paraphrased sentenees with signifi- cant variance (Lin and Wan, 2021). Hosking and Lapata (Hosking and Lapata, 2021) continued to use the idea of variational autoencoder to encode the structure and semantics of the sentence sepa- rately, and generate paraphrases by perturbing the structure encoding. Different from these methods, this work learns to generate syntax- based prompts, which could induce the pre-trained model gener- ate diverse paraphrases.
Since the prompts are dependent on the in- put sentence, this work introduces sample-aware prompt encoder. For given sequence of tokens
T2, ..... Jn). we first get the embeddings fer, e2 encoder to take the sentence embeddings as inputs and output the M continuous prompts, given by
. ..... en). Then employ sentence
SentenceEncooerr(e... en), (1) where r stands for the continuous prompt (with length of M) for the sentence æ. We adopt the encoder of the T5 model (Raffel et al., 2020) as the sentence encoder.
In general, the prompt in our work for paraphrase generation illustrates the abstract rule of paraphrase transformation. Indeed, humans have summarized several abstract rules of the transformation between paraphrases. For instance, the abstract rule "what is the reason of Sx? -> why does $x happen?" could characterize a number of transformations of paraphrases. Therefore, we expect that the prompt could indicate the abstract transforming rules for paraphrase generation.
Therefore, we make the second assumption that the transforming rules of paraphrase generation are finite. Based on the assumption, we propose a prompt encoder that produces discrete rule rep- resentations by vector quantization (VQ) (Zhang et al., 2022; Bhardwaj et al., 2022).
Apart from the traditional methods working with language models (LMs) that have parameters less than IB, modern LLMs like ChatGPT can also generate paraphrases with high quality. However, it costs much more than traditional methods since they require a huge training corpus and learnable parameters.
Formally, the prompt encoder maintains code- book that comprises discrete prompt encodings. The above continuous prompt I'm are then quan- tized into discrete ones, selected from the codebook E TR KDD where K. is the number of the dis- crete code, and D is the dimensionality of each dis- crete code Cj., We measure the 1.2 distance between the continuous representation and code vectors in the codebook. For each vector set r E MxD where M is the number of the continuous vectors and D is the dimensionality of each continuous vector Tma the code vector that yields the mini- mum distance is taken to obtain the discrete rule
3 Method
3.1 Model Architecture
In this work, we propose VQPrompt, a novel model that generates paraphrases via prompt learning. is composed prompt encoder and pre-trained generative language model, which will be elabo- rated as follows.

---

Codebook
Prorpt
Enbedding
Outantibation
TS Encoder
T5 Decoder
Encoder
9m
X Input Sequance
Continous Prospt
Quantized Proupt
Exbedded Sequence
Output Sequence
Copy
Figure 2: Model Architecture.
representation qm. The detailed computations are
Algorithm K-means Update Algorithm
qm Cs:-
(2)
Require: Paraphrase dataset
((In, Un) 1,, Computing word embeddings ea
where k = arg min ||rm Je(I....k.)
2,
where Om is the closest quantized vector for the continuous vector Tm- Finally, M prompt vectors constitute paraphrase prompt, given by
EmbeddingLLaerr
Collecting embeddings E {e 1,... .N]
PromptEncoder(e) {4m/m = M).
Initializing code list C for e do
(3)
Q == Prompt Encodcr(e) for m in size(Q) do Caappend(qm))
where Q is the final prompt generated by the prompt encoder for a particular sentence æ. To make the above vector quantization work, we need to train both the neural networks and the code- book in the prompt encoder toward the minimum of the distance between the continuous and discrete representations. The objective function of vector quantization for particular data point is (x) [lsg(rm)) qm 2 |
7:
end for
9: end for
10: Obtaining codebook by computing the
K-means centers of the code list K-means(C)
Ensure: Codebook
sg( (g
11 (4)
where sg(.) is stop-gradient operation during com- putation. In this way, rule representations, which expected to be in-
learned to generate paraphrases nor able to capture the syntactic structure information of the target sen- tence. All of this information should be encoded by the prompt codes. That is say, work builds an information bottleneck where vector-quantized prompts are the only pathway to convey the syntac- tic structure information the generative LM. this specific and effective design, the acquisition the syntactic information in the codebook can be well guaranteed.
derive the discrete terpretable and -ivaariann...i.att...
Overall, the prompt encoder a deep network at- tached with discrete codebook C, which trained following the basic idea of VQ-VAE. The prompt encoder takes an embedded sequence as input and generates several prompts 7m as output, which contains syntactic structure information that guides the generative LM produce paraphrases.
3.1.2 Generative LM
Note that the parameters of the generative lan- guage model our work are fixed when we train the prompt encoder and the codebook. Therefore, the generative language model (LM) neither
generative language model (LM) prescribes the generation a sentence as a sequence of word predictions based on the context. The generative

---

LMs have made waves in the NLP community by demonstrating astounding few-shot capabilities on myriad language understanding tasks (Brown et al.. 2020). Also, the generative language models pos- sess with powerful decoding capacity; they could produce arbitrary contents if given suitable prompts. Therefore, the paraphrase generated by our model is given by
Dataset
#Train set
#Val set
#Test set
Quora
55,611
5,255
5,255
Paralex MSCOCO
222,223
27,778
27,778
113,287
5,000
5,000
Table Statistics of the benchmark datasets used in this work.
tion intercepts the gradients of backpropagation. Our preliminary experiments reveal that most of the codes in the codebook are rarely selected by the prompt encoder after the optimization based on the objective function J. which is called index collapse (Wu and Flierl, 2020). The index collapse usually happens text generation since its gradi- ents are not smooth enough.
P(-[x) = GLM @ e)})
(5)
ÿ P(-lx)
(6)
where GLM stands for the generative language model and the variable Q means the sequence of prompts Qm i.e., {qm/ m 1,.., .M.. @ is the vector concatenation operation and ý is the generated sentence VQPrompt.
Therefore, we propose new training strategy (called K-means training) to climinate the index collapse in the prompt encoder. K-means training contains the following two stages:
This work aims adapt the generative LM produce paraphrases given the input sentence. which belongs to the task of conditional sentence generation. Therefore, we adopt an instruction- based language model named Flan-T5 (Chung et al., 2022) to serve as our base model. The fine- tuned language model (i.e., Flan-T5) takes a se- quence of words as inputs and outputs several sen- tences (i.e., ÿ) needed.
Codebook warm-up. We first ignore the code- book of the prompt encoder and directly the continuous prompt to perform the paraphrase gen- eration. Thus, the training objective is only to min- imize the maximum likelihood objective JML K-means Update. Before the training this stage, We sample some sentences from the dataset and collect the corresponding prompt codes gener- ated by the randomly initialized VQPrompt model. Then we perform the K-Means algorithm those codes and collect set of codes as the primitive version of the codebook. Next, during the training, we prevent index collapse by updating the dead codes in the codebook with the clustered center, When the amount of active codes is lower than a threshold T, we will perform the replacement. If the codes are not used for a relatively long time when training, we say that the code is dead.
3.2 Training Strategy
Similar to most paraphrase generators, VQPrompt is trained to the mapping from the input sen- tences to their paraphrases. Also, the paraphrase datasets are constructed as paraphrase pairs. For- mally, let the dataset be ' with size N. VQPrompt aims to maximize the log- log-likeldhood (denoted by J 11) of the target paraphrases over all the training samples of D, that is.
JML log (o(n)zz))
Discussion of K-means Training. In essence, the K-means strategy is an update trick in the op- timization of the prompt encoder. However, the index collapse in VQ has been a long-standing problem in deep generative modeling (Lańcucki et al., 2020; Wu and Flierl, 2020). The proposed K-means strategy works well empirically and has the potential to benefit other vector quantization models. But figuring out the underlying theory is nontrivial, which we leave as future work.
N To
log Po Polyneelun.tt In),
where yn, stands for the 1-th word of the target paraphrase in the nith sample. Tn denotes the word length of the target paraphrase yn- 0 is the model's parameters.
Together with the objective of VQ. the final ob- jective function I of VQPrompt is
I MI. (m((a)
(8)
Experiments
However, the parameters the prompt encoder are difficult to optimize since the vector quantiza-
In this section, we first test the proposed model on the benchmarking datasets on both automatic

---

Quora
Paralex
MSCOCO
Model
BLEU
self-BLEU
iBLEU
BLEU
self- -LF-BBLUU
iBLEU
BLEU
leff-BEEU
iBLEU
Copy
34.52
100.00
7.61
37.10
100.00
9.68
19.85
100.00
4.12
tf-idf
24.05
62.49
6.75
25.08
25.25
15.01
18.26
38.37
6.93
AE
28.99
60.11
11.17
40.10
75.71
16.94
27.90
38.71
14.58
VAE
27.23
51.09
11.57
38,91
53.28
20.47
27.44
24.40
16.99
VQ-VAB
16.31
21.13
8.83
40.26
65.71
19.07
25.62
22.41
16.01
SOW/REAP
21.27
38.1
9.41
33.09
37.07
19.06
12.51
6.47
8.71
BTmPG
19.83
35.11
8.84
28.40
35.99
15.52
19.76
13.04
13.20
LBoW
23.51
42.08
10.39
34.96
35.86
20.80
21.65
16.46
14.02
Separator
23.68
24.20
14.10
36.36
35.37
22.01
20.59
12.76
13.92
HRQ-VAE
33.11
40.35
18.42
39.49
33.30
24.93
27.90
16.58
19.04
VQPrompt-PG
35.01
39.98
20.01
42.58
41.96
25.67
29.92
23.59
19.21
Table 2: Performance of individual paraphrase generation methods on the Quora and Paralex, and MSCOCO datasets.
evaluation and human evaluation. Then we provide several detailed analyses to elucidate its effective- ness in generating paraphrases.
4.2 Competing Methods
We will compare VQ-prompt with multiple ad- vanced paraphrase generation models, We describe several most competing models follows.
SOW/REAP. uses a two-stage model to derive set of syntactic rearrangements, which are then used to guide an encoder-decoder model (Goyal and Durrett, 2020).
4.1 Datasets
this work, we use three widely used benchmark- ing datasets, namely, Quora (Chen et al., 2017), MSCOCO (Lin et al., 2014), and Paralex (also named Wiki Answers) (Fader et al., 2013) in our experiments.
BTmPG. leverages a multi-round paraphrase generator to improve diversity and tack-translation to preserve semantic information (Lin and Wan, 2021).
Quora.The Quora dataset is collected from the question-answeriigg forum Quora (Chen et al., 2017). It contains over 400k pairs of ques- tions, some are paraphrases and others are non- paraphrases. There are about 1 150k paraphrase pairs in total.
LBoW. It grounds the semantics of discrete latent variable by the latent bag-of-words technique (LBoW) (Fu et al., 2019).
Separator. (Hosking and Lapata, 2021) take both the semantic sentence and syntax-informa sentence as inputs in the training process. It com- bines training objective with a principled informa- tion bottleneck, to induce latent space that disen- tangles meaning and form.
Paralex. Paralex is a dataset of question para- phrases datasets scraped from WikiAnswers (Fader et al., 2013). has a large number of question pairs but presents lower quality in syntactic structures and semantic similarity compared to Quora.
HRQ- Hierarchical refinement quantized variational autoencoders (HRQ- VAE) a method for learning the decomposition of dense encod- ings as sequence of discrete latent variables that make iterative refinements of increasing granular- ity (Hosking et al., 2022). HRQ- VREE serves the previous state-of mate-of-the-ar paraphrasing method. We take it as our arch-rival.
MSCOCO. MSCOCO is benchmark dataset for image captioning (Lin et al., 2014). contains over 100k clusters five captions sentences. Con- sidering captions for images can involve different details or objects, the quality of these paraphrases is lower than those in Quora.
For the fairness of comparison, We the clus- ter version of these three datasets released by the previous best method (i.e., HRQ-VAE (Hosking et al., 2022)). The statistics of the training, valida- tion and test splits are shown in Table 1.
4.3 Evaluation Metrics
Many previous works adopt BLEU as a measure for evaluating several text generation tasks. But for paraphrase evaluation, the dissimilarity from

---

Prompt components
(b) Prompts
Figure 3: Visualization of the learned prompts and their components.
the input is also of vital importance. So, in order to take both paraphrase quality and similarity to the input into consideration, we also use iBLEU for our automatic evaluation. The calculation of iBLEU is given by
Semantic relevance
Fluency
Model
Variance
Mean Scure
Variance
Separator
1.94
0.30
0.60
0.35
HRQ-VAE
2.22
0.67
2.84
0.17
VQ "Q-Prompl
2.32
0.61
2,90
0.10
iBLEU a-BLEE(),
Table 3: Human evaluation.
(1-a) BLLUU<x Y) (9)
where Y stands for the set of reference paraphrases. Thus, the expression BLEU(x, indicates the BLEU score between the input sentence and the reference paraphrases, which also called the Self- BELU score. The coefficient balances the impor- tance between expression diversity and semantic similarity. Following the setting of (Hosking et al., 2022), we set a = 0.8.
4.5
Results
Table 2 presents the performance of all compet- ing methods on the Quora, Paralex, and MSCOCO datasets. Copy and tf-idf are typically repeating the input sentences and thus obtain the lowest iBLEU scores. The neural networks, including LBoW, VAE, and SEPARATOR, achieve higher iBLEU scores. But these improvements are obtained with the loss of the semantic meaning because the simi- larity with the references is decreased along with the improvements of iBLEU. HRQ-VAE is the pre- viously state-of-the-arr paraphrase generator, which obtains better performances than SEPARATOR and LBoW. However, HRQ-VAE prescribes that the dataset contains high-quality sentence pairs with similar syntax structures, which is not feasible in sentences with complex grammar dependence.
Overall, the BLEU, Self-BLEU, and iBELU scores constitute relatively comprehensive evalu- ation of the generated paraphrases. addition to the automatic evaluation metric, also conducted the human evaluation.
4.4 Implementation Details
As for VQPrompt, we observe that it consis- tently outperforms HRQ- VAE and the other base- lines on the three benchmark datasets. Consider- ing that HRQ-VAL VAE utilizes additional syntax su- pervision, the improvements on both BLEU and iBLEU demonstrate the effectiveness of the pro- posed method.
The hidden layer sizes the equation encoder and the expression generator are 768. The size of the codebook is set to 512. The length of the prompt (i.e., M) is 4. The threshold T in the K-means train- ing strategy is 256. The maximum input length of the feature vector is 256 and the maximum out- put length is 60. We evaluate the model for each half epoch and select the model that reaches the best performance on the validation set. Finally, we report the generation performance on the test set.
Human Evaluation. We also conducted human evaluation of the results. Due to the limit of bud- get and resources, we sampled 300 sentences from

---

Cluster
Input/Ourput
Sentence
Input
How can learn to speak Spanish flucntl?
Generation
What is the best way to learn Spanish?
Input
How can I lose 25 pounds in month in safe way?
Generation
What some ways
25 pounds month?
Input
How can you substitute turragon recipes?
Generation
What are some ways to substitute tarragon in recipes?
Input
What should you prepare for the RCMP?
Generation
How can prepare for the RCMP?
Input
What should follow keep myself fit without going gym?
Generation
How can stay without going gym?
Input
What should do make life worth living?
Generation
How can make my life worth living?
Input
What makes dog's 's vomit foamy? Is it dangerous?
Generation
Why is my puppy throwing up yellow liquid?
Input
What causes tides rise and fall?
Generation
Why the tides the sea risc?
Input
What is the story behind Ohama abstaining from the vote on Israeli?
Generation
Why did President Ohama abstain in the UN vote against Israeli?
Table 4: Paraphrases generated from the prompt clusters shown in Fig 3(b).
Model
BLEU
BeEf-BLEU
iBLEU
egy). These two variants together with VQPrompt share the same hyperparameters and data.
Generative LM
34.27
42.75
18.87
Generative LM (VQ)
32.51
46.04
16.80
As the Quora dataset is the most widely-used high-quality dataset, the ablation study is only con- ducted on Quora. As shown in Table 5, the gen- erative LM model reaches modest performance, owing to the decent initialization of the pre-trained language model. Next, simply adding discrete prompt the model would lead a side effect of the paraphrase generation, which is caused by the index collapse the VQ technique. With our train- ing scheme, the discrete presentation prompts could further boost the performance of the gener- ative LM. We also observe that more than half of the codes in the codebook are active after incorpo- rating the training scheme, which reflects the VQ computations works well and finally benefit the paraphrase generation.
VQPrompt
35.01
39.98
20.01
Table 5: The generation performances of individual VQPrompt variants.
the Quora test set and compared VQPrompt with Separator and HRQ-VAE, We asked three human annotators to evaluate the generated paraphrases in terms of semantic relevance and sentence fluency in a blind fashion; each aspect was scored from to 5. We report in Table 3 the average human scores and their variances. Table 4 shows that VQPrompt achieves the highest human satisfaction scores. The results are also consistent with the automatic met- rics in Table 2.
Ablation Study. order to investigate the reasons for the performance gain obtained by VQPrompt, we further build two variants of VQPrompt and evaluate their generation results. The two variants are the generative language model (denoted by generative LM) and the generative language model with traditional vector-quantized prompt (generative LM (VQ)). The difference between generative LM (VQ) and the proposed VQPrompt model lies the optimization of the prompt encoder (i.e., the K- K-meass training strat-
Prompt Visualization. For an intuitive visual- ization of generated prompts, we perform t-SNE on prompt component codes qm and the prompts Q. In this paper, M component codes constitute paraphrasing prompt (in experiments, M 4). Although
the same number of vectors conduct t-SNE, the dimension reduction results are varied. Generally, the points the paraphrase prompt tend to clump together in larger clusters, indicating that VQPrompt has learned several ab-

---

stract paraphrasing rules, which could induce the pre-trained model to produce paraphrases.
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models few-shot learners. Advances in NIPS, 13:1877-1
To demonstrate this point, we select three clus- ters and use them to perform paraphrasing. As shown in Table 5, we observe that these clusters contain a bunch of sentences that share similar syn- tactic structures, which validates that the learned prompts characterize the abstract transforming rule of paraphrase generation.
Zihang Chen, Hongbo Zhang, Xiaoji Zhang, and Leqi Zhao. 2017. Quora question pairs.
Jishnu Ray Chowdhury, Yong Zhuang, and Shuyi Wang. 2022. Novelty controlled paraphrase generation with retrieval augmented conditional prompt tuning. AAAI, pages 10535-105444
5 Conclusion
Hyung Won Chung, Le Hou, Shayne Longpre, Bar- ret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. 2022. Scaling instruction-finttune language models. arXiv preprint urXiv:2210 20. 1446..
Paraphrasing aims restate sentence other with the same meaning, but different word- ings. In this paper, we establish prompt learning framework, coined VQPrompt, for paraphrase gen- cration. VQPrompt leverages vector quantization to learn finite prompt components and thus possess modest interpretability. We introduce K-means training strategy to avoid index collapse in VQ. Experiments show VQPrompt achieves impressive generation performance on multiple datasets.
William Dolan, Chris Quirk, Chris Brockett, and Bill Dolan. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In COLING.
Zhendong Dong and Qiang Dong. 2003. Hownet-a hybrid language and knowledge resource. In Inter- national conference on natural language process- ing and knowledge engineering, 2003. Proceedings. 2003, pages 820-824. IEEE
6 Limitations
For ethnic concerns, the three datasets we use are publicly available and do not contain biased or discriminatory information. For resource concerns, our model is dependent on pre-trained model, which means a higher computation budget.
Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2013. Paraphrase-driven learning for open question answering. In ACL, pages 1608-1618.
Yao Fu, Yansong Feng, and John P Cunningham. 2019. Paraphrase generation with latent bag of words. Ad- vances in NIPS, 32.
Acknowledgements
Tanya Goyal and Greg Durrett. 2020. Neural syntac- tic preordering for controlled paraphrase generation. arXiv preprint urXiv:2005.000 02013..
This work was supported by the National Natural Science Foundation of China under Grant 62206192, the Natural Science Founda- tion of Sichuan Province under Grant 2023NS-36 FSC1408 and the MIIT Industrial Internet Innova- tion and Development Project.
Ankush Gupta, Arvind Agarwal, Prawaan Singh, and Piyush Rai. 2018. A deep generative framework for paraphrase generation. AAAI, pages 5149-5156.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 973) 1780.
References
Tom Hosking and Mirella Lapata. 2021. Factorising meaning and form for preent-preserving paraphrasing. In ACL-IJCNLP, pages 1405-1418.
Regina Barzilay and Lillian Lee. 2003, Learning to para- phrase: An unsupervised approach using multiple- sequence alignment. In ACL, pages 16-23.
Tom Hosking, Hao Tang, and Mirella Lapata. 2022. Hi- erarchical sketch induction for paraphrase generation. arXiv preprint arXiv:2203.034.
Rahul Bhagat and Eduard Hovy. 2013. Squibs: What is a paraphrase? Computational Linguistics, 49(3):463- 472.
David Kauchak and Regina Barzilay. 2006. Paraphras- ing for automatic evaluation. In Proceedings of the Human Language Technology Conference of the NAACL. Main Conference, pages 455-462.
Rishabh Bhardwaj, Amrita Saha, Steven H. Hoi. and Soujanya Poria. 2022. Vector-Quantized Contt-Contxttullizedd Soft Prompts for Natural Language Understanding. arXiv e-prints, page arXiv:2205.1 1004.
Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization step one: Sentence compression. AAAI, pages 703-710.

---

Adrian Lańcucki, Jan Chorowski, Guillaume Sanchez, Ricard Marxer, Nanxin Chen, Hans JGA Dolfing, Sameer Khurana, Tanel Alumäe, and Antoine Lau- rent. 2020. Robust training of vector quantized bot- tleneck models. In IJCNN, pages 1-7. IEEE.
Hanwei Wu and Markus Flierl. 2020. Vector quantization- based regularization for autoencoders. In AAAI, volume 34, pages 6380-6387.
Wenbo Zhang. Likai Tang, Site Mo. Sen Song, and Xianggen Liu. 2022. Learning robust rule representa- tions for abstract reasoning via internal inferences. In Advances in Neural Information Processing Systems, volume pages 33550 33562.
Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li. 2018. Paraphrase generation with deep reinforcement learn- ing. In EMNLP, pages 3865-3878.
Zichao Li, Xin Jiang, Lifeng Shang, and Qun Liu. 2019. Decomposable neural paraphrase generation. In ACL, pages 3403 3414.
Shiqi Zhao, Xiang Lan, Ting Liu, and Sheng Li. 2009. Application-driven statistical paraphrase generation. In ACL, pages 834-842.
Dekang Lin and Patrick Pantel. 2001. Discovery of inference rules for question-answert Natural Lan- guage Engineering. 7(4):343-360.
Shiqi Zhao, Cheng Niu, Ming Zhou, Ting Liu, and Sheng Li. 2008. Combining multiple resources improve SMT-hased paraphrasing model. In ACI, pages 1021-1029
Tsungyi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C Lawrence Zitnick. 2014. Microsoft COCO: Common objects context. In ECCV, pages 740- 755.
Zhe Lin and Xiaojun Wan. 2021. Pushing paraphrase away from original sentence: A multi-round para- phrase generation approach. In ACL-IJCNLP, pages 1548-1557
Xianggen Liu, Wenqiang Lei, Jiancheng Lv, and Jizhe Zhou. 2022. Abstract rule learning for paraphrase generation. In IJCAI, pages 4273-4279.
Xianggen Liu, Lili Mou, Fandong Meng, Hao Zhou, Jie Zhou, and Sen Song. 2020. Unsupervised paraphras- ing simulated anncaling, In ACL, pages 302-312.
Kathleen R Mckeown. 1983. Paraphrasing questions using given and new information. Computational Linguistics, 9(1):1-10.
Yuxian Meng, Xiang Ao, Qing He, Xiaofei Sun, Qinghong Han, Fei Wu, Jiwei Li, et al. 2021. Conrpg: Paraphrase generation using contexts as regularizer. arXiv preprint arXiv:2109.000
George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38( 1)139441.
Chris Quirk, Chris Brockett, and William B Dolan. 2004. Monolingual machine translation for paraphrase gen- eration. In EMNLP. pages 149.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee. Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter Liu, et al. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Much. Learn. Res., 21(140):1-67
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NewrlPS, pages 5998-6008.
Su Wang, Rahul Gupta, Nancy Chang, and Jason Baldridge. 2019. A task in suit and tie: Para- phrase generation with semantic augmentation. In AAAI, pages 7176-7183.


Reviews:
[{"review_id": "XqsZgKJOsU", "reviewer": "Reviewer_PqsA", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:\n\n**Q1. The assumption of the paper is not well explained.**\n\nOur paper assumes that the number of abstract transforming patterns of paraphrase generation is finite and usually not large. Bhagat [1] summarized all the categories of paraphrases in English and indicated that the high level of the transforming patterns of paraphrasing is limited. Transforming pattern means the change of syntactic structure between two sentences that have the same semantic meaning. Therefore, the above assumption is well established. We have also illustrated the paraphrasing process using different transforming patterns in Figure 1 for an intuitive understanding of the above claim.\n\n\t\n**Q2. The idea of using prompt learning is not new.**\n\nAdmittedly, the idea of prompt learning was actually not first invented by us. But we introduce it into paraphrase generation to produce instance-dependent prompts for the first time. In addition, we combine it with the vector quantizing (VQ) technique, which is no doubt a pretty novel idea. We also find that the traditional VQ does not work in prompt learning. This paper further proposes a new k-means-based approach for training the VQPrompt model which avoids index collapse. Therefore, we believe the proposed method is new and novel.\n\n**Q3. The meaning of  \"discrete rule representations by vector quantization\" and \"abstract transforming patterns\".**\n\nThe word “discrete” in this paper means that the representations that describe the paraphrasing transforming patterns are not continuous. We use this term to indicate the significant distinction with most of the deep learning frameworks where the deep representations are continuous and obtain gradients flexibly. Those discrete representations are depicted in Figure 3.\n\n\n“The abstract transforming patterns” stands for the patterns that characterize the high-level mapping from a sentence to its paraphrase. Considering that VQPrompt generates paraphrases according to the discrete rule representations, these learned rule representations are the abstract transforming patterns of the paraphrases. We will add the above discussion in the revised paper. Thanks for the reviewer’s helpful comment.\n \n[1] Bhagat R, Hovy E. What is a paraphrase?. Computational Linguistics, 2013, 39(3): 463-472.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The topic of the research is important for various tasks in NLP.", "reasons_to_reject": "1. The assumption of the paper is not well explained.\n2. The idea of using prompt learning is not new, which needs more concrete motivations.\n3. The terms such as \"discrete rule representations by vector quantization\", \"abstract transforming patterns\" should be discussed in more details for better understanding the paper.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors aim to exploit the knowledge of the pre-trained language model to balance expression diversity and semantic preservation. The key assumption is that the number of abstract transforming patterns of paraphrase generation (governed by prompts) is finite and usually not large, which motivates the paper to introduce vector-quantized prompts as the cues to control the generation of pre-trained models.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9A3bYVKGp6", "reviewer": "Reviewer_F6VU", "summary": "", "questions": "Q1a. Considering the current dominance of modern generative language models in paraphrasing tasks, how might the proposed VQPrompt framework compare in terms of efficiency and performance when applied to closed LLMs like GPT models and open LLMs? Could the paper explore potential trade-offs between VQPrompt's performance and inference costs, shedding light on the practicality of its utilization?\n\nQ1b. While the paper acknowledges concerns regarding the scalability and generalization of the VQPrompt model due to its limited evaluation on benchmark datasets, how might the proposed framework handle more diverse and challenging real-world paraphrasing tasks, different domains, and languages? Could the authors provide insights into addressing domain adaptability and potential concept drift to enhance the model's robustness and applicability beyond the evaluated datasets?\n\nQ1c. Lastly, what other applications of this vector-quantization style approach might be done? It seems interesting to port it to other use-cases where current generative LLMs might have huge inference costs.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the instructive feedback. We will first answer the questions, and then we will respond to the weakness.\n\n**Q1: Comparison between VQPrompt with LLMs.**\n\nFollowing the suggestion of the reviewer, we evaluate the efficiency and performance of the closed/open LLMs. ChatGPT is the most advanced closed LLMs and the Vicuna is an open-source LLMs finetuned on the LLaMA model. As shown in the following table, our method VQPrompt outperforms both ChatGPT and Vicuna on the Quora dataset. After analyzing the generation, we observe that the paraphrases generated by the ChatGPT and Vicuna are reasonable but still similar to the original sentence, thus yielding to lower iBLEU score. When we fine-tune the Vicuna with the training samples of Quora, Vicuna exhibits the best performance, indicating the generative power of LLMs. However, LLMs consume huge computational resources but bring marginal performance gains. Therefore, the proposed method, as the trade-off of the computational costs and the performance, stands as an effective paraphrase generator.\n|  Models\t| Type\t  | Parameters\t  | iBLEU | \n|  ----  | ----  | ----  | ----  |\n| Vicuna\t  | Open source | 70B\t| 10.21 |\n| Vicuna (fine-tuned) | Open source | 70B\t|21.11|\n|ChatGPT|Closed source |>175B|10.41|\n| VQPrompt| Open source |0.25B|20.01|\n\n**Q2: generalization ability**\n\nTo conduct experiments on more diverse and challenging paraphrasing tasks is a good idea, but there does not exist such a dataset that satisfies these requirements. The current benchmark datasets, such as Quora and Wiki, are collected from different domains and do not share reasonable semantics, which forbids the model from learning a transferable ability to paraphrase.\n\nTherefore, in this work, we build a new paraphrasing dataset for the evaluation of the generalization ability. We leverage ChatGPT to produce diverse paraphrases and ask native speakers to refine these samples. Finally, 500 high-quality samples are collected, each containing four paraphrases. We will make them open-source in the future. We observe that our method VQPrompt surpasses the previous SOTA method HRQVAE by a large margin, indicating the better generalization ability of our method.\n\n|  Method\t   |  BLEU| Self-BLEU| iBLEU|\n|  ----  | ----  | ----  |----  |\n| HRQ-VAE (random init.) | 12.84 | 27.52 | 4.76|\n| VQPrompt | 19.51 | 20.63 | 11.48|\n\n \n**Q3: Application of vector-quantization to LLMs.**\n\nR1c: In fact, our vector-quantize technique is used to build a finite set of transforming prompts, which has the potential to reduce the negative effect of noise during training and produce better performance and interpretability. It’s different from the quantized technique used in accelerating the inference of LLMs. We will clarify this point in the revised paper.\n\n## Responses to [Weakness]\n\n**Q4: Complex presentation & writing style.**\n\nWe are encouraged that the reviewer comments that our approach is well executed. We will explain the proposed techniques such as vector quantization and index collapse in more detail.\n\n**Q5. Vector quantization may backfire**\n\nAdmittedly, the expressive capacity of the vector quantization may be limited as the sacrifice to obtain the interpretability. However, except for the vector quantization, most of the parts in VQPrompt are continuous, which are highly non-linear, and capable of learning to fit the diversity and complexity of real-world paraphrases. Therefore, the influence of the vector quantization on the fitting power of our model is small and can be ignored.\n\nIn addition, as stated in the responses to Q2, the generalization ability of VQPrompt is stronger than the previous SOTA models.\n\n**Q6. Negative Scenarios and Limitations**\n\t\nWe thank the reviewer’s suggestions to further discuss the negative scenarios and the limitations of our method. In the submission phase, we are not able to provide more analyses on the main text due to the page limitation, we will add more results about the error cases and the application limitations in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "## Strengths\n\n1. **Interesting Approach:** The use of a vector-quantization method using a codebook and using k-means to control the codebook seems like an interesting approach that I have not personally come across before even for other tasks.\n\n2. **Potential for Generalization and Scaling:** The project's approach of utilizing pre-trained language models and incorporating them into the paraphrase generation process using vector quantization has the potential to be applied to other text generation tasks beyond paraphrasing, contributing to the broader field of natural language generation. This seems like an interesting way to prompt on more types of language models for more tasks which might have rule-based idiosyncracies to them.\n\n3. **Effective use of LMs T5 and Flan-T5**: The paper leverages pre-trained language models and demonstrates how they can be adapted for the task of paraphrase generation. \n\n4. **Interpretability:** The paper emphasizes the interpretability of the learned prompts, which is an important aspect of the proposed framework. The discrete rule representations obtained through vector quantization offer a degree of transparency and insight into the paraphrase generation process.\n\n5. **Reported SOTA:** The experimental results demonstrate that VQPrompt achieves new state-of-the-art performance on benchmark datasets, including Quora, Wikianswers, and MSCOCO.\n\n6. **Formulas and Pseudo code:** I like that the paper has well presented formulas and pseudo-code.", "reasons_to_reject": "## Weaknesses\n\n1. **Complex Presentation & Writing Style:** Although the approach seems to be executed well, I feel the writing style could be more well structured and a better insight as to the motivation of the approach could have been given. I personally did not have much experience with vector-quantization methods and complex nuances such as the index collapse issue and more such things were initially difficult to follow. A well-rounded understanding for this should have been presented.\n\n2. **Efficiency wrt current generative LLMs:** I also would have been interested in observing the performance of this approach when applied to modern generative LLMs, given their current superiority in paraphrasing tasks. It would have been valuable to have comparisons made between VQPrompt and both closed LLMs like the GPT models and the open LLMs. Even if the VQPrompt framework didn't directly outperform them, analyzing the trade-off between performance and inference costs on the Pareto front would have been insightful.\n\n3. **Vector Quantization may Backfire?:** The paper evaluates the proposed approach on a limited set of benchmark datasets, which may not fully represent the diversity and complexity of real-world paraphrasing tasks. The lack of evaluation on more diverse and challenging datasets raises concerns about the scalability and generalization of the VQPrompt model to different domains and languages. End-to-end text-to-text paraphrasing might be more generalizable on a myriad of text types, and I have concerns related to domain adaptability and concept drift in the future.\n\n- It appears that the solution space for this task may be large. Better Auto-Encoder based models which are more robust might outshine this approach, changing the generative decoding methods such as using beam-search trees or top sampling might generate the necessary valid perturbations needed (although I'm not sure how the outputs would compare). Beam search can help improve the diversity of generated paraphrases by considering multiple candidate sequences during decoding; this could enhance the model's ability to produce a wider range of paraphrase variations, modifying the decoding sampling technique, such as using nucleus sampling or temperature scaling, can provide more control over the level of diversity in the generated paraphrases. \n\n4. **Not Enough Discussion of Negative Scenarios:** The paper mainly highlights the positive results and state-of-the-art performance achieved by VQPrompt. However, the absence of a discussion on scenarios where the proposed approach might fail or produce suboptimal results limits the paper's comprehensive analysis and practical guidance for potential users.\n\n5. **Codebook Maintenance and Trade-offs:** I'm not sure how the codebook would be updated with time and concept drift scenarios. A more detailed analysis of the trade-offs and computational costs associated with codebook maintenance is needed. Would like to see a more critical assessment of statements made.\n\n6. **Minimal Discussion of Limitations**: The limitations section is negligible.", "questions_to_authors": "Q1a. Considering the current dominance of modern generative language models in paraphrasing tasks, how might the proposed VQPrompt framework compare in terms of efficiency and performance when applied to closed LLMs like GPT models and open LLMs? Could the paper explore potential trade-offs between VQPrompt's performance and inference costs, shedding light on the practicality of its utilization?\n\nQ1b. While the paper acknowledges concerns regarding the scalability and generalization of the VQPrompt model due to its limited evaluation on benchmark datasets, how might the proposed framework handle more diverse and challenging real-world paraphrasing tasks, different domains, and languages? Could the authors provide insights into addressing domain adaptability and potential concept drift to enhance the model's robustness and applicability beyond the evaluated datasets?\n\nQ1c. Lastly, what other applications of this vector-quantization style approach might be done? It seems interesting to port it to other use-cases where current generative LLMs might have huge inference costs.", "paper_topic_and_main_contributions": "The paper addresses the challenges in generating diverse and high-quality paraphrases by leveraging the power of pre-trained language models and instance-dependent prompts. It introduces a vector-quantized strategy to encode prompts and generate paraphrases using vector-quantization using a codebook of prompt templates which dynamically updates itself using k-means based on an explicit assumption that the abstract transforming patterns of paraphrase generation are finite and not numerous. They use the T5-encoder and the Flan-T5 models (text-to-text encoder-decoder). The research evaluates its proposed model, VQPrompt, on four paraphrasing datasets: Quora, Paralex (WikiAnswers), and MSCOCO, comparing it with several advanced paraphrase generation models. Their experimentation shows claimed sota on the paraphrasing tasks.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7viWUcISvL", "reviewer": "Reviewer_j1cc", "summary": "", "questions": "1.\tWhat is the size of T5 model used for prompt encoding and GLM ?\n2.\tIt is unclear how various parameters such as size of codebook K, number of prompts M chosen? Do the authors perform hyper-parameter tuning? What range of hyper-parameters were considered?\n3.\tWhy not include the fine-tuned T5 based paraphrase generator (which I believe is referred as Generative LM in Table 4) in the main Table 2 itself since its is a very relevant baseline? \n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, thanks for your valuable and constructive suggestions. And we carefully made the following point-to-point responses:\n\n\n**Q1: Size of T5 model**\n\nThe T5 model used in our work is T5-Flan-Base, which has 0.25B parameters. GLM is the intact T5-Flan-Base model, and the prompt encoder in our model only adopts the encoder part of T5-Flan-Base. And they do not share parameters.\n\n\n**Q2: Hyperparameters**\n\n The codebook is set to 256 and the number of prompts is set to 4. We do not perform hyper-parameter tuning. But we add a set of experiments that shows the performance of our model when the number of prompts M varies. Below is the result.\n| Prompt length\t\t| BLEU| Self-BLEU| iBLEU|\n|  ----  | ----  |----  | ----  |\n|1  |      33.30  |  34.26  |   18.79|\n|2\t| 33.32 |\t33.78 |\t19.10|\n|4\t|\t\t\t35.01\t|39.98|\t20.01|\n\n\n**Q3: including the fine-tuned T5 as baseline**\n\nActually, the performance of the fine-tuned T5 paraphrase generator is already reported in Table 4, which is considered to be a part of our ablation study. We will further clarify this point in the revised paper.\n\n## Rebuttals for reject reasons:\n\n**Q4. The writing can be improved.**\n\nSorry for causing confusion. R is actually a vector set that contains an M vector of dimension D. We will rectify this error in our paper.\n\nAs for how we transform the input sentence of variable length to fixed-length vectors, we first pad every input sentence to a fixed length (which is set to 64 in our experiment). Then We feed the input into the prompt encoder and compress it to a prompt with length M by an MLP. We will clarify this point in the revised manuscript.\n\n**Q5. Parameter sizes of models.**\n\nThe T5 model used in our work is T5-Flan-Base, which has 0.25B parameters. GLM is the intact T5-Flan-Base model, and the prompt encoder in our model only adopts the encoder part of T5-Flan-Base. The model sizes of VQPrompt and HRQ-VAE are similar, which are around 0.3B. In the revision, we will indicate the model sizes of all the methods.\n\n**Q6. Hyperparameters.**\n\nFor our experiment reported in Table 2, we train for 16 epochs and select the best one. The learning rate is set to be 0.0005 and the batch size is 48 * 10 (accumulation steps) = 480. Using the proposed techniques (such as K-means clustering), the training process of VQPrompt is stable. We will release all the code once our submission is accepted.\n\n**Q7. Presentation Improvements.**\n\nWe will provide more detailed captions in Figures 2 and 3 to explain the idea of our method. Thanks for your suggestion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe paper develops a novel paraphrase generation model based on pre-trained T5 and vector quantization of the intermediate prompt/ states and also provides a reasonable justification of why it works well\n2.\tThe paper proposes a new k-means based approach for training VQPrompt model which avoids index collapse\n3.\tThe VQPrompt model is thoroughly evaluated on benchmark paraphrase datasets and achieves SOTA results. Human evaluation is also performed to validate the quality of the generated paraphrases\n", "reasons_to_reject": "1.\tThe writing can be improved. Some notations in Section 3 are not very clear. From equation 1, $r$ is the SentenceEncoding of the given input $x$. What is the dimension of $r$, is it a single vector or a collection of $M$ vectors of certain dimension? In Line 233, the authors say $r$ is of dimension $M \\times D,$ but refers is as a vector. This is confusing, since if $r$ represents a collection of vectors, it should be referenced as a matrix or a set, not vector. I suggest using upper case $\\mathcal{R}$ to represent matrix or set of vectors. Also the notation $\\mathbb{R}^{M \\times D}$ mathematically refers the field of $M \\times D$ matrices over real numbers. Also how do you transform Sentence embedding of $n$ sequences into a set of continuous $M$ vectors? Figure 2 shows a FC layer, but it is not expressed mathematically.\n2.\tThe parameter sizes of various baseline models are not clear. Does the VQPrompt model use separate T5 encoders for prompt encoding and the GLM? This considerably increases the model size. \n3.\tDue to the intermediate K-means step, and stop gradients, the training stability is questionable. Can the authors comment on this? The paper doesn't mention how many epochs of training are needed and the learning rates used\n", "questions_to_authors": "1.\tWhat is the size of T5 model used for prompt encoding and GLM ?\n2.\tIt is unclear how various parameters such as size of codebook K, number of prompts M chosen? Do the authors perform hyper-parameter tuning? What range of hyper-parameters were considered?\n3.\tWhy not include the fine-tuned T5 based paraphrase generator (which I believe is referred as Generative LM in Table 4) in the main Table 2 itself since its is a very relevant baseline? \n", "paper_topic_and_main_contributions": "In this paper the authors propose a new model VQPrompt for paraphrase generation based on pre-trained model T5 along with vector quantized training strategy. The paper improves on traditional VQ training by using a novel K-means update which prevents index collapse. On three benchmark datasets and via automatic and human evaluation the proposed model is shown to perform better than state-of-the-art methods in paraphrase generation.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Please provide more detailed Figure caption (e.g. Figure 2 and 3)"}, "extra_text": {}}, {"review_id": "XDugnYVgzS", "reviewer": "Reviewer_CUyp", "summary": "", "questions": "1. In Eq.1, how can you get a continuous prompt with length M by taking a sentence with length n? Do you mean adding M tokens in front of the input sequence for prompt encoding?\n2. How the number of discrete vectors in the codebook affects the performance is not clear. \n3. It's not fair to compare the proposed model with other methods, as it uses Flan-T5 for model initialisation. Have you tried other pre-trained models, such as those used in HRQ-VAE.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First of all, we sincerely appreciate the reviewer CUyp for your helpful feedback. And we carefully made the following point-to-point responses:\n\n**Q1. How to get a continuous prompt?**\n\nWe first pad every input sentence to a fixed length (which is set to be 64 in our experiment). Then We feed the input into the prompt encoder and compress it to a prompt with length M by an MLP. \n\n**Q2.Influence of the number of discrete vectors in the codebook.**\n\nThe codebook is set to 256 and the number of prompts is set to 4 on all the datasets. Although the size of the codebook is not large, the combination space of the four codes is huge. Also, we find that increasing the codebook size from 256 does yield better performances. Therefore, the size of the codebook is dependent on the complexity of the paraphrasing problem, and a size of 256 is enough for most of the paraphrasing tasks.\n\n**Q3. Not fair to compare the proposed model with other methods.**\n\nThe previous SOTA method, HRQ-VAE uses the embedding layer of Bert, which is also a powerful pre-trained model. In addition, to fairly compare our method with HRQ-VAE, we also only use the embedding layer of our pre-trained model, and other parameters are randomly initialized. As shown in the following table, our method still outperforms HRQ-VAE by a noticeable margin.\n|  Method\t   |  Bleu  | Self-bleu | iBleu |\n|  ----  | ----  | ----  |----  |\n| HRQ-VAE  | 33.11 | 40.35 | 18.42|\n| VQPrompt (random init.) | 32.40 | 33.83 | 19.15|\n\nBy the way, HRQ-VAE has relatively high demands on the dataset, as it uses additional information about the syntax structure as inputs. Even in this condition, our method is still shown to be more effective.\n\n\n\n## Response to the weakness\n**Q4. Similarity to Bhardwaj et al., 2022.**\n\nOur method is different from Bhardwaj et al., 2022 for the following reasons:\n\n(1) We use instance-dependent quantized prompts to produce a finite set of transforming prompts, which also aims to reduce the negative effect of noise during training as the training data contains some records of low quality. \n\n(2) The usage of K-means during training is not the same. We record the output of the sentence of the encoder in each epoch and cluster all the codes recorded. The center of each cluster will become the code in the new codebook.  \n\nWe will cite this paper and discuss the difference in the revised paper considering the rigor of the research.\n\n**Q5: Syntax-related information in the learned prompts.**\nThe evidence of our prompt learning syntax-related information can be found in Table 5. We notice that different prompt clusters can perform different transformations on input sentences, but prompts in the same cluster produce sentences with similar syntactic structures. \n\n**Q6: The generalization ability of quantized prompts.**\n\nAs shown in Table 5 in the manuscript, the learned quantized prompts could characterize the syntax-related information. These prompts are generalizable to the other domains if the target domains share similarities with the training data. To demonstrate it, we build a new paraphrasing dataset for the evaluation of the generalization ability. We leverage ChatGPT to produce diverse paraphrases and ask native speakers to refine these samples. Finally,  500 high-quality samples are obtained, each containing four paraphrases. We will make them open-source in the future. We observe that our method VQPrompt surpasses the previous SOTA method HRQVAE by a large margin, indicating the better generalization ability of our method.\n\n|  Method\t   |  BLEU| Self-BLEU| iBLEU|\n|  ----  | ----  | ----  |----  |\n| HRQ-VAE  | 12.84 | 27.52 | 4.76|\n| VQPrompt | 19.51 | 20.63 | 11.48|\n\t\t\t\t\n**Q6: Typos**\nWe will further check the presentation and typos in the revised manuscript. Thanks for your careful inspection.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed quantized-prompt method improves the quality and diversity of paraphrase generation empirically.\n2. The motivation behind using quantized-prompts to encode paraphrase rules is reasonable.", "reasons_to_reject": "1. This paper shares similar ideas with Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding (Bhardwaj et al., 2022), in terms of both model architecture and employing k-means algorithm for quantized-prompt learning. How these two works differ is unclear, which limits the novelty of this paper.\n2. The author mentioned that the learned prompts could encode syntax-related information, but this claim is not supported well in experiments.\n3. The author argues that a limited number of quantized prompts are able to encode the information related to transforming rules for paraphrase generation, which should not be dedicated to a specific dataset. Thus, we have a concern that how well the quantized prompts learned in one dataset or one domain performs in the unseen domains. However, this essential property was not examined.", "questions_to_authors": "1. In Eq.1, how can you get a continuous prompt with length M by taking a sentence with length n? Do you mean adding M tokens in front of the input sequence for prompt encoding?\n2. How the number of discrete vectors in the codebook affects the performance is not clear. \n3. It's not fair to compare the proposed model with other methods, as it uses Flan-T5 for model initialisation. Have you tried other pre-trained models, such as those used in HRQ-VAE.", "paper_topic_and_main_contributions": "This paper presents a method to improve paraphrase generation using prompts. The main assumption is that the paraphrase generation has limited transformation formats, which can be captured by a small number of discrete vectors through quantization. During generation, an encoder model is first applied to obtain M continuous prompts based on current input, which are later used as queries to find\na set of relevant quantized vectors stored in a codebook. An encoder-decoer model takes the concatenation of the retrieved quantized vectors and the input sentence as the input to generate the desirable paraphrased sentence. In order to make sure the proposed discrete vectors can be learned jointly with other parts of the model, the author also employs an online k-means algorithm to update\nthe quantized vectors. The results on three benchmarks show that the proposed method outperforms a variety of baselines according to both automatic evaluation metrics and human evaluation. The ablation analysis also verifies the effectiveness of each proposed component. The case study also shows that some of the learned quantized prompts can encode information related to some specific types of transforming rules.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "[1] Rishabh Bhardwaj, Amrita Saha, Steven C.H. Hoi, and Soujanya Poria. 2022. Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding. EMNLP 2022.", "Typos_Grammar_Style_and_Presentation_Improvements": "line 200: \"a For\" -> Delete \"a\"\n\nlines 172-177: \"Hosking and Lapata continues to ..... (Hosking and Lapata, 2021)\" -> \"Hosking and Lapata (2021) continues to .....\""}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work introduces a method for enhancing paraphrase generation using quantized prompts. The method employs a vector-quantizer to encode prompts and generate paraphrases from a codebook of prompt templates that are updated with a k-mean algorithm. However, the novelty of the method and the evidence for the claims about the syntactic information captured by the learned prompts are questionable. Overall, this is important research top in NLP. It also provides comprehensive experiments.
Decision: Accept-Findings