Prompt-Based Editing for Text Style Transfer
Authors: Guoqing Luo
Yu Tong Han
Lili Mou
Mauajama Firdaus
Venue: EMNLP 2023 Findings (2023)
Keywords: Text style transfer; Prompt-based editing; Large language models
TL;DR: This paper proposes a prompt-based editing approach for text style transfer, transforming the generation problem into a classification one and being more controllable than autoregressive sentence generation.

Abstract: Prompting approaches have been recently explored in text style transfer, where a textual prompt is used to query a pretrained language model (PLM) to generate style-transferred texts word by word in an autoregressive manner. However, such a generation process is less controllable and early prediction errors may affect future word predictions. In this paper, we propose a prompt-based editing approach to text style transfer. Specifically, we prompt a PLM for style classification and use the classification probability to compute a style score. Then, we perform discrete search with word-level editing to maximize a comprehensive scoring function for the style-transfer task. In this way, we transform a prompt-based generation problem into a classification one, which does not suffer from the error accumulation problem and is more controllable than the autoregressive generation of sentences. In our experiments, we performed both automatic and human evaluation on three style-transfer benchmark datasets, and show that our approach largely outperforms the existing systems that have 20 times more parameters. Additional empirical analyses further demonstrate the effectiveness of our approach.

Prompt-Based Editing for Text Style Transfer
Guoqing Luol, Yu Tong Han', Lili Mou't Mauajama Firdaus'
Dept.. Computing Science, Alberta Machine Intelligence Institute (Amii) University of Alberta, Canada 'Canada CIFAR AI Chair, Amii {gluo, yhan22)@ualberta. ca
{doublepower. mou, mauzama. 03 @3)mmmiil.com com
Abstract
text simplification (Dong al., 2019; Kumar al.. 2020), and headline generation (Jin et al., 2020; Zhan et al., 2022).
Prompting approaches have been recently ex- plored in text style transfer, where textual prompt is used to query a pretrained language model (PLM) generate style-transferred texts word by word in an autoregressive manner. However, such a generation process less con- trollable and early prediction errors may affect future word predictions. In this paper, we pro- pose prompt- editing approach text style transfer. Specifically, prompt PLM for style classification and use the classification probability to compute a style score. Then, we perform discrete search with word-level editing to maximize a comprehensive scoring function for the style-transfer task. In this way, trans- form prompt- based generation problem into a classification one, which does not suffer from the error accumulation problem and is more controllable than the autoregressive generation sentences. our experiments, we performed both automatic and human evaluation on three style-transfer benchmark datasets, and show that our approach largely outperforms the exist- ing systems that have 20 times more parameters. Additional empirical analyses further demon- strate the effectiveness of approach.'
Early work text style transfer falls mainly into three categories: 1) Parallel supervision with labeled source-target sentence pairs in sequence- to-sequence manner (Zhu al., 2010; Rao and Tetreault, 2018; Zhang ct al., 2020), 2) Von-parlllll supervision with style labels only, including learn- ing latent representations of style and content sepa- rately (Shen et al., 2017; John et al., 2019; Goyal et al., 2021) and constructing aseudo-parallel train- ing data for learning (Luo al., 2019; Krishna et al., 2020; Reid and Zhong, 2021), and 3) Unsu- pervised learning methods that do not require style labels (Jain et al., 2019; Xu et al., 2020).
Very recently, prompting methods have been ex- plored in text style transfer (Reif et al., 2022: Suz- gun al., 2022), as large-scale pretrained language models (PLMs) enable us perform various natu- ral language generation tasks in a zero-shot (Wei et al., 2022a; Sanh et al., 2022) or exemplar manner (Brown et al., 2020; Schick and Schütze, 2021a). In this paper, we also follow the prompt- based setting. This does not require any training samples or labels, but directly performs inference with PLMs; thus, it is more challenging than the above three settings.
1 Introduction
Text style transfer aims to automatically rewrite a sentence by changing it from one style to an- other (John et al., 2019), such as transferring the sesttine-sentiment sentence "He loves eating sandwiches" into a negative one "He hates eat- ing sandwiches". During the transfer, the style of the sentence must be changed, whereas the style- independent content should be preserved. Text style transfer has wide range of real-world ap- plications, such personalized response genera- tion (Yang et al., 2017: Zheng et al., 2021), text debiasing (Xiang et al., 2012; Ma et al., 2020),
In previous work, a prompt (e.g., a piece of text "Rewrite the text to be positive:") is used to query PLM, which will then generate style-transferred sentence in autoregressive manner (Reif et al., 2022: Suzgun et al., 2022). However, such autore- gressive generation less controllable as words are generated one after another by the PLM. It has the error accumulation problem where early errors the PLM will affect its future predictions, leading to less satisfactory performance in general.
To this end, we propose prompt-based edit- ing approach to unsupervised style transfer. We first design a PLM--bssed style scorer. Specifically,
'Our code and resources are available at: htps://github com/MANGA- LAAAANGG--LLOAAPPPPpEEEEALL

---

we prompt a PLM for style classification and use the classification probability to compute a style score. Then, we perform steepest-ascent hill climb- ing (SAHC; Russell and Norvig, 2010) for discrete search with word-level editing (such as replace- ment, insertion, and deletion) to maximize heuris- tically defined scoring function for style transfer. In this way, we transform prompt-based generation problem into classification one, which involves only a style-word prediction and is generally be- lieved to be easier than multiple-wer word predictions for sentence generation.
natural language classification tasks (Schick and Schütze, b; Min et al., 2022), where PLMs are asked to predict the masked word given piece of text containing the token "IMASK]", and the predicted word is then projected to a label by a predefined verbalizer. With the emergence of var- ious PLMs (Devlin et al.. 2019; Radford et al., 2019; Brown et al., 2020; Raffel et al., 2020; Wei et al., 2022a), prompting methods have recently been widely applied to natural language generation tasks (Liu et al., 2021), such as text style trans- fer (Reif et al., 2022; Suzgun et al., 2022) and machine translation (Radford et al., 2019; Brown et al.. 2020; Raffel et al., 2020).
Our approach provides several additional advan- tages. First, it does not suffer from the error accu- mulation problem, because it performs word edits scattered throughout the entire sentence instead of generating a sentence word by word. Further, design discrete search algorithm that combines the PLM based style score with other scoring func- tions, including fluency and semantic similarity. Consequently, the search algorithm contributes to a more controllable and refined generation of sen- tences.
Text style transfer. Traditionally, style-transfer generation can be accomplished by supervised methods with parallel training data (Xu et al., 2012: Zhang et al., 2015; Rao and Tetreault, 2018). How- ever, obtaining parallel data is labor-intensive and coneucon,,,ng, which remains a significant chal- lenge for this task.
To mitigate the need for parallel data, one line of research is non non- per-parlllll supervision, where it trains the model on a non-parallel but style-labeled corpus (Shen et al., 2017). Early work focuses on learning latent representations of content and style separately (Hu et al., 2017; Fu al., 2018; John et al., 2019; Bao et al., 2019). Goyal et al. (2021) train multiple language models as discriminators for each of the target styles given the content repre- sentation. However, explicit separation of content and style is not always possible, because style can only be conveyed holistically for some sentences.
We used Eleuther Al's GPT-J-6B (an off-the- shelf 2LM)² and conducted both automatic and hu- man evaluations on three syle-transfer benchmark datasets. Results show that our coompt-based edit- ing approach largely outperforms existing prompt- ing systems that have 20 times more parameters. Additional empirical analysis shows the effective- ness of different scoring components and the search algorithm proposed in our approach.
2 Related Work
On the other hand, researchers construct pseudo- parallel training data for training the model in a supervised manner (Lample et al., 2018; Li et al., 2018; Luo et al., 2019; Krishna et al., 2020). Reid and Zhong (2021) first train an attentive style clas- sifier to synthesize source-target style pairs, which are then used to train a Levenshtein editor and per- form multi-span edits. However, the process of constructing meeudo-paralles data can sometimes yield poor-quality data, which would lead to sub- optimal training and low model performance.
Prompting. Prompting methods use a piece of text to query PLM to provide desired outputs (Liu et al., 2021). The simplest prompting method, per- haps, is zero-shot prompting (Wei et al., 2022a; Sanh et al., 2022; Suzgun et al., 2022), which di- rectly queries a PLM to perform natural language processing task, but this may result in less well- formatted or logical sentences (Reif et al., 2022). Another method is few-shot prompting (Brown et al., 2020; Schick and Schütze, 2021a,b; Wei et al., 2022b). It requires several task-specific ex- amples for PLM, but can achieve higher perfor- mance than zero shot prompting, and is thus more widely adopted in NLP tasks (Schick and Schütze, 2021a; Brown et al., 2020: Wei al., 2022b). Prompting methods were initially applied to
Another line of research is devoted to unsuper- vised learning methods, where the training samples contain no style labels (Jain et al., 2019; Xu et al., 2020). Jain al. (2019) train an encoder model with unlabeled texts and multiple control- ling scorers to perform formal style transformation. However, these unsupervised learning methods re-

---

Prompt-Based Generation
the beef, taco,
tasty
PLM
Rewrite the sentence to be more positive
the beef taco is bland
pr propttpo gom
Input Classification: negative/positive
Prompt-Based Editing
PLM
f The sentiment of the text
the beef taco is bland
beef
taco
tasty
is
Input
prompt
Candidate
prompt
Discrete Search
a) Prompt-based generation: previous work (Reif et al., 2022) uses prompt to query PLM, which generates a style-transferred sentence in an autoregressive manner. b) Our prompt-hased editing approach involves one-word classification positive or negative sentiment transfer).
Figure 1:
quire complicated training process, which is not efficient.
3.1 Prompt-Based Classifier
In previous work, researchers directly prompt PLM obtain style-transferred sentences (Figure la; Reif al., 2022: Suzgun et al.. 2022). However, this could be challenging process, as the PLM has to generate the sentence zero-shot or exemplar- based manner; such a process is autoregressive and less controllable.
Recently, researchers have developed several prompt-based approaches that generate style- transferred texts in zero-shot (Suzgun al., 2022) or exemplar-basee manner (Reif et al., 2022). Such methods do not require a learning process or any training labels. Reif et al. (2022) prompt large- scale PLMS to generate sentences in various styles. Suzgun et al. (2022) generate multiple candidate sentences and then choose one with the highest score as the final output.
To address this, we design a prompt- based classi- fier, transforming style-transfer text generation into classification problem. Specifically, prompt PLM for style word, which in turn yields style score. This involves only one single-step predic- tion and is much simpler than generating the entire sentence.
re-ranking mechanism
Our approach follows the prompt-based setting and directly performs style transfer without any training procedure. However, unlike other work that mainly performs autoregressive generation, our approach proposes new prompt-based editing paradigm for text generation, where we only design PLM-basee scoring function but also de- velop discrete search algorithm that is particularly suited to our scenario.
Given a candidate sentence [y], we intuitively design the prompt as
promptais (y) = The [t] of the text [y]] is: (1)
where [t] is the style-transfer task, i.e., sentiment or formality in our experiments, and "(" and "}" are text boundary markers (Reif et al., 2022). Notice that we have not performed prompt engineering, which is beyond the scope of this paper. Instead, our focus is to develop a prompt-based editing ap- proach for text style transfer.
3 Approach
Given an input sentence x (x1, .... Xm), our goal is to generate sentence y = (y1..... Yn) that transfers the style of x. Figure 1b depicts the framework of our prompt-based editing approach, where we propose to prompt a pretrained language model (PLM) to predict the style of candidate sentence. Then, we perform discrete search and iteratively edit the candidate sentence maximize a scoring objective that involves the PLM's clas- sification probability. Finally, the highest-scored candidate taken as the stlletrannsferee sentence.
Based on the above prompt, we perform the next-word prediction to obtain style proba- bility. Specifically, the PLM computes the conditional probability of the next word in the vocabulary given the prompt, denoted by P promptas(y)).
We denote 8 by the representative word of the ith style. Here, s; is simply chosen to be the most intuitive style word, namely,

---

positive and negative for sentiment transfer and formal and informal for formality trans- fer. In general, the predicted probabilities of the two styles are (SL (y)m and (8 pyompt.x(y)).
Algorithm 1 Prompt-Based Editing
1: Input: Original sentence x, iterative steps T
2:
y =
3:
for (1.... do
4:
Enumerate all edit positions and operations
5:
Obtain the highest-scored candidate y by Equ. (3)
To compute the style score, consider the ratio of the two styles. Suppose sentence in style s1 is be transferred $2, we design the style score as:
if fay (y") 1 b. PLM believes style transferred
7:
then: return y"
if fly""" p,x) 2 fly', .x) b Local optioun found then: return y(t-1)
9:
10: clse: y()) y
[yy(y) (82 prompte (y)) (s promptal(y))
11:
return (())
(2)
where a is a hyperparameter balancing fuu with other scoring functions*.
Such a ratio measures the candidate's relative affiliation with different styles.' It is more ro- bust than the predicted target-style probability PPLM (52) promptats (y)), which could be affected by the data sample per se.
Semantic Similarity. The semantic similarity scorer evaluates how output y captures the mantics of input x. our work, we adopt word- and sentence-level semantic similarities as in Li et al. (2020).
3.2 Search Objective
A word-level scorer focuses on keyword infor- mation, where the keywords in the input sentence are extracted by the Rake system (Rose et al., 2010). Then, the RoBERTa model (Liu et al., 2019) is adopted to compute the contextualized repre- sentation, denoted by RBT(w, s), for word w in some sentence The word-level semantic score defined as the lowest similarity among all the keywords, given by
We apply an edit-based search for unsupervised style transfer. This follows the recent development of search-based text generation (Li et al., 2020; Kumar al., 2020; Liu al., 2022; Dong al., 2021; Jolly et al., 2022), where local edits (e.g.. word changes) are performed to maximize heuris- tically defined objective function. However, differ- ent from previous search-basee based work, we propose to prompt an off-the-sher PLM to compute style score and do not require any task-specific training procedure. Overall, objective function involves three aspects:
xy.
min max cos (RBT (BT( keleywond()) y:Ey
(5)
A sentence-level scorer computes the cosine sim- ilarity of two sentence vectors as [yem(y, x) = cos(y, x)) = Twll | where the sentence vectors and x are also encoded by RoBERTa.
f(y: x) = Say(y) [nu(y) Ssem(y (y,x, x)
(3)
where the style scorer fay is designed in Sec- tion 3.1; fnu and fsem are fluency and semantic similarity scorers, mostly adopted from previous work and explained below.
Finally, the semantic similarity score is com- puted as the product of word- and sentence-level scores:
Language Fluency. A language model scorer provides an approximation of how fluent candi- date sentence is. We follow Suzgun et al. (2022) and use GPT-2 (Radford al., 2019) to obtain the fluency score of the candidate y by the geometric mean predicted probabilitics:
fsemly,x) fwordly, (x) x)8 fsem (y, x)7
(6)
where and y are the weighting hyperparameters.
3.3 Discrete Search Algorithm
We perform style-transfer generation by discrete local search using editing operations, such as word insertion, deletion, and replacement, following pre-
i 2
fnu(y)
(4)
vious work (Miao et al., 2019; Li et al., 2020). However, we propose to use steepest-ascen hill
"While our datasets only consider the transfer between two styles, our approach can he extended to multiple styles in one-ys-one or one-vs-all manner.
"Notice that weighting hyperparameter is not needed for the style scorer fay because the relative weight of different scorers are given in fa, and fum-

---

climbing (SAHC; Russell and Norvig, 2010) as our search algorithm.
have been widely used in previous work (Luo al., 2019; John et al., 2019; Suzgun al., 2022). YELP contains reviews for restaurants and other busi- nesses (Zhang et al., 2015), and AMAZON contains product reviews. Both the YELP and AMAZON datasets contain 500 positive and 500 negative sen- tences in the test set.
During development, we measured the edit dis- tance between the input sentences and the reference outputs for sentiment and formality transfer tasks. Our observation is that the average edit distance is 2.9 steps for sentiment transfer and 4.7 steps for formality transfer. Therefore, we set the maximum number of edit steps to 5 to maintain their resem- blance. This, unfortunately, makes previous search algorithms-aae such as simulated annealing (SA; Liu et al., 2020) and first-choice hill climbing (FCHC; Schumann et al., fully make use of the limited search steps.
For formality transfer, we used Grammarly's Ya- hoo Answers Formality Corpus (GYAFC; Rao and Tetreault, 2018). GYAFC consists sentences that were extracted from the Yahoo Answers forum. We chose the Family & Relationships domain follow- ing Luo et al. (2019) and Suzgun et al. (2022). The test set contains 500 formal and 500 informal sen- tences.
as they cannot
In our work, we use the SAHC algorithm: in search step 6, SAHC enumerates every editing posi- tion and performs every editing operation (namely, word delction, replacement, and insertion) Then it selects the highest-scored candidate sentence y* if the score fly", x) is higher than fy x) be- fore it reaches the maximum edit steps. Otherwise, SAHC terminates and takes the candidate (t-1) as the tyyee-rranffrreed output. In this way, our SAHC greedily finds the best edit for every search step and is more powerful than SA and FCHC our scenario, as will be shown in Section 4.6.
4,2 Implementation Details
We used Eleuther Al's off-the-shelf GPT-J-6B as the prompt-based classifier for computing the style score. We also used the off-the-shelf pretrained language model RoBERTa-Large (Liu et al., 2019) to encode sentences (Section 3.2) and to predict top-k words as candidate edits (Section 3.3). We set k = 50 for all sentiment and formality transfer datascts.
For the weighting hyperparameters a, 8, and Y of the search objective f(y) in Eqn. (3), they were 3 8 and for both YELP and AMAZON datasets, and 4. 8. and 2 for the GYAFC dataset. This shows that the style scorer is the most impor- tant one among all the scorers.
Moreover, we design an additional stopping cri- terion such that the search terminates when the prompted PLM predicts that the source style has changed into the target one even if it has not reached the maximum edit steps. This not only improves time efficiency but also encourages con- tent preservation.
We developed our prompt-based editing ap- proach with Python 3.7 and Pytorch 1.11.0. The experiments were conducted on NVIDIA A100 SXM4 GPUs.
Our approach summarized in Algorithm 1.
4 Experiments
4.3 Evaluation Metrics
In this section, will present empirical evaluation of our proposed prompt-based editing approach. First, we will introduce our datasets and setups. Then, we will show our main results, followed by detailed analyses.
We adopted the following automatic evaluation met- ries:
Style transfer accuracy. This measures whether generated output correctly transferred. Fol- lowing the practice in Reif et al. (2022) and Lai et al. (2021), we used SiEBERT (Hartmann et al., 2022) for sentiment classification, and finetuned RoBERTa-Large (Liu et al., 2019) for formality classification.
4.1 Datasets
We evaluated our approach on two standard style- transfer tasks: sentiment and formality.
We used Yelp reviews (YELP; Zhang al., 2015) and Amazon reviews (AMAZON; He and McAuley, 2016) for sentiment transfer. These two datasets
BLEU. The BLEU score measures the mantic similarity between generated outputs and human-written references. Following Luo et al. (2019) and Reif et al. (2022), we used multi-bleu perl to obtain the BLEU-4 score.
'For replacement and insertion, we follow Li et al. (2020) and choose top-k candidate words predicted hy RoBERTa due to efficiency concerns.

---

YELP
AMAZON
Setting
Method
Model
ACC%
BLEU
GM
HM
ACC%
BLEU
GM
HM
LLM
128
69.7'
28.6'
44.6
40.6
Vanilla
LLM-dialog
128
59.1*
17.6
32.3
27.1
Zero-shot
P&R
GPT-J-6B
68.6
19.8
35.2
30.1
57.1
21.7
35.2
31.4
Ours
GPT-J-6B
73.0
40.1
54.1
51.7
72.7
45.6
41.0
GPT-I-6B
52.8
35.8
43.5
42.7
51.0
27.1
37.2
35.4
GPT-3 curie
6.7
53.0"
48.3*
50.6
50.5
72.2
22.9
40.7
34.8
Distant
LLM
128
79.6'
16.1"
35.8
26.8
exemplars
Few-shol
LLM-dialog
128
90.6"
10.4"
30.7
18.7
GPT-3 danvinci
175
74.1"
43.8*
57.0
55.1
87.3
28.3
49.7
42.7
P&R
GPT-J-6R
75.0
42.5
56.5
54.3
66.8
20.5
37.0
31.4
Ours
GPT-I-6B
74.5
48.9
60.3
59.0
78.5
37.1
54.0
50.4
Table I: Results on YELP and AMAZON test sets. #Para: Number of parameters. GM and HM: Geometric mean and harmonic mean of ACC% and BLEU. We replicated Prompt & Rerank (Suzgun et al., 2022) by their released code, as the settings in Suzgun et al. (2022) are incompatible with other previous work. "Quoted from Reif et al. (2022). Other results are given by our experiments. The performance of LLM and LLM-dialog not available for AMAZON because these PLMS
public.
Method
Model
#Para (B)
ACC%
BLEU
GM
HM
Distant exemplars
GPT-J-6B
39.4
33.1
36.1
36.0
P&R
GPT-J-6B
44.4
32,9
38.2
37.8
Ours
GPT-J-6B
44.4
33.4
38.5
38.1
Table Four-shot performance on the GYAFC dataset, considering both directions of informal ++ formal.
Geometric mean (GM) and harmonic mean (HM). They are the average of the above- mentioned metries, evaluating the overall perfor- mance of text style transfer. Again, this follows the standard practice in previous work (Luo et al., 2019; Li et al., 2020).
transfer exemplars in few-shot manner. How- ever, their exemplars have different target styles from the test cases in the inference task, and thus we call it distant-exemplar prompting.
Prompt & Rerank. Suzgun et al. (2022) pro- pose a method that generates multiple candi- date outputs from different manually designed prompts; then, they rerank the outputs by a heuristically defined scoring function. It should be mentioned that the paper (Suzgun et al., 2022) adopts setting that is non-compatible with other work: they report different directions of sentiment transfer separately, while excluding informal-to-formal transfer the formality ex- periment. Therefore, we replicated their work under the standard settings (Luo et al., 2019; Reif et al., 2022).
We also performed human evaluation on selected style-transfer systems, detailed in Section 4.6.
4.4 Baselines
Since our approach is based prompting and does require training process, we compare our approach with the following existing prompting systems:
Vanilla prompting. This baseline method prompts PLM with "Here is some text: { [x] }. Here is rewrite of the text, which is more [s]: I" where [x] is the input and [s] is the style word. This bascline directly obtains style-transferred sentence, shown in Figure la. No exemplars used here.
To the best of our knowledge, Reif et al. (2022) and Suzgun et (2022) are the only prior studies of prompting methods on text style transfer.
Distant-exemplat prompting. We adopted the approach in Reif et al. (2022), which queries a large PL.M (such as the LLM, LLMM-diaoog, and 175B-parameter GPT-36) with several style-
4.5 Main Results
Table shows the performance of different prompt- ing systems on YELP and AMAZON datasets. Com-
obtain results on the off-the-shelf GPT-3 babbage and GPT-J- 6B for the YELP,, AMAZON, and GYAFC datuscts.
"We use the same prompt provided by Reif et al. (2022) to

---

Dataset
Method
Style
Content
Fluency
Average
Daraset
Model
ACC%
BLEU
GM
HM
PPL
Prompt & Rerank
3.64
3.55
3.04
3.41
Full model
73.0
40.1
54.1
51.7
122.7
YELP
Our appreach
3,76
4.24
3,13
3.71
w/o style
17.9
25.1
21.2
33.9
29.3
Prompt & Rerank
3,46
3.52
3,38
3.45
YELP
woo semanttcc
74.0
39.0
53.7
51.1
124.0
AMAZON
appreach
3.67
3.97
3.62
3.75
w/o fluency
81.3
39.3
56.5
53,0
223.6
w/o criterion
78.3
25.2
44.4
38.1
192,4
Table 3: Human evaluation on the sentiment transfer datasets. We show human ratings of style transfer strength (Style), content preservation (Content), and fluency.
Full model
72.7
28.6
45.6
41.0
137.2
w/o style
33.6
20.3
26.1
25.3
31.5
AMAZON
w/o semantic
71.1
28.1
44.7
40.3
116.3
w/o fluency
78.0
28.6
47.2
41.8
229.9
w/o stop critecion
79.9
19.3
39.3
31.1
176.3
Table 4:
Ablation study on the sentiment transfer datasets in the zero-shot setting. PPL: Perplexity (the smaller, the better). In the "wlo style" setting, the model mainly optimizes toward fuu- so achieves an extraor- dinarily low PPL: however, its style is usually not trans- ferred, shown by extraordinarily low ACC%, Therefore, this not a meaningful style-transfer setting and is grayed out.
pared with the recently proposed Prompt & Rerank system (Suzgun et al., 2022), our approach achieves a performance improvement 14 and 3 points for GM, as well as 15 and 5 points for HM in the zero- and few-shot settings, respectively, averaged across the two datasets. Further, compared with 175B parramereet GPT-3 with distant exemplars (i.e., style-transfer exemplars containing source texts and outputs written in non-target styles), our ap- proach yields higher GM and HM by more than and points, respectively, also averaged across the two datasets. This is compelling result, as our approach yields a better balance between content preservation and style transfer strength while using 20x smaller PLM.
Dataset
Algorithm
ACC%
BLEU
GM
HM
SAHC
73.0
40.1
54.1
51.7
YELP
FCIIC
67.2
31.8
46.2
43.1
SA
66.0
28.7
43.5
40.0
SAHC
72.7
28.6
45.6
41.0
AMAZON
FCHC
64.1
24.8
39.8
35.7
SA
63.2
23.7
38.7
34,4
Table 5: Results of different search algorithms the sentiment transfer datasets.
Table 2 shows the results of different prompt- ing systems on the GYAFC dataset, where both coormal-to-formalt and formal-to-nnformal direc- tions are considered (Luo et al., 2019; Reif et al., 2022). For fair comparison with previous prompt- ing systems, we followed Suzgun et al. (2022) and conducted experiments in four-shot setting. As seen, our method outperforms previous approaches in GM and HM scores, which consistent with the results Table 1. It is also noticed that our approach achieves less improvement on GYAFC than on YELP and AMAZON, as formality transfer is more challenging than sentiment transfer.
our approach is based on the same GPT-J-6B model. Following Luo et al. (2019) and Krishna al. (2020), we asked three human annotators, who were instructed to rate each sentence based on a 1-5 Likert scale (Stent et al., 2005) in terms of style transfer strength, content preservation, and flu- ency (Briakou al., 2021). Our annotations were strietly blind; the samples from the two prompting approaches were randomly shuffled and the anno- tators did not know which approach generated the sample.
4.6 Detailed Analyses
We measured the inter-rater agreement by Fleiss' Kappa score (1971) for the Likert scale ratings. They are 0.37, 0.42, and 0.39 for style transfer strength, content preservation, and fluency, respec- tively. These scores are considered fair correla- tion?.
In this subsection, we conduct in-depth analyses assess the effectiveness of our prompt-based editing approach. Due the limit of time and resources, we chose the sentiment transfer datasets (YELP and AMAZON) as our testbed.
Table presents the results of human evaluation. We observe that our prompt-based editing approach outperforms P&R all three aspects. particularly in terms of content preservation. This is because with the proposed stopping criterion and discrete
Human Evaluation. We conducted human evaluation via pairwise comparison of system out- puts to further confirm the superiority of our ap- proach. Specifically,, we randomly selected 100 outputs from the recently proposed Prompt-and- Rerank (P&R) system (Suzgun et al., 2022) and
http://www....iiiio.i/v///////ee _appa

---

YELP
Negative
Positive
Positive Negative
Source
far not really impressed
their lunch special is great value
P&R
The text good now
but their lunch is a great value
Ours
so far i' I'm really impressed
their lunch special is not a great value
A AAZON
Negative - Positive
Positive + Negative
like neutrogena products as a rule,
Source
for my purpose this the perfect item.
this was disappointment.
like neutrogena products, this was a disappointment.
for my purpose this is the perfect item. So this text has two different purposes: to be a text and to be rewrite...
P&R
overall like neutrogena products rule, this was success.
Ours
but for my purpose this is not the perfect item.
GYAFC
Informal Formal
Formal Informal
Source
think about what good it brought about.
i'm unsure concerning what should do.
P&R
think about what good it will bring about
i' m not certain about what to do next..
please think about what all the good news
Ours
yeah lol really i'm unsure concerning what i 'll do.
has brought about.
Table 6: Example outputs on the YELP, AMAZON, and GYAFC datasets. Improperly generated expressions are italicized.
search, we avoid unnecessary edits and preserve the original content well. Our approach also achieves a higher average score, which is consistent with the automatic evaluation results in Table 1, further demonstrating the effectiveness of our approach. Ablation Study. To evaluate the contribution key components in our model, we conducted an ablation study of different scoring functions and our proposed stopping criterion.
and replacement (where top-50 candidate words are considered for efficiency concerns). Then, SAHC selects the best one for the next round of editing, shown in Algorithm 1.
We compare our SAHC with two stochastic optimization algorithms, first-choice hill climb- ing (FCHC; Schumann et al., 2020) and simulated annealing (SA; Liu et al., 2020), which are used in previous search-based text generation. Both FCHC and SA perform stochastie local changes to obtain candidate sentence. If the proposed sentence is bet- ter than the current one, the algorithms will accept the new candidate. Otherwise, FCHC will keep the current candidate, while SA may still accept the candidate with small probability.
Table 4 shows that all the scorers play a role in our approach, and that the prompt-based style scorer is the most important one. This makes sense, as it is the only signal of the style, without which we would not be able to perform meaningful style transfer. Moreover, we find that the fluency scorer slightly hurts style accuracy and BLEU scores, which are the standard metries in Luo al. (2019). However, it significantly improves language model probability (i.c., lower perplexity). which roughly estimates the fluency of text (John et al., 2019). Therefore, we deem the fluency scorer fau essen- tial to our text style transfer model.
From Table 5, observe that SAHC algo- rithm significantly outperforms FCHC and SA in both style-transfer accuracy and the BLEU score. This is likely due to the limited number of edit steps, requiring that the algorithm should make an effective edit at every search step. The results con- firm that SAHC is more suited in our scenario than other discrete search algorithms
In addition, criterion that terminates the search process if the PLM believes the style successfully transferred. As seen from the last row of Table 4, more edit steps (w/o stop criterion) improve the style accuracy but drastically hurt BLEU scores. This shows that our stopping criterion is able to seek balance between style transfer accuracy and content preservation.
approach involves a stopping
Case Study. We show Table 6 that our method is able to avoid issues that arise from the error accumulation problem in an autoregressive gen- eration. This is observed through several exam- ple outputs by P&R and our approach for YELP, AMAZON, and GYAFC datasets. We see that the previous approach, which performs autoregressive generation, yields less controllable and satisfactory sentences. For example, given the source input "for my purpose this is the perfect item" in the
Discrete Search Algorithms. Our steepest- ascent hill climbing (SAHC) algorithm enumerates candidate edits, including word deletion, insertion,

---

positive-to-negativ sentiment transfer of the AMA- ZON dataset, P&R generates an unrelated sentence starting with "So this text has", leading to the sub- sequent improper word predictions "a text and to be a rewrite".
search outputs. In this way, our approach can be more computationally efficient.
Another limitation is the need for manually de- signed prompts, which is inevitable in zero-shot prompting. Our current work adopts the most in- tuitive prompt and has not performed prompi engi- neering. In the future, we would like to investigate prompt tuning (Schick and Schütze, 2021b; Li and Liang, 2021; Wei et al., 2022a) mitigate the liance on designing prompts.
However, our prompt-based editing approach transfers the sentiment of a source sentence from positive negative by inserting the words hut and not, while maintaining other semantic content. This shows that our approach is capable of generating more sensible and controllable sentences,
Acknowledgments
In addition, we find that our approach able to convert the style of source input with multiple edits. For example, given the source sentence "i' m unsure concerning what i should do" in formal- to-informal transfer, our approach inserts multiple tokens (yeah, lol, really, and "...") the beginning and replaces should with 'Il at the end, and the sen- tence is transferred to an informal one. By allowing iterative edits and examining all possible positions and editing operations, we are able to have multiple word edits scattered throughout the sentence and experience a gradual transfer of style.
We thank all reviewers and chairs for their valuable comments. This research is supported in part by the Natural Sciences and Engineering Research Coun- cil of Canada (NSERC) under Grant No. RGPIN- 2020-04465, the Amii Fellow Program, the Canada CIFAR AI Chair Program, the Alberta Innovates Program, a UAHJIC project, donation from Deep- Mind, and the Digital Research Alliance of Canada (alliancecan.ca).
References
5 Conclusion
Bao, Hao Zhou, Shujian Huang, Lei Li, Lili Mou, Olga Vechtomova, Xin-yu Dai, and Jiajun Chen. 2019. Generating sentences from disentangled syn- tactic and semantic spaces. ACI, pages 6008- 6019.
In this paper, propose novel prompt-based editing approach text style transfer that turns a prompt-based generation problem into clas- sification one. does not suffer from the issue of error accumulation and more controllable than autoregressive generation. Our experiments on three benchmark datasets show that the pro- posed approach significantly outperforms the ex- isting prompting systems while using 20x fewer parameters. Additional empirical analysis shows the effectiveness of different scorers and the dis- crete search algorithm in our approach.
Eleftheria Briakou, Sweta Agrawal, Ke Zhang, Joel Tetreault, and Marine Carpuat. 2021. A review of human evaluation for style transfer. In Workshop on Natural Language Generation, Evaluation, and Metrics, pages 58-67.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models are few-shot learners. In NewrIPS, pages 1877-1901.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre- Pre-training of deep bidirectional transformers for language under- standing. In NAACL, pages 4171-4188
6 Limitation
Our paper transforms generation problem into a classification one text style transfer, but comes with a trade-off between output quality and infer- ence efficiency. Nevertheless, our algorithm can be implemented in a highly parallel manner when eval- uating different candidates, and we only need five iterations. Therefore, the efficiency of our SAHC is already much higher than other search algorithms (such as SA) which requires several hundred search steps (Liu et al., 2020). Further, the efficiency can be improved by learning from the search results (Li et al., 2020), i.e., finettunin& a PLM based on our
Chengzhang Dong, Chenyang Huang, Osmar Zaïane, and Lili Mou. 2021. Simulated annealing for emo- tional dialogue systems. In CIKM, pages 2984-2988.
Yue Dong, Zichao Li, Mehdi Rezagholizadeh, and Jackie Chi Kit Cheung. 2019. EdiINTS: An neu- ral programmer-interpeter model for sentence sim- plification through explicit editing. In ACL, pages 3393-3402.
Joseph L Fleiss. 1971. Measuring nominal scale agrec- ment among many raters. Psychological Bulletin, 76(5):378-382.

---

Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer text: Exploration and evaluation. In AAAI, pages 664-670.
Juncen Li, Robin Jia, He He, and Perey Liang, 2018. Delete, retrieve, generate: A simple approach to sen- timent and style transfer. In NAACI, pages 1865- 1874.
Navita Goyal, Balaji Vasan Srinivasan, N Anandhavelu, and Abhilasha Sancheti. 2021. Multi-style transfer with discriminative feedhack on disjoint corpus. In NAACL, pages 3500-330 3510.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In ACL-IJCNLP, pages 4582-4597.
Jochen Hartmann, Mark Heitmann, Christian Siebert, and Christina Schamp. 2022. More than feel- ing: Accuracy and application of sentiment analy- sis. International Journal of Research in Marketing, 40(1):75-87.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang. Hiroaki Hayashi, and Graham Neubig, 2021. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv: 107.566.7.33566
Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. WWW, pages 507-517.
Puyuan Liu, Chenyang Huang, and Lili Mou. 2022.
Learning non- non-autoreeressvve models from search for unsupervised sentence summarization. In ACI, pages 7916-7929.
Zhiting Hu, Zichao Yang, Xiaodan Liang Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward con- trolled generation of text. In ICML, pages 1587- 1596.
Xianggen Liu, Lili Mou, Fandong Meng. Hao Zhou, Jie Zhou, and Sen Song. 2020. Unsupervised paraphras- ing by simulated anncaling. ACL, pages 302-312.
Parag Jain, Abhijit Mishra, Amar Prakash Azad, and Karthik Sankaranarayanan. 2019. Unsupervised con- trollable text formalization. In AAAI, pages 6554- 6561.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer. and Veselin Stoyanov. 2019. RoBERTa: robustly optimized BERT pretraining approach. arXiv preprint arXiv: 7 1692.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Pe- ter Szolovits. 2020. Hooks in the headline: Learning generate headlines with controlled styles. In ACL, pages 5082-5093.
Fuli Luo, Peng Li, Jie Zhou, Pengeheng Yang, Baobao Chang, Zhifang Sui, and Xu Sun. 2019. A dual reinforcement learning framework for unsupervised text style transfer. In IJCAI, pages 5116-5122
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In ACL, pages 424-434.
Xinyao Ma, Maarten Sap, Hannah Rashkin, and Yejin Choi. 2020. PowerTransformer: Unsupervised con- trollable revision for biased language correction. EMNLP, pages 7426-7441.
Shailza Jolly. Zi Xuan Zhang, Andreas Dengel, and Lili Mou. 2022. Search and learn: Improving semantic coverage for data-to-text generation. In AAAI, pages 10858-10866.
Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li. 2019. CGMH: Constrained sentence generation by Cetropolis-Hastiing sampling. In AAAI, pages 6834-6842.
Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating unsupervised style transfer as para- phrase generation. In EMNLP, pages 737-762.
Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Noisy channel language model prompting for few-shot text classification. In ACL, pages 5316-5330.
Dhruv Kumar, Lili Mou, Lukasz Golab, and Olga Vech- tomova. 2020. Iterative edit- based unsupervised sen- tence simplification. In ACL, pages 7918-791 7928.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAl Blog.
Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2021. Thank you BART! Rewarding pre-trained models improves formality style transfer. In ACL-IJCNLP, pages 484-494.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. JMLR, 21((14)):167,
Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc' Aurelio Ranzato, and Y-Lan Boureau, 2018. Multiple-atrriute text rewriting. In ICLR.
Jingjing L., Zichao Li, Lili Mou, Xin Jiang, Michael Lyu, and Irwin King. 2020. Unsupervised text gen- eration by learning from search. In NeurIPS, pages 10820-10831.
Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may introduce the GYAFC dataset: Corpus, bench- marks and metrics for formality style transfer. NAACL, pages 129-140.

---

Machel Reid and Victor Zhong. 2021. LEWIS: Leven- shtein editing for unsupervised text style transfer. In Findings of ACI-IJCNIPP, pages 3932-3944.
Peng Xu, Jackie Chi Kit Cheung, and Yanshuai Cao. 2020. On variational learning of controllable rep- resentations for text without supervision. In /CMI., pages 10534-10543.
Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Seltiioon-Burehle and Jason Wei. 2022. recipe for arbitrary text style transfer with large language models. In ACL, pages 837-848.
Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In COL- ING, pages 2899-2914.
Stuart Rose, Dave Engel, Nick Cramer, and Wendy Cowley. 2010. Automatic keyword extraction from individual documents. Text Mining: Applications and Theory, pages 1-20.
Min Yang. Zhou Zhao, Wei Zhao, Xiaojun Chen, Jia Zhu, Lianqiang Zhou, and Zigang Cao. 2017. Per- sonalized response generation via domain adaptation. In SIGIR, pages 1021-1024.
Stuart J Russell and Peter Norvig. 2010. Artificial In- lelligence: A Modern Approach. Pearson Education Limited.
Jiaao Zhan, Yang Gao, Yu Bai, and Qianhui Liu. 2022. Stage-wise stylistic headline generation: Style gen- eration and summarized content insertion. In IJCAI, pages 4489 4495.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, al. 2022. Multitask prompted training enables zero-shot task generalization. ICLR.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In NIPS, pages 649-657.
Yi Zhang, Tao Ge, and Xu Sun. 2020. Parallel data augmentation for formality style transfer. In ACI. pages 1-22-8.
Timo Schick and Hinrich Schütze. 2021a. Few-shot text generation with natural language instructions. In EMNLP, pages 390-402.
Yinhe Zheng. Zikai Chen, Rongsheng Zhang. Shilei Huang. Xiaoxi Mao, and Minlie Huang. 2021. Styl- ized dialogue response generation using stylized un- paired texts. In AAAI, pages 14558 14567.
Timo Schick and Hinrich Schütze. 2021b. It's just size that matters: Small language models are also few-shot learners. In NAACL, pages 2339-2355 2352.
Raphael Schumann, Lili Mou, Yao Lu, Olga Vechto- mova, and Katja Markert. 2020. Discrete optimiza- tion for unsupervised sentence summarization with word-level extraction. In ACL, pages 0032-5042.
Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In COLING, pages 1353- 1361.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In NIPS, pages 6833-6844.
Amanda Stent, Matthew Marge, and Mohit Singhai. 2005. Evaluating evaluation methods for generation the presence variation. In CICLing, pages 341- 351.
Mirac Suzgun, Luke Melas- Kyriazi, and Dan Juraf- sky. 2022. Coanpt-and-Rerane method for zero- shot and few-shot arbitrary textual style transfer with small language models. In EMNLP, pages 2195- 2222.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du An- drew M Dai, and Quoe V Le. 2022a. Finetuned language models are zero-shot learners. In ICLR.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou. 2022b. Chain of thought prompting elicits reasoning large language models. In NeurlPS, pages 24824-244 24837.
Guang Xiang, Bin Fan, Ling Wang, Jason Hong, and Carolyn Rose. 2012. Detecting offensive tweets via topical feature discovery over large scale twitter corpus. In CIKM, pages 1980-1984.


Reviews:
[{"review_id": "1h2Yp5BqlT", "reviewer": "Reviewer_PLiK", "summary": "", "questions": "1. Please address my questions above regarding the accuracy of the method i.e. whether it will be feasible with more complex timestep variant styles (say author style or discourse)\n2. Is this extendable to the setting where a style is not concretely defined? For example, if a style is represented as a few-shot scheme i.e. a group of sentences of one style, is it possible to then come up with an accurate style score?\n3. Why wasn't a non-generative classifier considered to produce the style score? This would need training but it might have been able to perform better in a low-resource setting.\n4. Is there any other work which uses an edit search algorithm in a generative process similar to this?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**General Response**:\n\nWe thank the reviewer for the detailed feedback and questions. We surmise that a general theme that the reviewer’s questions revolve around is: sentiment and formality transfer are too simple, will the approach be extended to more complex timestep variant styles (author style)?\n\nWe acknowledge that sentiment transfer is relatively simple to perform, but formality is a style that is not concretely defined in a sentence, which is as challenging as the author style transfer task, i.e., Shakespeare-to-modern transfer. Nevertheless, we are conducting additional experiments on this task. While it takes time to obtain quantitative results, we show some example outputs here:\n\n| Input | Output | Reference |\n|------|------|------|\n|Wisely and slow .| Do Wisely and also slowly . |Go wisely and slowly .|\n|What hast thou found ?|What have you found out ? |What have you found out ?|\n| And is he a man to encounter Tybalt ? | Is he man enough to see Tybalt ? | Is he man enough at this point to face off with Tybalt ?\n\nApart from formality transfer, we believe our prompt-based editing approach has the potential to perform other complex style transfer tasks, such as author style transfer.\n\nWe answered other questions individually. \n\n**Part 3, Point 2**: “This also in a way suffers from error accumulation in the same way that prompting-based approaches do. If the output at t is non-optimal, the algorithm from timestep t will deteriorate more with time suffering exactly the same problem as prompting-based approaches. The suggestion that the error accumulation problem is alleviated using examples in Table 6 is simply not valid evidence of this. More experiments are required to justify this claim.”\n\nWe acknowledge that our edit-based approach still occasionally encounters the error accumulation issue where non-optimal sentences are chosen for future search. However, our approach still greatly alleviates the issue compared with the prompt-based generation, where an LM performs the next word prediction simply based on its categorical prediction distribution without any explicit control over the quality of generated sentences. On the contrary, our approach applies search objectives for explicit control during the edits of sentences, and this has been proved to be effective in numerous previous works [Kumar et al., 2020; Li et al., 2020; Liu et al., 2020]\n\n\n**Part 3, Point 4**: “It is computationally more expensive and potentially a lot slower and practically infeasible. At every time step t there are 4 extra scoring prompts (one for each edit type) to generate the next token, and this might dramatically increase inference time. I do not see any discussion about the feasibility of this in a practical setting or how it compares computationally to other models.”\n\nFirst, we would like to point out that there is a misunderstanding. We only use one prompt for an off-the-shelf PLM to perform style classification (Line 225), and we do not use any other prompts to perform the edit types. \n\nSecond, we mention in the limitation section that our approach can be implemented in a highly parallel manner when evaluating different candidates, and the search algorithm only needs a few iterations, so our approach is more efficient than some stochastic search algorithms requiring several hundred search steps. (Lines 602–612)\n\n**Part 4, Q2**: “Is this extendable to the setting where a style is not concretely defined? For example, if a style is represented as a few-shot scheme i.e. a group of sentences of one style, is it possible to then come up with an accurate style score?”\n\nYes, it is extendable. In fact, formality is a style that is not concretely defined and is thus a challenging task. We mention in Lines 451–456 that we use four exemplars as a few-shot scheme. Specifically, we group two formal and two informal sentences together and feed them to an off-the-shelf PLM to help it understand what “informal” and “formal” mean. The results in Table 2 show that it is possible to give an accurate style score.\n\n**Part 4, Q3**: “Why wasn't a non-generative classifier considered to produce the style score? This would need training but it might have been able to perform better in a low-resource setting.”\n\nIn our approach, PLM-based search objectives are introduced to perform text style transfer without any training samples or labels (Lines 60–64), allowing us to achieve remarkable performance. However, if we replace the PLM with another style classifier, such as a BERT-based model, it would require a large labeled dataset to finetune the classifier and may achieve a better performance, but this does not constitute a fair comparison to previous prompt-based methods.\n\n**Part 4, Q4**: “Is there any other work that uses an edit search algorithm in a generative process similar to this?”\n\nYes. However, different from previous work in the edit-based generation, our approach designs a prompt-based classifier, which serves as a style scorer during discrete search and does not require any training procedure (Lines 262--265). Further, we involve an appropriate modification of the search algorithm and use the steepest-ascent hill climbing variant as well as a stopping criterion to make the style transfer process more controllable.\n\n**Part 5, Point 1** “The writing in this paper can be substantially improved. Generally in the punctuation, grammer and style of writing too sometimes. Please consider going through all the text again and trimming now excess punctuation.”\n\nWe thank the reviewer for the suggestion. However, we would like to point out that this paper has been carefully proofread by multiple professionals and native speakers, and we could not easily find format issues or errors in grammar and punctuation. In addition, we wrote the paper following the guidelines in [1], and we did not use excess punctuation. If the reviewer could give any concrete examples of grammatical errors or excess punctuation, we would be happy to address them in the revision.\n\n[1] Strunk, William Jr., and E.B. White. The Elements of Style. 4th ed., Longman, 2000.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea is interesting and seems novel. It is an intuitive way of breaking down the style transfer problem and guiding the tokens appropriately at every time step t.\n2. It shows to perform the best experimentally.\n3. The main reason I think this might be helpful to the community is simply the idea of using a score to greedily determine the next token. This simple idea might be borrowed in future work to aid generation, not necessarily in the context of style transfer.", "reasons_to_reject": "1. The approach has an inherent flaw in that manually tuning the mean edit distance is needed for every style/dataset. Therefore it might not be usable at all in the situation in a low resource setting and relies too much on fine-tuned hyperparameters. These max token thresholds are fundamentally problematic, especially in the case of a complex style which does not involve simple word edits and we can assume that the output length ~ input length. Perhaps there is a more elegant want to determine these rules and give the model more flexibility in sentence structure and length\n\n2. This also in a way suffers from error accumulation in the same way that prompting-based approaches do. If the output at t is non-optimal, the algorithm from timestep t will deteriorate more with time suffering exactly the same problem as prompting-based approaches. The suggestion that the error accumulation problem is alleviated using examples in Table 6 is simply not valid evidence of this. More experiments are required to justify this claim. \n\n3. It is only feasible for simple styles which rely on simple token edits. This approach might work well in a zero-shot setting for more complex styles as well. However this cannot be assumed, and this work only tests two very simple styles.\n\n4. It is computationally more expensive and potentially a lot slower and practically infeasible. At every time step t there are 4 extra scoring prompts (one for each edit type) to generate the next token and this might dramatically increase inference time. I do not see any discussion about the feasibility of this in a practical setting or how it compares computationally to other models.\n\n5. The underlying greedy assumption that any output at a given timestep t seems like an over assumption and can fail for more complex styles. It relies solely on the fact that the style score at timestep t is an accurate oracle in informing all future decisions. This might fail in the context of a style that contains long-term logical dependencies in a more bidirectional work. Style and formality are more time invariant in that sense and so this approach is more suited to this kind of setting.\n\n6. The style considered i.e. sentiment and formality are outdated and too simple to justify SOTA performance in the style transfer task.", "questions_to_authors": "1. Please address my questions above regarding the accuracy of the method i.e. whether it will be feasible with more complex timestep variant styles (say author style or discourse)\n2. Is this extendable to the setting where a style is not concretely defined? For example, if a style is represented as a few-shot scheme i.e. a group of sentences of one style, is it possible to then come up with an accurate style score?\n3. Why wasn't a non-generative classifier considered to produce the style score? This would need training but it might have been able to perform better in a low-resource setting.\n4. Is there any other work which uses an edit search algorithm in a generative process similar to this?", "paper_topic_and_main_contributions": "This paper introduces a discrete token edit algorithm to generate a token at every timestep t using a lightweight PLM to construct a sentence of the target style. In this greedy approach, the best token edit is chosen w.r.t a style score predicted by the underlying PLM for that phrase at time t. Hence a typically generative process for style transfer is converted into a sort of greedy search without training the underlying model.\n\nThis work addresses the error accumulation problem commonly seen with simple prompt-based approaches. It experimentally seems to perform the best w.r.t its SOTA baselines. \n\nThe approach is simple but effective-seeming and a novel direction for performing token generation.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None come to mind."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The writing in this paper can be substantially improved. Generally in the punctuation, grammer and style of writing too sometimes. Please consider going through all the text again and trimming now excess punctuation."}}, {"review_id": "F32PiszW6O", "reviewer": "Reviewer_xk33", "summary": "", "questions": "1.\tWhy do the authors choose the three factors – style, fluency, and semantic similarity? They are necessary. But are they enough?\n2.\tWhat will happen if the authors replace the PLM with another style classifier (eg., BERT-based models) even though prompt-based classifier is currently popular?\n", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**General Response**:\n\nWe thank the reviewer for the detailed feedback and questions, and for saying that “the statement of the methodology is good”!\n\nWe surmise that a general theme that the reviewer’s questions revolve around is the novelty of our work, such as a similar editing framework used in [Kumar et al., 2020], the PLMs and prompt-based editing seem not so important in the approach, and the method is like the Prompt & Rerank method.\n\nWe acknowledge that our work follows some practice in previous work, but still provides new insights into the prompt-based paradigm of text style transfer. We first design a prompt-based classifier for style classification, which is recognized by the reviewer. Further, we involve an appropriate modification of the search algorithm and use the steepest-ascent hill climbing variant as well as a stopping criterion to make the style transfer process more controllable. The search algorithm is not only very important in our approach, as shown in Table 5, but also a substantial novelty compared with previous prompting literature (e.g., Prompt & Rerank) that is mainly about prompt engineering. We sincerely hope that this difference is able to set us apart from other literature enough to make a meaningful contribution to this line of work.\n\nWhile the reviewer points out that the SAHC algorithm, which is one of the main contributions, plays a significant role in our prompt-based editing approach, it is concerned that the PLMs here seem not so important. In actuality, PLMs play an integral part in our approach along with SAHC. We use PLMs to measure different aspects of the search objectives. In particular, the prompt-based classifier also serves as a style scorer, being a novel insight into our work. The contributions of different PLM-based scorers are shown in Table 4, and detailed in “Ablation Study” in Section 4.5 (Lines 500–516). Therefore, we believe that both the SAHC algorithm and the PLMs are contributive enough to our prompt-based editing approach. \n\nWe answered other questions individually. \n\n\n**Part 3, Point 2**: \"The completeness of the work cannot cover the weak flaws in the core content...Sentence editing seems more controllable than generation. However, we can use it to do the sentiment and formality style transfer work. But it may be difficult to do a modern-to-Shakespeare style transfer task...\"\n\nWe want to point out that formality is a style that is not concretely defined in a sentence, which is as challenging as the Shakespeare-to-modern task. Nevertheless, we are conducting additional experiments on it. While it takes time to obtain quantitative results, we show some example outputs here:\n\n| Input | Output | Reference |\n|------|------|------|\n|Wisely and slow .| Do Wisely and also slowly . |Go wisely and slowly .|\n|What hast thou found ?|What have you found out ? |What have you found out ?|\n| And is he a man to encounter Tybalt ? | Is he man enough to see Tybalt ? | Is he man enough at this point to face off with Tybalt ?\n\nOverall, our approach has the potential to perform the Shakespeare-to-modern transfer task.\n\n**Part 4, Q1**: “Why do the authors choose the three factors – style, fluency, and semantic similarity? They are necessary. But are they enough?”\n\nWe follow [1-2] and use these three factors to evaluate the quality of sentences comprehensively. We think they are enough as these factors are believed to be a common practice of evaluation [3], and we will clarify this point in the revision.\n\n[1] Mirac Suzgun, Luke Melas-Kyriazi, and Dan Jurafsky. 2022. Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models. In EMNLP, pages 2195–2222.\n\n[2] Jingjing Li, Zichao Li, Lili Mou, Xin Jiang, Michael Lyu, and Irwin King. 2020. Unsupervised text generation by learning from search. In NeurIPS, pages 10820–10831.\n\n[3] Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep Learning for Text Style Transfer: A Survey. Computational Linguistics, 48(1):155–205.\n\n\n**Part 4, Q2**: “What will happen if the authors replace the PLM with another style classifier (eg., BERT-based models) even though prompt-based classifier is currently popular?”\n\nIn our approach, PLM-based search objectives are introduced to perform text style transfer without any training samples or labels (Lines 60–64), allowing us to achieve remarkable performance. However, if we replace the PLM with another style classifier, such as a BERT-based model, it would require a large labeled dataset to finetune the classifier and may achieve a better performance, but this does not constitute a fair comparison to previous prompt-based methods.\n\nWe thank the reviewer again for the detailed comments, and will thoroughly revise our paper according to the suggestions provided. We hope our response could alleviate all the concerns and are looking forward to your stronger support!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe goal of this paper to generate a sentence with a different style is very clear and the path to solving the problem is also very clear.\n2.\tThe idea of using the PLM as a style classifier is relatively new to some extent, even though it follows the same routine as the authors’ previous work in sentence simplification [1].\n3.\tThe statement of the methodology is good. The implementation is comprehensive. The results are solid.\n\n[1] Dhruv Kumar, Lili Mou, Lukasz Golab, and Olga Vechtomova. 2020. Iterative Edit-Based Unsupervised Sentence Simplification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7918–7928.\n", "reasons_to_reject": "1.\tThe work is old wine in a new bottle. The core is the SAHC algorithm to ‘enumerate every editing position and perform every editing operation’. We can see from Table 5 that it plays a very significant role. The PLMs and prompt-based editing here seem not so important.\n2.\tThe completeness of the work cannot cover the weak flaws in the core content. This is essentially a sentence editing work rather than a sentence generation work. Sentence editing seems more controllable than generation. However, we can use it to do the sentiment and formality style transfer work. But it may be difficult to do a modern-to-Shakespeare style transfer task. I don’t mean this is not a good work. It just tends to be conservative, especially there’s already similar framework used in their previous work.\n3.\tThe method of this paper is like the Prompt & Rerank [2] method, while the latter is more generation-based.\n\n[2] Mirac Suzgun, Luke Melas-Kyriazi, and Dan Jurafsky. 2022. Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2195–2222.\n", "questions_to_authors": "1.\tWhy do the authors choose the three factors – style, fluency, and semantic similarity? They are necessary. But are they enough?\n2.\tWhat will happen if the authors replace the PLM with another style classifier (eg., BERT-based models) even though prompt-based classifier is currently popular?\n", "paper_topic_and_main_contributions": "This paper proposes a combination of modules to do text style transfer. The authors apply an edit-based search method, the steepest-ascent hill climbing (SAHC),  to generate a series of candidate sentences by inserting, deleting, and replacing words of the original sentences. And they choose the best candidate by defining a scoring function with three aspects – style (using the GBT-J-6B model), fluency (using the GPT2 model), and semantic similarity (using the RoBERTa model). From the experiments, the proposed method outperforms three PLM-based baslines.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nbeXr7raBp", "reviewer": "Reviewer_bmvG", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reading our paper and providing valuable feedback. Additionally, thank you for saying our approach is “novel” and our experimental analysis is “detailed and comprehensive”!\n\n**Part 4, Q1**: “How to use the search method to generate candidate text is not very clear. It is recommended to provide a more detailed description.”\n\nWe thank the reviewer for the valuable suggestion. Should the paper be accepted, we will make sure to explain in more detail about our search method in Section 3.3 and highlight it in Algorithm 1 as well.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.The prompt-based editing method proposed in this paper can be applied to similar problems that involve generating outputs using large models, particularly in scenarios where the generated outputs need to mostly retain consistency with the input. It has a certain degree of universality.\n2.The experimental results of this paper demonstrate excellent performance, and the analysis provided is also detailed and comprehensive. ", "reasons_to_reject": " How to use the search method to generate candidate text is not very clear. It is recommended to provide a more detailed description.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on how to better utilize large langeuage models for text style transfer. The specific approach involves using prompt-based editing approach as alternatives to traditional generation methods. \n\nThe contributions are that this paper proposes a novel prompt-based editing approach to text style transfer that turns a prompt-based generation problem into a classification one. It does not suffer from the issue of error accumulation and is more controllable than the autoregressive generation. The experiments on three benchmark datasets show that the proposed approach significantly outperforms the state-of-the-art prompting systems.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents an editing mechanism which first prompts a PLM to extract style class of a given text, which is then used along with word-level editing for style transfer. The motivation is claimed to be a better more control during the style transfer phase, in contrast with conventional prompt-based style transfer solutions. On 3 style-transfer benchmarks, the paper outperforms previous SotA. The discussion with xk33 and PLiK clarified some of the missing steps and better contextualised the contribution of the work compared to previous sentence-editing literature. I encourage the authors to take these suggestions on board.
Decision: Accept-Findings