Prefix-Tuning Based Unsupervised Text Style Transfer
Authors: Huiyu Mai
Wenhao Jiang
Zhi-Hong Deng
Venue: EMNLP 2023 Findings (2023)
Keywords: Unsupervised text style transfer
Prefix-Tuning

Abstract: Unsupervised text style transfer aims at training a generative model that can alter the style of the input sentence while preserving its content without using any parallel data. 
In this paper, we employ powerful pre-trained large language models and present a new prefix-tuning-based method for unsupervised text style transfer. 
We construct three different kinds of prefixes, i.e., shared prefix, style prefix, and content prefix, to encode task-specific information, target style, and the content information of the input sentence, respectively.
Compared to embeddings used by previous works, the proposed prefixes can provide richer information for the model.
Furthermore, we adopt a recursive way of using language models in the process of style transfer.
This strategy provides a more effective way for the interactions between the input sentence and GPT-2, helps the model construct more informative prefixes, and thus, helps improve the performance.
Evaluations on the well-known datasets show that our method outperforms the state-of-the-art baselines. Results, analysis of ablation studies, and subjective evaluations from humans are also provided for a deeper understanding of the proposed method.

Prefix-Tiie Prefix-Tuning Based Unsupervised Text Style Transfer
Huiyu Mai Wenhao Jiang? Zhihong Deng'
Peeking University
PGuangming Laboratory
eswhjiang @ gmail.com, zhdeng pku.edu.cn
Abstract
tangled representations of style and content. They usually use recurrent neural networks (RNNs) or Transformer (Vaswani et al., 2017) to encode style and content information of the input sentence with two independent embeddings. And the decoder takes the content embedding and the target style embedding as inputs to generate a new sentence with the expected content and style. But (Lample et al., 2018) has proved that it is not necessary separate the style and content from the input sen- tence. The specially designed model can generate text with the target style by overwriting the original style. Therefore, we adopt a similar strategy in our model and directly feed the sentence and the target style information into the model for style transfer.
Unsupervised text style transfer aims at train- ing generative model that can alter the style of the input sentence while preserving its content without using any parallel data. In this paper, we employ powerful pre-trained large language models and present a new prefix-tuning wwt--ttnnnnnssee method for unsupervised text style transfer. We construct three different kinds of prefixes, i.c.. shared prefis, style prefix, content prefix, to encode task-specific information, target style, and the content information the input sen- tence, respectively. Compared embeddings used by previous works, the proposed prefixes can provide richer information for the model. Furthermore, adopt recursive way of us- ing language models in the process of style transfer. This strategy provides a more effec- tive way for the interactions between the input sentence and GPT-2, helps the model construct more informative prefixes, and thus, helps im- prove the performance, Evaluations the well- known datasets show that our method outper- forms the state-of-the-art baselines. Results, analysis of ablation studies, and subjective eval- uations from humans are also provided for deeper understanding of the proposed method.
In previous works, the encoder encodes the sen- tence into fixed-size, such as 768-dimensional latent representation, which is small and hard capture the full information of the sentence, lead- ing to information loss. Thus, models without en- coding the sentence to latent representation, e.g.. decoder-only model, might be a better choice for the task of text style transfer. Recently, pre-trained auto-regressive language models have shown great success in text generation. Both GPT (Radford et al., 2019; Brown et al., 2020) and LLaMA (Tou- vron et al., 2023) use the decoder part in Trans- former as their model framework. However, their ability in the task of UTST has not been fully inves- tigated before. In this paper, we design a novel model to exploit auto-regressive model's high- quality text generation ability to achieve text style transfer. Moreover, to avoid damaging the text gen- eration ability, we adopt prefix-tuning paradigm (Li and Liang, 2021).
1 Introduction
Style transfer is an important task in computer vi- sion and natural language processing. It aims to convert the input into the target style (e.g., season artist style image and sentiment or formality style text) while preserving its content informa- tion. In addition, text style transfer, the model also needs to ensure the fluency of the output sen- tence. Due to the lack of parallel data, the studies of text style transfer tasks mainly focus on unsuper- vised settings.
According to the above objectives, propose novel prefix-tuning framework based on GPT- for unsupervised text style transfer. Following previous works in the area of UTST (Zhu et al., 2017), we also employ the framework of adversar- ial learning due to the lack of parallel data. Thus,
The mainstream unsupervised text style transfer (UTST) methods based on the idea of disen-
The work was done when the author was with Tencent as un intern.
Corresponding author

---

our framework contains generator and discrimi- nator. In the generator, three prefixes, i.c., shared prefix, style prefix, and content prefix, are proposed, and each of them serves different roles. The shared prefix provides task-specific information, which is independent of the input. The style and content prefix provide input-dependent information. The style prefix encodes the target style, and the content prefix extracts the content information in the input sentence. These prefixes provide information for GPT-2, guiding the model for style transfer. For the content prefix, no extra encoder is used. We simply use the pre-trained language model recursively to obtain the content prefix. In our model, the dis- criminator is also based on prefix-tuned language model. Therefore, only fraction of the parameters of the whole model needs to be trained.
Sudhakar et al., 2019) follow this pipeline, and they also employ transformer to improve performance. Pscudo-parallel corpus construction methods use sentence embedding similarity to find sentence pairs with similar content but different styles from the corpus. Some works like LaMer (Liu et al., 2022) introduced scene alignment and reinforce- ment learning improve performance. Other work (Luo et al., 2019) also used RL for UTST. IMaT (Jin et al., 2019) proposed the Iterative Matching and Translation method to construct a parallel cor- pus with high quality.
Disentanglement methods focus on separating the style and content of the input sentence. Cross- alignment (Shen al., 2017) used variational auto-encoder and style classifier encourage learning style-independent content space. The reverse attention method was applied the paper (Lee et al., 2021) to enhance content preservation. Some works also focus on adding the target style to the representation of the input sentence, like using trained classifier to edit the latent representation obtained by the encoder through gradient-basee optimization (Wang et al., 2019; Liu et al., 2020). Since the ability of transformer has been demon- strated in many fields, some studies (Dai et al.. 2019; Fan et al., 2022; Reif et al., 2022) used transformer-basee
The contributions of our paper can be summa- rized as follows:
We propose new method based on prefix- tuning and pre-trained language model for unsupervised text style transfer. It is more expressive than previous methods, and only 2% of the parameters need to be trained.
A strategy of recursively using language model to extract content information as con- tent prefix that will be fed into the same LM for generation is proposed, which provide effective way for the interactions between trainable and frozen parameters.
mncoder-decoder models. Style Transformer (Dai et al., 2019) is the most related work to ours. It treated the target style as extra token, which is added to the input sentence, and the model modified the sentence according the target style. However, it does not exploit the power of pre-trained models.
Experiments conducted on well-known datasets show that our proposed method outperforms the state-of-the-art baselines. Evaluations by human and ablation stud- ies are also provided to obtain a deeper understanding of our method.
2.2 Prompt Learning Methods
Our method is also related to prompt learning, which is a class of methods that learn the input prompt with the pre-trained language models kept frozen. Prompt-leamingg based methods can exploit the knowledge obtained from the large-scale train- ing data during the pre-training process. Since only the parameters of prompt are trainable, the num- ber of parameters of prompt learning methods is usually quite small.
2 Related Work
In this section, we present brief summarization of topies related our method.
2.1
Unsupervised Text Style Transfer
The existing UTST methods can be classified into three categories, i.e., prototype editing, pseudo- parallel corpus construction, and disentanglement method. Here give a brief description of them. A pipeline called Delete, Retrieve and Generate for prototype editing was proposed in (Li et al., 2018), which designed a process to revise the style markers in the target sentences. Works (Lee, 2020;
P-tuning was proposed in (Liu et al., 2021), and it enables GPT-2 to solve natural language under- standing(NLU) tasks by employing trainable con- tinuous prompt embeddings. It can automatically search prompts in the continuous space to bridge the gap between GPT-2 and NLU applications. Rather than using embeddings, Prefix- Prefix-tuning (Li

---

GPT-
CPT-2
Fuind HEASERA
Juck
Aupietion Lagut
Geueratio Network fx(X,F)
Dicemmsator Network da(
Figure 1: The architecture of the generator and discriminator. For the generator on the left, the shared and style information is first initialized virtual token embeddings and then mapped prefixes. The style embeddings are fed into FFN(-.)) together and passed into the projection layer to generate PRE Then PREpre and the sentence are used as the input GPT-2 to generate content prefix. Finally, the GPT-2 takes the three prefixes and X as input for style transfer. The discriminator takes the prefix and X as input and predicts its style s. The orange blocks represent trainable embeddings and networks in our model. The pink "Projection Layer" blocks denote shared trainable parameters.
and Liang, 2021) uses key-value pairs in Trans- former as prefixes, as a lightweight alternative to fine-tuning for the generation tasks, e.g., transla- tion, summarization, and table-to-text. It uses small number of trainable parameters to provide task-specifie information for pre-trained language models, while still achieving comparable perfor- mance to fine-tuning the entire model.
3.2 Model Overview
We use GPT-2 (Radford et al., 2019) as our back- bone. Rather than using the encoder-decoder model such RNN BART (Lewis al., 2019) and T5 (Raffel et al., 2020), choose decoder-based model GPT-2, so as encode the style and input text into prefixes. The overall framework of our model is shown in Fig. 1.
Prompt-lcaring based methods have never been investigated for the setting of unsupervised text style transfer. this paper, aiming exploiting the knowledge in the pre-trained language models, we propose a new prompt-learning based method for unsupervised text style transfer.
Like other UTST models, our model also adopts the framework of adversarial learning. As shown in Fig. 1, our model consists of a generator network and a discriminator network. The generator takes the input sentence X and expected style gen- erate the output sentence Y, and it expected that the style of 8. The role of the discriminator is to predict the style the input sentence.
3 The Proposed Method
In the generator, a special token, called style to- ken 8, is employed to provide the style information. Like other tokens, the style token is first trans- formed into an embedding, which is single vector, with embedding table. To obtain the style prefix for the language model, the style embedding is pro- jected into L vectors with projection layer, which stands for the prefixes for the pre-trained language models with I. layers. To provide task-specific in- formation for the pre-trained language model, we design N shared token to achieve this goal. Simi- larly, the shared tokens are first transformed into N embedding vectors, and then each of them is pro- jected into prefixes for the L layers. To reduce the number of parameters and facilitate the interaction among trainable parameters and frozen parameters, the style prefix and input sentence are fed into the same pre- trained language model to generate the
this section, we will describe the task settings of unsupervised text style transfer briefly and then present our method.
3.1 Problem Formalization
this paper, we denote training set as D [(X,, s) where X, = I1, T2, In is a sen-
tence, n is the sentence length, and s; E (0, 1} is the corresponding style label. The goal of text style transfer is to change the style of a sentence and keep its content. Specifically, the input sentence X with style is converted into new sentence Y with target style with model fe(X, 3). In the unsupervised setting, there no paired data (X, Y) given, and the goal is to obtain the model with only unpaired training data D.

---

content prefix. In the discriminator, M discrimi- nator tokens are designed to provide information necessary for discrimination, and the procedure of prefix generation is similar to the shared prefix in the generator. The details of the proposed genera- tor, the discriminator, and the training strategy will be described in the following subsections.
to all Transformer activation layers and rightward to imput tokens. Similar to constructing the shared prefix, we use the projection layer to map the style embeddings to style prefix PREstyles where the style embeddings are derived from target styles.
3.3.3 Content Prefix and Recursively Applying LM
3.3 Tuningg Based Generator Network
The role of the content prefix is enhance the con- tent information for the generation process. Liven if the original sentence is also fed into the pre-trained language model, we found that adopting the pro- posed context prefix can improve the performance, which will be shown in the ablation study section. The details about the generation of prefixes can also be found in Fig. 1. We can see that the same pre-tren mre-trained model takes the PREpre and the input sentence X inputs, and the sentence tokens from the last layer are passed through a projection layer to produce the context prefix PREcmtent. The PREpre is a representation of style information. The embeddings of all styles are concatenated and are passed through a weed-forward neural network to reduce the dimensionality, followed by a pro- jection layer to yield PREpre. Finally, the three prefixes PRE PRE and PREeeteent are concatenated together as the prefix of the GPT-2 model to generate the sentence Y.
The overall architecture of our generator Y fo(X, 8) is shown in Fig. 1. Three different pre- fixes, i.e., shared prefix, style prefix, and content prefix, are proposed to guide the style transfer of GPT-2. The prefix is the input, excluding the sen- tence tokens, for a certain layer of the pre-trained language model. Unlike the sentence tokens in the I-th layer are from the / - 1-th layer, the proposed prefix for the I-th layer is from the projection layer. We will describe them in detail.
3.3.1 Shared Prefix
The role of the shared prefixes is to provide the pre-trained language model with task-related in- formation. For specific text style transfer task son different datasets, the shared prefixes expected capture different information needed by the pre- trained language model.
In our model, the shared prefix is generated from N shared tokens, denoted by ..., TN. Similar to the ordinary tokens, the N shared tokens are first transformed into N shared embeddings, which are then projected into N shared prefixes, denoted as PREghared- with a feed-forward neural network. Each shared prefix L is the number layer of the pre-trained language model. The I-th prefixes vector is the input for the I-th layer. Note that the prefix for the 1-th layer is not from the / - 1-th layer of the language model. Instead, it comes from the projection layer directly.
3.3.4 Generator Losses
Three different losses are adopted for the genera- tor, including relf-reconttruction loss, style transfer loss, and cycle generation loss. Fig. shows the cal- culation of losses for our style transfer model. The training process of our model similar to Cycle- GAN (Zhu et al., 2017) and Style Transformer (Dai et al., 2019), which are commonly used in unsuper- vised style transfer.
set of L vectors, where
Self reconstruction loss. The generator takes the sentence X and its original style as input so that we can train the generator to reconstruct the in- put sentence. This helps the model to maintain sentence content as much as possible, and the cor- responding loss function can be expressed as:
3.3.2 Style Prefix
previous works, the target style would be con- catenated to the input sentence as a special token (Dai et al., 2019), or converted into latent represen- tation and fed into the decoder (Lee et al., 2021). Unlike them, the target style is converted into pre- fix and then fed into the generator in our method. Thus, the component for generating style prefixes contains more trainable parameters. Hence, it is more expressive. Therefore, using prefixes rather than embeddings can provide more information of style and is more beneficial to style transfer. The effects of the prefixes will be propagated upward
L Pog (X),ss)
(1)
where Po() is the distribution over sequences from the generator.
Style transfer loss. The style quality of the gen- crated sentence is supervised by the discriminator, Specifically, the sentence X and the target style $

---

vision for the generator. Specifically, if the dataset contains two styles, the discriminator is a classifier with three classes. The first two classes are the two styles, and the last class labeled as 8 fake represents that the sentence comes from the generator fo(:;) )).
sX,
s
(x)
styyte
Y
The architecture of (x) is shown on the right side of Fig. 1. To maintain the consistency of gener- ation and discrimination, the discriminator (x) also based on GPT-2 and prefix-tuning. It takes the prefix dRE and X as input, where dRE is generated from token embeddings and projection layer just as the shared prefix. Then the average value of the last hidden layer is passed into a linear layer to calculate the probability that the sentence belongs to each category.
SX,
Leyclo
(Y,
An illustration of losses adopted by the gen- erator. The input sentence is transferred X on the top and Y on the right for calculating Caels and Latyles Y is passed into the generator again generate Y for calculating Leyele-
Figure 2:
3.4.2 Discriminator Loss
are fed into the generator. Then the discriminator dø aims maximize the probability that the gener- ated sentence belongs to the target style, and the corresponding loss can be expressed:
During training, the sentence X comes from either the corpus or the generator. If X comes from the corpus, the target Starget is the style of the sentence. If X is generated by the generator, the target is fake. The loss function defined follows:
Sstgle -logP (s)Y).
(2)
Cycle reconstruction loss. Except for the above losses, we also adopt the cycle reconstruction loss to encourage the generator to preserve the content. Specifically, Y and the original style s are fed into the generator to generate Y. Thus, Ý is expected to be the same as X. So the loss is expressed as
Lais log Po(gPoottreee X),
(5)
where Pa is the conditional distribution over styles obtained from the discriminator da (X). Finally, the discriminator is updated while the generator remains frozen. Similar to generator training, the trainable parameter of the discriminator e only in- cludes the prefix part.
Layele log (X
The final loss is the weighted sum of the above three losses, which is
3.5 Training Strategy
During the learning process, the generator and dis- criminator are trained alternately. In our imple- mentation, the discriminator is trained for ten steps, and the generator is updated for five steps in each iteration. Since prefix-tuning is employed, only the parameters for constructing prefixes will be up- dated, which account for a tiny part of the total model parameters.
Egen = -A + AgCrydde + Aghstyles
(4)
while X; is the weight of each loss. Both Leelf and Logdl help the style transfer model to achieve con- tent preservation, and Leef also makes the model output as fluent as possible. And Lstyle helps mod- els perform style transfer better.
Notice that the above three loss functions are only used to update the parameters of the gener- ator, while the discriminator's parameters remain frozen. Since contains only the parameters used to construct the prefixes, we only need to train and save only 2% of the parameters the pre-trained language model.
4 Experimental Results
4.1 Datasets
We evaluate our method on two commonly used review datasets, i.c., the Yelp Reviews Dataset (Yelp) and the IMDb Movie Review Dataset (IMDb)² (Maas et al., 2011). The Yelp dataset is a restaurant and business reviews dataset with
3.4 Discriminator Network
3.4.1
Discriminator Structure
The discriminator aims to classify the sentence style from the generator, which will provide super-
https://www. wwwp://wwwwwlll.... www..yelpp.ooooom com/dataset Phttp://at stanford, www.aa/daaaaaseeeaaaaaaaaaa

---

Yelp
IMDb
Measure of fluency. The perplexity (PPL) of the generated sentence is adopted as the fluency metric. The model with lower PPL can generate more flu- ent sentences. We achieve this by training a 3-gram language model on the training set two datasets using KenLM (Heafield, 2011).
Dataset
Positive
Negative
Positive
Negative
Train
266,041
177,218
178,869
187,597
Dev
2,000
2,000
2,000
2,000
Test
500
500
2,000
2,000
Table 1: Statistic for Yelp and IMDb datasets.
4.2.2 Human Evaluation
sentiment labels provided by the Yelp Dataset Chal- lenge. The IMDb dataset consists of movie reviews written by online users. Both datasets contain two styles, i.e., positive sentiment and negative senti- ment. We followed the pre-processing procedure of Style Transformer (Dai et al., 2019) and used only highly polar sentences to train the model. The statistics of the two datasets after pre-processing are shown in Table 1.
Many previous works (Mir al., 2019; Jin et al., 2022) have claimed that automatic evaluation meth- ods have some shortcomings. Therefore, we also conduct human evaluation to evaluate the results more accurately. For each dataset, we randomly select 200 sentences (100 for each class) from the test set. We will provide four annotators with the original sentences, the target style, and the output sentences of our model and other models. The an- notators are asked to score the model output by considering the three aspects, i.e., style, content, and fluency. Scores for each aspect range from (lowest) to (highest). We use the average scores as the final scores.
4.2 Evaluation Metries
Following the previous works (Dai et al., 2019; Fan et al., 2022), evaluated our generated sentences from three perspectives, i.e., style, content, and fluency. In addition to the automatic evaluations, we also compare the performance of our method with other methods through human evaluation.
4.3 Implementation Details
We use GPT2-large" as our base generator and dis- criminator, which contains 36 layers, 20 attention heads, and an embedding size of 1280. The shared and the style prefix lengths are 10 and 20, respec- tively. The size of our virtual token embedding is 1280, the same as the embedding size of GPT- 2. The projection layer is a two-layer MLP with a hidden size of 128. We train our model on an NVIDIA Tesla V100 GPU with a batch size of 8, The Adam optimizer (Kingma and Ba, 2014) is used, and the learning rate is set to 0.0001. We tested different weights of the three losses on the validation set find the best weight ratio. The ex- periments showed that our method is not sensitive to the three trade-off weights. The final weights of relftrrcontttuttioo loss, cycle loss, and classifica- tion loss are set to 0.25, 0.5, and 1.0, respectively.
4.2.1 Automatic Evaluation
Measure of style. We measure the ability of style controlling of competing models by evaluating the accuracy of commonly used classifier on the task of predicting the styles of generated sentences. Us- ing fastText (Joulin al., 2016), we train two clas- sifiers for Yelp and IMDb on their training set, re- spectively, which are used the compute the style classificatiin accuracy. A higher accuracy indicates that more output sentences have the same style as the target, which means that the model can transfer the sentence style better. The accuracy obtained is denoted as ACC in the rest of this paper.
Measure of content. We evaluate content preser- vation by calculating the BLEU score (Papineni et al., 2002) between the generated sentences and the original input sentences. A higher BLEU score means that the generated sentence retains more of the words in the original sentence, which indicates that the generating model achieves better content preservation. We also calculated the BLEU score between the generated sentence and the correspond- ing reference sentence for the Yelp dataset, which is provided by human. These two BLEU score met- rics are referred to as self-BLEU and ref- -EEUBBLEURA in the rest of the paper, respectively.
4.4 Experimental Results
4.4.1 Automatic Evaluation
We compare model with other disentanglement and prompt-based methods, including Deep Latent (He et al., 2020), Style Transformer (Dai et al., 2019). RACoLN (Lee et al., 2021), and LaMDA (Reif et al., 2022). The automatic evaluation re- sult in Table shows that our method significantly improved content retention and fluency.
hhttps://hugiingface.u co/ co/gpt2-large

---

Yelp
IMDb
Model
ACC(%)f
ref- BEf-BEUU
Belt-BLBUT
PPL.
ACC(%)
self Biff-BEUT
PPL.
Deep Latent
83.3
18.0
50.9
153
77.9
60.8
147
Style Transformer
83.7
20.1
58.7
148
78.6
66.1
145
RACOLN
84.9
20.3
58.2
142
79.5
70.9
133
LaMDA
90.6
8.3
20.4
79
/
Ours
84.3
21.9
59.3
139
81.0
74.3
130
Table Automatic evaluation results on Yelp and IMDb datasets. ACC the accuracy of the style classifier on the generated sentences. ref-BLEU and self-BL self-BLEU are the content preservation metrics calculated by NLTK. PPL is the perplexity the transferred sentences measured by KenLM.
Yelp
IMDb
helps it output more fluent sentences, improving the PPL metric. However, since these methods do not introduce any supervision, they perform poorly in content preservation.
Model
MyylCContencFluenccce
StyleTransformr
3.9
4.2
4.3
3,6
4.2
4.1
RACOLN
4.0
4.5
4.2
3.5
4,3
4.1
Ours
4.0
4.6
4.5
3,5
4.6
4.2
4.4.2 Human Evaluation
Table 3: Human Evaluation results both datasets. Each score is the average score from the four annotators. A higher score indicates better performance.
The human evaluation results are shown in Table 3. We choose two other methods compare with ours. As seen from the table, the human evalua- tion results are basically consistent with the auto- matic evaluation results. Our model outperforms the other two models in content preservation and fluency while keeping style control metrics close to those of the other two models. It can be seen from the results of the automatic evaluation and human evaluation that our method can better preserve the content of the original text and ensure the fluency of the output sentence while also achieving a good style transfer effect.
The style measuring metric ACC of our model on the Yelp dataset is competitive with other mod- els, and on the IMDb dataset, it is higher than the bascline models. We find that our method outper- forms other models style control on the IMDb dataset, which has a long average sentence length. There are two main reasons. First, we use prefix- tuning to encode the target style, which contains richer information than the latent representations used by other methods, and thus can better cope with longer texts. In addition, our discriminator also uses GPT-2, which can handle longer texts bet- ter than RNN, thus achieving better style control.
4.5 Ablation Study
Ablation studies are performed to investigate the impacts of each module on performance. We move each prefix by turns and retrain model while keeping other settings unchanged. To com- pare the performance using different ways to encode styles, a model using style embeddings in- stead of prefixes is also designed. Lastly, a model with all parameters fine-tuned was trained for com- parison with prefix-tuning. These models are eval- uated on Yclp, and the results shown in Table 4. Without the shared prefix, our model shows a slight drop style control and content preservation metrics. This indicates that it loses part of the task information because the shared prefix is a prompt of the whole task. In addition, the shared prefix also provides common information between different styles, which is also beneficial to style transfer. Removing the style prefix has a significant im- pact on style transfer. Without the style informa- tion, the model easily degenerates into a model that
Our model shows noticeable improvement in the BLEU score, especially the ref-BLEU score on the Yelp dataset, which is 1.6 higher than RACoLN, a relative improvement of 7%. And our self -elf-BLEU score on IMDb 3.4 higher than the baselines. We attribute this improvement to content prefixes. On the one hand, the content prefix contains richer information than the latent representation. On the other hand, by recursively using GPT-2, there is more sufficient interaction between the input text and the language model that it can obtain more content information. At the same time, since GPT2- large used, the outputs are more fluent and have lower perplexity.
LaMDA (Reif al., 2022) is zero-shot method based on large language model (L.LM). As can be seen in the table, achieves considerable ACC results. In addition, using large language model

---

Model
ACCT
r-BLEUt s-BLEUT PPL.
Negative Positive
Shared Prefix
82.1
21.2
55.7
152
Input
There chips are ok, but their salsa is really bland.
Style Prefix
25.8
23.3
82.5
130
Style Embedding
82.2
21.3
59.4
135
StyleTrans
There chips are ok, but their salsa is really outstanding.
Content Prefix
84.0
18.5
44.1
167
RACoLN
There chips delicious, and their salsa is really really.
Ours
84.3
21.9
59.3
139
Ours Full fine-tune
85.0
22.3
59.6
139
Ours
There chips are ok, and their salsa is really delicious.
Table 4:
Ablation studies on the Yelp dataset. (-) in- dicates that this part of the prefix is removed from model. r-BLEU and s-BLEU indicate ref-BLEU and self-BLEU, respectively.
Input
The wine very average and the food was even less.
StyleTrans
The wine very average and the food was even less delicious wine.
RACoLN
The wine was very average and the food is even good.
directly copies the imput sentence without modify- ing the style. Therefore, the model performs better on the BLEU score than the full model, with sig- nificant drop in the ACC score. The performance of the model using style embedding on ACC met- ric is worse than using the prefix, indicating that style prefix can provide more style information and improve the performance of the model.
Ours
The wine was very excellent and the food was even better.
Positive to Negative
Input
love this place, the service is always great! I guess this place, the service is not great!
StyleTrans RACOLN
I avoid this place, the service is always hor- rible!
Ours
I hate this place, the service is always bad!
After removing the content prefix, the self- BLEU score significantly drops by 15.1. There are two main reasons that the content prefix can improve performance. First, it provides more infor- mation about the input sentence for the generator, which helps the model retain content information better. Secondly, the input sentence will be passed into GPT-2 twice during style transfer, which al- lows GPT-2 to fully extract content information and achieve better content preservation.
Input
The best mexican food in the phoenix area. The this mexican food in the phoenix area. The great mexican food in the phoenix area. The worst mexican fond in the phocnix area.
StyleTrans RACoLN
Ours
Table 5: Some examples generated by the models Yelp dataset. The red words indicate the grammar error. The blue words indicate bad tramsfer.
replace the words with the wrong words, such "but" in the first sentence and "best" in the last. The possible reason is that both of them use embed- dings to represent the target style. Different from them, the target style is encoded as the style prefix in our model, which contains more parameters and can significantly improve style transfer quality.
Fine-tuning the whole model can be regarded as the upper bound of the performance. Experimental results show that using prefix-tuming can achieve close results to training the whole model. Since we only need to train 2% parameters compared to GPT-2, this performance is quite competitive.
5 Conclusion
In this paper, a novel method for unsupervised text style transfer based on prefix-tuning was proposed, in which we take advantage of the pre-trained lan- guage model. We encode both target style and input text with prefixes that guide style transfer using prefix-tuning. We design a style transfer method that uses the language model recursively, using GPT-2 in both the content extraction and style transfer stages. In this way, the input sentence can fully interact with the language model, fully utilizing the language model's 's capabilities for style transfer. Experiments show that our method signif- icantly improves content preservation and fluency of the style-transferred sentences compared the baselines while achieving competitive results on the style control. In the future, we plan to extend our approach to more broad application settings.
4.6 Case Study
Finally, to better understand the characteristics of different models, several sentences are sampled from the Yelp dataset and tested on different mod- els, including Style Transformer, RACoLN, and ours. The outputs are shown in Table 5. Some sentences generated by the baselines appeared in- coherent or made some grammar errors, which do not appear in ours. This is mainly due to the back- bone we use. The GPT-2 with solid expressive ability makes the output sentences of outputs more fluent. In addition, these grammar errors in the baseline methods also affect the content preserva- tion of the original input, which results lower BLEU scores. On style transfer, the baseline mod- els miss some words that need to be modified, or

---

Limitations
conditional layer normalization. arXiv preprint arXiv:2108:00445
The proposed method needs to be trained for new style transfer tasks. Thus, we will explore combin- ing the concept of in-context learning in the future and employ the more powerful pre-train models, i.e. ChatGPT, to improve the performance and ex-
Joosung Lee. 2020. Stable style transformer: Delete and generate approach with encoder-decoder for text style transfer. arXiv preprint arXiv:2005. 120866.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: De- noising sequence-to-sege -e-euueece pre- pre-training for natural language generation, translation, and comprehension. arXiv preprint 1910.11910.144
tend it the zero-shot setting.
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah. Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901.
Juncen Li, Robin Jia, He He, and Perey Liang- 2018. Delete, retrieve, generate: simple ap- proach to sentiment and style transfer. arXiv preprint 1004v/800.066
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv: 100.00990.
Ning Dai, Jianze Liang. Xipeng. Qiu, and Xuanjing Huang. 2019. Style transformer: Unpaired text style transfer without disentangled latent representation. arXiv preprint arXiv: 190.059..
Dayiheng Liu, Jie Fu, Yidan Zhang, Chris Pal, and Jiancheng Lv. 2020. Revision in continuous space: Unsupervised text style transfer without adversarial learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8376-8383.
Chenghao Fan, Ziao Li, et al. 2022. Gradient-guides un- supervised text style transfer via contrastive learning. arXiv preprint arXiv:2202.000 20499
Ruibo Liu, Chongyang Gao, Chenyan Jía, Guangxuan Xu, and Soroush Vosoughi. 2022. Non-parallel text style transfer with self- self-parallel supervision. arXiv preprint arXiv:2204.08/22
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg- Kink--iikaatrick 2020. probabilistic formulation of unsupervised text style transfer. arXiv preprint arXiv:2002.03912
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding. Yujic Qian, Zhilin Yang, and Jie Tang. 2021. Gpt understands, too. arXiv preprint 103.1:3003.
Kenneth Heafield. 2011. Kenlm: Faster and smaller language model queries. In Proceedings of the sixth workshop on statistical machine translation, pages 187-197.
Fuli Luo, Peng Li, Jie Zhou, Pengcheng Yang, Baobao Chang. Zhifang Sui, and Xu Sun. 2019. dual reinforcement learning framework for unsupervised text style transfer. arXiv preprint arXiv: 1905.
Di Jin, Zhijing Jin Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: A survey, Computational Linguistics. 48(1):155-203
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. pages 140, Portland, Oregon, USA. Association for Computational Lin- guistics.
Zhijing Jin, Jin, Jonas Mueller, Nicholas Matthews, and Enrico Santus. 2019. Imat: Unsupervised text at- tribute transfer via iterative matching and translation. arXiv preprint
Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. arXiv preprint 16Xiv:1607.01759)
Remi Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. 2019. Evaluating style transfer for text. arXiv preprint 19X4v:990.002290
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.698%)
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguistics, pages 311-318.
Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc' Aurelio Ranzato, and Y-Lan Boureau. 2018. Multiple-atrribte text rewriting. In International Conference on Learning Representa- tions.
Alec Radford, Jeffrey WU, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8)99
Dongkyu Lee, Zhiliang Tian, Lanqing Xue, and Nevin L Zhang. 2021. Enhancing content preservation in text style transfer using reverse attention and

---

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter Liu, et al. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Mach. Learn. Res., 31(140)11-61
Emily Reif, Daphne Ippolito, Ann Yuan, Andy Coenen, Chris Callison- and Jason Wei. 2022. A recipe for arbitrary text style transfer with large language models. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume Short Papers), pages 837-848, Dublin, Ireland. Association for Computational Linguistics.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. Advances neural information processing systems, 30.
Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Mah- eswaran. 2019. Transforming delete, retrieve, gener- ate approach for controlled text style transfer. arXiv preprint arXiv:
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-ture tuned chat models. arXiv preprint vrXiv:2307.09256
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
Ke Wang, Hang Hua, and Xiaojun Wan. 2019. Control- lable unsupervised text attribute transfer via editing entangled latent representation. Advances in Neural Information Processing Systems, 32.
Jun- Yan 7hu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. Pro- ceedings of the IEEE international conference on computer vision, pages 2223-2232


Reviews:
[{"review_id": "UDFpZ39upl", "reviewer": "Reviewer_2nkN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. Here are our responses.\n\n**1. Evaluated only on two datasets and two-style transfering task, which are relatively simple.**\n\nWe agree with the reviewer that it would be interesting to evaluate our method under more complex settings. Actually our method is not restricted to the task of sentiment transfer. In this paper, we aim to illurstrate the method clearly. Thus, we used relatively simple tasks to evalute our method. We are also working on evaluating it on more complex tasks, and the results will be included in the extention verion of our paper.\n\n**2. Did not describe the problem clearly.**\n\nWe will describe the necessary background, motivations for each step of our method more clearly in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The method provided improvements compared to three different models. Compared to LaMDA, the improvement is mainly in BLEU score. \n* The paper is written clearly (but there are parts that need to be explained better (e.g., why do we need the recursive exactly and what happens if we  do not do that?)", "reasons_to_reject": "* The paper is only evaluated on two datasets that target the same task (positive to negative sentences or vice versa). The approach would be more powerful if different types of styles have been considered. The provided examples focus on changes for the language (e.g. but) or antonym changes (e.g. love vs. hate) but not clear how does it perform in more complex scenarios. \n* The paper has focused on transferring only two styles to each other. Not clear what happens with scaling the number of styles. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper suggested a method of unsupervised text style transfer with prefix tunning approach. they followed adversarial learning approach where they have a generator and discriminator. For generator, they trained three types of prefixes (style, content, and task) and for the discriminator they also trained prefix tokens. Prefix tuning is beneficial to reduce the number of trainable parameters in both generator and discriminator. \nThe paper attempts to transfer styles between texts but they didn't describe the problem clearly - hence readers are assumed to understand why the problem is needed from their past experience (e.g., why do we need text style transfer from the first place? how encoder leads to information loss? why generative model is important? .... etc. ). ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "URwVSgR3D7", "reviewer": "Reviewer_p4xo", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. Here are our responses.\n\n**1. More baselines should be compared.**\n\nThanks for pointing out that. We will include the comparison with related methods under the same settings as ours in the revised version.\n\n**2. There is no analysis of the three losses to train the generator.**\n\nThanks for the suggestion. The trade-off parameters in Eq.(4) (0.25, 0.5 and 1) are selected on the valid dataset. We just fix the first one and tune the others by multiplying a factor of 2. And they are not carefully tuned. Our experiments showed that our method is not sensitive to the three trade-off parameters. We will add the descriptions in the revised version of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThis work is clearly motivated to do text style transfer and proposes a quite new and interesting.\n2.\tThe adversarial framework of the methodology is reasonable. The prefixes are quite significant according to the experimental results. Moreover, the losses designed for the generator are quite thoroughly considered.\n3.\tThe statement of the methodology and the structure of the paper are good. The implementation is comprehensive. The results are solid.\n", "reasons_to_reject": "1.\tBaselines in this paper are limited. There are no baselines from 2022. There a also no baselines of editing-based [1] or prompt-based methods [2].\n2.\tThere is no analysis of the three losses to train the generator.\n\n[1] Machel Reid and Victor Zhong. 2021. LEWIS: Leven- shtein editing for unsupervised text style transfer. In Findings of ACL-IJCNLP, pages 3932–3944. \n[2] Mirac Suzgun, Luke Melas-Kyriazi, and Dan Jurafsky. 2022. Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2195–2222.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes an unsupervised text style transfer method which is an adversarial framework consisting of a generator and a discriminator. Both the generator and discriminator are using GPT-2 as base models. Moreover, the authors use prefix-tuning to finetune the models. They design shared prefix, style prefix, and content prefix to encode corresponding information of the input sequence. The proposed method outperforms baselines in terms of both automatic evaluation and human evaluation from the aspects of style, content, and fluency. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wHvluZ2pQW", "reviewer": "Reviewer_cYMW", "summary": "", "questions": "- Can the authors provide more details on the specific differences between their proposed method and the framework presented in (Dai et al., 2019)? How does the incorporation of prompt learning techniques contribute to the proposed method?\n- While the evaluation of the proposed method focuses on sentiment transfer tasks, are there any plans to explore its performance on other tasks, such as formality transfer? If so, what are the potential challenges and considerations in adapting the method to different tasks?\n- The experiments in this paper only compare a limited number of baselines. Are there any specific reasons for choosing these particular baselines? Would it be possible to include additional baselines, such as DelRetri [Li et al., 2018], Template [Li et al., 2018], UnsuperMT [Zhang et al., 2018], and DualRL [Luo et al., 2019], to provide a more comprehensive comparison?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments. Here are our responses.\n\n**1. Differences with Style Transformer (Dai et al., 2019).**\n\nOur method is based on the core idea to employing the knowledge from the pre-trained and frozen model with prefix-tunning. Thus, the design of prefixes is important to achieve such a purpose. We proposed 3 prefixes to encode content, style and task-specific information. Moreover, we also proposed to use LM recursively to provide an effective way for the trainable and frozen parameters to interact. Thus, our method is totally different from Style Transformer.\n\n**2. Evaluation on more complex tasks.**\n\nWe agree with the reviewer that it would be interesting to evaluate our method under more complicated settings. It seems that our method is not restricted to the task of sentiment transfer. In this paper, we aim to illurstrating the method clearly, which is the reason why we used relatively simple tasks to evaluate our method. We are still working on evaluating it on more complex tasks, and the results will be included in the extension version of our paper.\n\n\n**3. Compare with more methods.**\n\nThanks for the suggestion. The methods in [Li et al., 2018] have been compared in Style Transformer, and our method is better than the Style Transformer. Thus, our method should be better than the methods in (Li et al., 2018). We will include the discussions and analysis of all the methods mentioned here in our revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Proposing a new method based on GPT-2 for unsupervised text style transfer.\n- Introducing three different types of prefixes to encode task-specific information, target style, and content details, thereby providing more comprehensive information to the model compared to the embeddings used in previous works.\n- Adopting a recursive approach to improve the interactions between the input sentence and GPT-2.\n- Providing comprehensive results, conducting ablation studies, and including subjective evaluations from human assessors to gain a deeper understanding of the proposed approach.", "reasons_to_reject": "- This paper presents incremental work. The proposed method is highly similar to the framework of (Dai et al., 2019) and incorporates prompt learning techniques that have been previously proposed.\n- The evaluation of the proposed method is limited to sentiment transfer tasks. It would be beneficial to explore the performance of the proposed method on other tasks (such as formality transfer) and datasets.\n- The experiments in this paper only compare a small number of baselines. It would be advantageous to include additional baselines, such as DelRetri [Li et al., 2018], Template [Li et al., 2018], UnsuperMT [Zhang et al., 2018], and DualRL [Luo et al., 2019].", "questions_to_authors": "- Can the authors provide more details on the specific differences between their proposed method and the framework presented in (Dai et al., 2019)? How does the incorporation of prompt learning techniques contribute to the proposed method?\n- While the evaluation of the proposed method focuses on sentiment transfer tasks, are there any plans to explore its performance on other tasks, such as formality transfer? If so, what are the potential challenges and considerations in adapting the method to different tasks?\n- The experiments in this paper only compare a limited number of baselines. Are there any specific reasons for choosing these particular baselines? Would it be possible to include additional baselines, such as DelRetri [Li et al., 2018], Template [Li et al., 2018], UnsuperMT [Zhang et al., 2018], and DualRL [Luo et al., 2019], to provide a more comprehensive comparison?", "paper_topic_and_main_contributions": "This paper focuses on unsupervised text style transfer, which involves modifying the style of input sentences while preserving their content, without relying on parallel data. Building on the framework of (Dai et al., 2019), the authors aim to utilize pre-trained language models (such as GPT-2) and prompt learning techniques to construct generators and discriminators. Furthermore, the paper adopts a recursive approach to improve the interactions between the input sentence and GPT-2. The proposed method is evaluated on Yelp and IMDb datasets, and the experimental results demonstrate its effectiveness.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "- Style Transfer as Unsupervised Machine Translation. Zhang et al., 2018.\n- A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer. Luo et al., 2019."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a method for unsupervised style transfer, using prefix-tuning of GPT-2 with 2 types of prefixes: style, content, and task. The proposed approach was properly evaluated using both automatic metrics and human evaluation, as well as ablation studies, but it was not evaluated against recent and similar models. Another concern is that the work seems incremental and the results are only slightly better than the baselines, and the illustrated examples are probably cherry-picked to favor the proposed model. In addition, it was only evaluated on sentiment task (for transferring sentiment from positive no negative and vice versa), so it's not clear how well it would work for other types of style transfer. Overall, while the paper is well written and the approach is somewhat sound, the paper needs extensive evaluations to validate the method for different style transfer tasks compared to other recently proposed approaches.
Decision: Accept-Findings