TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling
Authors: Parker Riley
Noah Constant
Mandy Guo
Girish Kumar
David Uthus
Zarana Parekh
Venue:  (2021)
Keywords: style transfer
text style
text generation
generative models
conditional generation

Abstract: We present a novel approach to the challenging problem of label-free text style transfer. Unlike previous approaches that use parallel or non-parallel labeled data, our technique removes the need for labels entirely, relying instead on the implicit connection in style between adjacent sentences in unlabeled text. We show that T5 (Raffel et al., 2020), a strong pretrained text-to-text model, can be adapted to extract a style vector from arbitrary text and use this vector to condition the decoder to perform style transfer. As the resulting learned style vector space encodes many facets of textual style, we recast transfers as "targeted restyling" vector operations that adjust specific attributes of the input text while preserving others. When trained over unlabeled Amazon reviews data, our resulting TextSETTR model is competitive on sentiment transfer, even when given only four exemplars of each class. Furthermore, we demonstrate that a single model trained on unlabeled Common Crawl data is capable of transferring along multiple dimensions including dialect, emotiveness, formality, politeness, and sentiment. 

Under review as a conference paper at ICLR 2021
TEXTSETTR: LABEL-FREE TEXT STYLE EXTRACTION AND TUNABLE TARGETED RESTYLING
Anonymous authors Paper under double-blind review
ABSTRACT
We present a novel approach to the problem text style transfer. Unlike previous approaches that use parallel or non-parallel labeled data, our technique removes the need for labels entirely, relying instead on the implicit connection in style be- tween adjacent sentences in unlabeled text. We show that T5 (Raffel et al., 2020), a strong pretrained text-to-text model, can be adapted to extract a style vector from arbitrary text and use this vector to condition the decoder perform style transfer. As the resulting learned style vector space encodes many facets of textual style, we recast transfers as "targeted restyling" vector operations that adjust specific attributes of the input text while preserving others. When trained over unlabeled Amazon reviews data, our resulting TextSETTR model is competitive on senti- ment transfer, even when given only four exemplars of each class. Furthermore, we demonstrate that a single model trained on unlabeled Common Crawl data is capable of transferring along multiple dimensions including dialect, emotiveness. formality, politeness, and sentiment.
INTRODUCTION
There has been a recent surge of interest in text style transfer, with the aim of training models able to modify specific attributes of input text (e.g., sentiment or formality) while preserving the remaining content. For example, sentiment transfer model might transform the input "best book ever!" into "worst book ever!", while formality transfer model might change the same input into "This the best book have ever read."
Work in this area falls into three categories. Supervised approaches like Thamtani et al. (2017) transfer between pre-selected styles, and rely aligned parallel training data teach the model the desired input/output correspondence. This method is limited by the availability of parallel corpora. So-called "unsupervised" approaches like Li et al. (2018) and Lample al. (2019) remove the requirement for parallel data, but still require labeled training examples of each style, and are limited to transfer between pre-specified set of styles. Label-free approaches like the recent Xu et al. (220)) remove the need for any training labels. While the most technically challenging, this offers the potential for transferring between arbitrary styles at inference time and has significant value, as curated datasets are not available for many style attributes.
In this work, we explore the hypothesis that large pretrained text-to-text models like T5 (Raffel eet al., (2020) already contain a strong representation of textual style, which can be extracted and used condition the decoder of a style transfer model through a relatively lightweight fine-tuning procedure. To isolate style information in the absence of labels, we rely on the observation that style a "slow-moving" feature, which tends to consistent over large spans of text. Specifically, given two adjacent sentences from an unlabeled corpus, we train our model to extract "style vector" from the first and use that vector to perform denoising and other reconstruction tasks on the second. This technique extends the unsupervised approach of Lample et al. (2019) to the label-free setting, and allows us to reformulate the style transfer operation as a directional operation in style vector space using the difference between target and source style vectors; call this "targeted restyling". When combined with a novel "tunable inference" " technique for controlling token add/delete rates, this gives final model: Text Style Extraction and Tunable Targeted Restyling (TexISETTR).

---

Under review as a conference paper at ICLR 2021
Training
Target really love it
Inference
Output works great
Decoder
Decoder
Style Extractor
Tuning Ranges Delete
Style Target
Tuning Ranges
Encoder
Add 40-70% 25-35%
(A B) Inp
Encoder
Add 10-50% 10-50%
Delete
really
Ex
Ex
Ex
Corruption
Style A Exemplars
Style B Exemplars
It doesn't work
great product. realty Context
Input
Input
Figure 1: Tex(ETTR architecture for label-free style transfer. The Encoder, Decoder and Style Extractor (Ex) transformer stacks initialized from pretrained T5. During training, the model reconstructs corrupted imput, conditioned on a fixed-width "style vector" extracted from the pre- ceding sentence. At inference time, new style vector formed via "targeted restyling": adding a directional delta to the extracted style of the input text. Stochastic tuning ranges provide extra conditioning for the decoder, and enable fine- one-grained control of inference.
Our main contributions to: (1) demonstrate the viability of label-free style transfer (2) sentence adjacency as means for inducing text style representations, (3) reframe style transfer as "targeted restyling" directional operations in style space, (4) introduce "tunable inference" for finer- grained control of transfers, (5) show the effectiveness of "noisy" back-translation training, and (6) illustrate few-shot generalization range of style attributes including dialect, emotiveness, formality, politeness, and sentiment.
2
METHOD
Figure T illustrates our proposed TexISETTR architecture. At high level, our approach follows Lample (2019), who train denoising auto-encoder conditioned on fixed-width style vector. The key difference in our case is that the true style is unknown at training time. To overcome this, we jointly train "style extractor" component to induce useful style representation (that can aid in reconstruction) from text in the nearby context. We describe this in more detail below.
2.1 MODEL ARCHITECTURE
We conduct our experiments using a modified version of the Text-to-Text Transfer Transformer (T5) (Raffel et al. 2020). Like T5, our model includes transformer-basee encoder and decoder. As in T5 pretraining, the input to the encoder is a corrupted/noised version of the target, resulting in reconstruction task. Our goal resembling style transfer, despite the lack of labeled training data.
design type of corruption that results in this training task
Our core addition to T5 the style extractor. Based on the encoder's architecture, this component's input is an uncorrupted sentence in the same style the target; relying on our assumption that style slow-moving feature, we use the sentence preceding the target (the "context") for this ? This encourages extracting a style representation that is useful for repairing the corrupted input.
The only architectural difference between the encoder and style extractor that we mean-pool the style extractor's 's hidden state sequence into single fixed-width vector (the "style vector"); in our experiments, the dimensionality of this vector and the encoder hidden states 1024. To incorporate the style vector into the rest of the model, simply add it to each of the final encoder hidden states.
Our work is concurrent with Xu et 2020), who offer substantially different approach label-free
style transfer, as discussed in Sections 3 and
This approach is similar to the use of adjacent sentences for weak supervision in Devlin et al. 2019 and
Zhang al. (2020
2

---

Under review as a conference paper at ICLR 2021
We initialize the weights of our model with those of a pretrained T5 model. We initialize both the style extractor and encoder from the pretrained encoder, but the weights are not tied during training.
2.2 CORRUPTION STRATEGIES
We experiment with combinations of three different reconstruction tasks, each contributing loss term. All three share the same overall structure, where sentence in the dataset is corrupted by some function f to produce 6; = (s.). The cross-entropy loss is calculated using the uncorrupted sentence 8; as the target, the corrupted sentence &; as the input, and the uncorrupted preceding sentence 8;-1 as the context. The three choices of f are Noise (N), Tack-Translation (BT), and Noisy Back-Translation (NBT), described below.
Noise (N) This function corrupts the input by (i) dropping, (ii) replacing, and/or (iii) shuffling tokens, in that order. For each example sample a separate noise probability p for each sub-type of noise from a uniform distribution in the range 20-60%. We do this because the model should be able to "undo" varying degrees of noise, as some types of style transfer may require changing only a few tokens, while others may require larger rewrites.
For drop noise, we drop each token in s with independent probability p. For replace noise, let sik be the k-th token within s, For each s,, a random other example 8; is chosen, and then each token Sck is replaced by $js with probability p. If 8j has fewer than A tokens, then the replacement does not occur. For shuffle noise, each token in 8; is chosen with probability p. and then all chosen tokens are randomly shuffled to the position of another chosen token, leaving non-chosen tokens in place.
The use of drop and shuffle noise results in a loss term similar to the denoising loss used by Lample al. (2019). Their motivation for this loss was encourage language modeling. As we fine-tune an already-strong T5 language model in our experiments, our motivation is rather to introduce a conditional element to the language model, in the form of the extracted style vector input.
Back-Transkanl Translation (BT) This corruption function, used by Lample et al. (2019), runs the current version of the model in inference mode to transfer 8; into different style, giving the corrupted & In prior work using labels, specifying a different target style was straightforward. In our case, because we do not have access labels, we simply sample a random sentence 8, to use as the context. To increase diversity of the generated examples, we decode with sampling instead of greedy decoding.
Because & is produced by a strong language model, BT should result in training examples where both the input and output are coherent sentences, matching our intended inference setting. By con- trast, the text produced by "Noise" corruption does not resemble test-time inputs.
Noisy Tack-Translation (NBT) This novel corruption function is composition of the previous two. Noise is first applied to 8; as described above, and the result is used as the input (with randomly- sampled sj as the context) to the model in inference mode to produce 8, via sampling, in BT.
Once the model has learned to undo random noise from the associated loss term, NBT should pro- duce training examples where some of the tokens are preserved from s, while others were generated by the model itself under the influence of the "incorrect" context 85. This is similar to BT,, but we hypothesize that it may be better suited to style transfer. BT was originally used for machine transla- tion (Sennrich et al. 2016), a setting where most or all input tokens need to be changed. In contrast, style transfer within a single language usually requires only changing subset of input tokens; the training examples resulting from the NBT procedure should have this property. We believe that this will encourage the model to identify which tokens in the input do not match the target style indicated by 8:-1 and change them, which is exactly what we want style transfer model to do. This is con- ceptually similar to the work of Clark et al. (2020), who pretrain language models to discriminate which input tokens were originally present and which were altered by a simpler language model.
Final Loss The final loss term used for training is the sum of the above loss terms, each calculated from the same input s,. However, not every model we experiment with includes all three losses.
2.3 INFERENCE PROCEDURE
Tunable Add/Delete Rates In preliminary experiments, we observed a recurring problem that the model would often change either far too little (failing to achieve the target style), or far too much
3

---

Under review as a conference paper at ICLR 2021
(failing to preserve the imput content). To address this problem, we introduce "tunable inference" mechanism to constrain how much content should be added and deleted at inference time.
For every input/output pair during training, we calculate the proportions of tokens that were added and deleted. The "add rate" is the proportion of output tokens absent from the input, and the "delete rate" is the proportion of input tokens absent from the output Rather than provide these rates directly to the decoder, we provide ranges, covering but not necessarily centered around the true rates. Specifically, we sample each range width uniformly from [0.1], and uniformly sample the "alignment" of the true rate within the range. The final ranges are clipped to [o.1], and passed to the decoder as four values: [min [min_add _addaaddettee rate, max_add_ras rate, min deeedrerreeee max rate] This approach provides more flexibility at inference time, so we can enforce tight or loose constraints on each rate.
Targeted Restyling While previous work on style transfer has largely assumed fixed set of discrete styles, our method can extract "ready-made" style vector from any sentence. We expect this learned representation to capture a rich summary of the sentence, covering many attributes. For example, a given style vector might encode that a sentence is informal, humorous, in British English, and so on.
In this framework, transferring single attribute (e.g., informal - formal) not as simple as just providing a vanilla "formal" style target, this would ignore all the other attributes that defined the original input. Rather, we must operate in style space to construct new target style that simuitaneously formal, humorous, British, and so on.
Concretely, at inference time, we assume access to a small set of "exemplar" sentences (between I and 100) for both the source value (e.g., informal) and target value (e.g.. formal) of the attribute being modified. We infer style vectors for each exemplar using the style extractor, and take the mean of each class, giving vectors y and ,/7 Assuming the exemplar pools are relatively diverse, this averaging should "wash out" most attributes not being targeted.
To transfer input sentence v, we apply targeted restyling in the appropriate direction. After extracting the original style from the input itself, v", we compute the target output style by moving in the direction of the delta between the source and target attributes values, in ( producing the style vector used for decoding. In practice, we find that the delta scale À is an important hyperparameter to tune. Generally values in the range [1.0, 10.0] work well, with the best values depending on the attribute and the exemplars in question.
2tth + A x (1-19 - tar))
(I)
3 EXPERIMENTS
To evaluate our approach and better understand the effects of our various design choices, we test on label-free sentiment transfer, using the Amazon reviews dataset of Li et al. (018). However, as their training split doesn't 't indicate which sentences were adjacent in the original reviews, we make use of a different source of raw review text.
Training Procedure Our unlabeled training data comes from the 233.1M Amazon reviews provided by Ni et al. (2019). Ignoring the star ratings completely, we extract adjacent lines from multi-line reviews to use as the context and input for our training procedure, giving 23, 3.6 examples. We also preprocess all text to match the format of the Li et al. (2018) data, as detailed in Appendix [A.4 we fine-tune on these examples, optimiz-
Initializing our model from pretrained T5 (t5.1. laare ing the joint reconstruction loss from Section 2 Our default TexiSETTR configuration is selected based on preliminary experiments (on development data) varying the set of reconstruction tasks and inference procedures. The model uses an equally weighted combination of the Noise (N) and Noisy (NBT) tasks. For both tasks, we use drop and replace noise, but no shuffle noise. We fine-tune for 10k steps, with a batch size of 65,536 tokens, and a fixed learning rate of Ic-3.
Thii calculation ignores word order. As one example, if a token appears three times in the input and five times in the output, two of the five occurrences are counted as "added". *More specifically, a vector is prepended to the sequence encoder hidden states, holding the tuning rates as the first four values, and zeros elsewhere.
4

---

Under review as a conference paper at ICLR 2021
100
Model TeXISETTR
Acc.
Content
55-700%
40-60%
73.7
34.7
30-50%
23.4
84.4
20-40%
B+BT BT
NBT
70.0
13.3
27.8
98.7
Sentiment Transfer Accuracy
80
small-train
1000-exemplar ars
-tuunalle
N8T + shuffle
replaace noise
66.1
42.1
CressAligned
rerplacc
+huuflle noise
70.3
34.1
60
BBGGST
manual exemplars
52.4
44.2
10-30%
1000 exemplars
74.5
37.2
CP-G manual
Delete&Retrieve
tunable inference
71.5
39.4
40
N(50k)
0-20%
overwrite style
25.3
55.8
CP-3
N+BT (50K)
0 1.00
small train set
74.5
33.4
CP-G
51.1
35,5
20
TeXtSETTR
N
CP-B
36.3
39.8
TextSETTP ETR ablations
CrossAligned
68.2
2.9
Other label-free models
N+BT
Delete&Retriev
49.4
56.9
B-GST
60.2
54.2
20
40
80
100
Content Preservation (Self-BLEU) BLEU)
Figure 2: Automatic evaluation metrics comparing our TexISETTR model, ablations, and previous work. Up-and-right is better. We train for 10k steps and use add/delete:20-40% unless otherwise specified. We recalculate metrics for previous approaches, using our BERT classifier for accuracy. ensuring direct comparability with our models.
Sentiment Transfer Accuracy
A50-70% 70%
TexISETTR (0-20%%
Model
Accuracy
63.4
Content
76.9
40-50%k
30-50% A
Lample al. 2019
20-40%
TexISETTR (10-30%)
72.7
60.2
TexISETTR (20-40%)
83.6
39.4
10-30%
TextSETTR (0%))
89.7
21.5
60
0-20%
TexISETTR (000-60)))))
94.3
11.3
TexISETTR (50-70%)
96.6
5.0
40
TextSETTR
trained
Lample et al. 2019
82.6
54.8
30
20
60
100
Content
Preservation (Self BLEU)
Figure 3: Comparison with Lample al. (2019) on the evaluation setting that includes pos pos-ppos >pos and neg-tne transfers. Note, a model that simply copies its input can achieve 50% accuracy.
Evaluation Procedure Following previous work, we use automatic metrics to assess attribute con- trol (sentiment) and content preservation the Li al. (2018) test data. To estimate the sentiment of the transferred output text, we fine-tune a BERT-Large classifier (Devlin et al. 2019) on the Li al. (2018) train split, scoring 87.8% accuracy on the dev split. For content preservation, we follow Sudhakar et al. (2019) and Xu et al. (2020) and report self- BLEU between the output and input text, calculated using SacreBI Post 2018 Some prior work reports instead BLEU scores between outputs and human-gent generated transfers from [i et al. (2018); we observe this metric to be highly correlated with self-BLEU but report it in Appendix A.3 for completeness.
To perform transfers, we follow the procedure from Section 2.3 For our default setup, sample 100 positive and 100 negative exemplars from Li et al. (2018) train. We also experiment with (i increasing to 1000 exemplars of each class, and (ii) decreasing just four manually selected exemplars of each class. Unless otherwise specified, we use greedy decoding, a delta scale of A=8, and add/delete tuning ranges of 20-40%.
Core Results Figure shows our core results. Our default TexISETTR configuration (using N+NBT training and tuning ranges 20-40% achieves 73.7% accuracy swapping sentiment (as judged by the classifier), while still staying somewhat close to the original input text (self-BLEU 34.7). Due to our tunable inference technique, we can also trade off accuracy for content preservation by adjusting the add/delete rates, as seen in the different points along the green line. Notably, TexISETTR out- performs the label-free CP-G and CP-B models of Xu al. (2020). More remarkably, Tex(SETT
Version string: BL (TTfcccce.mxed-mmmmee. +smooth.exp+tol 1atpttookk..44aeee 1344verionn.1.4
5

---

Under review as a conference paper at ICLR 2021
Negative - Positive
Positive + Negative
Model
Sentiment
Preservation
Fluency
Sentiment
Preservation
Fluency
TeXISETTR
2.8
2.4
4.2
2.3
2.8
3.8
Delrie&Retrieve
2.7
2.9
3.2
2.3
3.4
3.4
B-GST
2.3
2.8
3.6
2.1
3.0
3.6
Table I: Human evaluations on sentiment, content preservation, and fluency.
outperforms several approaches that rely training labels:
CrossAligned (Shen et al. 2017) and et al.) 2018). However there is still a small gap between our label-tree approach
Delete& Retrieve et and the best-performing labeled model, B-GST (Sudhakar al. 2019).
In Figure 3. we compare with Lample et al. (2019) on the evaluation setting including pos-> and neg -ee transfers. Note, this type of evaluation doesn't 't match our inference procedure, which assumes that the input and output styles differ. Nevertheless, Tex(SETTR comes fairly close to the performance Lample et al. (2019), despite not benefiting from training labels.
Human Evals As automatic metrics are known to diverge from human judgment (Sudhakar al. 2019), we conduct human evaluations of the strongest models from Figure 2 TexISETTR, Retriee and B-GST. We sample 200 examples per transfer direction from the Li et al. (2018) test set, and ask three annotators to evaluate each input/output pair on three metrics: sen- timent transfer (how well the model changed the sentiment), content preservation, and fluency, on scales of 1-5. Table shows that TexSETTR has superior fluency and performs similarly or better on sentiment, but is worse content preservation. These results support the view from the auto- mated metrics that TexISETTR achieves similar quality to models that benefit from training labels.
3.1 ABLATIONS
Modifying inference procedure To better understand the value of our proposed "targeted restyling" mechanism, we consider an alternative inference procedure whereby we ignore the style of the input text, and simply feed the average target exemplar style urra as the conditioning style vector. Our expectation is that since our learned style space covers multiple attributes, this will have the effect of setting the target attribute (e.g. sentiment), while simultaneously overwriting all other style attributes (e.g. formality) with unintended values determined by the average style of the target exemplars. This is borne out in our "overwrite style" ablation, which performs significantly worse than our baseline: accuracy drops from 54.0% to 25.3% while holding Self-BLEU constant.
To assess the value of tunable add/delete rates, we also train model (-tunable) without this feature. While the automatic metries are slightly above the TextETTR line, we observe several advantages to the tunable model. For one, we observe it significantly reduces the variance in self BLf- across different inputs. For example, focusing on the case of overly high self-BLEUU, we find that without tunable inference, 14.6% of dev eval outputs are identical their inputs, whereas with tunable inference, this goes to 0.9%. Additionally, through qualitative analysis in Section 4 we find that tunable inference allows more flexibility for controlling different types of transfer.
Adjusting data sizes While our unlabeled training data set consists of 23.6M examples, our model only sees M of these over its 10k steps of training. Yet this is still nearly 10x more data than the 0.6M examples in the Li et al. (2018) training set used by previous approaches. For a more direct comparison, we experiment with a "small train set", sampling 0.6M examples from our training set. Remarkably, the results in Figure 2 are nearly identical to our baseline, supporting our hypothesis that a fairly lightweight adaptation is sufficient to allow T5 to extract and transfer textual style.
To test the limits of our model's generalization, we reduce the set of exemplars to four manually selected examples of each class 7 In this setting, we also find reducing delta scale to X=4 is bene- ficial. The results, shown as "manual exemplars" in Figure 2 are still competitive, indicating that our approach generalizes well to the few-shot inference setting. In the other direction, we find that increasing the number of sampled exemplars from 100 to 1000 only provides small additional gains.
"five stars amazing / really love this product. / works great will buy again / it is well made and pleasure to use. // zero stars, horrible / really hate this product. / does not work, will not buy again / it is poorly made, and a pain to use.

---

Under review as a conference paper at ICLR 2021
Before TexISETTR train- ing (pretrained T5 initialization)
Fashian Somtware
Postive
Fastion
Postive
After TEXISETTR training
Figure 4: 2D UMAP embeddings of the style vectors extracted by our TextSETTR model before and after training, for text inputs from Amazon reviews covering three product categories and two sentiment labels. Within each row, the same embeddings are visualized with product category labels (left) and sentiment labels (right).
Reserved => Emotive
Emotive >> Reserved
liked the movie,
loved every minute of the movie!
cannot describe how amazing this movie was!!
> liked the movie,
was impressed with the results,
was shocked by the amazing results
was absolutely blown away with the results!!
was surprised the results,
American British
British American
The elevator in my apartment 't working.
The lift in my flat isn't working.
=>
The lift in my flat isn't working.
>
The elevator in my apartment isn't working.
The senators will
Washington week.
MPs will return Westminster next week.
>> The MPs will return to Westminster next week.
+ Representatives will return Washington next week.
Polite = Rude
Rude Polite
Are you positive 'e understood my point?
What the hell is wrong with your attitude?
+ you've never understood my point!
Perhaps the question is more about your attitude,
could care less, go find somebody clse to do this crap.
Could you ask before using my phone?
could be wrong, but would try find somebody to do this,
+ I ask you to stop using my phone!
Formal Informal
Informal Formal
herehy commit to never purchase anything from institution in the future.
best book ever!!
= The book highly recommended.
gonna never huy anything from this place again.
couldnt figure out what author tryna say
couldn't figure out what the author was trying say.
> The reader couldn't figure out what the author was trying say.
dont know what ur trying to say.
Positive Negative
Negative * Positive
was pretty impressed with results. # was pretty disappointed with the results.
was pretty disappointed with the results.
* was pretty impressed with the results.
I will definitely buy this brand again. + will definitely not buy this brand again.
definitely won't 't buy brand again.
+ definitely won't hesitate to buy this brand again.
Table 2: Examples of transferring along five different axes of style. The same model is used across all examples, with no additional training. Words deleted from the input are red, and words added in the output are blue. Within each category, fixed tiny set of exemplars is chosen, and fixed delta scale and tuning rates are used. The exemplars and settings are provided Appendix A.2
7

---

Under review as a conference paper at ICLR 2021
Modifying training task Lample et al. (2019) showed promising results by combining noise (N) with back-tramslation (BT). However we find this combination unstable. When training for 10k steps, our N and N+BT models nearly always copy their input. By increasing the training steps to 50k, we can recover reasonable performance, but the metrics still fall below the TextSETTR line, using novel NBT task. We also experiment with using NBT in isolation, but this again underperforms our baseline. We expect that the denoising task helps to ensure the NBT inputs (themselves the outputs of denoising) consist of realistic well-formed text. Finally, while Lample et al. (2019) use drop and shuffle noise, we find that only drop and replace are valuable, while shuffle noise can be removed without negative effect.
3.2 EMBEDDING VISUALIZATION
To demonstrate that our learned style extractor encodes multiple aspects of textual style, we compute style vectors for 12,000 lines of text from three review categories (Fashion, Software, Pantry) from the Ni et al. (2019) Amazon data. Within each category, we sample 2,000 positives (4 or 5 star) and 2,000 negatives (I or 2 star), filtering examples where our BERT classifier disagrees with the label. Figure F (bottom) plots a 2D UMAP dimensionality reduction (Mcinnes et al.] 2018) of the vectors, and shows clear separations among sentiments and product categories The top row runs UMAP with the same settings, but over style vectors from our model before training, where the style extrac- tor is initialized from pretrained T5. The contrast is clear indication that our training procedure is helping to learn representation space where sentiment and topic values are well separated.
4 QUALITATIVE ANALYSIS
One advantage of label-free style transfer is that, in theory, a single model can be used to perform transfer along any "dimension" of style, given only a few exemplars, and without the need for addi- tional training. In this section, we investigate the degree to which our approach achieves this goal in practice. For this purpose, we train a single general-purposs TextETTT model, with the same configuration as our model from Section 3 except fine-tuned for 200k steps on English Common Crawl data (the same "C4" data that T5 pretrained on) as opposed to Amazon reviews.
Transferable Attributes Given that our architecture limits the style representation to 1024 dimen- sions, one may ask how the unsupervised model will make use of this capacity, and which style attributes will be encoded in the learned space. Encouragingly, we find that our model trained on unlabeled Common Crawl data is capable of transferring along many independent axes of style. Ta- ble [ shows selected successful examples of our Common Crawl model transferring emotiveness, dialect, politeness, formality and sentiment. The same model is used in each case, with no additional training. At inference time, tiny set of exemplars (1-5 examples of each class) is the only labeled data used to compute the style vector delta; these exemplars are presented in Appendix A.2
Across each type of transfer, we see evidence of generalization beyond the specifics of the chosen exemplars. In making text more emotive, the model uses amazing and blown away, despite these terms not occurring in the exemplars. In making text more polite, the model inserts novel hedges like perhaps and I could be wrong. In transferring between American and British styles, the model gen- eralizes to unseen vocabulary items (elevator <+ lift)) and draws sound analogies (senators ++ MPs).
We observe three other noteworthy abilities of our model, which we outline briefly here, providing further examples of each in Appendix [. First, if inference tuned to add rather than delete tokens, we find the model is capable of generating style-sensitive completions. For instance, if we reuse the American and British exemplars used to produce the results in Table P but set the tuning ranges to add:40-70%, delete:0%, the model completes the input My favorite hot drink: with either Starbucks Coffee (American) or a mug of tea (British).
Beyond Style Transfer
In the other direction, if we tune inference for heavy deletion but apply no modification to the input style vector, we find the model can perform coherent shortening of text while leaving most or all
'We sub-sample to 3,000 points after dimensionality reduction for clearer visualization. Note, we don't 't expect perfect separation, as inputs may be underspecified for category ("I love this product") or for sentiment ('I bought this last month"). "). Additionally, since we aim for the learned emhedding space to encode many style attributes simultaneously, don't expect to see crisp linear separation within each attribute.
8

---

Under review as a conference paper at ICLR 2021
of the meaning intact. For instance, with add:0-5% delete:40- 90% the model shortens: "They do this without any prior knowledge of cats, for example, that they have fur, tails, whiskers and cat-like faces." "They do not know that cats have fur, tails, whiskers and cat-like faces.".
Finally, if instead of applying a targeted style delta vector, we add a small random delta to the input style, we find model can output a range of largely meaning-preserrinn preserving random augmentations. For example depending on tuning ranges, the model may transfer the input What Il the weather be tomorrow? into outputs What's the weather forecast for tomorrow? (add/delete: 30-50%) What's the weather like for the next day? (add/delete: 50-70%). Further examples are in Appendix A.
RELATED WORK
As mentioned at the outset, recent work on text style transfer falls into three classes: supervised, unsupervised, and label-free. Supervised style transfer has seen limited research due to the difficulty of obtaining parallel data. Examples can be seen in Jhamtani et al. (2017) and Carlson et al. (2018).
Unsupervised Approaches The bulk of research has focused on "unsupervised" approaches, which rely on labeled but non-parallel data. Typically, labels are assumed to be available for both source and target styles (Dai et al. 2019 He et al. [ Lample et al. 2019 Li et al., 2018 Niu et al., Prabhumoye et al. [2018 Shen et al. [2017 Sudhakar et al. 2019 Wu et al. 2019; Yang et al,,
2018
2018 However Zhao et al. (018)) also explore the case where only the target style is labeled. The
use of labels at training time can aid modeling, but limits the applicability these methods, as
labeled datasets not readily available for many attributes of interest.
Our work differs from the above by removing the need for training labels, and offering a single model that can target unrestricted set of style attributes. Despite these differences, our work shares some similarities with past work. For example, our encoder-decoder architecture and corruption methods are similar to Lample et al. (2019), and we leverage strong pretrained language model, as in Sudhakar et al. (2019) and Wu et al. (201)).
Label-Free Approaches label-free approach has recently been explored by Xu et al. (2220) The authors train a variational auto-encoder on unlabeled text, where "manipulable" portion of the latent representation constrained to fall on a k-dimensional simplex. To perform transfer, they identify empirically the basis vector that most strongly corresponds to the target attribute, and manipulate its magnitude. Compared to our approach, key difference that the number of latent factors must be chosen ahead of time, and this limits the number of attributes that may be controlled. Additionally, there is no guarantee that single basis of the learned simplex will correspond to a target attribute such as dialect or politeness.
Controlled Generation separate strand of research explores "controlled generation" methods for supplementing generative language models allow control of specifie attributes of the output text. As with style transfer, this can be achieved either through labeled examples, as in CTRL (Keskar al. 2019) and PPL.M (Dathathri al. 2020), label-free, in CoCon (Chan et al.) 2020). The key difference between these models and style transfer models that they aim to generate plausible continuations following prompt, as opposed to transferring attributes of fully-formed input while preserving as much content as possible. It is not clear if controlled generation models could be used perform style transfer, and they have not to our knowledge been evaluated in this context.
Tunable Transfer Our delta scale parameter À is similar to the "modification weight" of Wang et al. (2019) controlling the strength of the transfer operation applied in latent space. However our tunable add/delete rates are novel in offering fine- tne-grain control over the change expected in the surface form. This degree of control enables applications such as style-sensitive completion and shortening.
6 CONCLUSION
We have presented unique approach to label-free text style transfer that can achieve results com- parable to systems trained with labels (an easier setting), while allowing control of how much of the input is changed. We demonstrate with qualitative results that this approach produce a single system capable of performing many different types of style transfer, requiring only a handful of exemplars inference time.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Keith Carlson, Allen Riddell, and Daniel Rockmore. Evaluating prose style transfer with the bible. Royal Society Open Science, 1(10),17120,, 2018. doi: 10. 10.1008/rsos.179999
Alvin Chan, Yew-Soon Ong, Bill Pung, Aston Zhang, and Jie Fu. CoCon: A self-supervised ap- proach for controlled text generation. arXiv preprint arXiv:2006.03533 2020.
Kevin Clark, Minh- Thang Luong, Quoc Le, and Christopher D. Manning. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations, 2020. URL https: /pw.reeeeeeeeeeeeeeeeeeeeeeeeeee fortt//orummiiice leterrxxxMiBBBB Bt
Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang. Style transformer: Unpaired text style transfer without disentangled latent representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5997-6007, Florence, Italy, July 2019, Asso- ciation for Computational Linguistics. doi: 10. 10.18863/11PPPP 1/P19-161.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to con- trolled text generation. In International Conference on Learning Representations, 2020. URL https:://penrer////
net httt:/ooommiid-Heeeeee
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers). pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 20.18853/v1/N199
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkppatrcck probabilistic formulation of unsupervised text style transfer. In International Conference on Learning Representations, 2020. URL http::///pppeeeeeeeeeeeeeeeeeeee
Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. Shakespearizing modern language using copy-enriched sequence to sequence models. In Proceedings of the Workshop on Stylistic Variation, pp. 10-19, Copenhagen, Denmark, September 2017, Association for Computational Linguisties. doi: 10. 11.18665///W17--4022
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. CTRL: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05885 2019.
Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc' Aurelio Ranzato, and Y-Lan Boureau. Multiple-atrribute text rewriting. In International Conference on Learning Representations, 2019. URL /octpas///ppmrreevee...eee// wtwp::/wwwweeettttttt/eamm
gNg2kCCKK
Juncen Li, Robin Jia, He He, and Percy Liang. Delete, retrieve, generate: simple approach to the 2018 Conference of the North American Chap- ter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long Papers), pp. 1865-1874, New Orleans, Louisiana, June 2018. Association for Computa- tional Linguistics. doi: 10..1855/1/118.. 1169.
sentiment and style transfer. In Proceedings the
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Groberger. UMAP: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29):861, 2018. doi: 10. 21 21105/ooss.00811 URL / 2ww.0////..............ooo 00861
Jianmo Ni. Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language
Processing (EMNLP-IJCNLP)) pp. 188-197 Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/DD 1/D19- 1018. URL https: www.. aclweb.
lorg/antho wwwpnthhooooy/yoooo ogy//19--011
10

---

Under review as a conference paper at ICLR 2021
Xing Niu, Sudha Rao, and Marine Carpuat. Multi-task neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computa- tional Linguistics, pp. 1008-1021, Santa Fe, New Mexico, USA, August 2018. Association for Computational Linguistics.
/https::/www..c. ac www.:aawwww..or..
Matt Post. A call for clarity in reporting BLEU scores. Proceedings of the Thirl Conference on Machine Translation: Research Papers, pp. 186-191, Belgium, Brussels, October 2018. Associa-
tion for Computational Linguistics. Lingutsstissssi URL https: www acwwablwww..... wtw.://wb..rggnnoe www.ggggnnoooogoy/ooooooo
6319
Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. Style transfer through fack-translation. In Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics (Volume I: Long Pupers), pp. 866-876, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 10.18653/v1/P18 1080..
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to- text transformer. Journal of Machine Learning Research, 21(140): 2020. URL http:
7/jmlr /
html
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation mod- els with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 86-96, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10. 10.18653/v1/P16-000)
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. Style transfer from parallel text by cross-alignment. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Infor- mation Processing Systems 30, pp. 6830-6841. Curran Associates, Inc., 2017. URL
http::/papest hwtpaaaprrs.iis.cccepppppaeaeeee wwwpppppppppp.eeeeeeeeeeeeeeeeee 7250 strssss-ssylltttrassteee -eraassfr-ffoomnooo from-non- wtw.bbyycoosssaalaaaaaaaaaaaaaaa
pdf.
Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Maheswaran. "Transforming" delete, retrieve, generate approach for controlled text style transfer. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Confer- ence on Natural Language Processing (EMNLP-IJCNLP?) pp. 3269-3279, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.1855// URL
https: www. acl web org.b.orrg/ee weaggg/ntthllll
wgg.hooggooooa...gooooo ogy 9-
Ke Wang, Hang Hua, and Xiaojun Wan. Controllable unsupervised text attribute transfer via editing entangled latent representation. In H. Wallach, H. Larochelle, A. Beygelzimer, F. F'd' d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, pp. 11836-11046 Curran Associates, Inc., 2019. cc/ htc/papper2211//11//BB00 http:///.....aa......eee.....ra.
neurips.
Xing Wu, Tao Zhang. Liangjun Zang, Jizhong Han, and Songlin Hu. Mask and infill: Applying masked language model for sentiment transfer. In Proceedings of the Twenty-Eighth Interna- fional Joint Conference on Artificial Intelligence, IJCAI-19, pp. 5271-5277. International Joint Conferences on Artificial Intelligence Organization, 7 2019. doi: 10.24963ficca.221197773
Peng Xu, Jackie Chi Kit Cheung, and Yanshuai Cao. On variational learning of controllable repre- sentations for text without supervision. In Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria, 2020. PMLR 119.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-KKrkpatriik. Un- supervised text style transfer using language models discriminators. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Disa-Bianchi, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems 31. pp. 7287-7298. Curran Associates, http:ppppprrr..i.cccpppp.... wtw.ippp.ccpppppppc
Inc., 2018. URL sttpleetransffr-usiing

---

Under review as a conference paper at ICLR 2021
Yizhe Zhang, Xiang Gao, Sungjin Lee, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. Consistent dialogue generation with self--uperrvied feature learning. arXiv preprint arXhv:1903.05759, 20200
Yanpeng Zhao, Wei Bi, Deng Cai, Xiaojiang Liu, Kewei Tu, and Shuming Shi. Language style transfer from sentences with arbitrary unknown styles. CoRR, abs/1808.04477) 2018. URL.
httt:///axii/ org/ abs / 1088 04071
A APPENDIX
A.1 BEYOND STYLE TRANSFER
In this section, provide additional examples illustrating the abilities of our TextSETTR model trained on Common Crawl data, beyond typical style transfer.
Examples of dialect-sensitive completion given in Table 3 input of the form "My favorite X: Despite the fact that TexISETTR not trained specifically for completions, can the add and delete rates to encourage the model to insert few additional tokens, while leaving the original prompt largely unchanged
In each case, the model is passed an
The completions show a detailed knowledge of American and British culture. Furthermore, it is remarkable that the model is able to generalize to these "deeper" cultural differences, given only the shallow vocabulary differences presented in the limited set of exemplars in Table
is also worth highlighting that, thanks to our directional transfer procedure, these completions are merely "typical American" or "typical British" continuations such we would expect from conditional language model trained on each sub-domain of text. Rather, since our inference proce- dure pushes the style away from one domain and towards the other, the resulting completions are distinctive representations of each dialect. As one example, we expect "quinoa" would not only be a common American favorite, but also an uncommon British favorite.
American + British
British + American
My favourite food: fish and chips.
My favorite food: quinoa.
My favourite hot drink: mug of tea.
My favorite hot drink: Starbucks Coffee.
My favourite dessert: scone!
My favorite dessert: brownic.
My favourite city: Cardiff.
My favorite city: Diego.
My favourite band: The Beatles.
My favorite band: The Black Keys.
My favourite sports league: the English Premier League.
My favorite sports league: the NFI.
My favourite newspaper: The Daily Telegraph.
My favorite newspaper: The Washington Post.
My favourite museum: the British Museum.
My favorite museum: The National Air and Space Museum.
Table 3: Examples of dialect-sensitive completion (A=8, add:40-70%, delete:0%). each case, the
input text consists of unfinished phrase, for example: "My favorite food: ". The three exemplars used for each dialect are the same as those used for the transfers in Table as listed Table 7
Examples of shortening are given in Table 4 with inputs taken from the first five sentences of the Wikipedia article "Artificial neural network". As successful shortening may require minor rephrases, we set our tuning ranges to add:0-5%, delete:40-909s Since our intention is to leave the textual style unchanged (apart from length), we extract the target style directly from the input text, with no delta added. In each case, the model is largely successful at identifying and removing "superfluous" content, and finding ways of rephrasing to shorten while preserving meaning.
Examples of random augmentations given in Table 5 In each case, transfer the input sen- tence "What'l Il the weather be tomorrow?" to a slightly different style. Specifically, for each transfer, extract this sentence's s style vector and apply small amount of noise, with each component of the noise vector sampled from a Gaussian (0, 0.08). Note that apart from the noise in the style greedy decoding.
vector, the transfer process is deterministic,
Note however, that in transferring from American to British, the model does prefer to change the prompt from favorite favourite.
12

---

Under review as a conference paper at ICLR 2021
Artificial neural networks (ANN) connectionist systems are computing systems that are inspired by, but not identical to, biological neural networks that constitute animal brains. Artificial neural networks (ANNs) are computing systems that are inspired by the biological neural networks that constitute animal brains.
Such systems "Iearn" to perform tasks by considering examples, generally without being programmed with task-spocific rules.
Such systems learn perform tasks by considering examples, generally withour explicit rules.
For example, in image recognition, they might learn to identify images that contain analyzing example images that have been manually labeled as "cat" "no cat" and using the results identify in other images.
For example, image recognition systems might learn to identify images that contain cats hy analyzing images that have been manually classified "cat" or "no cat".
They do this without any prior knowledge of cats. for example, that they have für, tails, whiskers and cat-like faces.
They do not know that cats have fur. tails, whiskers and cut-like faces.
Instead, they automatically generate identifying characteristics from the examples that they process.
> Instead, they automatically generate identifying characteristies.
Table 4: Examples of shortening (add:0-5%, delete:40-90%) using the first five sentences from the Wikipedia article "Artificial neural network". For each sentence, the target style is extracted directly from the input text, and no delta is added.
The cells of Table 5apply different tuning ranges, conditioning the model to change a little or a lot. Within each cell, repeatedly sample the noised style, and present the first five unique outputs. The results indicate that many random changes in style are largely meaning preserving, especially when a small change is requested. With larger add/delete rates, the outputs are still closely related in meaning, despite low lexical overlap.
Add/Delete: 10-30%
Add/Delete: 30-50%
What'll the weather be like?
What's the weather like?
What'll the weather be like tomorrow?
What will the weather be like tomorrow?
What's the weather like tomorrow?
Will the weather be better tomorrow?
What'll Ill the weather be tomorrow?
What's the weather forecast for tomorrow?
What's the weather supposed tomorrow?
How will the weather he tomorrow?
Add/Delete: 50-70%
Add/Delete: 70-90%
Will the weather be perfect tomorrow?
How do you know what the weather will like? Is it supposed to be cold tomorrow!
What's the weather for tomorrow?
What's the weather like on the course?
What will the weather be like in the South? I'm not fan of the weather.
Hopefully the weather will be better tomorrow.
What's the weather like the next day?
What the temperature and what is the humidity.
Table 5:
Random augmentations of input text "What'll the weather be tomorrow?", using random style vector deltas with components sampled from (0, 0.08).
A.2 SETTINGS USED FOR QUALITATIVE ANALYSIS
For each of the transfer types (c.g., formal <+ informal) in Table we specify the intended target styles to the model through a tiny set of exemplars. These exemplars are provided in Tables 10 Additionally, for each transfer type, we select delta scale À and add/delete rates. These settings selected through initial experiments, and are held fixed across all examples of transfer shown.
A.3 HUMAN REFERENCE BLEU
Li et al. 2018) provide human reference transfers for their Amazon test data, and report BLBU scores of model outputs against these targets. In principle, we believe this metric is less informative than self- BLEU, as style transfer is a relatively open-ended task, and successful transfers may differ significantly from the single human reference. However, for completeness, we report "reference BLEU" of models and those of prior work in Figure
We observe BLEU and self-BLEU
13

---

Under review as a conference paper at ICLR 2021
Reserved Exemplars
Emotive Exemplars
That very pretty painting.
OMG, that's such beautiful painting!
I'm excited see the show.
I'm sooo excited see the show, it's going to stel- lar!!
I'm surprised they rescheduled the meeting.
4. This specimen is an example of the baroque style.
3. T absolutely can not helieve that they rescheduled the meeting!
5. After the performance. we meal.
This wonderful specimen truly spectacular ple baroque style.
After the superb performance, we ate delicious meal.
Table 6: Emotiveness transfer exemplars. Transfer settings: X=9, add/delete rates: 0-100%
American Exemplars
British Exemplars
It bucks.
1. It quid.
My neighbor apologized.
My neighbour apologised.
I'm heading out with some friends.
I'm heading out the pub with some mates.
Table 7: Dialect transfer exemplars. Transfer settings: X=8, add/delete rates: 10-30%.
Polite Exemplars
Rude Exemplars
No thank you, I'd prefer to.
1. Hell no, you can't 't make do that.
This game could have been better designed.
This game
a piece garbage!
Do you know why they might have delayed the launch?
Why in god's name would they delay the damn launch?
Sorry, wasn't 't certain you were joking.
Are you frigging kidding me?
Table 8: Politeness transfer exemplars. Transfer settings: λ=5, add/delete rates: 20-50%.
Formal Exemplars
Informal Exemplars
This remarkably thought-pmeg provoking read.
reading this makes think
It certainly amongst my favorites.
def one of my favs
3. We humbly request your presence at our gala on the 12th.
come swing by our bbq next week if ya can make it
Table Formality transfer exemplars. Transfer settings: >=4, add/delete rates: 40-80%
Positive Exemplars
Negative Exemplars
stars, love it.
Zero stars, hate
Table 10: Sentiment transfer exemplars. Transfer settings: A=3, add/delete rates; 0-100%.
14

---

Under review as a conference paper at ICLR 2021
Model
BLEU
Self-BLER
100
450-70%
CrossAligned
440-60% 60%
30-50%
Relete&Retrieve
29.7
56.9
20-40%
B-GST
29.0
54.2
X
CP-G
17.0
35.5
CrossAligned
GST
10-30%
CP-B
19.4
TexISETTR (0-20%)
39.0
73.3
Detriee.
TexISETTR (10-30%)
30.7
55.8
Sontimank
0-20%
TexISETTR (20-40%)
20.0
34.7
TextSETTR
TexISETTR (30-50%)
10.6
Other la lebe--fre models
TextSETTR (40-60%)
5.5
9.1
Models trained with labels
TeXISETTR (50-70%)
2.2
3.6
15
25
35
Reference BLEU
Figure 5: BLEU scores between model outputs and human references provided by et al. (2018), along with self-BLEU for comparison. The first group of models in the table had access to labels training time, while the second group did not. TextSETTR (X-Y%) refers our model with add/delete rate ranges set to X-Y%.
are highly correlated, and the "Accuracy vs. BLEU" plot conveys the same relationships we saw in Figure As before, all BLEU scores calculated using SacreBLEU (Post 2018
A.4 AMAZON REVIEWS PREPROCESSING
We use the code in Figure 6 to process raw Amazon reviews from the Ni et al. (2019) dataset and extract pairs adjacent lines, preprocessed to have similar format to Li et al. (2018) dataset. split reviews newlines, and clip lines to 100 characters, always ending with period. This gives results similar ei al. (2018), where one line may contain multiple sentences, and may consists of "half-sentence" ending with "e.g." similar non-sentence-fieg period. Additionally, various tokenization and normalization operations to roughly match the observed Li et al. (2018) text.
apply
15

---

Under review as a conference paper at ICLR 2021
import re
from html parser import HTMLParser
html_parser" HTMLParser
def preprocess (line)
"""Simulate Li et al. preproceasing of one review line. Lowercase.
line = line.lower
Replace apostroppee, parens and quotes with spaces.
line = sub'""'
line)
Replace dollar values =
line = sub("\
"$", line)
Replace percent values
line = re. sub"" "$", line)
Replace digits
digits => num_num
line = re. sub(" \d[,
num_num ", line)
# Replace multi-digits and codes =-> num num_extend
line = re. \d[" ]*", num_extend", line)
# Remove remaining numbers, including decimals. "", line)
line sub
# Add spaces around certain punctuation marks. r" \1 ", line)
line re.
# Remove double spaces after periods before words. return re. sub ([a-z)) r". \1" line)
def acceptable (line) """Check if text looks like an acceptable line from Li et al. """ if not line or len (line) 30 or (1ine) >= 100:
return False
Avoid lines with any char absent from Li et al. train. if re. re.search ('[" : 'a-2(1)]', line):
return False return True
def latp_to_lstt_priidddooooo (1ine) return ::nen:len (line) dinel::-1) index (*.())
def Silacent_liness (review):
"""Extract list of adjacent 1ine pairs from review text. """ review = Baaml_parrer.ueeee
(review)
review review. replace A)", ("()
Simulate Li et al. splitting and filtering.
if Ann not in review:
return
lines = review.split ('An')
lines [preprocess for in lines if and "." in 1[:100]]
lines [preprocess (1) for in lines]
lines = [1 for 1 in lines if acceptable [1)]
if len (lines) 2:
return
return 1ist (zip (lines [: :-1], lines
Figure 6: Python code to extract adjacent lines of text from raw Amazon reviews, producing outputs in similar style to Li al. (2018).
16


Reviews:
[{"review_id": "UmAsk5M1yes", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tackles the problem of extracting and modeling text (writing) style without using labeled data.\n\nTraditionally, modeling text style requires either paired sentences (supervised) or two pools of unpaired sentences (so-called unsupervised). This paper exploits 1) language model pretraining and 2) free supervision signals in text to achieve modeling text styles without labeled data.\n\nFor the 1st point, the authors (correctly) hypothesize that a large pre-trained language model (e.g. T5) already “knows” about style information and one can isolate the style information using the right fine-tun signal.\nFor the 2nd point, the authors assume that text style (e.g., sentiment) is slow-moving and consistent for adjacent sentences (I guess It’s a similar signal is exploit by Next Sentence Prediction in BERT, and CBOW in word2vec?). And this is used as the “free” supervision signal to their model finetune.\n\nIn the experiments section, the authors test their model on transfer learning tasks. The experiments (Fig 2 &3) seem to suggest that at at given high content preservation score ( > 50), the proposed model is not as accurate as other supervised models. But with low content preservation, the model can steadily improve accuracy by modifier more words (Fig 2). \n\nIn Figure 2, the TextSETTR accuracy has almost (inverse) linear response wrt to the content preservation score. But in Figure 3, the plot for TextSETTR stopped at “30-50%”. What would happen if the modification percentage is higher? Would TextSETTR get closer to 100% accuracy?\n\nAnother small issue with Figure 3: I believe the task is binary (pos vs neg). It might be more useful to plot the accuracy from 50%~100% instead of from 0% ~100% since 50% is the practical lower bound performance.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "S3l_DW0jEw6", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\nReasons for score: \n\n\nThis paper proposes a novel approach to the label-free style transfer task where an input is corrupted via different strategies and fed into an auto-encoder which is additionally conditioned on its prior adjacent context sentence via a \"style encoder\" which adds its mean pooled hidden state to the former before decoding.  Both encoders are initialized to and leverage the strength of pre-trained T5 model.  Additionally the amount of addition/deletion of tokens is tunable at both training and inference time.  \n\nThe overall idea is quite compelling, but the paper's argument could be improved greatly with revisions to its existing experimental setup and more evaluation overall to better and more thoroughly back its claims.  \n\n \n##########################################################################Pros: \nPros:\n1) The authors propose a novel approach to the label-free style transfer task that is based on evaluating how training under different combinations of 3 noising strategies  ( Noise, Back Translation and Noisy Back Translation ) on input texts can be used in conjunction with an auto-encoder and style encoder over the prior sentence context to then do inference given an input text and a small number of examplars of the source and target styles.  The idea is laid out fairly clearly both for training and inference though certain particulars there and in the experiments section were a little unclear and could have benefited from some formal notation ( see next section ). \n\n2) The quantitative results on the Amazon dataset for their best model on both the full data and few shot regimes are quite impressive compared with the other label-free style transfer paper they compare against ( Xu 2020 ) \n\n3) The few qualitative examples shown are impressive ( particularly the American <-> British ones ) \n\n4) The tuning hyper parameter is a useful addition ( though it'd be interesting to see how dataset dependent it is )\n \n##########################################################################\nCons: \n1) Overall the writing was a little unclear at certain spots and could have benefitted greatly from some equations explicitly stating the setup.  For instance i was unclear if the context representation was added ( which the text suggests ) or concated to the noisy encoding before being decoded ( the later is suggested by Figure 1 especially since the 4 float values for the tuning rate ranges are said to be prepended ).  Similarly the sampling strategy used ( as opposed to greedy decoding )\n\n2) Doing quantitative evaluation on only one dataset ( Amazon ) and then only showing examples of how the model does qualitatively over another dataset ( English Common Crawl \"C4\" ) without doing any human eval is a little disappointing.  The idea is novel enough where even just doing some more automated evals would be suffice for me.  For instance, why weren't automated metrics given for the English Common Crawl dataset?  Those results and having information on training set size and the average token size of each example for C4 should be given.  Also the authors compare against Lample 19 for the pos->pos and neg->neg setup for the Amazon data, why not show the results for the SYelp data as well?   Does the 20-40% add/delete tuning work better there as well or is it dataset dependent?\n\n3) There are two issues with your use of the Amazon dataset.  First it doesn't really provide apples to oranges comparison against the prior papers as they train/test on the same data from Li 18 which has ~ 270 K training examples whereas the work here generates 23.6 M training examples.  It seems you should either see how those papers do with that much data or limit your dataset to be of at least comparable size to be fair.   Second, the Amazon test set is only of size 500 so assessing results on that alone seems in-suffice.\n\n4) The paper hypothesizes that style is a \"slow moving\" feature consistent over large spans of text hence the use of only the prior adjacent sentence as context.  The paper shows that using just an adjacent sentence gives promising results, but doesn't show that its necessarily better than just using examplars or using a leading paragraph to derive the style from.   I don't think this is exactly necessarily to address here, but for future work it would be nice to see such a comparison.  Additionally, how would using a 1000 examplars as opposed to 100 at inference time affect performance?  A graph showing how accuracy and content preservation were affected by that would be interesting to gain better understanding.  Similarly showing how just the NBT strategy did alone (as opposed to N + NBT ) would be interesting. \n\n5) I didn't find the multiple aspect UMAP embedding visualization particularly convincing for how well the embeddings separate the \"sentiment\" aspect as there is substantial overlap within each category ( particularly software ).  I don't know if this is particularly necessary for your argument in my opinion ( especially compared with evals on other datasets), but if so then it'd be interesting to have quantitative numbers for those separations and compared with how it differs from just taking the T5 embeddings and doing the same UMAP? \n\n6) The \"replace\" noise strategy feels pretty arbitrary.  Is there any motivation behind using that as opposed to using a LM or another strategy to replace tokens?\n\n7) A citation for using Self Bleu as opposed to Multi-Bleu in the Evaluation Procedure section would be helpful.  Additionally a citation of Ke Wang, Hang Hua, Xiaojun Wan \"Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation\" ( Neurips 19 ) particularly for its \"tunable\" aspect could be an addition to the Related Work section.\n\n8) This is nitpicky and probably for future work, but the use of examplars doesn't necessarily limit the user to a pre-defined set of styles ( like the unsupervised case does ), however it would be interesting to see what would happen given out of domain examplars for either the source or target classes at inference time\n\n##########################################################################\n\nQuestions during rebuttal period: \n\nPlease address and clarify the cons above \n \n#########################################################################\nPossible prior citation \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CRuZ4_C-DtW", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method for text-style transfer where they dont need label information of the interested style.\nThe extend t5 model to develop their architecture which models style extract a style vector from arbitrary text and\n use this vector to condition the decoder to perform style transfer.\n\n\n\nHowever the current presentation of the paper is hard to follow. Which raises the following concern:\n\n1. As they need to provide two sentences which has to be chronological sentences, it is not possible to obtain always. \nHence, they randomly select sentence pair, but then two sentences may not bear same style. How the authors are incorporating the same?\n\n2. For inference they need sentence exemplar both both style. This contradicts their previous claim.\nThey have not compared with\n\n3. How noise introduction in helpful for style corrupted sentence generation? They do not use any heuristic and from a single sentence there can be multiple variation of the corrupted version, are all those taken at the time of training? Then the style it is learning is possibly not the intended one as different corrupted sentences might  need different style to reconstruct the sentence back.\n\n4. Model portion is extremely cryptic. What is back translation etc? At least should be explained in one line.\n\n5. Due to the unreadability of the model, I cannot provide judgement on the result section.", "extra_scores": {}, "extra_text": {}}, {"review_id": "-JFyPegpsl", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the author proposed a transformer-based encoder-decoder framework for label-free text style transfer. The described task under the unsupervised setup is important and instructive for the text style transfer domain. The model architecture is well demonstrated and the writing is easy to follow up. The experiment results show satisfying performance even comparing with state-of-the-art supervised methods. \n\nHowever, I have some concerns that may lead to the weakness of the paper:\n\n1. About the assumption: The author claimed the method is label-free. However, the \"unsupervised\" model is based on an assumption that two adjacent sentences should have the same style. With the assumption, the training of the model is actually weak-supervised because in each step the paired sentences are provided with the same style. This assumption is actually utilizing the context-level supervision instead of the sentence-level labels. This idea is also previously used in [1].\n\n2. About the framework: The model adds the exacted style vector to all the hidden states of the encoder. How can the author guarantee that the encoder will not extract the style information of the input? Also, is it possible that the style vectors still contain the content information from context?\n\n3. About the style vector: The model changes the style of the sentence by adding a direction from the source style vector to the target style vector. The approach may work under the assumption that the style vector space is linear to the semantic meanings. But there is no regularizer or training loss to guarantee the linearity assumption of the style extractor. Why didn't the author directly replace the sentence style vector with the target style vector?\n\n4. About the dataset: The model is only evaluated on one dataset. It could be more solid if the author conduct experiment on other commonly-used style transfer datasets such as Yelp and Personality-Captions [2]. Besides, the split Amazon review dataset only has two sentiment classes as \"positive\" and \"negative\". It could be more persuasive if the model is tested on other datasets with multiple sentiments, to verify the effectiveness of the proposed re-styling strategy. \n\n5. About the evaluation: The author only reported the performance of content and style preservation (Acc and self-BLEU). The sentence generation quality is expected to report by testing the BLEU score of the generated sentences.\n\n6. In Figure 4, there is no clear pattern between positive and negative sentence embedding. The difference in embedding space is mainly caused by different topics, which in my understanding are the content of sentences. This means the style vectors cannot eliminate the content information and also failed to separate sentences with different sentiments.\n\n\nReference:\n\n[1] Zhang et al. Improving the Dialogue Generation Consistency via self-supervised Learning, 2019\n\n[2] Shuster et al. Engaging Image Captioning Via Personality, 2018", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new method for label-free text style transfer. The method employs the pre-trained language model T5 and makes an assumption that two adjacent sentences in a document have the same style. Experimental results show satisfying results compared with supervised methods.

Pros. • The paper is generally clearly written. • The proposed method appears to be new. • Experiments have been conducted.

Cons • The fundamental assumption of the method is not convincing enough. (Issue 1 of R3, Issue 4 of R4, Issue 1 of R2) • The proposed model is also not convincing enough. (Issues 2 and 3 of R3, Issue 3 of R2) • There are problems with the experiments. For example, it would be better to use more datasets in the experiments. (Issue 4 of R3, Issue 2 of R4)

Discussions have been made among the reviewers. The reviewers appreciate the efforts made by the authors in the rebuttal, including the additional experiments. However, they are not fully convinced and still feel that the submission is not strong enough as an ICLR paper.


Decision: Reject