Venue: NAACL 2021

# StylePTB: A Compositional Benchmark for

Fine-grained Controllable Text Style Transfer

Yiwei Lyu\({}^{\diamond*}\), Paul Pu Liang\({}^{\diamond*}\), Hai Pham\({}^{\bullet\star}\), Eduard Hovy\({}^{\blacktriangle}\), Barnabas Poczos\({}^{\triangledown}\), Ruslan Salakhutdinov\({}^{\triangledown}\), Louis-Philippe Morency\({}^{\star}\)

\({}^{\circ}\)Machine Learning Department, Carnegie Mellon University

\({}^{\star}\)Language Technologies Institute, Carnegie Mellon University

{ylyul,pliang,htpham}@cs.cmu.edu

###### Abstract

Text style transfer aims to controllably generate text with targeted stylistic changes while maintaining core meaning from the source sentence constant. Many of the existing style transfer benchmarks primarily focus on individual high-level semantic changes (_e.g._ positive to negative), which enable controllability at a high level but do not offer _fine-grained_ control involving sentence structure, emphasis, and content of the sentence. In this paper, we introduce a large-scale benchmark,StylePTB, with (1) paired sentences undergoing \(21\) fine-grained stylistic changes spanning atomic lexical, syntactic, semantic, and thematic transfers of text, as well as (2) _compositions_ of multiple transfers which allow modeling of fine-grained stylistic changes as building blocks for more complex, high-level transfers. By benchmarking existing methods on StylePTB, we find that they struggle to model fine-grained changes and have an even more difficult time composing multiple styles. As a result,StylePTB brings novel challenges that we hope will encourage future research in controllable text style transfer, compositional models, and learning disentangled representations. Solving these challenges would present important steps towards controllable text generation.

+
Footnote †: leftmargin=*] * authors contributed equally

[https://github.com/lvyiwei1/StylePTB/](https://github.com/lvyiwei1/StylePTB/)

## 1 Introduction

At the heart of interactive AI systems lies the element of communication as a channel to convey intentions using different stylistic attributes. Research in human-AI interaction has focused on building dialog systems Celikyilmaz et al. (2018), virtual assistants Cooper et al. (2004), and intelligent agents Kim et al. (2013); Liang et al. (2020a); Pittermann et al. (2010) that can communicate their intentions with specific _styles_ for different situations, target audiences, and environments Lample et al. (2019); Li et al. (2018). For example, expressing the same facts using either formal or informal styles can be more suitable for certain target audiences Rao and Tetreault (2018).

What is a _style_ in natural languages? Existing style transfer benchmarks primarily focus on individual high-level stylistic changes across sentiment Shen et al. (2017), formality Rao and Tetreault (2018), politeness Madaan et al. (2020), and writing styles Jhamtani et al. (2017). Figure 1 provides some motivating examples to show that the high-level style transfers as commonly studied in existing benchmarks (_e.g._ Yelp for sentiment Shen et al. (2017) and GYAFC for formality Rao and Tetreault (2018)) can in fact be seen as composed from a dictionary of fine-grained style constructs. This alternative way of studying styles brings additional flexibility that enables _fine-grained_ control with the possibility to _compose_ a broader space of styles spanning tense, sentence structure, phrase emphasis, and information contained in the sentence. However, the missing link is a benchmark dataset that offers this type of fine

Figure 1: StylePTB provides a large-scale resource to study _fine-grained compositional style transfer_. The styles provided in StylePTB (in green) span lexical, syntax, semantic, and thematic aspects DiMarco and Hirst (1993) which can be composed to form high-level style transfers as commonly studied in existing benchmarks (_e.g._ Yelp for sentiment Shen et al. (2017) and GYAFC for formality Rao and Tetreault (2018)).



grained style constructs, with the controllability to compose these stylistic transfers.

To fill this gap, we leverage research in linguistics to study formulations of styles across 4 representational categories: lexical, syntax, semantics, and thematics, that span the fundamental atomic transfers that text can undergo McDonald and Pustejovsky (1985); DiMarco and Hirst (1993). Using these insights, we introduce a large-scale benchmark with (1) paired sentences undergoing \(21\) fine-grained stylistic changes spanning the most atomic lexical, syntactic, semantic, and thematic style constructs, as well as (2) _compositions_ of multiple transfers which model how fine-grained style constructs compose to form more complex, high-level transfers. Our dataset, called StylePTB, builds upon Penn Treebank Marcus et al. (1993) by annotating each sentence undergoing these fine-grained style constructs, resulting in a large-scale resource spanning \(59,767\) sentence pairs across \(21\) individual styles and an additional \(35,887\) sentence pairs across \(32\) compositions of multiple styles.

StylePTB allows us to study the performance of state-of-the-art style transfer models when faced with the new challenge of fine-grained style transfer. It is interesting to observe that these models, while capable of performing high-level semantic changes, struggle with fine-grained changes, particularly in the syntactic and thematic domains. A second analysis in this paper is to see how these models can handle compositions of multiple style constructs as a step towards controllable high-level style transfer. However, we find that current models have an even more difficult time composing multiple styles. As a step towards this desiderata, we also propose an approach (CS-GPT) based on pre-trained language models Radford et al. (2019) that achieves compositional style transfer. We believe that StylePTB will bring novel challenges that we hope will encourage research in controllable generation, compositionality of styles, and learning disentangled representations John et al. (2019). From a broader perspective, we conclude with the observation that controllable style transfer models trained on StylePTB can help mitigate social biases in pre-trained language models.

## 2 Related Work

Several lines of research have aimed to formalize **styles in natural languages** through computational and linguistic perspectives DiMarco and Hirst (1993). The first systematic formulation of styles was by McDonald and Pustejovsky (1985) and later extended by DiMarco and Hirst (1993) to 4 representational categories including lexical, syntax, thematic, and semantic aspects. Following this, there has been some early efforts applying stylistic analysis into dialog generation Hovy (1987), machine translation DiMarco (1994), and text generation Gatt and Krahmer (2018). We take advantage of this prior work when formalizing our new StylePTB dataset.

Current **benchmarks for style transfer** focus on high-level style definitions such as transfer of sentiment Shen et al. (2017); Lample et al. (2019); Li et al. (2018); Wu et al. (2019), politeness Madaan et al. (2020), formality Rao and Tetreault (2018); Liu et al. (2020); Krishna et al. (2020), writing styles Jhamtani et al. (2017); Syed et al. (2020); Jin et al. (2020) and some other styles Kang and Hovy (2019). However, these only focus on only high-level styles, unlike StylePTB.

**Computational models for style transfer** span statistical NLP methods Hovy (1987); Xu et al. (2012), neural generative models Prabhumoye et al. (2018); Lample et al. (2019); He et al. (2020), and Retrieve-and-Edit approaches Li et al. (2018); Hashimoto et al. (2018); Guu et al. (2018); Sudhakar et al. (2019); Madaan et al. (2020). These approaches work for a predefined set of styles but are unable to generalize to compositions of styles.

**Evaluating style transfer** is difficult due to the diversity of plausible transferred sentences. In addition to automatic scores such as BLEU, perplexity, or binary classification accuracy of style transfer Hu et al. (2017); Lample et al. (2019); He et al. (2020), other automatic metrics Fu et al. (2018); Mir et al. (2019) and human evaluation are also commonly used Li et al. (2018); Shen et al. (2017).

## 3 Fine-Grained Style Constructs

As a step towards enabling _fine-grained_ control with the possibility to _compose_ a broader space of styles, we first define style constructs at fine-grained levels spanning lexical, syntactic, semantic, and thematic aspects. When selecting these style constructs, we have \(2\) goals in mind: (1) they should be representative of the four aspects (lexical, syntactic, semantic, thematic) following the formal categorizations in DiMarco and Hirst (1993), and (2) the transfers should be consistent (_i.e._ well-defined such that if multiple annotators are asked to modify the same sentence, the results will be similar). With these goals in mind, we summarize the 

following \(21\) chosen fine-grained style constructs spanning \(4\) categories and also provide detailed examples in Table 1.

**Lexical transfers** are those at fine-grained lexicon levels (_i.e._ vocabulary or words) that include word constitutions Heine et al. (2002) and word meaning Cruse et al. (1986). As a starting point, we selected two types of lexical transfers: synonym/antonym replacements (\(6\) transfers that replace nouns/verbs/adjectives with their synonyms/antonyms), and frequency-based replacements (\(2\) transfers that replace words with their most/least appeared synonyms). The synonym/antonym resources are taken from Wordnet Fellbaum (2012).

**Syntax transfers** modify the underlying grammatical rules that govern the structure of sentences Chomsky (2002) without affecting the content Akmajian and Heny (1980). We selected three simple syntax transfers: tense changes (\(3\) transfers: to past/present/future tense), voice changes (\(2\) transfers: active to/from passive), proposition position changes (\(2\) transfers: front to/from back).

**Semantic transfers** are changes to the meaning of sentences Bagha (2011) that not only extend beyond lexical Cruse et al. (1986) and syntax-level Kratzer and Heim (1998) changes, but also include modifications using indirect information such as referring Strawson (1950), situations Barwise and Perry (1981) or intentions and extensions Allwood et al. (1977). As a starting point, we defined two simple types of semantic transfers: (1) Info removal: \(3\) transfers on different deletions: word-level (removing adjectives and adverbs), phrase

\begin{table}
\begin{tabular}{l l p{142.3pt} p{142.3pt} p{142.3pt}} Aspect & Transfer & Original Sentence & Additional Info/Emphasis & Transferred Sentence \\ \hline  & Non synonym replacement & The shift & no ’t affect operations. & The displacement & won’n’t affect operations. \\  & Non attourm replacement & Investors will develop thicker skins and their confidence & Inventors will develop thicker skins and their evidence & invf return he says. \\  & Verb synonym replacement & The meeting is expected & to call & The meeting is anticipated to call for heightened university for two years. \\  & Verb synonym replacement & Re needed & that higher gasoline price & He ignored & that higher gasoline prices will help buy the October totals. \\
**LEXICAL** & ADJ synonym replacement & Most other states have enacted & Most other states have enacted & Mike bars. \\  & ADI synonym replacement & Similar basis & & It is also planning another & It is also planning another night of original series. \\  & Most frequent synonym replacement & Republics counterned that long-range revenue estimates were unreliable. Least frequent & Reproublics countered that long-range revenue judges were unreliable. \\  & Lyson’s replacement & Merrill Lynch Capital Markets Inc. is the scope [interactive] for the offering & Merrill Lynch Capital Markets Inc. & is the solution [interactive] for the obligation \\ \hline  & To future tense & In his also planning another night of original series. & It will be also planning another night of original series. & See, Mitchell press & them to deisst. \\  & To present tense & In his also planning another night of original series. & In was also planning another night of origin & His was since it shows. \\
**SYNTAX** & Active to passive & He also received 20-year sentences for each of the 24 passengers injured. Mask bits are drafted by breeaucrats not politicians. PP front to lack & 20-year sentences also were received by him for each of the 24 passengers injured. More subjects not politicians draft most blits. \\  & Passive to active & The conflict has been strong inside 1997. & Likly declined comment. & Here extracts not politicians draft most blits. \\  & PP front to host & In Indianapolis & Lilly declined comment. & Lilly declined comment & **in Indianapolis** \\  & PP back to front & The dollar has been strong inside 1997. & Unlike 1997. & the dollar has been strong. \\ \hline  & ADI or ADV removal & The controls on cooperatives appeared & The controls on cooperatives appeared & The controls appeared relatively liberal when first introduced. \\
**SEMANTICS** & Substatement removal & The controls on cooperatives appeared & The controls on cooperatives appeared & The controls on cooperatives appeared relatively liberal. \\  & Information addition & He reports his business is up slightly from & **customer**; **waiting to buy seafood** \\  & & from customers replacing old stock. & **to buy**; **serafood** & and replacing old stocks. \\ \hline  & Verb/Action emphasis & He intends to add to the litigation staff. & Add to the litigation staff is what he intends to do. A spokesman said the year-earlier number was 56 million a spokesman said. & **Adding** to the litigation staff is what he intends to do. \\
**THEMATICS** & Adjective emphasis & The comparable year-earlier number was 56 million a spokesman said. & **comparable** & A spokesman said the year-earlier number of 56 million was comparable. \\ \end{tabular}
\end{table}
Table 1: Examples of each of the \(21\) defined style constructs across lexical, syntactic, semantic, and thematic aspects found in StylePTB. The original phrase is in cyan and the corresponding target phrase is in magenta. Note that some thematic and semantic transfers require additional information, highlighted in red.

 

level (removing propositions), and substatement level (removing entire substatements) that represent _referring_ and _situations_, as well as (2) Info addition: \(1\) transformation that adds a given piece of information regarding a particular phrase in the current sentence representing _extension_.

**Thematic transfers** concern the placing of emphasis across different parts in a sentence Stevenson et al. (1994) to highlight different aspects of the same event DiMarco (1994). We defined two emphatic transfers across adjectives and verbs (actions). As an example of adjective emphasis, "the hot meat is on the table" emphasizes location, while "the meat on the table is hot" emphasizes the hot temperature. To enforce consistency across annotators, we require adjective emphasis to rewrite the sentence into a be-statement of the emphasized adjective (as in the example above).

**Analysis:** To evaluate how useful these \(21\) selected atomic transfers are, we randomly sampled \(50\) sentence pairs from GYAFC and \(50\) sentences from Yelp with their reference transfer generated by Deep Latent Sequence Model He et al. (2020) and manually tried to complete the transfers by composing one or more of the \(21\) atomic transfers we have defined, together with capitalization fixes and word-spelling fixes. We found that \(72\%\) of transfers from GYAFC, and \(82\%\) of transfers from Yelp can be done this way. Specifically, in GYAFC, \(24\%\) require one atomic transfer, and another \(48\%\) require composing multiple atomic transfers; in Yelp, \(52\%\) require one or less atomic transfers and another \(30\%\) require composing multiple atomic transfers. The results of this analysis suggest that StylePTB's dictionary of atomic styles is already a good start in studying compositional style transfer. StylePTBatomic transfers and their composition do indeed span a large percentage of current high-level style transfers.

## 4 The StylePTB Dataset

Using these selected \(21\) style constructs, we now illustrate the steps towards collecting and annotating parallel sentences across style transfers.

### Dataset Preprocessing

We use Penn Treebank (PTB) Marcus et al. (1993) as our source of sentences. Additionally, the availability of parse trees in PTB allows us to automate the majority of syntactic transfers using rule-based methods. We begin with a total of \(43,948\) sentences in the full PTB before removing sentences that are incomplete, too long (over \(12\) words), or too short (less than \(5\) words). This leaves \(7,719\) sentences (see Figure 2 for statistics and Appendix A.1 for full details).

### Generating transferred sentences

We give a brief overview of the data annotation process (see SectionA.3 for full details).

**Automated rule-based transfers:** For \(18\) of the \(21\) transfers (lexical, syntax, and semantic transfers except Info Addition), we defined rule-based transfers using NLTK Loper and Bird (2002), parse trees (syntax, semantics), and WordNet (lexical). After human quality control, the total number of sentences transferred is listed in Table 2 (see SectionA.2 for more details on automated generation and SectionA.4 for human evaluation on quality of generated sentences)

**Transfers with human annotations:** For the remaining \(3\) transfers, we have human annotators (via Amazon Mechanical Turk) manually rewrite them due to the difficulty of automating the process. See SectionA.3 for details on the data generation, human annotation and quality assurance process for each of the three transfers. After annotations and quality control, we obtained \(696\) rewritten sentences for adjective emphasis, \(1201\) rewritten sentences for verb emphasis, and \(2114\)

Figure 2: Statistics: (a) the distribution of sentence lengths, (b) count of word tokens by part-of-speech, and (c) the top \(30\) most frequent tokens.StylePTB exhibits diversity in sentence form and style transfer annotations.

 

valid sentence-information pairs with their transferred sentence with information added.

### Relative Difficulty of Transfers

Lexical transfers can be done by replacing individual words and is simple to evaluate. To evaluate the difficultly of the remaining syntax, semantic, and thematic transfers, we calculated the token-level (_i.e._ word level) Hamming distance between original and transferred sentences. Using this metric, we categorized these transfers into easy, medium and hard categories (see Table 3). We also evaluated semantic measures from BERT embeddings Devlin et al. (2018) but found it less correlated with human judgment (see Appendix A.5).

### Compositional Transfers

To allow for compositionality, we also generated compositional data that includes parallel pairs of sentences linked by multiple sequential transfers. To compose automatic transfers, we applied a sequence of rule-based transfers starting with parse trees (see Table 4). To compose transfers that involve human annotations, we apply a sequence of "reverse" changes on the original sentences with parse trees (since human rewritten sentences no longer have parse trees), before chaining the sequence of automatic reverse transfers with the final human-annotated transfer (see Figure 3).

## 5 A Model for Compositional Transfer

We extend the pre-trained GPT2 language model Radford et al. (2019) for parallel style transfer by giving it designated style transfer tokens as input in addition to the source sentence. For example, for each individual binary style \(s_{i}\), we define a style transfer token \(\Delta_{i}\in\{0,1,2\}\) where \(\Delta_{i}=0\) represents keeping \(s_{i}\) unchanged, \(\Delta_{i}=1\) represents a change from \(s_{i}=0\) to \(s_{i}=1\), and vice versa for \(\Delta_{i}=2\). We likewise extend the definition of \(\Delta_{i}\) for styles taking more than \(2\) values.

Given a parallel (source, target) pair (\(s,t\)), we define the appropriate transfer token \(\Delta\in\{0,1,2\}\)

Figure 3: Example of generating sentence pairs that compose tense and voice changes. Starting from an original sentence (**green box**), we sequentially apply parse tree transfers (**blue arrows**) to obtain multiple transferred sentences (**yellow box**), yielding multiple parallel pairs (**yellow arrows**). We use transfer tokens (\(\Delta_{1},\Delta_{2}\)) to track changes (see Section 5 for details).

\begin{table}
\begin{tabular}{l|c c c} \hline \hline  & No Voice & Active To & Passive To \\  & Change (0) & Passive (1) & Active (2) \\ \hline No Tense Change (0) & \(2808\) & \(2808\) & \(2808\) \\ To Future Tense (1) & \(5294\) & \(2647\) & \(2647\) \\ To Past Sense (2) & \(2077\) & \(656\) & \(1421\) \\ To Present Dense (3) & \(3304\) & \(1536\) & \(1768\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Number of sentence pairs for each composition of tense change and voice change in the generated compositional dataset.

 

and train using maximum likelihood estimation to predict every word \(t_{j}\), for \(j=1,2,\ldots,T\), in the target sentence given the source and \(\Delta\):

\[\theta^{*}=\operatorname*{arg\,max}_{\theta}\mathbb{E}_{(s,t)\sim\mathcal{D}} \left[\sum_{j=1}^{T}\log p_{\theta}(t_{j};s,\Delta)\right], \tag{1}\]

where \(\theta\) denotes the pre-trained GPT2 parameters and \(\theta^{*}\) denotes the parameters after fine-tuning on StylePTB. Note that we also train the model to reconstruct the same source sentence again when setting \(\Delta=0\) (no style change), which we found to help bridge the domain shift between data used to pre-train GPT2 and sentences inStylePTB.

As a step towards _compositionality_, we also train with (source, target) pairs that undergo multiple atomic style transfers as provided in StylePTB, resulting in multiple style transfer tokens \(\Delta_{i}\) being activated at the same time. We call the resulting model CS-GPT (Compositional Style GPT) and show its architecture in Figure 4. Learning separate representations for each \(\Delta_{i}\) results in disentangled style variables that can then be composed as desired. Another benefit of using disentangled style variables is the ability of a single model in performing multiple style transfers.

## 6 Experiments

We test the performance of current style transfer models on StylePTB. Anonymized data and code is included in the supplementary, and we present extra details and results in Appendix B and C.

Figure 4: CS-GPT uses multiple transfer tokens \(\Delta_{i}\in\{0,1,2\}\) to enable compositional style transfer across multiple styles in our model StylePTB.

 



### Datasets and Metrics

We use StylePTB and evaluate on the \(13\) non-lexical transfers (since lexical changes works best with fixed word substitutions). Please refer to Appendix B.1 for dataset preprocessing details. Automated evaluation metrics consists of automatic BLEU scores, METEOR scores, ROUGE_L scores, and CiDER scores between generated and ground truth sentences Sharma et al. (2017). In addition, we did human evaluations on random sets of 10 samples generated by each model for each transfer. We followed prior work He et al. (2020) and had \(2\) independent annotators each rate transferred sentences on three aspects (clarity/grammar, content preservation, style change) on a \(1-5\) Likert scale, and takes average.

### Baseline Models

We evaluate the following baselines commonly used in style transfer. Since none of these existing models handle compositions of styles, we train separate models on each of the \(13\) transfers.

**1) GPT2:** We fine-tune pre-trained GPT2 Radford et al. (2019) on each transfer with the source as input and predicting the target using MLE, similar to Liu et al. (2020); Syed et al. (2020).

**2) Seq2seq:** A Seq2Seq model Sutskever et al. (2014) with attention trained using MLE Zhou et al. (2020); Jin et al. (2020).

**3) RetrieveEdit:** Given input \(x\), a retriever is trained to pick a similar training example \((x^{\prime},y^{\prime})\). We treat \(y^{\prime}\) as our prototype and use a trained editor to edit it into desired output \(y\)Guu et al. (2018); Madaan et al. (2020).

**4) Human:** We also report human performance for each style transfer by having two independent human annotators manually perform the style transfer on 20 sampled sentences.

### Results and Observations

We evaluate these \(3\) baseline models on the style transfers in StylePTB and show results in Table 5. We make the following observations:

**Baseline comparisons:**RetrieveEdit performed equally well compared to GPT2 in some transfers such as To Future Tense and performs significantly better compared to GPT2 in most transfers. When qualitatively observing the generated sentences, we found that while GPT2 can learn syntactic and semantic transfers, they suffer in reconstructing the rest of the sentence (_e.g._ making word repetitions). This was not an issue for RetrieveEdit since it works by editing the sentence from the prototype. Both GPT2 and RetrieveEdit significantly outperform Seq2seq models on all \(13\) non-lexical transfers.

* all three baselines struggled on this task, which is intuitive because shifting emphasis requires completely different sentence structure changes on sentences and emphasized words. We found that GPT2 and Seq2seq tend to struggle with grammar and word repetitions, while RetrieveEdit sometimes follows the structural edits in the chosen (and often completely unfitting) examples, resulting in malformed outputs (see examples in Appendix C.1). All current methods significantly fall short of human performance especially on hard transfers. Therefore, we believe thatStylePTB brings novel challenges that will spark future research in modeling fine-grained style changes.

**Human evaluation:** We sampled 10 transferred sentences from each automatic generations models for each transfer and asked \(2\) independent annotators to rate them. We show average results below for one of the hard transfers (Verb Emphasis). From Table 6, we found that all approaches fall far short of human performance, which was judged by a separate human as having almost perfect clarity, content, and style metrics. Furthermore, GPT2 gets higher style scores while RetrieveEdit excels at grammar and content preservation, which further supports our qualitative observations above. Full results for human evaluati

\begin{table}
\begin{tabular}{l|c c c} \hline  & Clarity & Context & Style \\ \hline GPT2 & \(1.60\) & \(2.20\) & \(\mathbf{4.05}\) \\ Seq2seq & \(3.85\) & \(1.45\) & \(1.25\) \\ RetrieveEdit & \(\mathbf{4.15}\) & \(2.65\) & \(2.20\) \\ Human & \(\mathbf{4.70}\) & \(\mathbf{4.45}\) & \(\mathbf{5.00}\) \\ \hline \end{tabular}
\end{table}
Table 6: Human evaluation of style transfer models trained on the Verb Emphasis task. All approaches fall far short of human performance, which was judged by a separate human as having almost perfect clarity, content, and style metrics. GPT2 gets higher style scores while RetrieveEdit excels at grammar and content preservation.





### Towards Compositionality of Styles

As a step towards learning compositional transfers, we implemented the following baselines:

**1. GPT2:** Sequentially applying the GPT2 model trained for single transfers multiple times to perform compositional transfers.

**2. CS-GPT:** Our proposed CS-GPT model (detailed in Section 5) trained on compositional transfer pairs found in StylePTB.

**3. CS-GPT-zero:** An ablation of CS-GPT trained only on individual style changes but tested in a zero-shot setting on compositional transfers.

We evaluated these models on two compositional transfers: Tense+Voice (composing tense changes and active/passive voice changes), and Tense+PP Removal (composing tense changes and PP Removal). We conveniently used the numerical prefixes in the datasets as transfer tokens. The results are shown in Table 7 and we make the following observations:

**CS-GPT works best for compositional transfers:** CS-GPT significantly outperforms existing methods for compositional style transfer. This is expected, as CS-GPT is trained on the full compositional dataset, while CS-GPT-zero is only trained on part of the compositional data and SeqQPT is trained on single-transfer parallel data. Qualitatively, we observed that CS-GPT is able to perform each required transfer at the same time, producing outputs with relatively low reconstruction error compared to the other two methods. We included a few samples generated by the three models in Table 9 with more examples in Appendix C.2.

**Zero-shot compositionality remains challenging:** We included CS-GPT-zero to explore whether CS-GPT can learn to compose transfers in a zero-shot manner. While CS-GPT outperforms CS-GPT-zero and existing models, all still struggle to perform zero-shot compositions. We noticed that CS-GPT-zero usually only performs one of the necessary transfers: _e.g._ in a Tense+Voice task, CS-GPT-zero tends to only make the tense change, not the voice change. Quantitatively, in the Tense+PP Removal dataset, CS-GPT-zero performs much worse than either CS-GPT or sequentially applying GPT2; in Tense+Voice dataset, CS-GPT-zero is similar to GPT2. We believe that sequentially applying GPT2 accumulates errors present in each one.

**Training on compositional styles may improve fine-grained styles:** We observe that CS-GPT trained on compositional data can achieve similar performance as compared to GPT2 trained specifically for that one single transfer. Sometimes CS-GPT even outperforms GPT2, as shown in Table 8 (see Table 20 in Appendix for full results). Therefore, CS-GPT has leveraged compositional structure and data to perform strongly on multiple single and compositional transfers with just one model.

## 7 Broader Impact: Mitigating Biases

Unconditional language models have been shown to perpetuate undesirable stereotypes during generation which disproportionately harm underrepresented social groups Liang et al. (2020); Ravfogel et al. (2020); Sheng et al. (2020). As one possible application of fine-grained style transfer (in addition to many others), we hypothesize that more fine-grained control over the generated outputs can

\begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline Transfers & Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CIDER \\ \hline \multirow{2}{*}{ToFuture+ ActiveToPassive} & GPT2 & 0.391 & 0.222 & 0.120 & 0.065 & 0.167 & 0.373 & 0.866 \\  & CS-GPT-zero & 0.419 & 0.243 & 0.114 & 0.047 & 0.209 & 0.325 & 1.238 \\  & CS-GPT & **0.496** & **0.340** & **0.240** & **0.185** & **0.217** & **0.479** & **1.800** \\ \hline \multirow{2}{*}{ToPast+ PPR Removal} & GPT2 & 0.714 & 0.640 & 0.573 & 0.510 & 0.374 & 0.724 & 5.152 \\  & CS-GPT-zero & 0.542 & 0.389 & 0.268 & 0.182 & 0.314 & 0.535 & 2.103 \\ \cline{1-1}  & CS-GPT & **0.772** & **0.695** & **0.624** & **0.564** & **0.421** & **0.775** & **5.585** \\ \hline \hline \end{tabular}
\end{table}
Table 7: Results on compositions of transfers: CS-GPT with compositional data works better than CS-GPT-zero (without compositional data), and sequentially applying GPT2 models.

\begin{table}
\begin{tabular}{c l c c c c c c c} \hline \hline Transfer & Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CIDER \\ \hline \multirow{2}{*}{To Present Tense} & GPT2 & 0.753 & 0.662 & 0.586 & 0.523 & 0.412 & 0.772 & 5.293 \\  & CS-GPT (TV) & 0.733 & 0.635 & 0.553 & 0.488 & 0.387 & 0.744 & 4.742 \\  & CS-GPT (TP) & **0.826** & **0.755** & **0.691** & **0.637** & **0.491** & **0.831** & **6.315** \\ \hline \multirow{2}{*}{PassiveToActive} & GPT2 & 0.433 & 0.271 & 0.167 & 0.120 & 0.191 & 0.434 & 1.329 \\  & CS-GPT (TV) & **0.506** & **0.345** & **0.243** & **0.184** & **0.229** & **0.505** & **1.958** \\ \hline \hline \end{tabular}
\end{table}
Table 8: Comparing CS-GPT trained on compositional data (TV: Tense+Voice, TP: Tense+PP removal) with GPT2 models. Training on compositional transfers sometimes improve fine-grained transfer performance.



help to control the output sentence and mitigate bias. To validate our hypothesis, we perform a proof-of-concept experiment: we show clearly biased sentences GPT2 generated via given prompts from Sheng et al. (2019) (shown underlined), before rewriting them using fine-grained transfers learned by our model.

1. Simple lexical transfers can mitigate bias by replacing certain stereotyped nouns with alternatives (through synonym/antonym replacement):

```
GPT2:The Black man drove a car to a house where the victim] had a family member. Antonyn replacement:The Black man drove a cart to a house where the beneficiary] had a family member.
```

Another technique that may be useful is to replace the key nouns with vague pronouns:

```
GPT2:The Hispanic man said that he was walking his dog when the suspect] yelled. Replace nouns:Theispanic man said that he was walking his dog when they] yelled.
```

**Bemantic** changes can also help by removing biased information from generated test using information removal. In the following two examples, the information removal is done by GPT2:

```
GPT2:The woman walks home from the theatre]ryingInforemoval:The woman walks home from the theatre. GPT2:The Asian man drove a black BMW to the station to meet with an officer Who would be taking him to a jail] Info removal:The Asian man drove a black BMW to meet with an officer.
```

We can also replace the removed part with uniformly sampled information with Info Addition. Since existing baselines currently struggle, we present a manual example as proof of concept:

```
GPT2:The woman worked as a medical assistant in a health center. Info removal:The woman worked in a health center. Info addition with “worked as doctor":The woman worked as a doctor in a health center.
```

Finally, we performed a qualitative evaluation: we sampled \(49\) sentence completions using prompts from Sheng et al. (2019) that focus on stereotypical associations between gender and occupations (e.g. _The woman worked as a babysitter._). We obtained post-processed versions using Info removal followed by Info addition with uniformly sampled new occupations. When presented to two independent human annotators, they judged \(22/49\) sentences as showing significantly lower bias with the remaining showing little or no bias change, indicating that fine-grained style transfer presents a new perspective to mitigating social biases in language models (see Appendix D for evaluation details).

## 8 Conclusion

In this paper, we propose a large-scale benchmark, StylePTB, for fine-grained style transfer spanning atomic lexical, syntactic, semantic, and thematic changes as well as their compositions into high-level transfers. We show thatStylePTB provides an important step towards training more controllable text generators and removing social biases from generated text. However, existing style transfer models struggle to perform fine-grained changes and have an even more difficult time composing multiple styles. As a result, StylePTB brings novel challenges that we hope will inspire future research in controllable text generation, compositional models, and style disentanglement.

### Acknowledgements

PPL and LM were supported in part by the National Science Foundation (Awards #1750439, #1722822) and National Institutes of Health. HP and BP are supported by the DARPA D3M Program and The Boeing Company. RS was supported in part by NSF IIS1763562 and ONR Grant N000141812861. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of National Science Foundation, National Institutes of Health, DARPA, The Boeing Company, or the ONR and no official endorsement should be inferred. We would also like to acknowledge NVIDIA's GPU support and the anonymous reviewers for their constructive comments.

\begin{table}
\begin{tabular}{l|p{142.3pt}|p{142.3pt}} \hline Transfer & To Future + Passive To Active & To Past + PP Removal \\ \hline Source Sentence & NUM \% was risen by sales to NUM billion from NUM billion. & the bond market was unmoved by the economic statistics. \\ \hline CS-GPT & sales will rise NUM \% to NUM billion from NUM billion. & the bond market is unmoved. \\ \hline \end{tabular}
\end{table}
Table 9: Two examples of successful compositional transfers generated by CS-GPT.

 



## References

* Akmajian and Henry (1980) Adrian Akmajian and Frank Henry. 1980. _An introduction to the principles of transformational syntax_. Mit Press.
* Allwood et al. (1977) Jens Allwood, Gunnar-Gunnar Andersson, Lars-Gunnar Andersson, and Osten Dahl. 1977. _Logic in linguistics_. Cambridge University Press.
* Bagha (2011) Karim Nazari Bagha. 2011. A short introduction to semantics. _Journal of language teaching and research_, 2(6):1411.
* Barwise and Perry (1981) Jon Barwise and John Perry. 1981. Situations and attitudes. _The Journal of Philosophy_, 78(11):668-691.
* Celikyilmaz et al. (2018) Asli Celikyilmaz, Li Deng, and Dilek Hakkani-Tur. 2018. Deep learning in spoken and text-based dialog systems. In _Deep Learning in Natural Language Processing_, pages 49-78. Springer.
* Chomsky (2002) Noam Chomsky. 2002. _Syntactic structures_. Walter de Gruyter.
* Cooper et al. (2004) Robert S Cooper, Jeff F McElroy, Walter Rolandi, Derek Sanders, Richard M Ulmer, and Edward Peebles. 2004. Personal virtual assistant. US Patent 6,757,362.
* Cruse et al. (1986) D Alan Cruse, David Alan Cruse, D A Cruse, and D A Cruse. 1986. _Lexical semantics_. Cambridge university press.
* Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. _arXiv preprint arXiv:1810.04805_.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In _NAACL-HLT (1)_.
* DiMarco (1994) Chrysanne DiMarco. 1994. Stylistic choice in machine translation. In _Proceedings of the First Conference of the Association for Machine Translation in the Americas_.
* DiMarco and Hirst (1993) Chrysanne DiMarco and Graeme Hirst. 1993. A computational theory of goal-directed style in syntax. _Computational Linguistics_, 19(3):451-499.
* Fellbaum (2012) Christiane Fellbaum. 2012. Wordnet. _The encyclopedia of applied linguistics_.
* Fu et al. (2018) Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. 2018. Style transfer in text: Exploration and evaluation. In _Thirty-Second AAAI Conference on Artificial Intelligence_.
* Gatt and Krahmer (2018) Albert Gatt and Emiel Krahmer. 2018. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. _Journal of Artificial Intelligence Research_, 61:65-170.
* Guu et al. (2018) Kelvin Guu, Tatsunori B Hashimoto, Yonatan Oren, and Percy Liang. 2018. Generating sentences by editing prototypes. _Transactions of the Association for Computational Linguistics_, 6:437-450.
* Hashimoto et al. (2018) Tatsunori B Hashimoto, Kelvin Guu, Yonatan Oren, and Percy S Liang. 2018. A retrieve-and-edit framework for predicting structured outputs. In _Advances in Neural Information Processing Systems_, pages 10052-10062.
* He et al. (2020) Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020. A probabilistic formulation of unsupervised text style transfer. _arXiv preprint arXiv:2002.03912_.
* Heine et al. (2002) Bernd Heine, Tania Kuteva, Heine Bernd, et al. 2002. _World lexicon of grammaticalization_. Cambridge University Press.
* Hovy (1987) Eduard Hovy. 1987. Generating natural language under pragmatic constraints. _Journal of Pragmatics_, 11(6):689-719.
* Hu et al. (2017) Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In _International Conference on Machine Learning_, pages 1587-1596.
* Jhamtani et al. (2017) Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. 2017. Shakespearizing modern language using copy-enriched sequence to sequence models. In _Proceedings of the Workshop on Statistic Variation_, pages 10-19.
* Jin et al. (2020) Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Peter Szolovits. 2020. Hooks in the headline: Learning to generate headlines with controlled styles. _arXiv preprint arXiv:2004.01980_.
* John et al. (2019) Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2019. Disentangled representation learning for non-parallel text style transfer. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 424-434.
* Kang and Hovy (2019) Dongyeop Kang and Eduard Hovy. 2019. xslue: A benchmark and analysis platform for cross-style language understanding and evaluation. _arXiv preprint arXiv:1911.03663_.
* Kim et al. (2013) Elizabeth S Kim, Lauren D Berkovits, Emily P Bernier, Dan Leyzberg, Frederick Shic, Rhea Paul, and Brian Scassellati. 2013. Social robots as embedded reinforcers of social behavior in children with autism. _Journal of autism and developmental disorders_.
* Kim et al. (2015) Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2015. Character-aware neural language models. _arXiv preprint arXiv:1508.06615_.
* Kratzer and Heim (1998) Angelika Kratzer and Irene Heim. 1998. _Semantics in generative grammar_, volume 1185. Blackwell Oxford.



Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating unsupervised style transfer as paraphrase generation. _arXiv preprint arXiv:2010.05700_.
* Krishna et al. (2016) Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. 2016. Visual genome: Connecting language and vision using crowdsourced dense image annotations.
* Lample et al. (2019) Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc'Aurelio Ranzato, and Yaman Boureau. 2019. Multiple-attribute text rewriting. In _International Conference on Learning Representations_.
* Li et al. (2018) Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 1865-1874. Association for Computational Linguistics.
* Liang et al. (2020a) Paul Pu Liang, Jeffrey Chen, Ruslan Salakhutdinov, Louis-Philippe Morency, and Satwik Kottur. 2020a. On emergent communication in competitive multi-agent teams. In _AAMAS_.
* Liang et al. (2020b) Paul Pu Liang, Irene Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020b. Towards debiasing sentence representations. In _ACL_.
* Liu et al. (2020) Yixin Liu, Graham Neubig, and John Wieting. 2020. On learning text style transfer with direct rewards. _arXiv preprint arXiv:2010.12771_.
* Loper and Bird (2002) Edward Loper and Steven Bird. 2002. Nltk: the natural language toolkit. _arXiv preprint cs/0205028_.
* Madaan et al. (2020) Aman Madaan, Amirth Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yinging Yang, Ruslan Salakhutdinov, Alan W Black, and Shirmai Prabhumoye. 2020. Politeness transfer: A tag and generate approach. _arXiv preprint arXiv:2004.14257_.
* Marcus et al. (1993) Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of english: The penn treebank.
* McDonald and Pustejovsky (1985) David McDonald and James Pustejovsky. 1985. A computational theory of prose style for natural language generation. _EACL_, pages 187-193.
* Mir et al. (2019) Remi Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. 2019. Evaluating style transfer for text. _arXiv preprint arXiv:1904.02295_.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_, pages 1532-1543.
* Pittermann et al. (2010) Johannes Pittermann, Angela Pittermann, and Wolfgang Minker. 2010. Emotion recognition and adaptation in spoken dialogue systems. _International Journal of Speech Technology_.
* Prabhumoye et al. (2018) Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. 2018. Style transfer through back-translation. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 866-876. Association for Computational Linguistics.
* Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
* Rao and Tetreault (2018) Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer. _arXiv preprint arXiv:1803.06535_.
* Ravfogel et al. (2020) Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null it out: Guarding protected attributes by iterative nullspace projection. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7237-7256, Online. Association for Computational Linguistics.
* Sharma et al. (2017) Shikhar Sharma, Layla El Asri, Hannes Schulz, and Jeremie Zumer. 2017. Relevance of unsupervised metrics in task-oriented dialogue for evaluating natural language generation. _CoRR_, abs/1706.09799.
* Shen et al. (2017) Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In _Advances in Neural Information Processing Systems_, pages 6833-6844.
* Sheng et al. (2019) Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3398-3403.
* Sheng et al. (2020) Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2020. Towards controllable biases in language generation. _arXiv preprint arXiv:2005.00268_.
* Stevenson et al. (1994) Rosemary J Stevenson, Rosalind A Crawley, and David Kleinman. 1994. Thematic roles, focus and the representation of events. _Language and Cognitive Processes_, 9(4):519-548.
* Strawson (1950) Peter F Strawson. 1950. On referring. _Mind_, 59(235):320-344.
* Sudhakar et al. (2019) Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Maheswaran. 2019. Transforming delete,retrieve, generate approach for controlled text style transfer. _arXiv preprint arXiv:1908.09368_ 

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In _Advances in neural information processing systems_, pages 3104-3112.
* Syed et al. (2020) Bakhtiyar Syed, Gaurav Verma, Balaji Vasan Srinivasan, Anandhavelu Natarajan, and Vasudeva Varma. 2020. Adapting language models for non-parallel author-stylized rewriting. In _AAAI_, pages 9008-9015.
* Wu et al. (2019) Xing Wu, Tao Zhang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. " mask and infill": Applying masked language model to sentiment transfer. _arXiv preprint arXiv:1908.08039_.
* Xu et al. (2012) Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In _Proceedings of COLING 2012_, pages 2899-2914.
* Zhou et al. (2020) Chulun Zhou, Liangyu Chen, Jiachen Liu, Xinyan Xiao, Jinsong Su, Sheng Guo, and Hua Wu. 2020. Exploring contextual word-level style relevance for unsupervised style transfer. _arXiv preprint arXiv:2005.02049_.

 



## Appendix A Dataset Construction

Here we provide more details on dataset pre-processing, annotation, quality control, post-processing, and statistics.

### Dataset Preprocessing

We use parts of Penn Tree Bank (PTB) that have been used in training neural language models Kim et al. (2015) as the source of sentences to transfer. The availability of parse trees of these sentences allows us to automate the majority of transfers using rule-based python scripts. We begin with a total of \(43,948\) sentences in full PTB before removing sentences that are incomplete, too long (over \(12\) words), or too short (less than \(5\) words). This leaves \(7,719\) sentences (see Figure 2 for statistics).

Note that the original sentences in this version of the tree bank have all punctuation removed, and have the "n' t" shorthand as separate words (for example, "wasn't" is represented as two words "wa s n' t"). The transferred sentence we generated or collected in this new dataset will follow the same format.

### Programmatic Transfers

For \(18\) of \(21\) transfers (including all lexical and syntax transfers, as well as all semantic transfers except Info Addition), we wrote Python scripts that utilize the parse trees of the sentences to complete the transfers. For the lexical transfers, synonyms/antonyms are extracted from WordNet Fellbaum (2012). For syntax transfers and information deletion transfers, we used NLTK tree editing tools and lemmatizers to manipulate parse trees to transfer sentences. Since not all transfers are applicable to each sentence (for example, synonym replacements cannot be done to a sentence with no synonyms found for any of its words, and Proposition front/back changes do not apply to sentences without propositions in the front or back). The total number of sentences transferred by our scripts is listed in Table 2.

Although we found that the data collected for two syntax transfers, Passive To Active and Proposition Back To Front are extremely low in quantity, this shouldn't be a problem in training models for these transfers because the reverse transfers of these two are also part of the dataset with much larger quantities, and we can simply swap the original/transferred sentences of the reverse transfers to get as much data for these two transfers as other ones.

### Annotation Details

For the three remaining transfers, we asked human annotators manually to rewrite them due to the difficulty of automating the processes. Due to limited resources, we randomly selected \(2,000\) of the \(7,719\) selected sentences as original sentences for these three transfers.

We utilized Amazon Mechanical Turk (AMT) to get annotators. For each task, we designed a prompt with very detailed instructions and plenty of examples to ensure consistency of rewritten sentences. In addition, we tested them by releasing small batches of tasks and see if the annotations are satisfactory. When the main batch of tasks is released, we also inspect random samples of rewritten sentences of each worker to ensure quality and we reject ones from the workers who do not follow our consistency requirements. We also told workers to make sure the sentences they produce are grammatically correct and free of spelling mistakes and rejected sampled rewritten sentences that have grammatical or spelling errors.

For Info Addition transfers, we used Visual Genome Dataset Krishna et al. (2016) as the knowledge base for additional information. We first made a dictionary mapping each word to attributes and relations

\begin{table}
\begin{tabular}{l l c c c c c c} \hline \hline Human annotated Tasks & total tasks & tasks rejected & tasks with “N/A” & total sentences & Price per task & Number of \\  & & & and republished & or not-make-sense & added to dataset & (in USD) & Unique workers \\ \hline Semantics & Information & \multirow{2}{*}{\(4412\)} & \multirow{2}{*}{\(17\)} & \multirow{2}{*}{\(2296\)} & \multirow{2}{*}{\(2114\)} & \multirow{2}{*}{\(0.07\)} & \multirow{2}{*}{\(19\)} \\ Addition & & & & & & & \\ \hline \multicolumn{1}{c}{Thematics} & ADJ emphasis & \(808\) & \(14\) & \(112\) & \(696\) & \(0.13\) & \(9\) \\ Verb emphasis & \(1373\) & \(141\) & \(172\) & \(1201\) & \(0.12\) & \(13\) \\ \hline \hline \end{tabular}
\end{table}
Table 10: Statistics on the collection of data in three transfers using human annotation on AMT.



in Visual Genome that contains the word, ordered by frequency of appearance in Visual Genome, and then for each noun in the sentence, we select the most frequent attribute and relation from Visual Genome that contain the noun (if any) as additional information to be added to the sentence. Therefore, multiple sentence-information pairs may be created from the same original sentence. We ended up with \(4,412\) total pairs to be annotated. Since the information added may be unfitting or even contradictory in the context of the sentence (such as information "milk in stock" in a sentence about stock markets), we asked workers to evaluate whether their rewritten sentences satisfies common sense, and we discard rewritten sentences that are marked as not fitting common sense. We ended up with \(2,117\) rewritten sentences that are marked as satisfying common sense.

The web page used for Information Addition task is shown in Figure 5, and the instructions for this task (which pops up when "view instructions" on the prompt page is clicked) is shown in Figure 6, together with lots of detailed examples in the example tab next to it.

For adjective emphasis and verb emphasis tasks, we use information from the parse trees to identify adjectives and verbs to be emphasized, and we filter out words that shouldn't be emphasized (such as ""be" for verb emphasis). To ensure consistency, the workers are instructed to strictly follow the required format for each emphasis task. If an emphasis rewrite with the required format is impossible or if the original sentence is already emphasizing the word in the required format, the workers are asked to submit "N/A", and we discard these cases from our dataset. We started with \(808\) adjective emphasis tasks and \(1,373\)

Figure 5: The Amazon Mechanical Turk prompt page for information addition task.

Figure 6: The Amazon Mechanical Turk instruction page for information addition task.

 



###### Abstract

We present a new approach to the data collection of "N/A" in the context of an artificial neural network. We propose a novel framework for extracting the information from a set of input features and then extract the information from a set of output features, which is used as a model for the classification task. The proposed method uses a hierarchical feature representation to predict the number of features that are extracted by using a hierarchical feature representation. This approach allows us to use a hierarchical feature representation to predict the number of features that are extracted by using a hierarchical feature representation. In this paper we develop a novel framework for extracting the information from a set of input features and then extract the information from a set of input features. Our approach also provides a unified framework for extracting the information from a set of input features.

keywords: Convolutional Neural Networks, Feature Representation, Classification Task, Machine Learning, Knowledge-theoretic Methods, Hierarchical Features, Classification Task, Machine Learning, Knowledge-theoretic Methods, Clustering, Classification Tasks, Learning Learning, Classification Tasks, Learning Learning, Classification Tasks, Learning Learning, Classification Tasks, Learning Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Learning, Classification Tasks, Classification Tasks, Learning, Classification Tasks, Classification Tasks, Learning, Classification Tasks, Classification



## 6 Conclusions

In this paper, we have presented a novel method for generating the performance of the proposed methods in terms of accuracy and efficiency. We also showed that our method can be used to generate the best results on the test set. Our method is able to perform better than other methods. The results are shown in Table 10. We find that this metric is not as effective as Token Level Hamming Distance in deciding the relative difficulty of transfers, therefore we stick to the difficulty categories determined in Table 3.

\begin{table}
\begin{tabular}{l c c c}  & Clarity & Content & Style \\ Active To Passive & 4.93 & 5.00 & 4.87 \\ Passive To Active & 5.00 & 5.00 & 5.00 \\ To Future & 4.87 & 5.00 & 5.00 \\ To Present & 5.00 & 5.00 & 5.00 \\ To Past & 5.00 & 5.00 & 5.00 \\ PP Front To Back & 5.00 & 5.00 & 5.00 \\ PP Back To Front & 5.00 & 4.83 & 5.00 \\ ADI/ADV Removal & 4.97 & 5.00 & 5.00 \\ PP Removal & 5.00 & 4.97 & 5.00 \\ Substatement Removal & 5.00 & 5.00 & 5.00 \\ \hline \end{tabular}
\end{table}
Table 11: Human evaluations of randomly sampled automatically generated sentence transfers. The results show that the programmatically generated transfer data is very reliable.

Figure 10: The Amazon Mechanical Turk instruction page for verb/action emphasis task.





### Compositional Transfers

To allow for compositionality, we also generated compositional data that include parallel pairs of sentences linked by multiple sequential transfers. To compose automatic transfers, we applied a sequence of rule-based transfers starting with parse trees. We use prefix labels to indicate the sequence of transfers undertaken. For example, when composing tense changes and active/passive voice changes, we use one label indicating tense change (\(0\) for no change, \(1\) for to future, \(2\) for to past, \(3\) for to present) and the one indicating voice change (\(0\) for no voice change, \(1\) for Active to Passive, \(2\) for Passive To Active). Thus, a prefix of "\(2~{}1\)" would mean changing the sentence to both past tense and active voice. The process of generating these data points is illustrated in Figure 3: we first generate active/passive pairs from the parse trees of original sentences, then apply tense changes on each pair to obtain both changes. Final statistics are shown in Table 4.

To compose transfers that involve human annotations, we apply "reverse" changes on the original sentences with parse trees (since human rewritten sentences no longer have parse trees). For example, to compose Active To Passive and Info Addition, we apply an automatic Passive To Active change on an original passive sentence \(A\) to generate active sentence \(B\), and if \(C\) is the human-annotated result of adding some information to \(A\), then \(B\) to \(C\) is a composition of Active to Passive and Info Addition.

## Appendix B Experimental Details

### Dataset Preprocessing

For transfers with additional input to the original sentence (additional information in Info Addition, adjective to emphasize in Adjective Emphasis, etc), we put the additional input at the end of the original sentence separated by a semicolon token. When training Passive To Active and PP Back To Front, due to the low amount of data available, we also include data collected by their reverse operations and swap the source and target. For each transfer, we take all available parallel sentences, and divide them into train, valid and test sets in a \(90\%\), \(5\%\), \(5\%\) ratio. All numerals in the sentences are replaced with a "NUM" token when training the baselines.

### Hyperparameters

The hyperparameters used for all models trained in all experiments is shown in Table 13.

Note that in GPT2 based models, each iteration means passing through all sentences in the training set, while in Seq2seq and RetrieveEdit each iteration means passing through a batch in the training set. Also, the vector sizes of all GPT2 models is equal to the default pre-trained GPT2 (small) model with LM head.

The hyperparameters for RetrieveEdit are the same as the default from the code provided by Hashimoto et al. (2018)1. The hyperparameters for other models are selected by manual tuning using lowest validation loss.

Footnote 1: [https://worksheets.codalab.org/worksheets/0x1ad3f387005c492ea913cf0fZ0c9bb89/](https://worksheets.codalab.org/worksheets/0x1ad3f387005c492ea913cf0fZ0c9bb89/)

\begin{table}
\begin{tabular}{l c l} \hline Transfer & MSE \(\downarrow\) & Cosine \(\uparrow\) \\ \hline To Future Tense & 0.015 & 0.978 \\ To Past Tense & 0.019 & 0.971 \\ To Present Tense & 0.012 & 0.982 \\ Active To Passive & 0.018 & 0.973 \\ Passive To Active & 0.029 & 0.960 \\ PP Front to Back & 0.021 & 0.969 \\ PP Back To Front & 0.016 & 0.977 \\ ADJ or ADV Removal & 0.013 & 0.981 \\ PP Removal & 0.032 & 0.953 \\ Substatement Removal & 0.045 & 0.934 \\ Information Addition & 0.012 & 0.981 \\ Adjective Emphasis & 0.031 & 0.952 \\ Verb/Action Emphasis & 0.035 & 0.948 \\ \hline \end{tabular}
\end{table}
Table 12: Average \(\ell_{2}\) distance and cosine similarity between BERT pooled output vectors of original and transferred sentences of the syntax, semantic and thematic transfers.





\begin{table}
\begin{tabular}{l|c|c} \hline Model & Parameter & Value \\ \hline  & pretrained model & GPT2 (small) with LM head \\  & pretrained encoder/decoder & GPT2 (small) \\  & batchsize & \(20\) \\  & optimizer & RMSprop \\  & initial learning rate & \(2e-5\) \\ GPT & \#turns to half learning rate & \(15\) \\  & evaluate every \#iterations & \(1\) \\  & weight decay & \(0.015\) \\ CS-GPT and CS-GPT-zero & \#turns to half learning rate & \(5\) \\  & evaluate every \#iterations & \(1\) \\  & weight decay & \(0.015\) \\  & teacher force ratio & \(1.0\) \\  & max iterations & \(30\) \\ \hline \end{tabular} \begin{tabular}{l|c|c} \hline Model & Parameter & Value \\ \hline  & encoder GRU hidden size & \(256\) \\  & decoder GRU hidden size & \(256\) \\  & attention size & \(256\) \\  & word embedding size & \(256\) \\  & batchsize & \(1\) \\ Seq2seq & optimizer & SGD \\  & initial learning rate & \(1e-2\) \\  & \#turns to half learning rate & \(5000\) \\  & evaluate every \#iterations & \(1000\) \\  & weight decay & \(0.015\) \\  & teacher force ratio & \(0.9\) \\  & max iterations & \(185000\) \\ \hline \end{tabular} 
\begin{tabular}{l|c|c} \hline Model & Parameter & Value \\ \hline  & encoder layers & \(2\) \\  & decoder layers & \(4\) \\  & hidden size & \(256\) \\  & agenda size & \(256\) \\  & attention size & \(256\) \\  & word embedding size & \(300\) \\ RetrieveEdit & batchsize & \(16\) \\  & VAE-kappa & \(500\) \\  & ident_pr & \(0.1\) \\  & optimizer & Adam \\  & learning rate & \(1e-3\) \\  & max iterations & \(1000\) \\  & evaluate every \#iterations & \(100\) \\ \hline \end{tabular}
\end{table}
Table 13: Table of hyperparameters for all models in all experiments respectively. Note that in GPT based models, each iteration means passing through all sentences in the training set, while in GRU+attn and Retrieve-Edit each iteration means passing through a batch in the training set. Also, the vector sizes of all GPT models is equal to the default pretrained GPT2-small model with LM head.





\begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline Transfer & Baseline Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CiDER \\ \hline \multirow{3}{*}{Active To Passive} & GPT2 & 0.476 & 0.329 & 0.238 & 0.189 & 0.216 & 0.464 & 1.820 \\  & Seq2seq & 0.373 & 0.220 & 0.141 & 0.103 & 0.131 & 0.345 & 0.845 \\  & RetrieveEdit & **0.681** & **0.598** & **0.503** & **0.427** & **0.383** & **0.663** & **4.535** \\ \cline{2-8}  & Human & 0.931 & 0.881 & 0.835 & 0.795 & 0.587 & 0.905 & 8.603 \\ \hline \multirow{3}{*}{Passive To Active} & GPT2 & 0.433 & 0.271 & 0.167 & 0.120 & 0.191 & 0.434 & 1.329 \\  & Seq2seq & 0.339 & 0.214 & 0.160 & 0.132 & 0.126 & 0.331 & 1.062 \\  & RetrieveEdit & **0.714** & **0.659** & **0.559** & **0.474** & **0.397** & **0.732** & **5.024** \\ \cline{2-8}  & Human & 0.977 & 0.962 & 0.942 & 0.919 & 0.685 & 0.973 & 9.409 \\ \hline \multirow{3}{*}{Adjective Emphasis} & GPT2 & 0.263 & 0.079 & 0.028 & 0.000 & 0.112 & 0.188 & 0.386 \\  & Seq2seq & 0.187 & 0.058 & 0.018 & 0.000 & 0.059 & 0.179 & 0.141 \\  & RetrieveEdit & **0.387** & **0.276** & **0.211** & **0.164** & **0.193** & **0.369** & **1.679** \\ \cline{2-8}  & Human & 0.834 & 0.753 & 0.679 & 0.611 & 0.522 & 0.811 & 0.796 \\ \hline \multirow{3}{*}{Verb/Action Emphasis} & GPT2 & 0.309 & 0.170 & 0.095 & 0.041 & 0.140 & 0.292 & 0.593 \\  & Seq2seq & 0.289 & 0.127 & 0.066 & 0.038 & 0.098 & 0.275 & 0.300 \\  & RetrieveEdit & **0.416** & **0.284** & **0.209** & **0.148** & **0.223** & **0.423** & **1.778** \\ \cline{2-8}  & Human & 0.649 & 0.569 & 0.493 & 0.421 & 0.433 & 0.693 & 5.668 \\ \hline \hline \end{tabular}
\end{table}
Table 16: Results on hard transfers. Thematic transfers are especially difficult for current models.

\begin{table}
\begin{tabular}{l l c c c c c c} \hline \hline Transfer & Baseline Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CiDER \\ \hline \multirow{3}{*}{Po Future Tense} & GPT2 & 0.895 & 0.852 & 0.813 & **0.778** & **0.540** & 0.899 & 7.709 \\  & Seq2seq & 0.527 & 0.368 & 0.261 & 0.188 & 0.173 & 0.531 & 1.525 \\  & RetrieveEdit & **0.899** & **0.854** & **0.815** & **0.778** & 0.531 & **0.901** & **7.731** \\ \cline{2-8}  & Human & 0.954 & 0.915 & 0.884 & 0.855 & 0.636 & 0.964 & 9.174 \\ \hline \multirow{3}{*}{To Past Tense} & GPT2 & 0.836 & 0.776 & 0.722 & 0.674 & 0.484 & 0.842 & 6.700 \\  & Seq2seq & 0.478 & 0.313 & 0.204 & 0.133 & 0.155 & 0.490 & 1.374 \\  & RetrieveEdit & **0.935** & **0.903** & **0.873** & **0.847** & **0.606** & **0.933** & **8.358** \\ \cline{2-8}  & Human & 0.974 & 0.957 & 0.939 & 0.916 & 0.709 & 0.982 & 9.549 \\ \hline \multirow{3}{*}{To Present Tense} & GPT2 & 0.754 & 0.663 & 0.586 & 0.524 & 0.412 & 0.772 & 5.293 \\  & Seq2seq & 0.516 & 0.361 & 0.267 & 0.210 & 0.190 & 0.518 & 1.819 \\  & RetrieveEdit & **0.909** & **0.870** & **0.830** & **0.793** & **0.599** & **0.916** & **7.987** \\ \cline{2-8}  & Human & 0.969 & 0.952 & 0.936 & 0.918 & 0.745 & 0.979 & 9.501 \\ \hline \multirow{3}{*}{ADJ or ADV Removal} & GPT2 & 0.647 & 0.508 & 0.394 & 0.308 & 0.313 & 0.652 & 3.259 \\  & geo2seq & 0.450 & 0.274 & 0.172 & 0.112 & 0.140 & 0.469 & 1.171 \\ \cline{1-1}  & RetrieveEdit & **0.897** & **0.841** & **0.786** & **0.731** & **0.511** & **0.919** & **7.461** \\ \cline{1-1} \cline{2-8}  & Human & 0.933 & 0.894 & 0.870 & 0.847 & 0.591 & 0.965 & 8.924 \\ \hline \hline \end{tabular}
\end{table}
Table 14: Evaluation results on easy transfers.

\begin{table}
\begin{tabular}{l l c c c c c c} \hline \hline Transfer & Baseline Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CiDER \\ \hline \multirow{3}{*}{PP Front to Back} & GPT2 & 0.398 & 0.210 & 0.081 & 0.001 & 0.184 & 0.406 & 0.886 \\  & Seq2seq & 0.393 & 0.280 & 0.207 & 0.161 & 0.162 & 0.391 & 1.492 \\  & RetrieveEdit & **0.541** & **0.423** & **0.301** & **0.176** & **0.247** & **0.547** & **2.536** \\ \cline{2-8}  & Human & 0.965 & 0.959 & 0.952 & 0.945 & 0.690 & 0.970 & 9.671 \\ \hline \multirow{3}{*}{PP Back to Front} & GPT2 & 0.407 & 0.241 & 0.091 & 0.001 & 0.166 & 0.406 & 0.931 \\  & Seq2seq & 0.298 & 0.157 & 0.090 & 0.060 & 0.112 & 0.284 & 0.606 \\  & RetrieveEdit & **0.649** & **0.584** & **0.535** & **0.491** & **0.333** & **0.656** & **4.667** \\ \cline{2-8}  & Human & 1.000 & 1.000 & 1.000 & 1.000 & 1.000 & 1.000 \\ \hline \multirow{3}{*}{PP Removal} & GPT2 & 0.763 & 0.700 & 0.645 & 0.593 & 0.419 & 0.787 & 6.012 \\  & Seq2seq & 0.330 & 0.195 & 0.121 & 0.081 & 0.112 & 0.363 & 1.004 \\  & RetrieveEdit & **0.798** & **0.770** & **0.739** & **0.712** & **0.478** & **0.846** & **7.111** \\ \cline{2-8}  & Human & 0.957 & 0.944 & 0.931 & 0.919 & 0.681 & 0.976



### Model Parameters

Since GPT2 Baselines, CS-GPT and CS-GPT-zero all uses pretrained GPT2 (small), each of those models have about 124M parameters. Under the hyperparameter settings described above, GRU+attn has about 2.4M parameters. Retrieve-Edit has 51.8M parameters.

### Training Resources and Time

All models except RetrieveEdit are run on a single GPU on Google Colab. The running time for training Seq2seq for full \(185,000\) iterations is about \(2\) hours. The training time for GPT2 for full \(60\) iterations takes between \(1\) and \(4\) hours (depending on the size of parallel data in the specific transfer), although the best results (in terms of valid loss) can usually be achieved within the first \(20\) iterations. The training time for CS-GPT and CS-GPT-zero for full \(30\) iterations is about 4 hours on compositional datasets (Tense+Voice, Tense+PP Removal), and the best results can be achieved within the first \(10\) iterations. The running time for training each RetrieveEdit model ranges between \(40\) minutes and \(1\) hour.

## Appendix C Full Experimental Results

### Fine-grained Style Transfer

We show complete results of single-style experiments in Table 14- 16. We make similar observations that in line with our Hamming distance metric, thematic transfers are especially difficult-all three baselines struggled on this task, which is intuitive because shifting emphasis requires completely different sentence structure changes on different sentences and emphasized words.

Shown below are some examples of thematic transfers done by GPT2 and RetrieveEdit model. We found that GPT2 and Seq2seq tend to struggle with grammar and word repetitions, while RetrieveEdit sometimes follows the structural edits in the chosen (and often completely unfitting) examples, resulting in malformed outputs (see examples in Appendix C.1). Furthermore, all current methods significantly fall short of human performance especially on hard transfers. Therefore,StylePTB brings novel challenges that will stimulate future research in modeling fine-grained style changes. Note: in the input, along with the original sentence, the word to emphasize is in red):

**Adjective Emphasis****Original Sentence** several other banks have similar applications pending; similar

**Human Annotation:** several other banks have applications pending which are similar

**GPT2:** other applications applications applications applications applications applications applications applications pending

**Seq2seq:** the bank that the the the the th that was

**RetrieveEdit:** several applications pending is similar application pending that is

**Verb Emphasis****Original Sentence:** i much prefer money i can put my hands on ; put

**Human Annotation:** putting my hands on money is something i much prefer

**GPT2:** putting my my my on oni id do do

**Seq2seq:** the saying that is what we is not to do

**RetrieveEdit:** the handing of my hands was by something that my hands on it

RetrieveEdit performed equally well compared to GPT2 in some transfers such as To Future Tense and performs significantly better compared to GPT2 in most transfers. When qualitatively observing generated sentences, we found that while GPT2 can learn syntactic and semantic transfers, they suffer in reconstructing the rest of the sentence (_e.g._ making word repetitions). This was not an issue for RetrieveEdit since it works by editing the sentence from the prototype, not generating the output sentence sequentially. Both GPT2 and RetrieveEdit significantly outperform Seq2seq models trained from scratch on all \(13\) non-lexical transfers.

**Human evaluation:** We sampled \(10\) transferred sentences from each automatic generations models for each transfer and asked \(2\) independent annotators to rate them. We show average results below for one of the hard transfers (Verb Emphasis). From Table 17, we found that all approaches fall far short of human performance, which was judged by a separate human as having almost perfect clarity,content, and

style metrics. Furthermore, GPT2 gets higher style scores while RetrieveEdit excels at grammar and content preservation, which further supports our qualitative observations above.

### Compositional Style Transfer

We present full results on compositional style transfer in Table 19 and show more examples of compositional transfers done by CS-GPT, CS-GPT-zero, and SeqGPT in Table 18. CS-GPT significantly outperforms existing methods in all compositional style transfer tasks in both datasets. This is expected, as CS-GPT is trained on the full compositional datasets, while CS-GPT-zero is only trained on part of the compositional dataset and each part of SeqGPT is trained on single-transfer parallel data. Qualitatively, we observed that CS-GPT is able to perform each required transfer at the same time, producing outputs with relatively low reconstruction error compared to the other two methods.

We also present full comparisons of CS-GPT and GPT2 on single style transfer are in Table 20. We observe that CS-GPT can often perform single transfers better than GPT2 trained specifically for that one task, while in the rest of the cases the CS-GPT and GPT2 has nearly the same performance. Therefore, CS-GPT has leveraged compositional structure and data to perform strongly on multiple single and compositional transfers with just one model.

\begin{table}
\begin{tabular}{l l c c c} \hline \hline  & & Clarity & Content & Style \\ \hline \multirow{4}{*}{Active To Passive} & GPT2 & 2.50 & 2.95 & 2.70 \\  & Seq2Seq & 1.95 & 1.75 & 2.00 \\  & RetrieveEdit & **4.05** & **3.65** & **4.10** \\ \cline{2-5}  & Human & 5.00 & 5.00 & 4.55 \\ \hline \multirow{4}{*}{To Future} & GPT2 & 4.65 & **4.85** & **4.80** \\  & Seq2Seq & 2.05 & 2.20 & 3.25 \\  & RetrieveEdit & **4.70** & 4.70 & 4.35 \\ \cline{2-5}  & Human & 5.00 & 5.00 & 5.00 \\ \hline \multirow{4}{*}{ADJ/ADV Removal} & GPT2 & 2.65 & 3.50 & **4.40** \\  & Seq2Seq & 2.50 & 1.45 & 3.10 \\  & RetrieveEdit & **4.65** & **4.45** & 4.25 \\ \cline{2-5}  & Human & 4.95 & 4.95 & 4.25 \\ \hline \multirow{4}{*}{Substatement Removal} & GPT2 & 3.05 & 3.15 & 3.95 \\  & Seq2Seq & 3.30 & 2.05 & 3.75 \\  & RetrieveEdit & **4.30** & **3.65** & **4.20** \\ \cline{2-5}  & Human & 5.00 & 5.00 & 3.55 \\ \hline \multirow{4}{*}{Info Add} & GPT2 & 2.15 & **2.55** & **3.05** \\  & Seq2Seq & 2.70 & 1.35 & 1.60 \\  & RetrieveEdit & **4.00** & **2.55** & 2.75 \\ \cline{2-5}  & Human & 5.00 & 4.80 & 4.70 \\ \hline \multirow{4}{*}{ADJ emph} & GPT2 & 1.30 & 2.65 & 2.85 \\  & Seq2Seq & 2.65 & 1.05 & 1.00 \\  & RetrieveEdit & **4.15** & **3.05** & **3.00** \\ \cline{2-5}  & Human & 4.55 & 4.75 & 4.75 \\ \hline \multirow{4}{*}{VB mph} & GPT2 & 1.60 & 2.20 & **4.05** \\ \cline{2-5}  & Seg2Seq & 3.85 & 1.45 & 1.25 \\ \cline{1-1}  & RetrieveEdit & **4.15** & **2.65** & 2.20 \\ \cline{1-1} \cline{2-5}  & Human & 4.70 & 4.45 & 5.00 \\ \hline \hline \end{tabular}
\end{table}
Table 17: Human evaluation for single atomic style transfer on 7 selected transfers (the 7 transfers with BLEU scores appearing in main part of paper). The result shows that on harder transfers, all approaches fall short of human performance, and that GPT2 excels at style while RetrieveEdit is better at grammar and content preservation.

\begin{table}
\begin{tabular}{p{56.9pt}|p{113.8pt}|p{113.8pt}} \hline \hline Transfer & To Future + Passive To Active & To Past + PP Removal \\ \hline Source Sentence & NUM % was risen by sales to NUM billion from NUM billion & the bond market was unmoved by the economic statistics \\ \hline Target Sentence & sales will rise NUM % to NUM billion from NUM billion & the bond market is unmoved \\ \hline SeqGPT & willasalesalesalesales to billion from from NUM billion & the bond market is is \\ \hline CS-GPT-zero & NUM % % \% risen rises sales & the bond market is unmoved by the economic statistics \\ \hline CS-GPT & sales will rise NUM % to NUM billion from NUM billion & the bond market is unmoved \\ \hline \hline \end{tabular}
\end{table}
Table 18: 2 examples of composition transfers generated by CS-GPT, SeqGPT and CS-GPT-zero. CS-GPT successfully models compositional transfers across multiple styles.

 



\begin{table}
\begin{tabular}{l l l c c c c c c c} \hline \hline Dataset & Transfers & Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CiDER \\ \hline To Future Tense & GPT2 & **0.895** & **0.851** & **0.812** & **0.777** & **0.539** & **0.898** & **7.708** \\  & CS-GPT (TV) & 0.727 & 0.614 & 0.614 & 0.450 & 0.362 & 0.731 & 4.386 \\  & CS-GPT (TP) & 0.810 & 0.731 & 0.663 & 0.606 & 0.446 & 0.818 & 6.026 \\ \hline \multirow{3}{*}{To Past Tense} & GPT2 & **0.835** & **0.776** & **0.721** & **0.673** & 0.484 & **0.842** & 6.699 \\  & CS-GPT (TV) & 0.694 & 0.586 & 0.494 & 0.420 & 0.353 & 0.700 & 4.051 \\  & CS-GPT (TP) & 0.834 & 0.771 & 0.718 & 0.672 & **0.486** & 0.841 & **6.704** \\ \hline \multirow{3}{*}{To Present Tense} & GPT2 & 0.753 & 0.662 & 0.586 & 0.523 & 0.412 & 0.772 & 5.293 \\  & CS-GPT (TV) & 0.733 & 0.635 & 0.553 & 0.488 & 0.387 & 0.744 & 4.742 \\  & CS-GPT (TP) & **0.826** & **0.755** & **0.691** & **0.637** & **0.491** & **0.831** & **6.315** \\ \hline ActiveToPassive & GPT2 & **0.475** & **0.329** & **0.238** & **0.189** & 0.216 & **0.463** & **1.820** \\  & CS-GPT (TV) & 0.472 & 0.324 & 0.232 & 0.179 & **0.216** & 0.454 & 1.790 \\ \hline \multirow{3}{*}{PassiveToActive} & GPT2 & 0.433 & 0.271 & 0.167 & 0.120 & 0.191 & 0.434 & 1.329 \\  & CS-GPT (TV) & **0.506** & **0.345** & **0.243** & **0.184** & **0.229** & **0.505** & **1.958** \\ \hline \multirow{3}{*}{PP Removal} & GPT2 & **0.763** & **0.700** & **0.645** & **0.593** & 0.419 & **0.786** & **6.011** \\  & CS-GPT (TP) & 0.760 & 0.698 & 0.639 & 0.585 & **0.420** & 0.772 & 5.783 \\ \hline \hline \end{tabular}
\end{table}
Table 20: Comparing single transfer performances between CS-GPT and GPT2 baselines (where TV indicates the CS-GPT is trained on Tense+Voice dataset and TP indicates the CS-GPT is trained on Tense+PP Removal dataset). The result shows that CS-GPT can perform multiple single style transfers with similar performance to GPT2 trained specifically for that one transfer, and sometimes even outperforms GPT2.

\begin{table}
\begin{tabular}{l l l c c c c c c c} \hline \hline Dataset & Transfers & Model & BLEU-1 & BLEU-2 & BLEU-3 & BLEU-4 & METEOR & ROUGE\_L & CiDER \\ \hline \multirow{6}{*}{Tense} & TopPast+ & SeqQPT & 0.332 & 0.155 & 0.057 & 0.024 & 0.144 & 0.300 & 0.636 \\  & ActiveToPassive & CS-GPT-zero & 0.337 & 0.163 & 0.075 & 0.029 & 0.154 & 0.283 & 0.760 \\  & CS-GPT & **0.409** & **0.238** & **0.133** & **0.064** & **0.180** & **0.378** & **1.029** \\ \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.391 & 0.222 & 0.120 & 0.065 & 0.167 & 0.373 & 0.866 \\  & ActiveToPassive & CS-GPT-zero & 0.419 & 0.243 & 0.114 & 0.047 & 0.209 & 0.325 & 1.238 \\  & ActiveToPassive & CS-GPT & **0.496** & **0.340** & **0.240** & **0.185** & **0.217** & **0.479** & **1.800** \\ \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.401 & 0.212 & 0.097 & 0.048 & 0.163 & 0.385 & 0.888 \\ \multirow{3}{*}{Tense} & PositiveToActive & CS-GPT-zero & 0.399 & 0.245 & 0.123 & 0.047 & 0.212 & 0.349 & 1.075 \\  & CS-GPT & **0.528** & **0.364** & **0.259** & **0.197** & **0.234** & **0.524** & **2.020** \\ \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.381 & 0.210 & 0.098 & 0.045 & 0.156 & 0.368 & 0.876 \\  & PassiveToActive & CS-GPT-zero & 0.365 & 0.181 & 0.073 & 0.025 & 0.156 & 0.343 & 0.752 \\  & CS-GPT & **0.474** & **0.297** & **0.175** & **0.099** & **0.206** & **0.473** & **1.513** \\ \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.348 & 0.189 & 0.085 & 0.037 & 0.142 & 0.343 & 0.745 \\  & PassiveToActive & CS-GPT-zero & 0.424 & 0.257 & 0.118 & 0.046 & 0.208 & 0.389 & 1.025 \\  & CS-GPT & **0.523** & **0.366** & **0.264** & **0.210** & **0.243** & **0.522** & **2.118** \\ \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.396 & 0.256 & 0.177 & 0.136 & 0.179 & 0.384 & 1.209 \\  & ActiveToPassive & CS-GPT-zero & 0.445 & 0.254 & 0.120 & 0.059 & 0.212 & 0.348 & 1.271 \\  & ActiveToPassive & CS-GPT & **0.503** & **0.358** & **0.271** & **0.223** & **0.233** & **0.491** & **2.118** \\ \hline \multirow{3}{*}{Tense} & ToyFuture+ & SeQGPT & 0.722 & 0.644 & **0.581** & **0.524** & 0.385 & **0.755** & 5.562 \\  & PPR removal & CS-GPT-zero & 0.465 & 0.335 & 0.221 & 0.137 & 0.313 & 0.496 & 1.907 \\ \cline{1-1}  & CS-GPT & **0.738** & **0.652** & 0.578 & 0.518 & **0.393** & 0.755 & **5.289** \\ \cline{1-1} \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.714 & 0.640 & 0.573 & 0.510 & 0.374 & 0.724 & 5.152 \\ \cline{1-1}  & PPPR removal & CS-GPT-zero & 0.542 & 0.389 & 0.268 & 0.182 & 0.314 & 0.535 & 2.103 \\ \cline{1-1} \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.727 & **0.695** & **0.624** & **0.564** & **0.421** & **0.775** & **5.585** \\ \cline{1-1} \cline{2-10}  & \multicolumn{2}{c}{SegQPT} & 0.618 & 0.518 & 0.435 & 0.368 & 0.338 & 0.663 & 4.119 \\ \cline{1-1}





## Appendix D Mitigating Social Biases: Qualitative Evaluation

We created two prompts "The man worked as" and "The woman worked as", and generated \(50\) sentences with each prompt from GPT2. Next, we determine biased words by taking the \(1,000\) closest vectors in GloVe word embeddings Pennington et al. (2014) to "man" and "woman". Then, we determine a sentence as biased if the phrase describing the occupation in the sentences contains any biased words. With this standard, we found that \(21\) out of \(50\) sentences for man and \(28\) out of \(50\) sentences are biased. Then, we replaced the occupations in these \(49\) biased sentences with occupations sampled uniformly randomly from all \(100\) generated sentences, and then asked two independent human annotators to evaluate the \(49\) replaced sentences on a five-point scale of **Significantly More Biased, Slightly More Biased, The Same, Slightly Less Biased**, and **Significantly Less Biased**. On average, the annotators reported \(22\) sentences being significantly less biased compared to before the replacements, while all other sentences are either slightly less biased or neutral. The full results of this experiment are shown in Table 21. A few examples that were deemed **Significantly Less Biased** by both annotators are shown in Table 22.