A Recipe for Arbitrary Text Style Transfer with Large Language Models (2022)

Abstract: In this paper, we leverage large language models (LLMs) to perform zero-shot text style transfer. We present a prompting method that we call augmented zero-shot learning, which frames style transfer as a sentence rewriting task and requires only a natural language instruction, without model fine-tuning or exemplars in the target style. Augmented zero-shot learning is simple and demonstrates promising results not just on standard style transfer tasks such as sentiment, but also on arbitrary transformations such as ‘make this melodramatic’ or ‘insert a metaphor.’

## introduction text style transfer is the task of rewriting text to incorporate additional or alternative stylistic elements while preserving the overall semantics and structure. although style transfer has garnered increased interest due to the success of deep learning, these approaches usually require a substantial amount of labeled training examples, either as parallel text data @xcite or non-parallel text data of a single style. @xcite @xcite . even bleeding-edge approaches that tackle the challenging problem of label-free style transfer are limited in that they require at least several exemplar sentences that dictate a given target style @xcite . hence, recent survey papers have identified a need for new methods that both reduce the training data requirements and expand the scope of styles supported @xcite . in this work, we present augmented zero-shot learning, a prompting method that allows large language models to perform text style transfer to arbitrary styles, without any exemplars in the target style. our method builds on prior work showing ## augmented zero-shot prompting although large lms are trained only for continuation, recent work has shown that they can perform a variety of nlp tasks by expressing the task as a prompt that encourages the model to output the desired answer as the continuation @xcite @xcite ; schick and schütze, 2021, inter alia; see @xcite for a survey). the simplest approach, zeroshot prompting, directly uses natural language to ask the large lm to perform a task, as shown in figure 1a . zero-shot prompting, however, can be prone to failure modes such as not returning wellformatted or logical outputs (see §6). few-shot prompting, as shown in figure 1b , has been shown to achieve higher performance, but requires exemplars for the exact task that we want the model to perform. such few-shot examples can be easily obtained if the desired style transformation is known ahead of time, but this ultimately limits style transfer to a set of pre-specified style tasks. to remove the need for these labeled exemplars for each style transfer task, we propose augmented zero-shot learning, a method for performing multitask style transfer using a single set of exemplars. instead of prompting the model with exemplars specific to the exact style transfer task we wish to perform, we prompt the model with examples of a variety of sentence rewriting operations, as shown in figure 1c . this intuition is inspired by @xcite 's observation that successful prompts constrain the behavior of the large lm away from failure modes-in our case, we aim to preserve the flexibility of a zero shot prompt while encouraging the model to produce outputs of a specific template. we keep the the format of the exemplars constant and insert the de-original there, in the middle of the street, stood an old man in a weatherbeaten brown coat. ## experimental setup style transfer tasks. we consider six style transfer tasks that we deem non-standard, listed in we also evaluate our method on two standard style transfer tasks: sentiment and formality. we use the yelp polarity dataset @xcite for sentiment and grammarly's yahoo answers formality corpus (gyafc) dataset for formality (rao and tetreault, 2018). 1 these datasets allow us to evaluate performance of augmented zero-shot learning in the context of prior supervised methods which have been used on these tasks. model. augmented zero-shot learning requires a large language model. we primarily use lamda, a left-to-right decoder-only transformer language model @xcite with a non-embedding parameter count of 137b @xcite . the pre-trained lamda model, which we refer to as llm, was trained on a corpus comprising 1.95b public web documents, including forum and dialog data and wikipedia. the dataset was tokenized into 2.49t bpe tokens with a sentencepiece vocabulary size of 32k @xcite . we also use llm-dialog, the final lamda model which was finetuned on a curated, high-quality subset of data identified to be in a conversational format. decoding was done with top-@xmath0. to show that the success of augmented zero-shot learning is not restricted to these two large lms, we also perform experiments with gpt-3 ( the prompts used for llm and gpt-3 are shown in figure 1 . for llm-dialog, the prompt was instead formulated as a conversation between one agent who is requesting rewrites and another who is performing the rewrites. see ## potential of arbitrary styles one promising application of augmented zero-shot learning is an ai-powered writing assistant that can allow writers to transform their text in arbitrary ways that the writer defines and controls. as a qualitative case study to explore what arbitrary re-write styles may be requested, we built an ai-assisted story-writing editor with a "rewrite as" feature that uses our augmented few-shot method. our editor has a freeform text box for users to specify how they would like a selection of their story to be rewritten (see figure 6 in the appendix). we asked 30 people from a creative writing group to use our ui to write a 100-300 word story, collecting 333 rewrite requests in total. ## limitations and failure modes this section details several qualitative limitations with our method. unparsable answers a frequent problem that arises when using large lms for other nlp tasks is their outputs cannot be automatically parsed into usable answers. for example, when given a prompt like "here is some text: that is an ugly dress. here is a rewrite of the text, which is more positive" llm-dialog might return something like "sounds like you are a great writer!" similar error modes exist for llm, which might output something like "here are more writing tips and tricks." other 5.1 424 multidecoder @xcite 50.3 27.7 1,703 delete-only @xcite 81.4 28.6 606 delete-retrieve @xcite 86.2 31.1 948 unpaired rl @xcite 52.2 37.2 2,750 dual rl @xcite 85.9 55.1 982 style transformer @xcite to be a little less angsty • to be about mining • to be better written • to be less diabolical • to be more absurd • to be more adventurous • to be more dickensian • to be more emotional • to be more magical • to be more melodramatic • to be more philosophical • to be more revolutionary • to be more surprising • to be more suspenseful • to be more technical • to be more whimsical • to be warmer • to fit better grammatically with the rest of the story • to make more sense hallucinations large lms are known to hallucinate text content; we saw this happen frequently for style transfer. while this is an advantage in some contexts like creative writing, it is undesirable for applications like summarization. ## conclusions we introduced augmented zero-shot learning, which we find shows shows strikingly promising performance considering its simplicity. this prompting paradigm moves the needle in text style transfer by expanding the range of possible styles beyond the currently limited set of styles for which annotated data exists. more broadly, we also hope that the strategy of prompting a large lm with nontask specific examples can inspire new inferenceonly methods for other nlp tasks. raul puri and bryan catanzaro. 2019. zero-shot text classification with generative language models. arxiv preprint arxiv:1912.10165. sudha rao and joel tetreault. 2018. dear sir or madam, may i introduce the gyafc dataset: corpus, benchmarks and metrics for formality style transfer. in proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long papers), pages 129-140, new orleans, louisiana. association for computational linguistics. laria reynolds and kyle mcdonell. 2021. prompt programming for large language models: beyond the few-shot paradigm. parker riley, noah constant, mandy guo, girish kumar, david c. uthus, and zarana parekh. 2021. textsettr: label-free text style extraction and tunable targeted restyling. proceedings of the annual meeting of the association of computational linguistics (acl). keisuke sakaguchi and benjamin van durme. 2018. efficient online scalar annotation with bounded support. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 208-218, melbourne, australia. association for computational linguistics.