Generic resources are what you need: Style transfer tasks without task-specific parallel training data (2021)

Abstract: Style transfer aims to rewrite a source text in a different target style while preserving its content. We propose a novel approach to this task that leverages generic resources, and without using any task-specific parallel (source–target) data outperforms existing unsupervised approaches on the two most popular style transfer tasks: formality transfer and polarity swap. In practice, we adopt a multi-step procedure which builds on a generic pre-trained sequence-to-sequence model (BART). First, we strengthen the model’s ability to rewrite by further pre-training BART on both an existing collection of generic paraphrases, as well as on synthetic pairs created using a general-purpose lexical resource. Second, through an iterative back-translation approach, we train two models, each in a transfer direction, so that they can provide each other with synthetically generated pairs, dynamically in the training process. Lastly, we let our best resulting model generate static synthetic pairs to be used in a supervised training regime. Besides methodology and state-of-the-art results, a core contribution of this work is a reflection on the nature of the two tasks we address, and how their differences are highlighted by their response to our approach.

## introduction text style transfer is, broadly put, the task converting a text of one style into another while preserving its content. in its recent tradition within natural language generation (nlg), two tasks and their corresponding datasets have been commonly used @xcite @xcite @xcite . one dataset was specifically created for formality transfer and contains parallel data (gyafc @xcite ), while the other one contains a large amount of non-parallel sentiment labelled texts (yelp @xcite ), with parallel pairs for test, and is used for the task of polarity swap. examples from these datasets are shown in the two tasks are usually conflated in the literature under the general style transfer label and addressed with the same methods, but we find this an oversimplification. formality transfer implies rewriting a formal sentence into its informal counterpart (or viceversa) while preserving its meaning. polarity swap, instead, aims to change a positive text into a negative one (or viceversa); and while the general theme must be preserved, the meaning is by definition not maintained (e.g. "i hated that film" → "i loved that film"). in line with previous work, we also address both tasks in a similar way, but this is actually to unveil how their different nature affects modelling and evaluation. due to the general scarcity of parallel data, previous works mainly adopted unsupervised approaches, dubbed unpaired methods @xcite since they do not rely on labelled training pairs. however, it has also been shown that best results, unsurprisingly, can be achieved if parallel training data (such as the formality dataset @xcite ) is available @xcite . for this reason, substantial work has gone into the creation of artificial training pairs through various methods (see section 2); approaches using synthetic pairs are thus still considered unsupervised in the style transfer literature, since they do not use manually labelled data. we explore how parallel data can best be derived and integrated in a general style transfer framework. to do so, we create pairs in a variety of ways and use them in different stages of our framework. a core aspect of our approach is leveraging generic resources to derive training pairs, both natural and synthetic. on the natural front, we use abundant data from a generic rewriting task: paraphrasing. as for synthetic data, we leverage a general-purpose computational lexicon using its antonymy relation to generate polarity pairs. in practice, we propose a framework that adopts a multi-step procedure which builds upon a general-purpose pre-trained sequence-to-sequence (seq2seq) model. first, we strengthen the model's ability to rewrite by conducting a second phase of pre-training on natural pairs derived from an existing collection of generic paraphrases, as well as on synthetic pairs created using a general-purpose lexical resource. second, through an iterative backtranslation @xcite ) approach, we train two models, each in a transfer direction, so that they can provide each other with synthetically generated pairs on-the-fly. lastly, we use our best resulting model to generate static synthetic pairs, which are then used offline as parallel training data. contributions using a large pre-trained seq2seq model (1) we achieve state-of-the-art results for the two most popular style transfer tasks without taskspecific parallel data. we show that (2) generic resources can be leveraged to derive parallel data for additional model pre-training, which boosts performance substantially and that (3) an iterative back-translation setting where models in the two transfer directions are trained simultaneously is successful, especially if enriched with a reward strategy. we also offer (4) a theoretical contribution over the nature of the two tasks: while they are usually treated as the same task, our results suggest that they could possibly be treated separately. 1 ## related work style transfer is most successful if task-specific parallel data is available, as in the case of formality transfer @xcite . like in most nlp tasks, large pre-trained models have been shown to provide an excellent base for finetuning in a supervised setting @xcite . since parallel data for fine-tuning such large models for style transfer is scarce, a substantial amount of work has gone into methods for creating artificial sentence pairs so that models can be trained in a supervised regime. one way to do this is to artificially generate parallel data via back-translation, so that training pairs are created on-the-fly during the training process itself @xcite @xcite . in these systems, one direction's outputs and its inputs can be used as pairs to train the model of the opposite transfer direction. another common strategy is to use style-wordediting @xcite @xcite to explicitly separate content and style. these approaches first detect relevant words in the source and then do operations like deleting, inserting and combining to create the pair's target. back-transferring is generally used to reconstruct the source sentence for training, so that pairs are also made on-the-fly. @xcite provide evidence that disentangling style and content to learn distinct representations @xcite @xcite is not necessary. reconstructing the source, instead, appears beneficial: it is used by @xcite who pre-train a model on style transfer data with the transformer architecture @xcite ; and by @xcite , who use an attentional seq2seq model that pre-trains the model to reconstruct the source sentence and re-predict its word-level style relevance. @xcite pre-train a lstm-based seq2seq model @xcite using sentence pairs generated by a template-based baseline. more recently, @xcite proposed a twostage strategy of search and learning for formality transfer where they perform a simulated annealing search @xcite to obtain output sentences as pseudo-references, and then fine-tune gpt-2 @xcite with the resulting pairs. the methods above create task-specific artificial pairs, some using pre-crafted manual rules or templates. we aim to overcome this by exploiting generic resources. additionally, it is not evident which strategy works best for creating parallel data, whether offline or on-the-fly, and the simultaneous advantage of both strategies has not been fully explored. lastly, chawla and yang (2020) develop a semi-supervised model based on sequence-to-sequence pre-trained model (bart, @xcite ) using parallel training data and large amounts of non-parallel data, which achieves a significant performance. in previous work, we have also shown that a sequence-to-sequence pretrained model (bart) outperforms a language model (gpt-2) in content preservation and overall performance when task-specific parallel training data is available @xcite . therefore, we use bart as generic base model; we enrich it with iterative back-translation to cre- dataset style sentence-pair gyafc informal that is just my gut feeling. no different between ages if the mind is near to eachother formal that is my personal opinion. there is no difference between ages if the intellect is similar. yelp negative this branch is getting worse and worse. bad service in these areas and really ruined our visit. positive this branch is getting better and better. good service in these areas and really made our visit. parabank 2 source the bank is coming up on your left. i guess i've always been pretty good with words. target you have the bank on the left side. i think narrating has always been my strong suit. ## approach we propose a framework that adopts a multi-step procedure on top of the large pre-trained seq2seq model bart @xcite . given a source sentence @xmath0, • • • , x n } of length n with style s 1 , the goal of text style transfer is to generate a sentence y with style s 2 , preserving the source sentence's meaning in formality transfer or the source sentence's theme in polarity swap. 2 formally, the objective is to minimize the following negative log likelihood: our framework can be conceived as a pipeline, visualised in figure 1 . at the core of the framework are two bart models (model a and model b), one for each transfer direction. since the main challenge in unpaired style transfer is that we cannot directly employ supervision (i.e. task-specific parallel training pairs), we explore and evaluate different ways of creating and using sentence pairs at different stages of the pipeline. first, we strengthen the model's ability to rewrite by conducting a second phase of pre-training on natural pairs derived from an existing collection of generic paraphrases, as well as on synthetic pairs created using a general-purpose lexical resource (step 1, section 4.1). second, we use iterative back-translation with several reward strategies to train the two models in both transfer directions simultaneously; sentence pairs are created on the fly (step 2, section 4.2). third, we create high-quality synthetic pairs using our best systems from the previous step, to create a static resource of parallel data that can be used to train new transfer models (step 3, section 4.3). ## experiments all experiments are implemented atop huggingface transformers @xcite , taking the bart base model (139m parameters) for our experiments. we train our framework using the adam optimiser @xcite with the initial learning rate 1e - foot_3 . the batch size is set to 32. the final values λ for style and content rewards are both set to 1 based on validation results. both wordnet and sentiwordnet are used from nltk 5 . ## conclusions we proposed an unpaired approach that adopts a multi-step procedure based on the general-purpose pre-trained seq2seq model bart. achieving state-of-the-art results on the two most popular "style transfer" tasks, we have shown the benefit of further pre-training using data derived from generic resources as well as the advantage of back-translation, paired with rewards, especially towards content preservation. we have also seen how leveraging paraphrases can enhance both variability and naturalness in the generated text. through experimental settings as well as the introduction of bleurt and comet as metrics, we have also highlighted how the two tasks we addressed differ, and should probably not be conflated into a single "style tranfer" label. indeed, we show that they benefit from partially different modelling, and react differently to evaluation metrics, both key aspects to improve future modelling of these tasks.