TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling

We present a novel approach to the problem of text style transfer . Unlike previous approaches that use parallel or non-parallel labeled data , our technique removes the need for labels entirely , relying instead on the implicit connection in style between adjacent sentences in unlabeled text . We show that T5 ( Raffel et al. , 2020 ) , a strong pretrained text-to-text model , can be adapted to extract a style vector from arbitrary text and use this vector to condition the decoder to perform style transfer . As the resulting learned style vector space encodes many facets of textual style , we recast transfers as “ targeted restyling ” vector operations that adjust specific attributes of the input text while preserving others . When trained over unlabeled Amazon reviews data , our resulting TextSETTR model is competitive on sentiment transfer , even when given only four exemplars of each class . Furthermore , we demonstrate that a single model trained on unlabeled Common Crawl data is capable of transferring along multiple dimensions including dialect , emotiveness , formality , politeness , and sentiment . 1 INTRODUCTION . There has been a recent surge of interest in text style transfer , with the aim of training models able to modify specific attributes of input text ( e.g. , sentiment or formality ) while preserving the remaining content . For example , a sentiment transfer model might transform the input “ best book ever ! ” into “ worst book ever ! ” , while a formality transfer model might change the same input into “ This is the best book I have ever read. ” Work in this area falls into three categories . Supervised approaches like Jhamtani et al . ( 2017 ) transfer between pre-selected styles , and rely on aligned parallel training data to teach the model the desired input/output correspondence . This method is limited by the availability of parallel corpora . So-called “ unsupervised ” approaches like Li et al . ( 2018 ) and Lample et al . ( 2019 ) remove the requirement for parallel data , but still require labeled training examples of each style , and are limited to transfer between a pre-specified set of styles . Label-free approaches like the recent Xu et al . ( 2020 ) remove the need for any training labels . While the most technically challenging , this offers the potential for transferring between arbitrary styles at inference time and has significant value , as curated datasets are not available for many style attributes . In this work , we explore the hypothesis that large pretrained text-to-text models like T5 ( Raffel et al. , 2020 ) already contain a strong representation of textual style , which can be extracted and used to condition the decoder of a style transfer model through a relatively lightweight fine-tuning procedure . To isolate style information in the absence of labels , we rely on the observation that style is a “ slow-moving ” feature , which tends to be consistent over large spans of text . Specifically , given two adjacent sentences from an unlabeled corpus , we train our model to extract a “ style vector ” from the first and use that vector to perform denoising and other reconstruction tasks on the second . This technique extends the unsupervised approach of Lample et al . ( 2019 ) to the label-free setting , and allows us to reformulate the style transfer operation as a directional operation in style vector space using the difference between target and source style vectors ; we call this “ targeted restyling ” . When combined with a novel “ tunable inference ” technique for controlling token add/delete rates , this gives our final model : Text Style Extraction and Tunable Targeted Restyling ( TextSETTR ) . Our main contributions are to : ( 1 ) demonstrate the viability of label-free style transfer,1 ( 2 ) use sentence adjacency as a means for inducing text style representations , ( 3 ) reframe style transfer as “ targeted restyling ” directional operations in style space , ( 4 ) introduce “ tunable inference ” for finergrained control of transfers , ( 5 ) show the effectiveness of “ noisy ” back-translation training , and ( 6 ) illustrate few-shot generalization to a range of style attributes including dialect , emotiveness , formality , politeness , and sentiment . 2 METHOD . Figure 1 illustrates our proposed TextSETTR architecture . At a high level , our approach follows Lample et al . ( 2019 ) , who train a denoising auto-encoder conditioned on a fixed-width style vector . The key difference in our case is that the true style is unknown at training time . To overcome this , we jointly train a “ style extractor ” component to induce a useful style representation ( that can aid in reconstruction ) from text in the nearby context . We describe this in more detail below . 2.1 MODEL ARCHITECTURE . We conduct our experiments using a modified version of the Text-to-Text Transfer Transformer ( T5 ) ( Raffel et al. , 2020 ) . Like T5 , our model includes a transformer-based encoder and decoder . As in T5 pretraining , the input to the encoder is a corrupted/noised version of the target , resulting in a reconstruction task . Our goal is to design a type of corruption that results in this training task resembling style transfer , despite the lack of labeled training data . Our core addition to T5 is the style extractor . Based on the encoder ’ s architecture , this component ’ s input is an uncorrupted sentence in the same style as the target ; relying on our assumption that style is a slow-moving feature , we use the sentence preceding the target ( the “ context ” ) for this.2 This encourages extracting a style representation that is useful for repairing the corrupted input . The only architectural difference between the encoder and style extractor is that we mean-pool the style extractor ’ s hidden state sequence into a single fixed-width vector ( the “ style vector ” ) ; in our experiments , the dimensionality of this vector and the encoder hidden states is 1024 . To incorporate the style vector into the rest of the model , we simply add it to each of the final encoder hidden states . 1Our work is concurrent with Xu et al . ( 2020 ) , who offer a substantially different approach to label-free style transfer , as discussed in Sections 3 and 5 . 2This approach is similar to the use of adjacent sentences for weak supervision in Devlin et al . ( 2019 ) and Zhang et al . ( 2020 ) . We initialize the weights of our model with those of a pretrained T5 model . We initialize both the style extractor and encoder from the pretrained encoder , but the weights are not tied during training . 2.2 CORRUPTION STRATEGIES . We experiment with combinations of three different reconstruction tasks , each contributing a loss term . All three share the same overall structure , where a sentence si in the dataset is corrupted by some function f to produce s̃i = f ( si ) . The cross-entropy loss is calculated using the uncorrupted sentence si as the target , the corrupted sentence s̃i as the input , and the uncorrupted preceding sentence si−1 as the context . The three choices of f are Noise ( N ) , Back-Translation ( BT ) , and Noisy Back-Translation ( NBT ) , described below . Noise ( N ) This function corrupts the input by ( i ) dropping , ( ii ) replacing , and/or ( iii ) shuffling tokens , in that order . For each example we sample a separate noise probability p for each sub-type of noise from a uniform distribution in the range 20–60 % . We do this because the model should be able to “ undo ” varying degrees of noise , as some types of style transfer may require changing only a few tokens , while others may require larger rewrites . For drop noise , we drop each token in si with independent probability p. For replace noise , let sik be the k-th token within si . For each si , a random other example sj is chosen , and then each token sik is replaced by sjk with probability p . If sj has fewer than k tokens , then the replacement does not occur . For shuffle noise , each token in si is chosen with probability p , and then all chosen tokens are randomly shuffled to the position of another chosen token , leaving non-chosen tokens in place . The use of drop and shuffle noise results in a loss term similar to the denoising loss used by Lample et al . ( 2019 ) . Their motivation for this loss was to encourage language modeling . As we fine-tune an already-strong T5 language model in our experiments , our motivation is rather to introduce a conditional element to the language model , in the form of the extracted style vector input . Back-Translation ( BT ) This corruption function , used by Lample et al . ( 2019 ) , runs the current version of the model in inference mode to transfer si into a different style , giving the corrupted s̃i . In prior work using labels , specifying a different target style was straightforward . In our case , because we do not have access to labels , we simply sample a random sentence sj to use as the context . To increase diversity of the generated examples , we decode with sampling instead of greedy decoding . Because s̃i is produced by a strong language model , BT should result in training examples where both the input and output are coherent sentences , matching our intended inference setting . By contrast , the text produced by “ Noise ” corruption does not resemble test-time inputs . Noisy Back-Translation ( NBT ) This novel corruption function is a composition of the previous two . Noise is first applied to si as described above , and the result is used as the input ( with randomlysampled sj as the context ) to the model in inference mode to produce s̃i via sampling , as in BT . Once the model has learned to undo random noise from the associated loss term , NBT should produce training examples where some of the tokens are preserved from si while others were generated by the model itself under the influence of the “ incorrect ” context sj . This is similar to BT , but we hypothesize that it may be better suited to style transfer . BT was originally used for machine translation ( Sennrich et al. , 2016 ) , a setting where most or all input tokens need to be changed . In contrast , style transfer within a single language usually requires only changing a subset of input tokens ; the training examples resulting from the NBT procedure should have this property . We believe that this will encourage the model to identify which tokens in the input do not match the target style indicated by si−1 and change them , which is exactly what we want a style transfer model to do . This is conceptually similar to the work of Clark et al . ( 2020 ) , who pretrain language models to discriminate which input tokens were originally present and which were altered by a simpler language model . Final Loss The final loss term used for training is the sum of the above loss terms , each calculated from the same input si . However , not every model we experiment with includes all three losses .

Summary: This paper proposes a novel approach to the label-free style transfer task where an input is corrupted via different strategies and fed into an auto-encoder which is additionally conditioned on its prior adjacent context sentence via a "style encoder" which adds its mean pooled hidden state to the former before decoding.  Both encoders are initialized to and leverage the strength of pre-trained T5 model.  Additionally the amount of addition/deletion of tokens is tunable at both training and inference time.