TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling

We present a novel approach to the problem of text style transfer . Unlike previous approaches that use parallel or non-parallel labeled data , our technique removes the need for labels entirely , relying instead on the implicit connection in style between adjacent sentences in unlabeled text . We show that T5 ( Raffel et al. , 2020 ) , a strong pretrained text-to-text model , can be adapted to extract a style vector from arbitrary text and use this vector to condition the decoder to perform style transfer . As the resulting learned style vector space encodes many facets of textual style , we recast transfers as “ targeted restyling ” vector operations that adjust specific attributes of the input text while preserving others . When trained over unlabeled Amazon reviews data , our resulting TextSETTR model is competitive on sentiment transfer , even when given only four exemplars of each class . Furthermore , we demonstrate that a single model trained on unlabeled Common Crawl data is capable of transferring along multiple dimensions including dialect , emotiveness , formality , politeness , and sentiment . 1 INTRODUCTION . There has been a recent surge of interest in text style transfer , with the aim of training models able to modify specific attributes of input text ( e.g. , sentiment or formality ) while preserving the remaining content . For example , a sentiment transfer model might transform the input “ best book ever ! ” into “ worst book ever ! ” , while a formality transfer model might change the same input into “ This is the best book I have ever read. ” Work in this area falls into three categories . Supervised approaches like Jhamtani et al . ( 2017 ) transfer between pre-selected styles , and rely on aligned parallel training data to teach the model the desired input/output correspondence . This method is limited by the availability of parallel corpora . So-called “ unsupervised ” approaches like Li et al . ( 2018 ) and Lample et al . ( 2019 ) remove the requirement for parallel data , but still require labeled training examples of each style , and are limited to transfer between a pre-specified set of styles . Label-free approaches like the recent Xu et al . ( 2020 ) remove the need for any training labels . While the most technically challenging , this offers the potential for transferring between arbitrary styles at inference time and has significant value , as curated datasets are not available for many style attributes . In this work , we explore the hypothesis that large pretrained text-to-text models like T5 ( Raffel et al. , 2020 ) already contain a strong representation of textual style , which can be extracted and used to condition the decoder of a style transfer model through a relatively lightweight fine-tuning procedure . To isolate style information in the absence of labels , we rely on the observation that style is a “ slow-moving ” feature , which tends to be consistent over large spans of text . Specifically , given two adjacent sentences from an unlabeled corpus , we train our model to extract a “ style vector ” from the first and use that vector to perform denoising and other reconstruction tasks on the second . This technique extends the unsupervised approach of Lample et al . ( 2019 ) to the label-free setting , and allows us to reformulate the style transfer operation as a directional operation in style vector space using the difference between target and source style vectors ; we call this “ targeted restyling ” . When combined with a novel “ tunable inference ” technique for controlling token add/delete rates , this gives our final model : Text Style Extraction and Tunable Targeted Restyling ( TextSETTR ) . Our main contributions are to : ( 1 ) demonstrate the viability of label-free style transfer,1 ( 2 ) use sentence adjacency as a means for inducing text style representations , ( 3 ) reframe style transfer as “ targeted restyling ” directional operations in style space , ( 4 ) introduce “ tunable inference ” for finergrained control of transfers , ( 5 ) show the effectiveness of “ noisy ” back-translation training , and ( 6 ) illustrate few-shot generalization to a range of style attributes including dialect , emotiveness , formality , politeness , and sentiment . 2 METHOD . Figure 1 illustrates our proposed TextSETTR architecture . At a high level , our approach follows Lample et al . ( 2019 ) , who train a denoising auto-encoder conditioned on a fixed-width style vector . The key difference in our case is that the true style is unknown at training time . To overcome this , we jointly train a “ style extractor ” component to induce a useful style representation ( that can aid in reconstruction ) from text in the nearby context . We describe this in more detail below . 2.1 MODEL ARCHITECTURE . We conduct our experiments using a modified version of the Text-to-Text Transfer Transformer ( T5 ) ( Raffel et al. , 2020 ) . Like T5 , our model includes a transformer-based encoder and decoder . As in T5 pretraining , the input to the encoder is a corrupted/noised version of the target , resulting in a reconstruction task . Our goal is to design a type of corruption that results in this training task resembling style transfer , despite the lack of labeled training data . Our core addition to T5 is the style extractor . Based on the encoder ’ s architecture , this component ’ s input is an uncorrupted sentence in the same style as the target ; relying on our assumption that style is a slow-moving feature , we use the sentence preceding the target ( the “ context ” ) for this.2 This encourages extracting a style representation that is useful for repairing the corrupted input . The only architectural difference between the encoder and style extractor is that we mean-pool the style extractor ’ s hidden state sequence into a single fixed-width vector ( the “ style vector ” ) ; in our experiments , the dimensionality of this vector and the encoder hidden states is 1024 . To incorporate the style vector into the rest of the model , we simply add it to each of the final encoder hidden states . 1Our work is concurrent with Xu et al . ( 2020 ) , who offer a substantially different approach to label-free style transfer , as discussed in Sections 3 and 5 . 2This approach is similar to the use of adjacent sentences for weak supervision in Devlin et al . ( 2019 ) and Zhang et al . ( 2020 ) . We initialize the weights of our model with those of a pretrained T5 model . We initialize both the style extractor and encoder from the pretrained encoder , but the weights are not tied during training . 2.2 CORRUPTION STRATEGIES . We experiment with combinations of three different reconstruction tasks , each contributing a loss term . All three share the same overall structure , where a sentence si in the dataset is corrupted by some function f to produce s̃i = f ( si ) . The cross-entropy loss is calculated using the uncorrupted sentence si as the target , the corrupted sentence s̃i as the input , and the uncorrupted preceding sentence si−1 as the context . The three choices of f are Noise ( N ) , Back-Translation ( BT ) , and Noisy Back-Translation ( NBT ) , described below . Noise ( N ) This function corrupts the input by ( i ) dropping , ( ii ) replacing , and/or ( iii ) shuffling tokens , in that order . For each example we sample a separate noise probability p for each sub-type of noise from a uniform distribution in the range 20–60 % . We do this because the model should be able to “ undo ” varying degrees of noise , as some types of style transfer may require changing only a few tokens , while others may require larger rewrites . For drop noise , we drop each token in si with independent probability p. For replace noise , let sik be the k-th token within si . For each si , a random other example sj is chosen , and then each token sik is replaced by sjk with probability p . If sj has fewer than k tokens , then the replacement does not occur . For shuffle noise , each token in si is chosen with probability p , and then all chosen tokens are randomly shuffled to the position of another chosen token , leaving non-chosen tokens in place . The use of drop and shuffle noise results in a loss term similar to the denoising loss used by Lample et al . ( 2019 ) . Their motivation for this loss was to encourage language modeling . As we fine-tune an already-strong T5 language model in our experiments , our motivation is rather to introduce a conditional element to the language model , in the form of the extracted style vector input . Back-Translation ( BT ) This corruption function , used by Lample et al . ( 2019 ) , runs the current version of the model in inference mode to transfer si into a different style , giving the corrupted s̃i . In prior work using labels , specifying a different target style was straightforward . In our case , because we do not have access to labels , we simply sample a random sentence sj to use as the context . To increase diversity of the generated examples , we decode with sampling instead of greedy decoding . Because s̃i is produced by a strong language model , BT should result in training examples where both the input and output are coherent sentences , matching our intended inference setting . By contrast , the text produced by “ Noise ” corruption does not resemble test-time inputs . Noisy Back-Translation ( NBT ) This novel corruption function is a composition of the previous two . Noise is first applied to si as described above , and the result is used as the input ( with randomlysampled sj as the context ) to the model in inference mode to produce s̃i via sampling , as in BT . Once the model has learned to undo random noise from the associated loss term , NBT should produce training examples where some of the tokens are preserved from si while others were generated by the model itself under the influence of the “ incorrect ” context sj . This is similar to BT , but we hypothesize that it may be better suited to style transfer . BT was originally used for machine translation ( Sennrich et al. , 2016 ) , a setting where most or all input tokens need to be changed . In contrast , style transfer within a single language usually requires only changing a subset of input tokens ; the training examples resulting from the NBT procedure should have this property . We believe that this will encourage the model to identify which tokens in the input do not match the target style indicated by si−1 and change them , which is exactly what we want a style transfer model to do . This is conceptually similar to the work of Clark et al . ( 2020 ) , who pretrain language models to discriminate which input tokens were originally present and which were altered by a simpler language model . Final Loss The final loss term used for training is the sum of the above loss terms , each calculated from the same input si . However , not every model we experiment with includes all three losses .

Summary: In this paper, the author proposed a transformer-based encoder-decoder framework for label-free text style transfer. The described task under the unsupervised setup is important and instructive for the text style transfer domain. The model architecture is well demonstrated and the writing is easy to follow up. The experiment results show satisfying performance even comparing with state-of-the-art supervised methods.