Exploiting Social Media Content for Self-Supervised Style Transfer (2022)

Abstract: Recent research on style transfer takes inspiration from unsupervised neural machine translation (UNMT), learning from large amounts of non-parallel data by exploiting cycle consistency loss, back-translation, and denoising autoencoders. By contrast, the use of selfsupervised NMT (SSNMT), which leverages (near) parallel instances hidden in non-parallel data more efficiently than UNMT, has not yet been explored for style transfer. In this paper we present a novel Self-Supervised Style Transfer (3ST) model, which augments SSNMT with UNMT methods in order to identify and efficiently exploit supervisory signals in non-parallel social media posts. We compare 3ST with state-of-the-art (SOTA) style transfer models across civil rephrasing, formality and polarity tasks. We show that 3ST is able to balance the three major objectives (fluency, content preservation, attribute transfer accuracy) the best, outperforming SOTA models on averaged performance across their tested tasks in automatic and human evaluation.

## introduction style transfer is a highly versatile task in natural language processing, where the goal is to modify the stylistic attributes of a text while maintaining its original meaning. a broad variety of stylistic attributes has been considered, including formality @xcite , gender @xcite , polarity @xcite and civility @xcite . potential industrial applications are manifold and range from simplifying professional language to be intelligible to laypersons @xcite , the generation of more compelling news headlines @xcite , to related tasks such as text simplification for children and people with disabilities @xcite . data-driven style transfer methods can be classified according to the kind of data they use: parallel or non-parallel corpora in the two styles @xcite . to learn style transfer on non-parallel monostylistic corpora, current approaches take inspiration from unsupervised neural machine translation (unmt) @xcite , by exploiting cycle consistency loss @xcite , iterative back-translation @xcite and denoising autoencoders (dae) @xcite . as these approaches are similar to unmt they suffer from the same limitations, i.e. poor performance relative to supervised neural machine translation (nmt) systems when the amount of unmt training data is small and/or exhibits domain mismatch @xcite . unfortunately, this is precisely the case for most existing style transfer corpora. in this paper, we follow an alternative approach inspired by self-supervised nmt @xcite that jointly learns online (near) parallel sentence pair extraction (spe), back-translation (bt) and style transfer in a loop. the goal is to identify and exploit supervisory signals present in limited amounts of (possibly domain-mismatched) nonparallel data ignored by unmt. the architecture of our system-called self-supervised style transfer (3st)-implements an online self-supervisory cycle, where learning spe enables us to learn style transfer on extracted parallel data, which iteratively improves spe and bt quality, and thereby style transfer learning, in a virtuous circle. we evaluate and compare 3st to current state-ofthe-art (sota) style transfer models on two established tasks: formality and polarity style transfer, where 3st is the most balanced model and reaches top overall performance. to gain insights into the performance of 3st on an under-explored task, we also focus on the civil rephrasing task, which is interesting as i) it has been explored only twice before @xcite and ii) it makes an important societal contribution in order to tackle hateful content online. we focus on performance and qualitative analysis of 3st predictions on this task's test set and identify shortcomings of the currently available data setup for civil rephrasing. on civil rephrasing, 3st generates more neutral sentences than the current sota model while being on par in overall performance. our contribution is threefold: style transfer can be treated as a supervised translation task between two styles @xcite . however, for most style transfer tasks, parallel data is scarcely available. to learn style transfer without parallel data, prior research has focused on exploiting larger amounts of monostylistic data in combination with a smaller amount of style-labeled data. one such approach is using variational autoencoders and disentangled latent spaces @xcite , which can be further incentivized towards generating fluent or style-relevant content by fusing them with adversarial @xcite or styleenforcing @xcite discriminators. chawla and yang (2020 ) use a language model as the discriminator, leading to a more informative signal to the generator during training and thus more fluent and stable results. @xcite argue that adversarially learned outputs tend to be low-quality, and that most sentiment modification is based on simple deletion and replacement of relevant words. the above approaches focus on separating content and style, either in latent space or surface form, however this separation is difficult to achieve (gonen and goldberg, 2019). @xcite instead train a transformer together with a discriminator, delicious cake. disgusting cake. without disentangling the style features before decoding. current approaches treat style transfer similar to an unsupervised neural machine translation (artetxe et al., 2019) task. jin et al. (2019) create pseudo-parallel corpora by extracting similar sentences offline from two monostylistic corpora to train an initial nmt model which is then iteratively improved using back-translation. luo et al. (2019) use a reinforcement approach to further improve sentence fluency. laugier et al. (2021) improve fluency without the need of any style-specific classifiers, giving their model a head start by initializing it on a pre-trained transformer model. wang et al. ( 2020 ) argue that standard nmt training cannot account for the small differences between informal and formal style transfer, and apply style-specific decoder heads to enforce style differences. our approach differs from the two step approach of @xcite , who first extract similar sentences from style corpora offline and then initialize their system by training on them. @xcite show that joint online learning to extract and translate in self-supervised nmt (ssnmt) leads to higher recall and precision of the extracted data. following this observation, our 3st approach performs similar sentence extraction and style transfer learning online with a single model in a loop. we further extend the ssnmt-based approach by combining it with unmt methods, namely by generating additional training data via online back-translation, and by initialising our models with dae trained in an unsupervised manner. figure 1 shows the 3st architecture, which uses the encoder outputs at training time as sentence representations to perform online (near) parallel sentence pair extraction (spe) together with online back-translation (bt) and style transfer. self-supervised nmt (ssnmt): ssnmt @xcite is an encoder-decoder architecture that jointly learns to identify parallel data in non-parallel data and bidirectional nmt. instead of using ssnmt on different language corpora to learn machine translation, we show how ideas from ssnmt can be used to learn a self-supervised style transfer system from non-parallel social media content. a single bidirectional encoder simultaneously encodes both styles and maps the internal representations of the two styles into the same space. this way, they can be used to compute similarities between sentence pairs in order to identify similar and discard non-similar ones for training. formally, given two monostylistic corpora s1 and s2 of opposing styles, e.g. toxic and neutral, sentence pairs (s @xmath0, s @xmath1, a transformer in our experiments. from the internal representations for the input sentences s s1 and s s2 , ssnmt uses the sum of the word embeddings w(s) and the sum of the encoder outputs e(s) for filtering. the embedded pairs {w(s s1 ), w(s s2 )} are scored using the margin-based measure @xcite . the same is done with pairs {e(s s1 ), e(s s2 )}. if a sentence pair is the most similar pair for both style directions and for both sentence representations, it is accepted for training, otherwise it is discarded. this sequence of scoring and filtering is denoted as sentence pair extraction (spe) in 3st. spe improves style transfer and style transfer improves spe online in a virtuous loop, resulting in a single system that jointly learns to identify its supervision signals in the data and to perform style transfer. to address the characteristics of the monostylistic corpora we extend basic ssnmt in two ways: large-scale extraction: ssnmt extracts parallel data from comparable corpora, which contain smaller topic-aligned documents {d s1 , d s2 } of similar content, thus reducing the search space during spe from |s1|×|s2| to |d s1 |×|d s2 |. however, style transfer corpora usually consist of large collections of (unaligned) sequences of a specific style, which forces the exploration of the full space. improving over the one-by-one comparison of vector representations, we index foot_0 our data using faiss @xcite .