Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus (2021)

Abstract: Style transfer has been widely explored in natural language generation with non-parallel corpus by directly or indirectly extracting a notion of style from source and target domain corpus. A common shortcoming of existing approaches is the prerequisite of joint annotations across all the stylistic dimensions under consideration. Availability of such dataset across a combination of styles limits the extension of these setups to multiple style dimensions. While cascading single-dimensional models across multiple styles is a possibility, it suffers from content loss, especially when the style dimensions are not completely independent of each other. In our work, we relax this requirement of jointly annotated data across multiple styles by using independently acquired data across different style dimensions without any additional annotations. We initialize an encoder-decoder setup with transformer-based language model pre-trained on a generic corpus and enhance its re-writing capability to multiple target style dimensions by employing multiple style-aware language models as discriminators. Through quantitative and qualitative evaluation, we show the ability of our model to control styles across multiple style dimensions while preserving content of the input text. We compare it against baselines involving cascaded state-of-the-art uni-dimensional style transfer models.

## introduction style transfer is a popular task in natural language processing and has been studied on attributes like age or gender @xcite , styles emanating from social construct like formality @xcite and politeness @xcite , linguistic styles based on author writing style @xcite , or psycho-linguistic styles based on personality types @xcite . while early style transfer frameworks were modeled as a supervised learning task on a parallel corpus, state-of-the-art models are semi-supervised/unsupervised and operate on nonparallel corpus. these models achieve style transfer by aligning source and target distribution of sentences from non-parallel corpus @xcite , disentangling content space from style space in latent representation @xcite or employing self-reconstruction @xcite and back translation @xcite objectives to achieve pseudo-supervision with non-parallel corpus. recent works have also modeled this in a self-supervised manner where rewriting (transfer) is achieved by utilizing corpus from the target style alone @xcite . these wide studies have also led to the curation and benchmarking of non-parallel dataset for various style dimensions, such as sentiment @xcite , formality @xcite , politeness @xcite , excitement @xcite , etc. but availability of data with joint tagging across multiple styles is limited and has restricted the ability of existing approaches to scale from single-dimensional transfer to multiple style dimensions. in this paper, we propose a multidimensional style transfer approach that can work off partially labelled data for style transfer across multiple dimensions simultaneously. the work by @xcite attempts style transfer with multiple attributes such as age, gender, and sentiment simultaneously. however, their approach avails corpus tagged with each of these three style dimensions. in contrast to this and other similar explorations in multi-style transfer, our approach does not require jointly labelled data across all the stylistic dimensions in source and/or target corpus. we focus on the problem where independent corpus is available across different stylistic dimensions (say sentiment and formality) and we achieve style transfer spanning different stylistic dimensions (say make a sentence more positive and formal). while state-of-the-art approaches can be extended to achieve this by sequentially transferring one style after another, it is limited as different style dimensions are not necessarily independent of each other. in aspects that are not independent, changing one style aspect of the text might affect another aspect considered, making a sequential brute-force approach non-ideal. as we show in our experiments later, the cascaded setup also lacks common grounding between the content from different styles leading to erratic changes in content. we circumvent this by grounding our framework on the linguistic understanding of a large language model. our model builds understanding of interplay between the different styles by incorporating multiple discriminative language models (lm) with language model-based encoder-decoder setup. the key contributions of this paper are: 1) an encoder-decoder setup with multiple language models as discriminator, with each entity harnessing the language understanding from a large pre-trained transformer model. ## related work one line of work in style transfer attempts to learn disentangled latent representation for style and content, and transfer style by manipulating latent representation of style @xcite . although these approaches perform well with one style at a time, they do not trivially scale to multidimensional style transfer. several other works develop unsupervised approach for style transfer by employing denoising autoencoding (dae) @xcite and back-translation (bt) @xcite loss to develop interaction and hence transfer between the source and target domain. @xcite extend this approach to multiple styles by conditioning on average of embedding of each target attribute and using combination of dae and back-translation techniques. dae takes as input a sentence x from style s and tries to reconstruct sentence x from its corrupted version x. this relies on the assumption that the input sentence x is from a certain style combination @xmath0, s 2 , . . . , s k }. similarly back translation (bt) objective with input sentence x from style s, first estimates @xmath1, s ), where @xmath2, s). thus, these approaches are inherently dependent on knowledge of annotation of each sentence with all the style combinations. @xcite achieve state-ofthe-art style transfer in single style dimensions by employing transformer-based model in conjunction with classifier-based discriminator. in addition to discriminator losses, their proposed technique uses self-reconstruction and cycle reconstruction losses, which similar to dae and bt losses are also reliant on availability of jointly annotated data to be extendable to multiple style setup. language modeling is integral to several natural language generation (nlg) tasks like text summarization, spelling correction, image captioning, etc. the model architecture for these tasks has evolved from n-gram based methods to recurrent neural networks to transformer architectures. the introduction of transformer-based architecture accompanied with generative pre-training @xcite capabilities have led to strong improvements in many downstream generation and glue @xcite tasks. generative pre-training aims to adapt a large transformer language model to large unsupervised corpus. this capability of generative pre-training is exploited in many large language models like bert @xcite , gpt-2 @xcite , ernie 2.0 @xcite which have the ability to perform tasks like reading comprehension @xcite , summarization @xcite , question-answering @xcite and translation @xcite in zero-shot and few-shot settings. recently these pre-trained generative language models have been explored in translation (conneau and lample, 2019) and style transfer tasks @xcite . @xcite develop cross-lingual models for unsupervised machine translation by initializing encoder and decoder with a pre-trained language model trained on masked language modeling (mlm) @xcite objective and fine-tuning the encoderdecoder framework with adversarial training. @xcite extend this to stylized re-writing task by employing dae during fine-tuning. the joint encoder-decoder framework learns to reconstruct sentences in target-domain from its noisy version using dae objective. as previously discussed, the dae objective is reliant on the corpus being tagged for the target domain style (or combination of style) and restricts the generalization of this setup to multiple attributes. we overcome this by employing discriminative language models to assist the decoder with feedback for various target styles. @xcite show that even with nonparallel data, the content distribution across source and target style is shared. based on this, a language model trained on target style will have high perplexity on transferred text if it does not match target style and low perplexity otherwise. @xcite exploit this ability of language models to replace standard binary classifier-based discriminators with an implicitly trained language model as discriminator. they show that using the language model as structured discriminator allows for more stable training by eliminating the adversarial step. we extend this idea to a multi-discriminator approach. training a lm on combination of target styles is not possible in absence of jointly labelled dataset. due to this, we attempt to use multiple discriminators for each of the target styles. since with multiple styles, the underlying corpus is independently acquired, the variation in content distribution across different styles is more noticeable. consequently, an independently trained lm on one of the target styles might have high perplexity even if the transferred sentence fits in the corresponding target style, due to the content space of source sentence. to equip discriminative lm with more generalized notion of content, we use large transformer-based lm pre-trained on large unsupervised corpus to establish generic content distribution before style-oriented fine-tuning. ## approach our proposed approach has two key elementsa transformer-based encoder-decoder model initialized with a pre-trained transformer language model and fine-tuned on dae loss to achieve style transfer (section 3.1) and the multiple language models as discriminators stacked together to enable multi-style transfer (section 3.2). ## experiments we experiment with a combination of sentiment and formality styles. for sentiment, we use a mixture of imdb @xcite and yelp dataset @xcite with 300k examples in the positive and negative sentiment each. for formality, we use gyafc corpus @xcite which has 104k examples in each formal and informal class. the test set has 3000 and 4849 examples for sentiment and formality respectively, following the data split available in @xcite ; @xcite . for both datasets, the training corpus is non-parallel and the test corpus has human written references available, which we use for content evaluation (section 4.2). for pre-training, we use 12-layer transformer model with 512 hidden units, 16 heads, a dropout rate of 0.1 and learned positional embedding. we train our models with the adam optimizer, and ## conclusion and future work we propose an approach to extend currently existing style transfer work to multiple style setting without imposing any extra constraints on availability of dataset. our method makes use of disjoint corpus from separate styles to enable one step transfer across multiple target styles. we exploit multiple discriminative language models with an encoder-decoder framework, all emerging from large transformer-based language models pretrained on masked language modeling objective and fine-tuned separately for transfer and discriminative purposes. we show that unified single step transfer approach is able to achieve better transfer while offering much better content preservation which is paramount to any style transfer task. further improvements are in scope for adding modularity to the proposed transfer module. in the current setup, each version of model is trained for a specific combination of target style(s). the utility of such a model increases manifold with added ease of transfer across multiple style combinations within a single model. this could be attempted by employing a controlled language model as a unified discriminator for multiple styles, which would be the subject of further research.