Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization (2020)

Abstract: Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve performance of many downstream NLP tasks. In this work, we propose a semi-supervised formality style transfer model that utilizes a language model-based discriminator to maximize the likelihood of the output sentence being formal, which allows us to use maximization of token-level conditional probabilities for training. We further propose to maximize mutual information between source and target styles as our training objective instead of maximizing the regular likelihood that often leads to repetitive and trivial generated responses. Experiments showed that our model outperformed previous state-of-the-art baselines significantly in terms of both automated metrics and human judgement. We further generalized our model to unsupervised text style transfer task, and achieved significant improvements on two benchmark sentiment style transfer datasets.

## introduction text style transfer is the task of changing the style of a sentence while preserving the content. it has many useful applications, such as changing emotion of a sentence, removing biases in natural language, and increasing politeness in text @xcite @xcite @xcite . there is a wide availability of "informal" data from online sources, yet current natural language processing (nlp) tasks and models could not leverage or achieve good performance for such data due to informal expressions, and grammatical, spelling and semantic errors. hence, formality style transfer, a specific style transfer task that aims to preserve the content of an informal sentence while making it semantically and grammatically correct, has informal i flippin' love that movie, sweeeet! formal i truly enjoy that movie. informal we was hanging out a little. formal we were spending a small amount of time together. the most widely-used models for formality style transfer are based on a variational auto-encoder architecture, trained on parallel text data of (informal, formal) style sentence pairs with same content @xcite . however, there is still a lot of inconsistencies between human-generated sentences and outputs of current models, largely due to the limited availability of parallel data. in contrast, large amount of data consisting of sentences with just either informal or formal labels is relatively easier to collect. to tackle the training data bottleneck, we propose a semi-supervised approach for formality style transfer, using both human-annotated parallel data and large amount of unlabeled data. following the success of generative adversarial nets (gan) @xcite , binary classifiers are often used on the generator outputs in unsupervised text style transfer to ensure that transferred sentences are similar to sentences in the target domain @xcite . however, @xcite showed that using a language model instead of a binary classifier can provide stronger, more stable training loss to the model, as it leverages probability of belonging to the target domain for each token in the sentence. we extend this line of work to semi-supervised formality style transfer, and propose to use two language models (one for source style and another for target) to help the model utilize information from both styles for training. moreover, style transfer models are usually trained by maximizing p (y|x), where (x, y) is a (informal, formal) sentence pair. such models tend to generate trivial outputs, often involving highfrequency phrases in the target domain @xcite . building on prior work, to introduce more diversity and connections between the input and output, we propose to maximize mutual information (mmi) between source and target styles, which take into account not only the dependency of output on input, but also the likelihood that the input corresponds to the output. while this has only been done at test-time so far, we extend this approach to train our model with mmi objective. we evaluate our proposed models that incorporate both the language model discriminators and mutual information maximization on grammarly yahoo answers corpus (gyafc) dataset @xcite . experiments showed that our simple semi-supervised formality style transfer model outperformed state-ofthe-art methods significantly, in terms of both automatic metrics (bleu) and human evaluation. we further show that our approach can be used for unsupervised style transfer, as demonstrated by significant improvements over baselines on two sentiment style benchmarks: yelp and amazon sentiment transfer corpus, where parallel data is not available. we have publicly released our code at https://github.com/ gt-salt/formalitystyletransfer . pre-training of the encoders on multiple tasks and datasets has been shown to be effective @xcite in improving performances of individual tasks. these models are often trained with the cross-entropy loss @xcite on the output tokens, or in other words, maximising p (y|x) where (x, y) is a pair of source and target style sentence respectively. li et al. (2016b) showed that maximising mutual information (mmi) m (x, y) during test-time between the source and target instead can lead to more diverse and appropriate outputs in seq2seq models. some other works (zhang et al., 2018) maximize a variational lower bound on pairwise mutual information. we use a denoising auto-encoder bart @xcite trained with mmi objective. semi-supervised and unsupervised style transfer some approaches like @xcite and @xcite focus on deleting style-related keywords to make content style-independent. however, other works hypothesize that content and style cannot be separated, and use techniques such as back-translation @xcite , cross-projection between styles in latent space @xcite , reinforcement learning-based one step model @xcite , and iterative matching and translation @xcite . following @xcite , using a generator along with a style classifier is often used for unsupervised tasks @xcite @xcite . however, recent work suggests @xcite that using language models instead of cnn discriminators can result in more fluent, meaningful outputs. maximizing likelihood of reconstruction of the input from the generated output has been used in both image generation @xcite ) and text style transfer @xcite @xcite to improve performance. motivated by these work, we use language models for our discriminator, and maximize cyclic reconstruction likelihood as part of our training objective. ## method this section presents our semi-supervised formality style transfer model. we detail the task and our base architecture in section 3.1. we add a language model-based discriminator to the model, described in section 3.2, and explain the maximization of mutual information in section 3.3. the final architecture for our model is summarized in section 3.4 and shown in figure 1 3.1 formality style transfer define t (="formal" in our case) as the target style and s (="informal") as the source style for the formality style transfer task. let d be the parallel dataset containing (source, target) style sentence pairs and u be the additional unlabeled data, denoted by u s for sentences with source style and u t for sentences with target style. our base model is a variational auto-encoder mechanism g that generates sentences of target style. the goal is to maximize p (y|x; θ g ) where θ g are the parameters of the model. this is done by cross-entropy loss over the target sentence tokens and generated output probabilities. to leverage maximum mutual information objective, as described in section 3.3, we make the model bidirectional. it can be used to transfer source style to target style as well as target style to source style. hence, an additional input @xmath0, t } is passed to g specifying the style to which the sentence is to be converted. hence, our objective for base model is to maximize p (y|x, t ; θ g ). ## model analysis and discussion although our model performed well on formality style transfer, there is still a gap compared to human performance. to understand why the task is challenging and how future research could advance this direction, we take a closer look at formality dataset, model generation errors, and certain challenges that existing approaches struggle with. ## conclusion this work introduces a semi-supervised formality style transfer model that utilizes both a language model based discriminator to maximize the likelihood of the output sentences being formal, and a mutual information maximization loss during training. experiments conducted on a large-scale formality corpus showed that our simple method significantly outperformed previous approaches in terms of both automatic metrics and human judgement. we also demonstrated that our model can be generalized well to unsupervised style transfer tasks. we also discussed specific challenges that current approaches faced with this task.