Rethinking Style Transformer with Energy-based Interpretation: Adversarial Unsupervised Style Transfer using a Pretrained Model (2022)

Abstract: Style control, content preservation, and fluency determine the quality of text style transfer models. To train on a nonparallel corpus, several existing approaches aim to deceive the style discriminator with an adversarial loss. However, adversarial training significantly degrades fluency compared to the other two metrics. In this work, we explain this phenomenon using energy-based interpretation, and leverage a pretrained language model to improve fluency. Specifically, we propose a novel approach which applies the pretrained language model to the text style transfer framework by restructuring the discriminator and the model itself, allowing the generator and the discriminator to also take advantage of the power of the pretrained model. We evaluated our model on three public benchmarks GYAFC, Amazon, and Yelp and achieved state-of-the-art performance on the overall metrics.

## introduction text style transfer is the task of converting a sentence from one style to another while preserving style-agnostic semantics. in solving the text style transfer task, three criteria must be considered: 1) style control, how well a style is transferred from the original sentence to the generated one, 2) content preservation, how well the generated sentence has retained the semantics of the original, and 3) fluency, how natural the generated sentence is. text style transfer is challenging, since fluently converting the style of a sentence often conflicts with content preservation @xcite @xcite . to address this challenge, several supervised text style transfer methods have been attempted @xcite @xcite . but style-labeled sentence pairs are often not available, making that approach less practical in a realworld setting. various unsupervised text style deceiving energy-based discriminator d requires 1) minimizing the energy e between the transferred sentence x ′ and the target style s ′ , and 2) maximizing the energy between the sentence x ′ and the original style s. however, the style s and s ′ are originated from the same corpus, so maximizing e( x ′ , s) degrades the overall fluency. it is an interpretation of eq. 6. in this paper, we review style transformer to investigate the reason behind the fluency degradation in adversarial models. to more precisely interpret what fluency is, we introduce the notion of energy @xcite , which is the entropy of variables. the energy function, which measures the energy of input variables with respect to a particular style, outputs low energy if the inputs are common in that style and outputs high energy otherwise. for example, formal/informal sentences would likely have low energy in the formality corpus, while sentences that a structural dilemma when applying pretrained models to adversarial learning. to propagate gradients to generated tokens, the generator, discriminator, and lm should have the same vocabulary and tokenizer. however, publicly available pretrained models (e.g., bert and gpt) use their own tokenizers, so the discriminator and lm need to be trained from scratch if we apply the pretrained model to the generator. to apply the lm-based regularizers, we can leverage pretrained models such as gpt-2 to generate fluent sentences. moreover, fluency is expected to further improve when the generator and the discriminator are also replaced with a pretrained model. however, as shown in figure 2 , the generator, discriminator, and lm must share the same vocabulary and tokenizer in order to propagate gradients successfully. thus, inefficiency can arise, in that two of the three modules may need to be re-trained from scratch because the existing pretrained models are based on different tokenizers. to address this issue, we restructured the discriminator and lm such that a single pretrained model is applied to all three modules: the generator, discriminator, and lm. by using a single pretrained model, our method not only solves the inconsistent vocabulary problem but also has advantages when applying further pretraining for domain adaptation @xcite or additional dataset @xcite . this is because additional pretraining is necessary when only a single model is used to improve style transfer performance. our contributions can be summarized as follows: ## method in an unsupervised setting, we assume the nonparallel corpus @xmath0, x (1) , • • • , x (m) } and x ′ = {x ′(0) , x ′(1) , • • • , x ′(n) }, and denote each style of the corpus as s and s ′ . the objective is to train a style transfer model g in an unsupervised way such that a sentence x is turned into a sentence x ′ having similar content but the other style. ## limitations as our model uses adversarial learning, training is somewhat unstable, like gan @xcite . for this reason, continuing training does not guarantee good results, and the text style transfer performance fluctuates. although we have paid attention to model selection to compensate for the unstable training, the instability of adversarial learning remains an issue. in addition, we have only conducted experiments on widely used datasets, to compare our work with previous studies. these datasets are composed of binary style classes, such as positive and negative sentiments. therefore, conducting experiments using multi-class datasets @xcite should be considered. ## ethical consideration our model may generate negative and rude expressions about a specific person or a commercial site because of the data distribution of the yelp, amazon, and gyafc datasets. however, we propose our work in anticipation of positive applicability as shown in previous studies. ## conclusion using energy-based interpretation, we found that fluency is inevitably degraded when deceiving the discriminator in style transformer @xcite . the problem is solved by adding an lmbased regularizer and training the pretrained generator, discriminator and lm together. our model shows comparable performance in text style transfer and content preservation while preserving fluency, and we demonstrated the robustness of our model by conducting extensive experiments on various styles in raw text. training details for model selection, we record the model checkpoint per 500 steps, and the model with the highest j(a, s) in a single run is selected as our final model. our model takes about 14 hours on a single nvidia rtx a6000 machine to train the gyafc dataset. we use the adamw optimizer (loshchilov and hutter, 2019) with a batch size of 64. the initial learning rate of the un-pretrained layers, such as the last linear layer of the discriminator, are set to 2e -4 , and all the others are set to 3e -5 . in addition, a cross entropy of the last linear layer in the discriminator is labelsmoothed @xcite with α = 0.1.