Text Style Transfer: Leveraging a Style Classifier on Entangled Latent Representations (2021)

Abstract: Learning a good latent representation is essential for text style transfer, which generates a new sentence by changing the attributes of a given sentence while preserving its content. Most previous works adopt disentangled latent representation learning to realize style transfer. We propose a novel text style transfer algorithm with entangled latent representation, and introduce a style classifier that can regulate the latent structure and transfer style. Moreover, our algorithm for style transfer applies to both single-attribute and multi-attribute transfer. Extensive experimental results show that our method generally outperforms state-of-the-art approaches.

## introduction text generation, which leverages knowledge in computational linguistics and artificial intelligence for automatically generating natural language texts, is the core problem for a number of natural language processing (nlp) applications such as speech to text, conversational/dialogue system @xcite , and text summarization @xcite . text style transfer can be thought of as a controllable text generation task, which aims to restyle a given sentence by changing specific attributes (sentiment, tense, formality, or politeness) while preserving the remaining attributes and the content. successful applications of text style transfer include paraphrasing @xcite , formality transfer @xcite , and text simplification @xcite . a good latent representation is essential to the performance of text style transfer. regarding the structure of the latent representation, the current work for text style transfer can be generally categorized into the disentangled representation and the entangled representation. in particular, the former method aims to learn disentangled latent representations by separating the style information from the content, while the latter method learns latent representations that entangle the style with the content. disentangled representations are often interpretable and consequently most of the current work adopts this method @xcite @xcite @xcite . however, learning disentangled representations is often challenging; and multiple attributespecific decoders are commonly required for text generation, which is undesirable especially when transferring multiple attributes. the entangled representations, on the other hand, has been shown to achieve promising performance on the content preservation and to produce fluent sentences with a much less complicated architecture @xcite @xcite . although existing models achieve adequate performance on text style transfer, most of them are designed specifically for style transfer @xcite @xcite @xcite @xcite , and meanwhile lack of explicit modeling of the latent space. we argue that the quality of latent representations is crucial for text generation. in this study, we focus on building a generative model that supports both text style transfer and text generation with regularized entangled latent representations. our contributions can be summarized as follows: (1) we extend the framework of adversarial autoencoder by including a classifier for both the regularization of the latent space and text style transfer. we show that the classifier can divide sentences with different attributes into different regions in the latent space and thus greatly improve the performance of style transfer. (2) we provide algorithms for both single-attribute and multi-attribute style transfer. we empirically compare with sev-eral state-of-the-art baselines and show that our proposed method achieves promising results. ## related work in this section, we first introduce several generative adversarial network (gan)-regularized autoencoders, which have been successfully used for text manipulation. then we review some recent methods for text style transfer focusing on the latent representation. ## our method we first briefly explain text style transfer as follows. generally, a source or an input sentence includes both the content and the attribute. in figure 1 , we use (x, y) to include both the input x and the attribute y. as a concrete example, a sentence "this place is a great place to live !" has two types of attribute: the positive sentiment and the present tense. the positive sentiment is reflected by the word "great" and the present tense is reflected by the word "is". all remaining words in this sentence are considered as the content. for single-attribute style transfer, only one attribute (e.g. sentiment) will be transferred, and the other attributes and the content will be kept the same. in the above example with the sentiment style transfer, the sentence will be converted into a negative sentence: "this place is a terrible place to live !" by flipping the sentiment label y from positive to negative. in contrast, for multi-attribute style transfer, two or more attributes will be transferred simultaneously while the rest will be preserved. again, in the above example the sentence will be converted into "this place was a terrible place to live !" by transferring both the sentiment and the tense. ## experiments in this section, we first visualize the latent representation of our method, and then compare our method with several baselines, namely, tae @xcite , arae @xcite , and daae @xcite for single-attribute and multipleattribute text style transfer. we then evaluate our model on text generation and compare it with arae. ## conclusion and future work in this paper, we proposed a new approach for text style transfer with entangled latent representations. we added a classifier to regularize the distribution of latent sentences in a probabilistic autoencoder. extensive experiments show that this regularized latent structure significantly improves the downstream text manipulation tasks. compared with benchmarks our method achieves impressive results on both single-attribute and multi-attribute text style transfer. moreover, both approaches of fast gradient and vector arithmetic style transfer outperform baselines on style transfer tasks. in addition, we demonstrated that the classifier regularization also improves other style transfer models. in the future, we would like to explore other methods to regularize latent representation in controllable text generation. moreover, text generation models have a wide range of applications in nlp tasks. besides style transfer, we will apply our model to other tasks such as text simplification and examine the latent structure in these applications.