VAE based Text Style Transfer with Pivot Words Enhancement Learning (2021)

Abstract: Text Style Transfer (TST) aims to alter the underlying style of the source text to another specific style while keeping the same content. Due to the scarcity of high-quality parallel training data, unsupervised learning has become a trending direction for TST tasks. In this paper, we propose a novel VAE based Text Style Transfer with pivOt Words Enhancement leaRning (VT-STOWER) method which utilizes Variational AutoEncoder (VAE) and external style embeddings to learn semantics and style distribution jointly. Additionally, we introduce pivot words learning, which is applied to learn decisive words for a specific style and thereby further improve the overall performance of the style transfer. The proposed VT-STOWER can be scaled to different TST scenarios given very limited and non-parallel training data with a novel and flexible style strength control mechanism. Experiments demonstrate that the VT-STOWER outperforms the state-of-the-art on sentiment, formality, and code-switching TST tasks.

## proposed method the training of vt-stower consists of two stages. the training stage i is a vae reconstruction task in which the input text x will be reconstructed together with external style embeddings. the latent space of content distribution is learned by vae, and the original and target style mapping will be learned and saved in style embeddings. the trained vae and style embeddings will also be utilized in the second training stage. to make the style transfer focus on pivot words (e.g., emotional words in sentiment tst) while maintaining other words unchanged (so that the fluency and semantics can be largely preserved), we fine-tune the vae with pivot word masking in training stage ii. the masking is based on the probability distribution of pivot words for specific styles, which is learned from a style classification task. in the inference stage, vt-stower uses the learned external target style embeddings to adjust the sampled latent vector of the original input to the target style. the adjusted sentence vector will then be input to the decoder to generate the target style text. ## conclusion we proposed the vt-stower, a model joinly trained with vae and style embeddings for content distribution and style information. the method successfully transfers several different text styles, including the code-switching tst task for the first time. taking advantage of the flexibility of style embeddings, our proposed model has the ability to adjust the style strength during the transfer by simply adjusting the style weights. to further enhance the transfer accuracy, we propose additional pivot words masking training scheme, which shows impressive improvement.