DGST : a Dual-Generator Network for Text Style Transfer (2020)

Abstract: We propose DGST, a novel and simple Dual-Generator network architecture for text Style Transfer. Our model employs two generators only, and does not rely on any discriminators or parallel corpus for training. Both quantitative and qualitative experiments on the Yelp and IMDb datasets show that our model gives competitive performance compared to several strong baselines with more complicated architecture designs.

## introduction attribute style transfer is a task which seeks to change a stylistic attribute of text, while preserving its attribute-independent information. sentiment transfer is a typical example of such kind, which focuses on controlling the sentiment polarity of the input text @xcite . given a review "the service was very poor", a successful sentiment transferrer should covert the negative sentiment of the input to positive (e.g., replacing the phrase "very poor" with "pretty good"), while keeping all other information unchanged (e.g., the aspect "service" should not being changed to "food"). without supervised signals from parallel data, a transferrer must be supervised in a way to ensure that the generated texts belongs to a certain style category (i.e., transfer intensity). there is a growing body of studies to intensify the target style by means of adversarial training @xcite , variational autoencoder @xcite @xcite , generative adversarial nets @xcite @xcite , or subspace matrix projection @xcite furthermore, in order to boost the preservation of non-attribute information during style transformation, some works explicitly focus on modify- * corresponding author ing sentiment words, which is so-called the "pivot word" @xcite . there are also works which add extra components for constraining the content from being changed too much. these include models like autoencoder @xcite , part-of-speech preservation, and the content conditional language model @xcite . in order to achieve highquality style transfer, existing works normally resort to adding additional inner or outer structures such as additional adversarial networks or data preprocessing steps (e.g. generating pseudo-parallel corpora). this inevitably increases the complexity of the model and raises the bar of training data requirement. in this paper, we propose a novel and simple model architecture for text style transfer, which employs two generators only. in contrast to some of the dominant approaches to style transfer such as cyclegan @xcite , our model does not employ any discriminators and yet can be trained without requiring any parallel corpus. we achieve this by developing a novel sentence noisification approach called neighbourhood sampling, which can introduce noise to each input sentence dynamically. the nosified sentences are then used to train our style transferrers in the way similar to the training of denoising autoencoders @xcite . both quantitative and qualitative evaluation on the yelp and imdb benchmark datasets show that dgst gives competitive performance compared to several strong baselines which have more complicated model design. the code of dgst is available at: https://xiao.ac/proj/dgst . ## methodology suppose we have two non-parallel corpora x and y with style s x and s y , the goal is training two transferrers, each of which can (i) transfer a sentence from one style (either s x and s y ) to another (i.e., transfer intensity); and (ii) preserve the styleindependent context during the transformation (i.e., preservation). specifically, we denote the two transferrers f and g. f : @xmath0. likewise, g : @xmath1. to obtain good style transfer performance, f and g need to achieve both a high transfer intensity and a high preservation, which can be formulated as follows: here d(x||y) is a function that measures the abstract distance between sentences in terms of the minimum edit distance, where the editing operations φ includes word-level replacement, insertion, and deletion (i.e., the hamming distance or the levenshtein distance). on the one hand, eq. 1 requires the transferred text should fall within the target style spaces (i.e., x or y). on the other hand, eq. 2 constrains the transferred text from changing too much, i.e., to preserve the style-independent information. inspired by cyclegan @xcite , our model (sketched in figure 1 ) is trained by a cyclic process: for each transferrer, a text is transferred to the target style, and then back-transferred to the source style using another transferrer. in order to transfer a sentence to a target style while preserving the style-independent information, we formulate two sets of training objectives: one set ensures that the generated sentences is preserved as much as possible (detailed in §2.1) and the other set is responsible for transferring the input text to the target style (detailed in §2.2). ## conclusion in this paper, we propose a novel and simple dualgenerator network architecture for text style transfer, which does not rely on any discriminators or parallel corpus for training. extensive experiments on two public datasets show that our model yields competitive performance compared to several strong baselines, despite of our simpler model architecture design.