Fine-grained Text Style Transfer with Diffusion-Based Language Models (2023)

Abstract: Diffusion probabilistic models have shown great success in generating high-quality images controllably, and researchers have tried to utilize this controllability into text generation domain. Previous works on diffusion-based language models have shown that they can be trained without external knowledge (such as pre-trained weights) and still achieve stable performance and controllability. In this paper, we trained a diffusion-based model on StylePTB dataset, the standard benchmark for fine-grained text style transfers. The tasks in StylePTB requires much more refined control over the output text compared to tasks evaluated in previous works, and our model was able to achieve state-of-the-art performance on StylePTB on both individual and compositional transfers. Moreover, our model, trained on limited data from StylePTB without external knowledge, outperforms previous works that utilized pretrained weights, embeddings, and external grammar parsers, and this may indicate that diffusion-based language models have great potential under low-resource settings.

## introduction diffusion probabilistic models @xcite have became the state-of-the-art technique in visual generative tasks. by starting from random gaussian noise and gradual denoising, they are able to generate images that look realistic in details. moreover, conditional diffusion models such as stable diffusion @xcite are able to achieve detailed control over the generated output by conditioning on text, layouts, etc. the generated images are faithful to the text description or layouts, often to the finest details. analogically, researchers have tried to utilize the controllability of diffusion models to achieve more controllable language generation. for ex-ample, diffuseq @xcite applies diffusion models to sequence-sequence text generation tasks such as paraphrasing, question generation and text simplification; diffusion-lm @xcite combined diffusion models with language models to control language generation by specifying generation length, syntax tree, semantic context, etc. what made these diffusion-based language models impressive is that they are trained from scratch with zero external knowledge (i.e. no pre-trained word embeddings or model weights, no external grammar parsers, etc) and on very few data (on the order of 10 5 tokens) compared to any large language models (for example, gpt-3's @xcite training data is on the order 10 11 tokens), so they have to learn representations at all levels (word embeddings, sentence structures, etc) from scratch with very limited data. however, while the earlier tasks assessed on diffusion-lm and diffuseq require a degree of control over the generated output, they are incapable of modifying the existing text to exhibit specific stylistic characteristics. in this paper, we would like to further examine the capabilities of diffusion-based language models on fine-grained text style transfer, an important task that requires more fine-grained control than the tasks from previous works on diffusion-based language modeling because it only allows changing the specified fine-grained stylistic properties of the input while leaving the rest unchanged. for example, "verb emphasis" is a fine-grained style transfer that requires the model to rewrite the sentence emphasizing a certain verb, without changing any other information that the original sentence conveys. in comparison, previous evaluation tasks such as controlling sequence length, semantic context, etc essentially control one aspect at a time and require no control over any other properties of generated text. we use 13 non-lexical transfers from styleptb @xcite ). we present one example sentence pair before/after each transfer, as well as the total number of sentence pairs available for each transfer in styleptb. as we can see, the transfers require changing one specific stylistic aspect of the sentence while leaving all other aspects unchanged, and the amount of data available for training is limited (compared to the typical amount of data required to train large language models nowadays). since identifying the grammatical structure of the sentence can be very helpful for most of these transfers (such as active-to-passive), some previous methods (such as neural qcfg @xcite ) utilizes external grammar parsers to gain such information. we trained a diffusion-based model on styleptb data without any pre-trained weights or external grammar parsers. therefore, our model has to start from zero grammar/linguistic knowledge and learn all of them from very limited training data (styleptb only has 7719 sentences from penn tree bank @xcite plus their transferred outputs). even under these hard conditions, our model still managed to outperform previous works that do utilize external weights or grammar parsers. moreover, we also evaluate the capabilities of diffusion-based language models on performing multiple transfers using one single model and composing multiple learned transfers on a single sentence. we list our contributions as follows: ## methodology we adapt diffuseq @xcite to be able to perform fine-grained text style transfer given a source sentence and specified transfer operation(s), as illustrated in figure 1 . we model the transfer as a conditional generation process, where the condition includes the source sentence and the specified transfer operation(s). we first define a set of special style tokens, one for each possible individual finegrained transfer. if we wish to perform one or more transfer on the source sentence, we will prepend the corresponding special token(s) to the beginning of the source sentence to form the condition s. we use bert tokenizer to tokenize the input into discrete token ids, and adopt a token embedding layer to encode both the source (including prepended style tokens) and the ground truth target sentence (during training) to obtain the embedded source z s and target z t rg 0 . for the diffusion pro- . cess, we use a transformer model to recover the target embedding. both the diffusion transformer and the token embeddings are initialized randomly and jointly optimized. in other words, our model does not rely on any prior knowledge about our task or the english language in general. we use the simplified diffusion objective during training: for each input (s, t rg) where s is the source sentence (with style tokens) and t rg is the ground truth target sentence, we randomly sample a step number t from 1, 2, ...t , where t is the maximum number of steps, and add t steps of random gaussian noise to z t rg 0 following a linear diffusion schedule to obtain z t rg t . we then concatenate z s and z t rg t and input the concatenated sequence into our diffusion transformer, where we only take the output embeddings at the locations corresponding to z t rg t as z ′t rg 0 . our training objective is simply going to be the mse loss between z t rg 0 and z ′t rg 0 . during inference, we randomly initialize z ′t rg t ∼ n (0, 1), and encode the condition (source sentence and style tokens) into z s . then we concatenate them and use our transformer to predict a temporary z ′t rg 0temp , add t -1 steps of noise back to the temporary z ′t rg 0temp to obtain z ′t rg t -1 . we repeat this process until we get z ′t rg 0 . for each embedding in z ′t rg 0 , we find the closest embedding in our token embedding layer by cosine distance, and decode the embedding to that token. then we combine the tokens to form the output sentence in natural language. styleptb was built with only 7719 different sentences from penn tree bank @xcite plus their stylistic variations, so both the amount and the diversity of training data are very limited, thus making this task even more challenging for diffuseq since it does not have access to external knowledge or pre-trained weights and have to extract all linguistic knowledge from limited data. for fair comparison, we preprocess the data following the same criterion as @xcite : we replace numbers with num token, and we replace each word that occurs less than 3 times in the training set with unk token. we also split the data into train/valid/test splits with proportions of 0.9/0.05/0.05 using the same splits as all previous works. ## limitations and future works one significant limitation of our work is that we only explored the capabilities of diffusion-based language models under a challenging circumstance where it is not allowed to use pre-trained weights or grammar parsers, which means we did not utilize this kind of model to its full potential, so a future research direction could be exploring possible ways to further improve the model's performance by leveraging pretrained weights or word embeddings, and train with enough data to find the full potential of these models. another limitation of our work is that we only explored one typical diffusion-based language model, so our conclusions may not generalize to special types of diffusion-based language models (such as ones that uses discrete state space). we also conducted all experiments using the exact same model architecture design. in the future, we plan to experiment with different architectures for the diffusion model, such as more sophisticated conditioning methods (currently we just concatenate the source to the target, but we would like to try other ways of conditioning on the source, such as cross attention, as these conditioning methods for diffusion models have promising performance in the image generation domain). lastly, we found that diffusion-based language models work well with limited data and no external knowledge or pre-trained weights, thus these mod-els may have great potential under low-resource settings, but we didn't apply them to any real lowresource settings (such as low-resource languages, rare domains, etc) in this paper, and we would like to do that in the future to explore the full potential of diffusion-based language models. ## conclusions in this paper, we explored the capabilities of diffusion-based models on fine-grained text style transfer, a task that requires a high level of control over generated text, with no external knowledge or pre-trained weights and with very limited training data. our diffusion-based language model, which builds upon diffuseq @xcite , achieves state-of-the-art performance on all transfers as well as composition of transfers, outperforming all previous works on this dataset, including ones that uses pre-trained weights, word embeddings, and external grammar parsers. it is even on par with human performance on many transfers. therefore, our model is a great step towards solving automated fine-grained text style transfer. moreover, our work, together with previous works such as diffusion-lm @xcite , demonstrates that diffusion-based language models could have great potential in controllable text generation under low-resource settings. under lowresource settings (such as rarely spoken language or uncommon tasks), it would be difficult to find existing large language models or pre-trained weights, and available training data will likely be very limited, so most approaches based on finetuning existing models or large amounts of training will not work well, and diffusion-based language models could be an alternative to consider.