Controllable Text Generation via Probability Density Estimation in the Latent Space (2023)

Abstract: Previous work on controllable text generation has explored the idea of control from the latent space, such as optimizing a representation with attribute-specific classifiers or sampling one from relevant discrete samples. However, they cannot effectively model a complex space with diverse attributes, high dimensionality, and asymmetric structure, leaving subsequent controls unsatisfying. In this work, we propose a novel control framework using probability density estimation in the latent space. Our method utilizes an invertible transformation function, the Normalizing Flow, that maps the complex distributions in the latent space to simple Gaussian distributions in the prior space. Thus, we can perform sophisticated and flexible controls in the prior space and feed the control effects back into the latent space owing to the bijection property of invertible transformations. Experiments on single-attribute and multi-attribute control reveal that our method outperforms several strong baselines on attribute relevance and text quality, achieving a new SOTA. Further analysis of control strength adjustment demonstrates the flexibility of our control strategy.

## introduction controllable text generation, a fundamental issue in natural language generation, refers to generating fluent and attractive sentences conditioned on target attributes @xcite . with the development of pre-trained language models @xcite , early work explores converting generative language models to conditional models by altering their parameters via fine-tuning @xcite or reinforcement learning @xcite . due to the high cost of modifying parameters @xcite , current control approaches prefer leaving pre-trained language models fixed @xcite . blue and red represent two attributes. prefix-tuning represents attributes with points in latent space and composes them by interpolation. latentops uses classifiers to estimate continuous distributions of attributes and control by optimizing in latent space. discrete maps sentences to discrete samples in latent space and controls with direct searching. our method deploys probability density estimation by transforming the complex latent space into a well-formed prior space, where common control strategies can be more effective. see §a for more details of the latent space's defects. recent studies perform impressive control by influencing the fixed language model from the latent space @xcite with prefix-tuning @xcite . however, inefficient and unreliable modeling of the complex latent space remains a problem that plagues control performance. as shown in the left part of figure 1 , @xcite reveal that distributions of attributes in high dimensional latent space are usually asymmetric and even non-convex, making simple control strategies inefficient, including interpolation methods like prefix-tuning @xcite and optimization approaches like latentops @xcite . for example, interpolation may exceed the support set of distributions, making generated sentences unable to acquire desired attributes. besides, the optimization process can stuck in the saddle or local optimal points. although mitigating the problem by discrete modeling and direct searching, discrete @xcite introduces a more complicated control process, where searching for the intersection of attributes is vulnerable to the high-dimensionality of space and noise in samples. in this paper, we alleviate problems above by better modeling the latent space. as shown in the right of figure 1 , we propose probability density estimation in latent space by invertible transformation, where complex distributions of attributes in latent space are mapped (bijection between continuous spaces) to simple ones, such as gaussian distributions, in prior space. thus, traditional control strategies such as interpolation can be tractable and explainable in this normalized prior space. in the inference stage, our paradigm becomes: control attributes in prior space, then activate the language model in latent space. furthermore, we explore the relationship between the latent space and our prior space and attempt to prove under what circumstances the control in the prior space can be effectively fed back into the latent space. we conduct experiments on single-attribute control and multi-attribute control. datasets we use are imdb movie reviews @xcite for sentiment, agnews @xcite for topic, and jigsaw toxic comment classification challenge dataset for detoxification. we measure the control ability of our method using the correlation of generated sentences with each attribute. for text generation quality, we evaluate sentences with perplexity and distinctness concerning fluency and diversity. results show that our method can significantly outperform baseline models and analytical experiments on control strength adjustment reveal our flexibility. the main contributions of our work are summarized as follows: ## methodology as illustrated in figure 2 , our framework is divided into three parts, where the former two are training phases, and the latter is the generation phase. ## conclusion in this work, we present a novel control framework by introducing a well-formed prior space converted from latent space via invertible transformation. we further provide some theoretical support to ensure that controls in the prior space can be fed back into the latent space. this allows our framework the potential to generalize to similar situations bothered by high-dimensional and complex latent spaces. experimental results confirm the superiority of our model on control effectiveness, control flexibility, and generation quality.