Exploring Controllable Text Generation Techniques (2020)

Abstract: Neural controllable text generation is an important area gaining attention due to its plethora of applications. Although there is a large body of prior work in controllable text generation, there is no unifying theme. In this work, we provide a new schema of the pipeline of the generation process by classifying it into five modules. The control of attributes in the generation process requires modification of these modules. We present an overview of different techniques used to perform the modulation of these modules. We also provide an analysis on the advantages and disadvantages of these techniques. We further pave ways to develop new architectures based on the combination of the modules described in this paper.

## introduction controllable text generation is the task of generating natural sentences whose attributes can be controlled. the attributes to control can range from being stylistic such politeness, sentiment, formality, etc.; demographic attributes of the person writing the text such as gender, age, etc.; content such as information, keywords, entities, etc.; ordering of information, events, like plot summaries etc. controlling various attributes of text generation has manifold applications. for instance in dialogue response generation task, work has been done in controlling persona @xcite , controlling various aspects of the response such as politeness @xcite , formality, authority etc, grounding the responses in external source of information @xcite @xcite , and controlling topic sequence @xcite . another application is story generation where you can control the ending @xcite , the persona @xcite , the plot @xcite , and the topic sequence @xcite . controllable text generation is also used to modulate the formality and politeness of emails @xcite . report generation can be controlled by pulling disparate source documents into a coherent unified whole, which can use a shared set of sources such as wikipedia article generation @xcite . although there is a large body of prior work in controllable text generation, there is no unifying theme. each work addresses a specific task in a specific context. in this paper we outline a new schema which connects prior work and provides an insight into various aspects of controllable text generation. the schema contains five modules that cover the overall generation pipeline and provide an understanding of the effect of each component on the generation process. prior work has focused on specific parts of the schema that we outline here and we provide insights into their similarities. we provide an overview of these modules and also present an exploration of the various techniques used to control and update each of these modules. most of the controllable text generation tasks can be framed as conditional language generation tasks. they have an input or a source sequence u and an output or a target sequence y to be generated. in this case, we model the probability of the target sequence conditioned on the source sequence given by p (y|u) = t @xmath0, @xmath1. the generation of the target tokens of the sequence y unfolds as a time series where each token y t is generated at a time step t. at a given time step t, a generative model takes in the previous hidden state h t-1 and the input x t at current time step. it performs a set of operations denoted by g to produce the output o t which is used to predict token xt . the ground truth token to be generated is denoted by y t . figure 1 this schema provides an insight into the contributions of the various modules for controllable text generation. the main advantage of this schema is that it can be used with any algorithmic paradigm like sequence-to-sequence, adversarial methods, reinforcement learning, etc. the schema can also be used with non-autoregressive algorithms which may generate text using graphical structures like trees @xcite . in this paper, we focus on how this schema can be used to describe controllable text generation focusing particularly on the use of autoregressive models. this work paves way to designing new architectures based on our schema. this can be done by identifying promising techniques for each module and then combining them. our schema can also be potentially used for applying these techniques on new tasks of similar nature. it also provides a standardized framework to position and compare new architectures with existing techniques. the prior work on unifying text generation models has mostly focused on building efficient tool-kits and modular views of generation. for instance, @xcite details seven sub-tasks which are conceptually distinct to describe the generation process. these sub-tasks can be modelled separately or in some cases they may interleave. in @xcite , these seven sub-tasks are primarily characterized as content or structure tasks. note that @xcite is not specific to neural text generation. our work focuses specifically on controlling attributes in neural text generation process. we don't divide the generation pipeline into several sub-tasks but we divide the neural text generation process into modules all of which are required for generation. in @xcite , the focus is on building a toolkit for various text generation tasks based on the three properties of versatility, modularity and extensibility. this work enlists few model architectures and learning paradigms for various text generation tasks. in our work, we focus only on the generation process of controllable text generation tasks. we specifically detail the inputs, outputs and operations of the generation process. we do not provide any specific examples of architectures but provide an overview of the basic underlying modules which can be used with any learning paradigm. @xcite provides a practical guide to the neural generation process describing it in terms of initialization, optimization, regularization and decoding strategies. our work on the other hand does not delve into the implementation details of the generation pipeline but provides an overall schema for understanding of the various components involved. in the remainder of the paper, we denote the representation of the control attribute by s and the representation of the input or source sentence returned by the encoder as h e . in what follows, we first describe the possible ways of controlling attributes by modulating the external input in §2, the sequential input in §3, the generator operations in §4, the output in §5 and the training objective in §6. at the end of each section, we provide an analysis of each of the techniques described and how they fit together. ## external input in this section we discuss the different techniques which can be used to control the generation process by updating the initialization of the generator h 0 . in the standard generation process, h 0 is equal to h e . this is marked as module (1) in figure 1 . ## sequential input in this section we discuss the different techniques which can be used to manipulate the sequential input x t to the decoder at each time step. x t here is used to denote the word embedding of the token at time step t. this is marked as position (2) in figure 1 . ## generator operations this module takes in the external input h 0 , the sequential input x t at time step t and performs the same set of computations (g) to return an output o t . changes can be made to the set of operations g to include the the control vector s in computing o t . this is shown as position (3) in figure 1 . ## output in the standard generation process, o t is the output of the generator module which is projected to the vocabulary space to predict the token xt . here, we discuss the various techniques used to modulate the sequential output o t at each time step t, before projecting it to the vocabulary space. this is marked as position (4) in figure 1 . ## training objective in this section we describe various methods used to control the generation using objective functions. the output o t at each time step t of the generation process is projected to the vocabulary space using a linear transform (ô @xmath2. a token xt is predicted from the vocabulary by passing ôt through a softmax function and taking the max value. the predicted token xt is compared with the reference token y t using a loss function. this loss function can be tweaked to ensure that the generated text carries the desired control attributes. ## discussion discrete space issues: the classifier loss ( §6.3) is used to determine if the generated tokens x are in accordance with the target control attribute s. to calculate the loss, the generated tokens x are provided as input to the classifier. if the tokens in this case are generated using the argmax then this function is not differentiable. hence, passing tokens effectively to the classifier is a challenge. in @xcite , the reinforce (williams, 1992) algorithm is used and rewards are calculated using monte carlo search sampling for the next tokens. this technique is known to be unstable due to the high variance of the sampled gradient during training @xcite . kusner and hernández-lobato (2016) introduce the gumbel-softmax distribution as a solution. it approximates the multinomial distribution parameterized in terms of the softmax distribution. here the predicted token is: combined module architecture: it is also possible to combine techniques from multiple modules to control the generation process. we mention some of the prior works that have successfully combined various modules here. @xcite combine stochastic changes ( §2.2), kl divergence loss ( §6.2) and a classifier loss ( §6.3). it adopts a variational auto-encoder along with kl divergence loss objective and further adds a discriminator loss which signifies if the generated sentence belong to the target attribute. as mentioned earlier, @xcite combine the decomposition of the external input ( §2.3) with external feedback provided to the external input ( §2.4). external feedback is used to ensure that the decomposed latent sub-spaces represent the desired target attributes. @xcite establishes formal connections between generative adversarial networks (related to §5.2 and §6.3) and variational auto-encoders (related to §2.2 and §6.2). determining the best possible combination of modules through empirical evaluation remains an open challenge. ## conclusion and future work in this paper we propose a new schema to organize the prior work in controllable text generation. the schema contains five modules, each of which plays an important role in the generation process. we detail the various techniques used to modulate each of the five modules to perform controllable text generation. we also provide theoretical understanding and qualitative analysis of these techniques. this understanding paves way to new architectures based on combinations of these modules. the future work will focus on empirical comparison of these techniques to gain an insight into their usefulness and strength.