Style Control for Schema-Guided Natural Language Generation (2021)

Abstract: Natural Language Generation (NLG) for task-oriented dialogue systems focuses on communicating specific content accurately, fluently, and coherently. While these attributes are crucial for a successful dialogue, it is also desirable to simultaneously accomplish specific stylistic goals, such as response length, point-of-view, descriptiveness, sentiment, formality, and empathy. In this work, we focus on stylistic control and evaluation for schema-guided NLG, with joint goals of achieving both semantic and stylistic control. We experiment in detail with various controlled generation methods for large pretrained language models: specifically, conditional training, guided fine-tuning, and guided decoding. We discuss their advantages and limitations, and evaluate them with a broad range of automatic and human evaluation metrics. Our results show that while high style accuracy and semantic correctness are easier to achieve for more lexically-defined styles with conditional training, stylistic control is also achievable for more semantically complex styles using discriminator-based guided decoding methods. The results also suggest that methods that are more scalable (with less hyper-parameters tuning) and that disentangle context generation and stylistic variations are more effective at achieving semantic correctness and style accuracy.

## introduction natural language generation (nlg) for taskoriented dialogue focuses on effectively generating responses based on inputs that are frequently in the form of a structured meaning representation (mr) @xcite @xcite . recent work has suggested a schema-guided paradigm for taskoriented dialogue by adding descriptions in natural language form @xcite @xcite . compared to structured mrs, dialogue schemata contain much richer contextual information, leading to better generated outputs. although the primary aim of task-oriented nlg is to effectively generate outputs that realize system dialogue actions and communicate their associated contents correctly, it is often desirable to control the stylistic variations of an output. for example, recognizing and reacting to emotions has been shown to enhance task outcomes and user engagement in task-oriented conversations @xcite . language generation systems that use corpora and methods without awareness of emotions may generate callous, generic or even biased responses @xcite . depending on the use case or type of system, it may be useful to stylistically vary responses, e.g., using shorter responses for spoken dialogue systems, longer responses if the system includes visual modality through a screen, or emotion-specific responses that appropriately address user sentiment. previous work on controlled text generation aimed at achieving stylistic goals has not focused on a schema-guided paradigm where specific content must be communicated correctly; instead, most work focuses more on unconstrained text-to-text generation without explicit meaning representations @xcite @xcite . meanwhile, work on schema-guided nlg has primarily focused on generating fluent outputs that achieve low semantic error rather than achieving stylistic goals @xcite @xcite . in this paper, we hope to fill the gap on stylistic control and evaluation for schemaguided nlg. our contributions in this paper are three-fold: specifically, we consider three types of methods: conditional training, guided fine-tuning, and guided decoding. we show that conditional training (ct) can be used for both very lexically-defined (e.g., point-of-view) and more semantically complex styles (e.g., empathy). however, ct introduces the need to re-train new models per style and is more effective at learning styles with strong lexical characteristics (e.g., specific language patterns or vocabulary). for guided fine-tuning, we explore the plug-and-play language model (pplm) @xcite , but show that it requires careful hyper-parameter turning and is prone to degeneration. for guided decoding, we evaluate the beam search weighted decoding (bswd) method and show that it performs best overall on measures of style accuracy for semantically complex styles. the results suggest that unlike style control for unconstrained text generation where no specific content needs to be communicated, style control under the schema-guided paradigm has stronger restrictions on the degree of freedom allowed for content generation. we show that methods that disentangle content generation and style variations, especially for more semantically complex styles, result in better overall performance on semantic and stylistic control. ## related work controllable text generation is an emerging research field. current methods for controlling styles in text generation involve learning a conditional generative model or designing an appropriate decoding strategy. there are many methods proposed for learning a good conditional generative model. these include conditional training @xcite @xcite , fine-tuning language models with external attribute models or side models @xcite , finetuning models with reinforcement learning and human feedback @xcite , training generative adversarial models @xcite , and training variational auto-encoders @xcite . others have worked on designing a good decoding strategy to guide generation, where the decoding procedure is guided by a scoring function or discriminator. these include weighted decoding @xcite @xcite and guided generation @xcite . other lines of work include curating training data with rich style markup to facilitate training models with explicit stylistic supervision @xcite . while this previous work does focus on controllable text generation, most work has been carried out in a text-to-text generation setting, without specific semantic constraints. instead, we focus on the task-oriented dialogue framework where specific values must be communicated, and conduct a rigorous evaluation of different methods and their efficacy on different forms of style generation. other recent work has explored adding additional information such as chit-chat data to task-oriented dialogue @xcite and could potentially provide new opportunities for stylistic control. ## baseline model language model given a sequence of tokens @xmath0, • • • , x t , the goal of the language model is to model the joint probability of the sequence p(x) = p(x 1 , • • • , x t ). the joint probability p(x) is often factorized in terms of the product of conditional probabilities using the chain rule of probability p (x) = t @xmath1, • • • , x t -1 ) @xcite . in recent years, transformer-based models have been used widely to model these conditional probabilities @xcite . in this work, we use gpt-2 foot_1 as our baseline language model @xcite and fine-tune the gpt-2 model with the processed sgd data using a flat representation with the beginning of sequence, separator, and end of sequence special tokens. 4 semantic style classifiers the classifiers used to annotate semantic parameters are single layer classifiers. to train each classifier, we encode the input @xmath2, • • • , x t of length t using the baseline gpt-2 model described above and obtain the last hidden layer o t for all time steps t. we then take the average representation across time, denoted ōt , and train the classifier to predict the target label (e.g., formal vs. informal) from the average repre ## style controlled text generation in this work, we require a controlled generation method that is able to simultaneously render the semantic content given the schemata while achieving the desired stylistic goals. we also require the method to be both stable (preserve the fluency of the response even when the stylistic goals are not met) and general-purpose (can be applied to many styles). under these requirements and constraints, we discuss three types of controlled generation methods to achieve these goals: conditional training, guided fine-tuning, and guided decoding, and compare their performance in section 6. to our knowledge, our work is the first to systematically study the effectiveness of these control methods for schema-guided nlg. ## experiments in this section, we show the experimental results of the methods described in section 5 for controlling the styles described in section 3 on the sgd data. the baseline gpt-2 model is fine-tuned on the training set with no control variables, and the conditional training model is fine-tuned with control variable special tokens, e.g., length_short. our evaluation is tested on the test set of 1,137 mrs. we focus on controlling a single style at a time in this experiment; however, it is also possible to control for multiple styles -we include details on multiple-style control experiments in appendix c (with sample outputs in appendix ## conclusion in this work, we focus on stylistic control and evaluation of schema-guided nlg. we discuss three different types of methods for style controlled text generation: conditional training (ct), guided finetuning (pplm), and guided decoding (bswd). we present a rich set of evaluations to quantify each method's ability to achieve various styles while preserving language fluency and minimizing slot errors. our analysis shows that, in general, styles that encapsulate abstract ideas are naturally harder to generate (e.g., empathy), and methods that require careful hyper-parameters tuning may run into the problems of instability and degeneration (e.g., pplm) while under-performing in style accuracy. the automatic and human evaluations suggest that simultaneously achieving stylistic goals and realizing schema information requires methods that allow us to separate content generation and stylistic variations. we show that ct and bswd overcome some of these challenges and are effective at controlling several styles while maintaining good fluency and semantic correctness in most cases. ct is effective for lexical styles with strong syntactic characteristics or distinctive vocabulary sets, while bswd excels at semantic styles that are more complex to characterize. for future work, we are interested in extending our analysis to a larger number of styles, and in exploring techniques for understanding and representing styles that are more abstract, such as "empathy", as well as methods for generating those styles under a task-oriented dialogue framework. additionally, since abstract styles may be characterized by multiple features (e.g., a combination of sentiment and descriptiveness), we are interested in studying how these underlying features can be represented and incorporated more accurately to improve overall semantic and stylistic control. layer (logit vector) to achieve the above two goals. the logit vector o t is obtained by a forward pass of the lm, i.e. o @xmath3, h t ). the next generated token x t+1 is sampled as the shift ∆h t is then updated with gradient descent as follows: