Controllable Neural Natural Language Generation: comparison of state-of-the-art control strategies (2020)

Abstract: Most NLG systems target text fluency and grammatical correctness, disregarding control over text structure and length. However, control over the output plays an important part in industrial NLG applications. In this paper, we study different strategies of control in triple-totext generation systems particularly from the aspects of text structure and text length. Regarding text structure, we present an approach that relies on aligning the input entities with the facts in the target side. It makes sure that the order and the distribution of entities in both the input and the text are the same. As for control over text length, we show two different approaches. One is to supply length constraint as input while the other is to force the end-ofsentence tag to be included at each step when using top-k decoding strategy. Finally, we propose four metrics to assess the degree to which these methods will affect a NLG system’s ability to control text structure and length. Our analyses demonstrate that all the methods enhance the system’s ability with a slight decrease in text fluency. In addition, constraining length at the input level performs much better than control at decoding level.

## introduction if many researches have focused on end-to-end neural nlg (nnlg), approaches depending on a pipeline architecture, where the generation can be divided into multiple steps and the steps are accomplished separately using either neural networks or other models, are still competitive @xcite @xcite . besides improvement in text fluency and grammatical correctness, succeeding in controlling different aspects of text such as style, structure and length is a key enabler for reliable systems fit for industrial applications and for better understanding of nnlg models. in end-to-end models, control can be applied at various stages of the neural generation process such as input, hidden states and decoding @xcite . plug and play language model (pplm) proposed by @xcite takes an external input and performs computations on hidden states. it combines a pretrained lm with one or more simple attribute classifiers that guide text generation towards the desired topic or sentiment. @xcite describe a training method based on diverse ensembling to encourage models to learn distinct text styles. @xcite intervene during the decoding stage to relieve the problem of infinite generation. the end-to-end models can be equipped with the ability to control style and length. in pipeline systems, control can be applied to various steps especially the step of text planning where controlling text content is the main objective. @xcite apply a series of rule-based graph-transducers and aggregation grammar while @xcite triples to be rearranged in line with target. to better understand the effect of such control in black-box models, it is necessary to look into stateof-the-art control strategies through independent quantitative analyses. in this paper, we focus on controlling text structure and text length for rdf triple-to-text generation. text structure consists in sentence split and entity order which care about how the facts are distributed in the sentences and in what relative order the entities are stated. text length is the word count of a produced text. in this work, we assume a two-step pipeline models which consist of text planning and realization @xcite . a text plan generated from the input facts in the first step is feeding the second step of realization. we make two main contributions. first, a systematic study of how the method of alignment affects control over text structure. the method aligns input with target in terms of sentence split and entity order. furthermore, we propose three metrics to evaluate its effects. second, a study of the text length control using two different approaches. one approach, applied to sentence planning, is to add the length constraint into input while the other, applied to surface realization, is to force the end-of-sentence tag to be included at each step when using top-k decoding strategy @xcite . we design one metric to evaluate the effects of the two different methods to control text length. none of the metrics we propose is used to optimize model training. ## method in this section, we discuss the methods of controlling sentence split, entity order and text length. ## conclusion and further-work in this study, we demonstrate that the alignment between the input and the target regarding sentence split and entity order leads to a substantial increase in the ability of nnlg models to control text structure. as for control of text length, we show that a control at the input level is highly preferable over a control at decoding level since it gives the model the opportunity to integrate the length constraint during the processing to avoid ending up generating an incomplete text. as shown in the study, different types of control do not seem independent (e.g. length influences number of sentences). the next step is to get more insight about these interdependencies.