Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021)

Abstract: Recent developments in neural networks have led to the advance in data-to-text generation. However, the lack of ability of neural models to control the structure of generated output can be limiting in certain real-world applications. In this study, we propose a novel Plan-then-Generate (PlanGen) framework to improve the controllability of neural data-to-text models. Extensive experiments and analyses are conducted on two benchmark datasets, ToTTo and WebNLG. The results show that our model is able to control both the intra-sentence and inter-sentence structure of the generated output. Furthermore, empirical comparisons against previous state-of-the-art methods show that our model improves the generation quality as well as the output diversity as judged by human and automatic evaluations.

## introduction generating natural language from structured data @xcite , i.e. data-to-text generation, is a research problem that is crucial to many downstream nlp applications. some examples are dialogue systems @xcite , restaurant assistant @xcite , and open domain question answering @xcite . to address this task, many researchers have designed sophisticated neural models based on various methods, such as soft-template @xcite , copy mechanism @xcite , and pre-trained language models @xcite . while achieving impressive results, most existing studies only focused on producing results that are close to the references. on the other hand, the controllability of such models is still under-explored, i.e. what to generate and in what order (the output structure) in their outputs cannot be explicitly controlled by the users. we argue that the model's ability to control the structure of its output is highly desirable for at least * work done while the author was an intern at apple. to control the output structure, we need an intermediate "planning" signal (i.e. a content plan) which informs the model what to generate and in what order. to this end, we propose a plan-then-generate (plangen) framework which consists of two components: a content planner and a sequence generator. given the input data, the content planner first predicts the most plausible content plan that the output should follow. then, the sequence generator takes the data and the content plan as input to generate the result. to further ensure the controllability of our model, we propose a structure-aware reinforcement learning (rl) objective that encourages the generated output to adhere to the given content plan. in this work, we formulate the intermediate content plan as an ordered list of tokens for its simplicity and wide applicability to data with different structures. for tabular data, each token in the content plan is a slot key from the table. as for graphical data with rdf structure, each token represents the predicate from an rdf triple. in figure 1 , we provide examples for both cases. to fully evaluate our approach, we test the proposed model on two benchmarks with different data structures: (i) totto dataset @xcite with tabular data, and (ii) webnlg dataset @xcite with graphical data. compared with previous state-of-the-art approaches, our model achieves better performance in terms of generation quality as judged by both human and automatic evaluations. in particular, the results also show that the outputs of our model are highly controllable and contain diverse structures. in summary, our contributions are: (1) a novel plan-then-generate (plangen) framework that consists of a content planner and a sequence generator for data-to-text generation. (2) extensive automatic and human evaluations reporting state-of-the-art results on two benchmark datasets. (3) in-depth analysis revealing the merits of the proposed approach in terms of controllability and diversity. ## related work data-to-text generation is a long-standing problem @xcite ) that aims at producing natural language descriptions of structured data. traditional systems are primarily built on templatebased algorithms @xcite @xcite . with recent advances in deep learning, researchers have shifted their attention to neural generation models that can be summarized into two categories. end-to-end models. many existing studies are dedicated to building end-to-end neural models with different strategies like soft-templates @xcite , attention awareness @xcite , and retrieved prototypes @xcite . @xcite , @xcite , and chen et al. (2020b) adopted copy mechanism for content selection to improve the information coverage of the outputs. with recent advance in pre-trained language models (plms) @xcite @xcite , several researchers @xcite @xcite have studied the ways to adapt plms into the data-to-text generation task. ## methodology figure 2 depicts the proposed plan-then-generate (p2g) framework. given the input data, the content planner ( §4.1) first predicts the most probable content plan. the sequence generator ( §4.2) then takes the structured data and the predicted content plan to generate the output. in the following, we elaborate the details of the proposed framework. ## results in this section, we report the experimental results. ## further analysis in this section, we present and discuss more empirical analyses of the proposed model. ## conclusion in this study, we propose a new plan-then-generate (plangen) framework for data-to-text generation which can be easily applied to data with different structures. extensive experiments and analyses are conducted on two benchmark datasets. both automatic and human evaluation results demonstrate that our model is highly controllable. furthermore, compared with previous studies, our model achieves better results both in terms of the generation quality as well as the output diversity. our code, models and other related resources can be found in https://github.com/yxuansu/ plangen/