Data-to-Text Generation with Style Imitation (2020)

Abstract: Recent neural approaches to data-to-text generation have mostly focused on improving content fidelity while lacking explicit control over writing styles (e.g., sentence structures, word choices). More traditional systems use templates to determine the realization of text. Yet manual or automatic construction of high-quality templates is difficult, and a template acting as hard constraints could harm content fidelity when it does not match the record perfectly. We study a new way of stylistic control by using existing sentences as “soft” templates. That is, a model learns to imitate the writing style of any given exemplar sentence, with automatic adaptions to faithfully describe the record. The problem is challenging due to the lack of parallel data. We develop a neural approach that includes a hybrid attention-copy mechanism, learns with weak supervisions, and is enhanced with a new content coverage constraint. We conduct experiments in restaurants and sports domains. Results show our approach achieves stronger performance than a range of comparison methods. Our approach balances well between content fidelity and style control given exemplars that match the records to varying degrees.

## introduction recent years have seen remarkable progress in neural natural language generation to produce well-formed coherent text @xcite . yet, controllability over various text properties, as an essential demand to ensure the utility of generations in real-world applications, has not attained the same level of advancement. data-to-text generation is one of such applications with ubiquitous practical use, in which natural language text is generated to describe a given data record such as a box score of a sports player or an infobox table of a restaurant. though current data-to-text neural approaches with encoder-decoder models could produce fluent text with high fidelity to content ("what to say"), they largely lack control over the writing style, such as sentence structures and word choices ("how to say"). many efforts have been made to promote the overall diversity in data-to-text generation through, e.g., latent variables @xcite or customized model architectures @xcite ). yet fine-grained style manipulation is not permitted. this contrasts with the traditional text generation systems which separate content planning and surface realization @xcite , and usually determine the realization with explicit templates @xcite or based on syntactic grammars @xcite . controlling writing style with "hard" templates could suffer from unscalable template creation and lack of generation flexibility. though previous work @xcite @xcite has enabled automatic template extraction, the templates usually act as hard constraints and could harm the content fidelity of generations when the template does not exactly match the content in a record. in this paper, we study a new way of stylistic control in data-to-text generation by using any existing sentences as "soft" templates. that is, we learn to imitate the writing style of a given exemplar sentence. the goal is two-fold: to generate text that not only faithfully describes all content in the record, but also inherits as many of the exemplar's stylistic characteristics as possible (figure 1 ). the new paradigm sidesteps the restrictions with traditional dedicated templates and allows us to use arbitrary exemplar sentences that could be describ- ## the task: data-to-text generation with style imitation for clarity, we first formally describe the problem of data-to-text generation with style imitation. we also establish the key notations used in the paper. consider a data record x which consists of a set of fields and their values (e.g., field "food" and its value "italian" in figure 1 ). note that different records can include different fields. for example, the field "customer rating" is included in some records but not the one in figure 1 . data-to-text generation aims to produce a sentence to describe the content in the record. we are additionally given an exemplar sentence y e which could be describing distinct content in the same domain. the goal of the task is thus to generate a new sentence y that achieves (1) content fidelity by describing the content in x accurately and completely, and (2) style embodiment by retaining as much of the writing style (e.g., sentence structure, word choice, etc) of y e as possible. a solution to the problem is required to balance well between the two objectives, by adaptively rewriting necessary portions of the reference y e to express the desired content in a correct and fluent way, while at the same time editing y e to a minimum extent to inherit its style. the demand for adaptive trade-off necessitates developing learning approaches for flexible imitation and generation. to the best of our knowledge, there is no large data containing the desired (x, y e , y) triples for supervised learning. instead, we often only have access to pairs of record and its description which was originally written without following any designated style. in the next section, we develop a neural approach that learns style imitation given only the paired data. ## the approach denote the proposed neural model as p θ (y|x, y e ). the model has a hybrid attention-copy mechanism (sec 4.1) for differentiated treatment of source content and style exemplar. we learn the model by constructing weak supervisions from the available non-parallel data (sec 4.2), and further encourage accurate content description with a content coverage constraint (sec 4.3). figure 2 presents an overview of the approach. ## experiments we study on two datasets in the restaurant recommendations and nba reports domains, respectively. we conduct both automatic and human evaluations to assess model performance. experiment results validate the proposed approach in learning an effective, balanced control of content and style. ## conclusion we have studied the new problem of data-to-text generation with style imitation. we developed a new approach with an attention-copy mechanism, weakly supervised learning, and a content coverage constraint. experiments show the approach achieves a good balance between content fidelity and style control, and is flexible to adapt exemplars that do not match the record perfectly. we are interested in applying the style imitation approach to control longer paragraphs given full data tables.