Abstract: In this paper, we focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer and aims to preserve text styles while altering the content. In detail, the input is a set of structured records and a reference text for describing another recordset. The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference. The task is unsupervised due to lack of parallel data, and is challenging to select suitable records and style words from bi-aspect inputs respectively and generate a high-fidelity long document. To tackle those problems, we first build a dataset based on a basketball game report corpus as our testbed, and present an unsupervised neural model with interactive attention mechanism, which is used for learning the semantic relationship between records and reference texts to achieve better content transfer and better style preservation. In addition, we also explore the effectiveness of the back-translation in our task for constructing some pseudo-training pairs. Empirical results show superiority of our approaches over competitive methods, and the models also yield a new state-of-the-art result on a sentence-level dataset.

Introduction: Data-to-text generation is an effective way to solve data overload, especially with the development of sensor and data storage technologies, which have rapidly increased the amount of data produced in various fields such as weather, finance, medicine and sports BIBREF0. However, related methods are mainly focused on content fidelity, ignoring and lacking control over language-rich style attributes BIBREF1. For example, a sports journalist prefers to use some repetitive words when describing different games BIBREF2. It can be more attractive and practical to generate an article with a particular style that is describing the conditioning content. In this paper, we focus on a novel research task in the field of text generation, named document-scale text content manipulation. It is the task of converting contents of a document into another while preserving the content-independent style words. For example, given a set of structured records and a reference report, such as statistical tables for a basketball game and a summary for another game, we aim to automatically select partial items from the given records and describe them with the same writing style (e.g., logical expressions, or wording, transitions) of the reference text to directly generate a new report (Figure 1). In this task, the definition of the text content (e.g., statistical records of a basketball game) is clear, but the text style is vague BIBREF3. It is difficult to construct paired sentences or documents for the task of text content manipulation. Therefore, the majority of existing text editing studies develop controlled generator with unsupervised generation models, such as Variational Auto-Encoders (VAEs) BIBREF4, Generative Adversarial Networks (GANs) BIBREF5 and auto-regressive networks BIBREF6 with additional pre-trained discriminators. Despite the effectiveness of these approaches, it remains challenging to generate a high-fidelity long summary from the inputs. One reason for the difficulty is that the input structured records for document-level generation are complex and redundant to determine which part of the data should be mentioned based on the reference text. Similarly, the model also need to select the suitable style words according to the input records. One straightforward way to address this problem is to use the relevant algorithms in data-to-text generation, such as pre-selector BIBREF7 and content selector BIBREF8. However, these supervised methods cannot be directly transferred considering that we impose an additional goal of preserving the style words, which lacks of parallel data and explicit training objective. In addition, when the generation length is expanded from a sentence to a document, the sentence-level text content manipulation method BIBREF1 can hardly preserve the style word (see case study, Figure 4). In this paper, we present a neural encoder-decoder architecture to deal with document-scale text content manipulation. In the first, we design a powerful hierarchical record encoder to model the structured records. Afterwards, instead of modeling records and reference summary as two independent modules BIBREF1, we create fusion representations of records and reference words by an interactive attention mechanism. It can capture the semantic relatedness of the source records with the reference text to enable the system with the capability of content selection from two different types of inputs. Finally, we incorporate back-translation BIBREF9 into the training procedure to further improve results, which provides an extra training objective for our model. To verify the effectiveness of our text manipulation approaches, we first build a large unsupervised document-level text manipulation dataset, which is extracted from an NBA game report corpus BIBREF10. Experiments of different methods on this new corpus show that our full model achieves 35.02 in Style BLEU and 39.47 F-score in Content Selection, substantially better than baseline methods. Moreover, a comprehensive evaluation with human judgment demonstrates that integrating interactive attention and back-translation could improve the content fidelity and style preservation of summary by a basic text editing model. In the end, we conduct extensive experiments on a sentence-level text manipulation dataset BIBREF1. Empirical results also show that the proposed approach achieves a new state-of-the-art result.

Question: How better are results of new model compared to competitive methods?
Answer: For Document- level comparison, the model achieves highest CS precision and  F1 score and it achieves higher BLEU score that TMTE, Coatt, CCDT, and HEDT. 
In terms of Human Evaluation, the model had the highest average score, the highest Fluency score, and the second highest Content Fidelity. 
In terms of Sentence-level comparison the model had the highest Recall and F1 scores for Content Fidelity.