Make Templates Smarter: A Template Based D ata2 T ext System Powered by Text Stitch Model (2020)

Abstract: Neural network (NN) based data2text models achieve state-of-the-art (SOTA) performance in most metrics, but they sometimes drop or modify the information in the input, and it is hard to control the generation contents. Moreover, it requires paired training data that are usually expensive to collect. Template-based methods have good fidelity and controllability but require heavy human involvement. We propose a novel template-based data2text system powered by a text stitch model. It ensures fidelity and controllability by using templates to produce the main contents. In addition, it reduces human involvement in template design by using a text stitch model to automatically stitch adjacent template units, which is a step that usually requires careful template design and limits template reusability. The text stitch model can be trained in self-supervised fashion, which only requires free texts. The experiments on a benchmark dataset show that our system outperforms SOTA NN-based systems in fidelity and surpasses template-based systems in diversity and human involvement.

## introduction data2text takes structured data like key-value pairs as inputs and generates corresponding texts. it has been used in various applications like automatically generating weather reports @xcite , restaurant descriptions @xcite , etc. recent works on data2text mainly focus on neural network (nn) based models @xcite . despite their great success,their fidelity (express all data correctly) and controllability (control the generated contents) are always their main issues. for example, for most nn-based generation models, we cannot guarantee that the generated texts do not drop or modify the information in the input. besides, it is hard to fix errors made by nn-based generation models. considering these issues, template-based data2text models @xcite @xcite are still widely used in real-world applications. as shown in fig. 1 , one of the most commonly used templates is the slot-filling style template @xcite . it does not require any linguistic knowledge. users just write a normal sentence but leave the changeable parts as slots. while this kind of system can produce faithful and controllable texts in specific domains, writing these templates typically requires a lot of human efforts, especially when we want to have both variety and fluency in the output texts. a main reason for intensive human involvement is the template reusability. it is difficult to reuse sentence-level templates in new tasks that usually have different requirements. therefore, people usually break long templates into smaller template units (tus). each tu contains candidate expressions expressing the same topic, and we randomly choose one to use during generation (see fig. 1 ). it allows us to recombine existing tus to cover new tasks. however, different expressions may require different contexts. as we include more candidates in a tu, the difficulty would rise rapidly to find suitable phrases to connect adjacent tus for all expressions. for example, in fig. 1 , expressions [price] price range and [price] price in tu-price require different prepositions ahead of them. however, if we instead put expressions requiring different contexts into different tus, the increased number of tus would raise the cost to use them. in this paper, we propose a text stitch powered template system (ts2), which reduces human involvement in template writing by removing the need to manually write phrases to connect adjacent tus. as shown in fig. 1 , once the step of manually writing connection phrases is removed, the whole template becomes much simpler, and more expressions can be added in the same tu. besides, the increased number of tu expressions and the automatically generated connection phrases enable us to generate more diversed and natural sentences. moreover, unlike nn-based models, since we only generate connection phrases, we can control most of the contents, especially the essential part containing input information. it guarantees output fidelity and delivers better generation controllability. since the vocabulary of the connection phrases is limited, we automatically generate text stitch training data by dropping certain words in free texts with simple rules. therefore, we can train a high-quality text stitch model in a self-supervised paradigm. by experimenting on a benchmark data2text dataset, we find that the text stitch model trained on an open-domain corpus can produce fluent text in most cases. when task-related data are added, our self-supervised ts2 system clearly outperforms the state-of-the-art (sota) template system. in addition, it outputs more faithful text compared with the sota nn-based system. ## experiments we aim to answer 4 research questions. q1) does ts2 produce more faithful texts than nn-based systems? q2) does ts2 produce more natural texts than template-based systems? q3) can the pretrained template stitch model be directly used in specific tasks? q4) does ts2 reduce human efforts compared with pure template-based systems? experimental setup we use the e2e dataset @xcite with 42063, 4672, 630 data for training, development and test. both template-and nn-based systems are extensively studied in e2e, which enables us to have a comprehensive comparison with existing data2text systems. we adopt bert tokenization, 70k max training steps, and batch-size of 8k tokens. more training details can be found in the appendix. ## conclusion we propose a novel text generation framework that combines the advantages of both template-and nn-based data2text systems. compared with nn models, it guarantees the fidelity of the output and improves system controllability. compared with pure template systems, it produces more varied texts and reduces human involvement in designing templates by removing the need to design connection texts for template units.