Two Birds, One Stone: A Simple, Unified Model for Text Generation from Structured and Unstructured Data (2020)

Abstract: A number of researchers have recently questioned the necessity of increasingly complex neural network (NN) architectures. In particular, several recent papers have shown that simpler, properly tuned models are at least competitive across several NLP tasks. In this work, we show that this is also the case for text generation from structured and unstructured data. We consider neural table-to-text generation and neural question generation (NQG) tasks for text generation from structured and unstructured data, respectively. Table-to-text generation aims to generate a description based on a given table, and NQG is the task of generating a question from a given passage where the generated question can be answered by a certain sub-span of the passage using NN models. Experimental results demonstrate that a basic attention-based seq2seq model trained with the exponential moving average technique achieves the state of the art in both tasks. Code is available at https://github.com/h-shahidi/2birds-gen .

## introduction recent nlp literature can be characterized as increasingly complex neural network architectures that eke out progressively smaller gains over previous models. following a previous line of research @xcite @xcite , we investigate the necessity of such complicated neural architectures. in this work, our focus is on text generation from structured and unstructured data by considering description generation from a table and question generation from a passage and a target answer. more specifically, the goal of the neural tableto-text generation task is to generate biographies based on wikipedia infoboxes (structured data). an infobox is a factual table with a number of fields passage: hydrogen is commonly used in power stations as a coolant in generators due to a number of favorable properties that are a direct result of its light diatomic molecules. answer: as a coolant in generators question: how is hydrogen used at power stations? automatic question generation aims to generate a syntactically correct, semantically meaningful and relevant question from a natural language text and a target answer within it (unstructured data). this is a crucial yet challenging task in nlp that has received growing attention due to its application in improving question answering systems @xcite @xcite , providing material for educational purposes @xcite , and helping conversational systems to start and continue a conversation @xcite . we adopt the widely used squad dataset @xcite for this task. prior work has made remarkable progress on both of these tasks. however, the proposed models utilize complex neural architectures to capture necessary information from the input(s). in this paper, we question the need for such sophisticated nn models for text generation from inputs comprising structured and unstructured data. specifically, we adopt a bi-directional, attentionbased seq2seq model @xcite equipped with a copy mechanism @xcite for both tasks. we demonstrate that this model, together with the exponential moving average (ema) technique, achieves the state of the art in both neural table-to-text generation and nqg. interestingly, our model is able to achieve this result even without using any linguistic features. our contributions are two-fold: first, we propose a unified nn model for text generation from structured and unstructured data and show that training this model with the ema technique leads to the state of the art in neural table-to-text generation as well as nqg. second, because our model is, in essence, the primary building block of previous models, our results show that some previous papers propose needless complexity, and that gains from these previous complex neural architectures are quite modest. in other words, the state of the art is achieved by careful tuning of simple and wellengineered models, not necessarily by adding more complexity to the model, echoing the sentiments of @xcite . ## related work in this section, we first discuss previous work for neural table-to-text generation and then nqg. ## model: seq2seq with attention and a copy mechanism in this section, we introduce a simple but effective attention-based seq2seq model for both neural table-to-text generation and nqg. figure 2 provides an overview of our model. ## experimental setup in this section, we introduce the datasets first, then explain additional implementation details, and finally describe the evaluation metrics. ## results and discussion in this section, we present our experimental results for both neural table-to-text generation and nqg. we report the mean and standard deviation of each metric across multiple seeds to ensure robustness against potentially spurious conclusions @xcite . in tables 2 and 3 , we compare previous work with our results for nqg and neural table-totext generation, respectively. all results are copied from the original papers except for @xcite in it is noteworthy that a similar version of our model has served as a baseline in previous papers @xcite @xcite . however, the distinctions discussed in section 2, especially the ema technique, enable our model to achieve the state of the art in all cases but bleu-4 on the squad split-2, where our score is very competitive; furthermore, @xcite only report results from a single trial. our results indicate that a basic seq2seq model is able to effectively learn the underlying distribution of both datasets. ## conclusions and future work in this paper, we question the necessity of complex neural architectures for text generation from structured data (neural table-to-text generation) and