Exploring Structural Encoding for Data-to-Text Generation (2021)

Abstract: Due to efficient end-to-end training and fluency in generated texts, several encoder-decoder framework-based models are recently proposed for data-to-text generations. Appropriate encoding of input data is a crucial part of such encoder-decoder models. However, only a few research works have concentrated on proper encoding methods. This paper presents a novel encoder-decoder based data-to-text generation model where the proposed encoder carefully encodes input data according to underlying structure of the data. The effectiveness of the proposed encoder is evaluated both extrinsically and intrinsically by shuffling input data without changing meaning of that data. For selecting appropriate content information in encoded data from encoder, the proposed model incorporates attention gates in the decoder. With extensive experiments on WikiBio and E2E dataset, we show that our model outperforms the state-of-the models and several standard baseline systems. Analysis of the model through component ablation tests and human evaluation endorse the proposed model as a well-grounded system.

## introduction data-to-text generation @xcite aims to produce humanunderstandable text from semi-structured data such as tables, concepts, etc. the input data consists of multiple records (or events), where each record represents a particular field (or attribute) of the data. due to efficient end-to-end training and fluency in generated texts @xcite , numerous data-to-text generation systems have adopted encoder-decoder based model @xcite . in such encoder-decoder based models, a proper meaningful encoding of input data is a real concern. as in most of the data-to-text generation, input data poses record-field structure (like in selecting appropriate content from input data (also known as content selection) is an important task. for example, according to we propose an encoder-decoder model for datato-text generation where the encoder encodes data based on record-field structures. we introduce structure-wise attention gates to capture appropriate content from the encoded data. as records in an input data don't pose any ordering among them (for example, in ## notations we follow the notations which are commonly used in popular data-to-text generation models @xcite @xcite . the goal of our task (t ) is to produce text representation @xmath0...y w (where y i is the i-th word in y) from a given an input data x. an input (x) consists of multiple records {r i } n @xmath1. a record (r j ) contains a field f j , with its name n j and the corresponding value ## approach we propose a neural network based encoderdecoder model for the task t , where the encoder structurally encodes the input (x). the decoder is a recurrent neural network with two sub-modules-(i) attention gates for appropriate content selection and (ii) copy module to handle the appearances of rare words in generating text. ## experiments the experiment considers two popular benchmark datasets for data-to-text generations-wikibio dataset and e2e dataset. ## related works the research presented in this paper is related to the recent data-driven data-to-text generation effort where text is generated from structured data @xcite @xcite . there are several types of data-driven text generation systems. @xcite used probabilistic context-free grammar for text generation from structured data. chen and mooney (2008) introduced a strategic text generation technique for sportscasting of a simulated soccer game. among data-driven text generations, @xcite was probably the first to propose a domain-independent approach with an application on weather forecasting. with the advent of recurrent neural network language model @xcite , neural text generation models are proposed several in number and successfully applied to several different text generation tasks, from poem generation @xcite to image captioning @xcite @xcite . after the seq2seq model @xcite and various attention mechanisms @xcite are reported in the literature, the encoderdecoder model in neural text generation become quite ubiquitous. for selective generation task where readers focus on a certain selective part of the input, @xcite proposed an encoderdecoder model with an attention mechanism. in a concept-to-text generation where aim lies in generating text descriptions from complex concepts, the encoder-decoder based models also achieve high accuracy @xcite @xcite . for the dialogue system too, this kind of data-driven approach finds some important results @xcite . the encoder-decoder model has also shown promising results on table-to-text generation task @xcite . ## conclusion in this paper, we propose an effective structural encoder for encoder-decoder based data-to-text generation, which carefully encodes record-field structured input data. with extensive experiments, we show that the proposed model is capable of handling both sequential and order-less (non-sequential) data. for selecting appropriate contents from encoded data, we incorporated attention gates in the proposed model. evaluation of the model on wikibio and e2e dataset brings out the potential of the proposed system in generating quality text. the immediate extension of this study may consider analysis of the model's behavior at sub-task levels, i.e., its effect on content selection, text/discourse planning, or on surface realization. these experiments may unveil more interesting features of the proposed model. moreover, further research is needed to improve the quality of output text. @xcite , clubs [blackpool fulham exeter city birmingham stoke], caps[0 1 2 49 0], goals[0 0 0 0 0], articletitle[bert crossthwaite] reference herbert crossthwaite -lrb-4 april 1887 -20 may 1944 -rrb-was an english footballer who played as a goalkeeper . generated text herbert bert crossthwaite -lrb-4 april 1887 -20 may 1944 -rrb-was an english footballer who played as a goalkeeper .