Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020)

Abstract: The neural attention model has achieved great success in data-to-text generation tasks. Though usually excelling at producing fluent text, it suffers from the problem of information missing, repetition and “hallucination”. Due to the black-box nature of the neural attention architecture, avoiding these problems in a systematic way is non-trivial. To address this concern, we propose to explicitly segment target text into fragment units and align them with their data correspondences. The segmentation and correspondence are jointly learned as latent variables without any human annotations. We further impose a soft statistical constraint to regularize the segmental granularity. The resulting architecture maintains the same expressive power as neural attention models, while being able to generate fully interpretable outputs with several times less computational cost. On both E2E and WebNLG benchmarks, we show the proposed model consistently outperforms its neural attention counterparts.

## introduction data-to-text generation aims at automatically producing natural language descriptions of structured database @xcite . traditional statistical methods usually tackle this problem by breaking the generation process into a set of local decisions that are learned separately @xcite @xcite . recently, neural attention models conflate all steps into a single end-to-end system and largely simplify the training process @xcite @xcite @xcite . however, the black-box conflation also renders the generation uninterpretable and hard to control @xcite . verifying the generation correctness in a principled way is nontrivial. in practice, it often suffers from the problem of information missing, repetition and "hallucination" @xcite . ## related work data-to-text generation is traditionally dealt with using a pipeline structure containing content planning, sentence planning and linguistic realization @xcite . each target text is split into meaningful fragments and aligned with corresponding data records, either by handengineered rules @xcite or statistical induction @xcite @xcite . the segmentation and alignment are used as supervision signals to train the content and sentence planner @xcite . the linguistic realization is usually implemented by template mining from the training corpus @xcite . our model adopts a similar pipeline generative process, but integrates all the sub-steps into a single end-toend trainable neural architecture. it can be considered as a neural extension of the pcfg system in @xcite , with a more powerful transition probability considering inter-segment dependence and a state-of-the-art attention-based language model as the linguistic realizer. @xcite tried a similar neural generative model to induce templates. however, their model only captures loose data-text correspondence and adopts a weak markov assumption for the segment transition probability. therefore, it underperforms the neural attention baseline as for generation. our model is also in spirit related to recent attempts at separating content planning and surface realization in neural data-to-text models @xcite @xcite . nonetheless, all of them resort to manual annotations or hand-engineered rules applicable only for a narrow domain. our model, instead, automatically learn the optimal content planning via exploring over exponentially many segmentation/correspondence possibilities. there have been quite a few neural alignment models applied to tasks like machine translation @xcite , character transduction @xcite and summarization @xcite . unlike word-to-word alignment, we focus on learning the alignment between data records and text segments. some works also integrate neural language models to jointly learn the segmentation and correspondence, e.g., phrase-based machine translation @xcite , speech recognition @xcite and vision-grounded word segmentation @xcite . data-to-text naturally fits into this scenario since each data record is normally verbalized in one continuous text segment. the neural attention architecture handles this task with an encode-attend-decode process @xcite . the input x is processed into a sequence of x 1 , x 2 , . . . , x n , normally by flatten-ing the data records @xcite . the encoder encodes each x i into a vector h i . at each time step, the decoder attends to encoded vectors and outputs the probability of the next token by p(y t |y 1:t-1 , a t ). a t is a weighted average of source vectors: ## approach suppose the input data x contains a set of records r 1 , r 2 , ..., r k . our assumption is that the target text y 1:m can be segmented into a sequence of fragments. each fragment corresponds to one data record. as the ground-truth segmentation and correspondence are not available, we need to enumerate over all possibilities to compute the likelihood of y 1:m . denote by s y the set containing all valid segmentation of y 1:m . for any valid segmentation s 1:τs ∈ s y , π(s 1:τs ) = y 1:m , where π means concatenation and τ s is the number of segments. for example, let @xmath2. one possible segmentation would be s 1:τs = {{y 1 , y 2 , @xmath0}, {y 4 , y 5 , @xmath1 is the end-of-segment symbol and is removed when applying the π operator. we further define c( * ) to be the corresponding data record(s) of * . the likelihood of each text is then computed by enumerating over all possibilities of s 1:τs and c(s 1:τs ): ## experiment setup dataset we conduct experiments on the e2e @xcite and webnlg @xcite datasets. e2e is a crowd-sourced dataset containing 50k instances in the restaurant domain. the inputs are dialogue acts consisting of three to eight slot-value pairs. webnlg contains 25k instances describing entities belonging to fifteen distinct dbpedia categories. the inputs are up to seven rdf triples of the form (subject, relation, object). ## results in this section, we first show the effects of the granularity regularization we proposed, then compare model performance on two datasets and analyze the performance difference. our model is compared against the neural attention-based pointer generator (pg) which does not explicit learn the segmentation and correspondence. to show the effects of the constrained decoding mentioned in §4, decoding. we run our model with only the first constraint to prevent empty segments (denoted by ours in experiments), with the first two constraints to prevent repetition (denoted by ours (+r)), and with all constraints to further reduce information missing (denoted by ours (+rm)). segmentation granularity we show the effects of the granularity regularization ( §4, segmentation granularity) in fig 3 . when varying the model size, the segmentation granularity changes much if no regularization is imposed. intuitively if the generation module is strong enough (larger hidden size), it can accurately estimate the sentence likelihood itself without paying extra cost of switching between segments, then it tends to reduce the number of transitions. vice versa, the number of transitions will grow if the transition module is stronger (larger embedding size). with the regularization we proposed, the granularity remains what we want regardless of the hyperparameters. we can thereby freely decide the model capacity without worrying about the difference of segmentation behavior. results on e2e on the e2e dataset, apart from our implementations, we also compare agianst outputs from the slug @xcite , the overall winner of the e2e challenge (seq2seqbased), dangnt (nguyen and tran, 2018), the best grammar rule based model, tuda @xcite , the best template based model, and the autoregressive neural template model (n temp) from @xcite . slug uses a heuristic slot aligner based on a set of handcrafted rules and combine a complex pipeline of data augmentation, selection, model ensemble and reranker, while our model has a simple end-toend learning paradigm with no special delexicalizing, training or decoding tricks. ## conclusion in this work, we exploit the segmental structure in data-to-text generation. the proposed model significantly alleviates the information hallucination, repetition and missing problems without sacrificing the fluency and diversity. it is end-to-end trainable, domain-independent and allows explicit control over the structure of generated text. as our model is interpretable in the correspondence between segments and input records, it can be easily combined with hand-engineered heuristics or user-specific requirements to further improve the performance.