Question: Why do they constrain the encoding to the underlying structure of the input data?
Evidence: Until recently, efforts to bring out semantics from structured-data relied heavily on expert knowledge BIBREF22, BIBREF8. For example, in order to better transcribe numerical time series of weather data to a textual forecast, Reiter et al. BIBREF8 devise complex template schemes in collaboration with weather experts to build a consistent set of data-to-word rules.
Modern approaches to the wide range of tasks based on structured-data (e.g. table retrieval BIBREF2, BIBREF23, table classification BIBREF24, question answering BIBREF25) now propose to leverage progress in deep learning to represent these data into a semantic vector space (also called embedding space). In parallel, an emerging task, called “data-to-text", aims at describing structured data into a natural language description. This task stems from the neural machine translation (NMT) domain, and early work BIBREF26, BIBREF27, BIBREF10 represent the data records as a single sequence of facts to be entirely translated into natural language. Wiseman et al. BIBREF10 show the limits of traditional NMT systems on larger structured-data, where NMT systems fail to accurately extract salient elements.
To improve these models, a number of work BIBREF28, BIBREF12, BIBREF29 proposed innovating decoding modules based on planning and templates, to ensure factual and coherent mentions of records in generated descriptions. For example, Puduppully et al. BIBREF12 propose a two-step decoder which first targets specific records and then use them as a plan for the actual text generation. Similarly, Li et al. BIBREF28 proposed a delayed copy mechanism where their decoder also acts in two steps: 1) using a classical LSTM decoder to generate delexicalized text and 2) using a pointer network BIBREF30 to replace placeholders by records from the input data.
Closer to our work, very recent work BIBREF11, BIBREF19, BIBREF13 have proposed to take into account the data structure. More particularly, Puduppully et al. BIBREF13 follow entity-centric theories BIBREF31, BIBREF32 and propose a model based on dynamic entity representation at decoding time. It consists in conditioning the decoder on entity representations that are updated during inference at each decoding step. On the other hand, Liu et al. BIBREF11, BIBREF19 rather focus on introducing structure into the encoder. For instance, they propose a dual encoder BIBREF19 which encodes separately the sequence of element names and the sequence of element values. These approaches are however designed for single-entity data structures and do not account for delimitation between entities.
Our contribution differs from previous work in several aspects. First, instead of flatly concatenating elements from the data-structure and encoding them as a sequence BIBREF11, BIBREF12, BIBREF10, we constrain the encoding to the underlying structure of the input data, so that the delimitation between entities remains clear throughout the process. Second, unlike all works in the domain, we exploit the Transformer architecture BIBREF21 and leverage its particularity to directly compare elements with each others in order to avoid arbitrary assumptions on their ordering. Finally, in contrast to BIBREF33, BIBREF13 that use a complex updating mechanism to obtain a dynamic representation of the input data and its entities, we argue that explicit hierarchical encoding naturally guides the decoding process via hierarchical attention.

First, instead of flatly concatenating elements from the data-structure and encoding them as a sequence [18,28,39], we constrain the encoding to the underlying structure of the input data, so that the delimitation between entities remains clear throughout the process.
Answer: To ensure the delimitation between entities remains clear throughout the process.