Question: What are the two main challenges when designing data-to-text models?
Evidence: Knowledge and/or data is often modeled in a structure, such as indexes, tables, key-value pairs, or triplets. These data, by their nature (e.g., raw data or long time-series data), are not easily usable by humans; outlining their crucial need to be synthesized. Recently, numerous works have focused on leveraging structured data in various applications, such as question answering BIBREF0, BIBREF1 or table retrieval BIBREF2, BIBREF3. One emerging research field consists in transcribing data-structures into natural language in order to ease their understandablity and their usablity. This field is referred to as “data-to-text" BIBREF4 and has its place in several application domains (such as journalism BIBREF5 or medical diagnosis BIBREF6) or wide-audience applications (such as financial BIBREF7 and weather reports BIBREF8, or sport broadcasting BIBREF9, BIBREF10). As an example, Figure FIGREF1 shows a data-structure containing statistics on NBA basketball games, paired with its corresponding journalistic description.
Designing data-to-text models gives rise to two main challenges: 1) understanding structured data and 2) generating associated descriptions. Recent data-to-text models BIBREF11, BIBREF12, BIBREF13, BIBREF10 mostly rely on an encoder-decoder architecture BIBREF14 in which the data-structure is first encoded sequentially into a fixed-size vectorial representation by an encoder. Then, a decoder generates words conditioned on this representation. With the introduction of the attention mechanism BIBREF15 on one hand, which computes a context focused on important elements from the input at each decoding step and, on the other hand, the copy mechanism BIBREF16, BIBREF17 to deal with unknown or rare words, these systems produce fluent and domain comprehensive texts. For instance, Roberti et al. BIBREF18 train a character-wise encoder-decoder to generate descriptions of restaurants based on their attributes, while Puduppully et al. BIBREF12 design a more complex two-step decoder: they first generate a plan of elements to be mentioned, and then condition text generation on this plan. Although previous work yield overall good results, we identify two important caveats, that hinder precision (i.e. factual mentions) in the descriptions:

Linearization of the data-structure. In practice, most works focus on introducing innovating decoding modules, and still represent data as a unique sequence of elements to be encoded. For example, the table from Figure FIGREF1 would be linearized to [(Hawks, H/V, H), ..., (Magic, H/V, V), ...], effectively leading to losing distinction between rows, and therefore entities. To the best of our knowledge, only Liu et al. BIBREF19, BIBREF11 propose encoders constrained by the structure but these approaches are designed for single-entity structures.
Arbitrary ordering of unordered collections in recurrent networks (RNN). Most data-to-text systems use RNNs as encoders (such as GRUs or LSTMs), these architectures have however some limitations. Indeed, they require in practice their input to be fed sequentially. This way of encoding unordered sequences (i.e. collections of entities) implicitly assumes an arbitrary order within the collection which, as demonstrated by Vinyals et al. BIBREF20, significantly impacts the learning performance.
To address these shortcomings, we propose a new structured-data encoder assuming that structures should be hierarchically captured. Our contribution focuses on the encoding of the data-structure, thus the decoder is chosen to be a classical module as used in BIBREF12, BIBREF10. Our contribution is threefold:

We model the general structure of the data using a two-level architecture, first encoding all entities on the basis of their elements, then encoding the data structure on the basis of its entities;
We introduce the Transformer encoder BIBREF21 in data-to-text models to ensure robust encoding of each element/entities in comparison to all others, no matter their initial positioning;
We integrate a hierarchical attention mechanism to compute the hierarchical context fed into the decoder.
We report experiments on the RotoWire benchmark BIBREF10 which contains around $5K$ statistical tables of NBA basketball games paired with human-written descriptions. Our model is compared to several state-of-the-art models. Results show that the proposed architecture outperforms previous models on BLEU score and is generally better on qualitative metrics.
In the following, we first present a state-of-the art of data-to-text literature (Section 2), and then describe our proposed hierarchical data encoder (Section 3). The evaluation protocol is presented in Section 4, followed by the results (Section 5). Section 6 concludes the paper and presents perspectives.

Designing data-to-text models gives rise to two main challenges: 1) understanding structured data and 2) generating associated descriptions.
Answer: 1) understanding structured data and 2) generating associated descriptions