Question: How many systems do they compare their hierarchical model against?
Evidence: We compare our hierarchical model against three systems. For each of them, we report the results of the best performing models presented in each paper.
$\bullet $ Wiseman BIBREF10 is a standard encoder-decoder system with copy mechanism.
$\bullet $ Li BIBREF28 is a standard encoder-decoder with a delayed copy mechanism: text is first generated with placeholders, which are replaced by salient records extracted from the table by a pointer network.
$\bullet $ Puduppully-plan BIBREF12 acts in two steps: a first standard encoder-decoder generates a plan, i.e. a list of salient records from the table; a second standard encoder-decoder generates text from this plan.
$\bullet $ Puduppully-updt BIBREF13. It consists in a standard encoder-decoder, with an added module aimed at updating record representations during the generation process. At each decoding step, a gated recurrent network computes which records should be updated and what should be their new representation.

We test the importance of the input structure by training different variants of the proposed architecture:
$\bullet $ Flat, where we feed the input sequentially to the encoder, losing all notion of hierarchy. As a consequence, the model uses standard attention. This variant is closest to Wiseman, with the exception that we use a Transformer to encode the input sequence instead of an RNN.
$\bullet $ Hierarchical-kv is our full hierarchical model, with traditional hierarchical attention, i.e. where attention over records is computed on the full record encoding, as in equation (SECREF16).
$\bullet $ Hierarchical-k is our full hierarchical model, with key-guided hierarchical attention, i.e. where attention over records is computed only on the record key representations, as in equation (DISPLAY_FORM17).
From a general point of view, we can see from Table TABREF25 that our scenarios obtain significantly higher results in terms of BLEU over all models; our best model Hierarchical-k reaching $17.5$ vs. $16.5$ against the best baseline. This means that our models learns to generate fluent sequences of words, close to the gold descriptions, adequately picking up on domain lingo. Qualitative metrics are either better or on par with baselines. We show in Figure FIGREF29 a text generated by our best model, which can be directly compared to the gold description in Figure FIGREF1. Generation is fluent and contains domain-specific expressions. As reflected in Table TABREF25, the number of correct mentions (in green) outweights the number of incorrect mentions (in red). Please note that, as in previous work BIBREF28, BIBREF12, BIBREF13, BIBREF10, generated texts still contain a number of incorrect facts, as well hallucinations (in blue): sentences that have no basis in the input data (e.g. “[...] he's now averaging 22 points [...]."). While not the direct focus of our work, this highlights that any operation meant to enrich the semantics of structured data can also enrich the data with incorrect facts.
Specifically, regarding all baselines, we can outline the following statements.
$\bullet $ Our hierarchical models achieve significantly better scores on all metrics when compared to the flat architecture Wiseman, reinforcing the crucial role of structure in data semantics and saliency. The analysis of RG metrics shows that Wiseman seems to be the more naturalistic in terms of number of factual mentions (RG#) since it is the closest scenario to the gold value (16.83 vs. 17.31 for resp. Wiseman and Hierarchical-k). However, Wiseman achieves only $75.62$% of precision, effectively mentioning on average a total of $22.25$ records (wrong or accurate), where our model Hierarchical-k scores a precision of $89.46$%, leading to $23.66$ total mentions, just slightly above Wiseman.
$\bullet $ The comparison between the Flat scenario and Wiseman is particularly interesting. Indeed, these two models share the same intuition to flatten the data-structure. The only difference stands on the encoder mechanism: bi-LSTM vs. Transformer, for Wiseman and Flat respectively. Results shows that our Flat scenario obtains a significant higher BLEU score (16.7 vs. 14.5) and generates fluent descriptions with accurate mentions (RG-P%) that are also included in the gold descriptions (CS-R%). This suggests that introducing the Transformer architecture is promising way to implicitly account for data structure.
$\bullet $ Our hierarchical models outperform the two-step decoders of Li and Puduppully-plan on both BLEU and all qualitative metrics, showing that capturing structure in the encoding process is more effective that predicting a structure in the decoder (i.e., planning or templating). While our models sensibly outperform in precision at factual mentions, the baseline Puduppully-plan reaches $34.28$ mentions on average, showing that incorporating modules dedicated to entity extraction leads to over-focusing on entities; contrasting with our models that learn to generate more balanced descriptions.
$\bullet $ The comparison with Puduppully-updt shows that dynamically updating the encoding across the generation process can lead to better Content Ordering (CO) and RG-P%. However, this does not help with Content Selection (CS) since our best model Hierarchical-k obtains slightly better scores. Indeed, Puduppully-updt updates representations after each mention allowing to keep track of the mention history. This guides the ordering of mentions (CO metric), each step limiting more the number of candidate mentions (increasing RG-P%). In contrast, our model encodes saliency among records/entities more effectively (CS metric). We note that while our model encodes the data-structure once and for all, Puduppully-updt recomputes, via the updates, the encoding at each step and therefore significantly increases computation complexity. Combined with their RG-# score of $30.11$, we argue that our model is simpler, and obtains fluent description with accurate mentions in a more human-like fashion.
We would also like to draw attention to the number of parameters used by those architectures. We note that our scenarios relies on a lower number of parameters (14 millions) compared to all baselines (ranging from 23 to 45 millions). This outlines the effectiveness in the design of our model relying on a structure encoding, in contrast to other approach that try to learn the structure of data/descriptions from a linearized encoding.

We compare our hierarchical model against three systems.
Answer: Three.