Venue: NAACL 2019

# Pragmatically Informative Text Generation

Sheng Shen\({}^{\dagger}\) Daniel Fried\({}^{\dagger}\) Jacob Andreas\({}^{\ddagger}\) Dan Klein\({}^{\dagger}\)

\({}^{\dagger}\)Computer Science Division, UC Berkeley

\({}^{\ddagger}\)Computer Science and Artificial Intelligence Laboratory, MIT

{sheng.s,dfried,klein}@berkeley.edu, jda@mit.edu

###### Abstract

We improve the informativeness of models for conditional text generation using techniques from computational pragmatics. These techniques formulate language production as a game between speakers and listeners, in which a speaker should generate output text that a listener can use to correctly identify the original input that the text describes. While such approaches are widely used in cognitive science and grounded language learning, they have received less attention for more standard language generation tasks. We consider two pragmatic modeling methods for text generation: one where pragmatics is imposed by information preservation, and another where pragmatics is imposed by explicit modeling of distractors. We find that these methods improve the performance of strong existing systems for abstractive summarization and generation from structured meaning representations.

## 1 Introduction

Computational approaches to pragmatics cast language generation and interpretation as game-theoretic or Bayesian inference procedures Golland et al. (2010); Frank and Goodman (2012). While such approaches are capable of modeling a variety of pragmatic phenomena, their main application in natural language processing has been to improve the _informativeness_ of generated text in grounded language learning problems Monroe et al. (2018). In this paper, we show that pragmatic reasoning can be similarly used to improve performance in more traditional language generation tasks like generation from structured meaning representations (Figure 1) and summarization.

Our work builds on a line of learned Rational Speech Acts (RSA) models Monroe and Potts (2015); Andreas and Klein (2016), in which generated strings are selected to optimize the behavior of an embedded listener model. The canonical presentation of the RSA framework Frank and Goodman (2012) is grounded in reference resolution: models of speakers attempt to describe referents in the presence of distractors, and models of listeners attempt to resolve descriptors to referents. Recent work has extended these models to more complex groundings, including images Mao et al. (2015) and trajectories Fried et al. (2018). The techniques used in these settings are similar, and the primary intuition of the RSA framework is preserved: from the speaker's perspective, a good description is one that picks out, as discriminatively as possible, the content the speaker intends for the listener to identify.

Outside of grounding, cognitive modeling Frank et al. (2009), and targeted analysis of linguistic phenomena Orita et al. (2015), rational speech acts models have seen limited application in the natural language processing literature. In this work we show that they can be extended

Figure 1: Example outputs of our systems on the E2E generation task. While a base sequence-to-sequence model (\(S_{0}\), Sec. 2) fails to describe all attributes in the input meaning representation, both of our pragmatic systems (\(S_{1}^{R}\), Sec. 3.1 and \(S_{1}^{D}\), Sec. 3.2) and the human-written reference do.



to a distinct class of language generation problems that use as referents structured descriptions of linguistic content, or other natural language texts. In accordance with the maxim of quantity [14] or the Q-principle [23], pragmatic approaches naturally correct underinformativeness problems observed in state-of-the-art language generation systems (\(S_{0}\) in Figure 1).

We present experiments on two language generation tasks: generation from meaning representations [56] and summarization. For each task, we evaluate two models of pragmatics: the reconstructor-based model of Fried et al. fried2018learning and the distractor-based model of CohnGordon et al. gordon2018language. Both models improve performance on both tasks, increasing rouge scores by 0.2-0.5 points on the CNN/Daily Mail abstractive summarization dataset and bleu scores by 2 points on the End-to-End (E2E) generation dataset, obtaining new state-of-the-art results.

## 2 Tasks

We formulate a conditional generation task as taking an input \(i\) from a space of possible inputs \(\mathcal{I}\) (e.g., input sentences for abstractive summarization; meaning representations for structured generation) and producing an output \(o\) as a sequence of tokens \((o_{1},\ldots,o_{T})\). We build our pragmatic approaches on top of learned _base speaker_ models \(S_{0}\), which produce a probability distribution \(S_{0}(o~{}|~{}i)\) over output text for a given input. We focus on two conditional generation tasks where the information in the input context should largely be preserved in the output text, and apply the pragmatic procedures outlined in Sec. 3 to each task. For these \(S_{0}\) models we use systems from past work that are strong, but may still be underminformative relative to human reference outputs (e.g., Figure 1).

Meaning RepresentationsOur first task is generation from structured meaning representations (MRs) containing attribute-value pairs [57]. An example is shown in Figure 1, where systems must generate a description of the restaurant with the specified attributes. We apply pragmatics to encourage output strings from which the input MR can be identified. For our \(S_{0}\) model, we use a publicly-released neural generation system [59] that achieves comparable performance to the best published results in Dusek et al. dussek2018language.

Abstractive SummarizationOur second task is multi-sentence document summarization. There is a vast amount of past work on summarization [57]; recent neural models have used large datasets (e.g., Hermann et al. hermann2015multilingual) to train models in both the extractive [10, 12] and abstractive [26, 27] settings. Among these works, we build on the recent abstractive neural summarization system of Chen and Bansal chen2018neural. First, this system uses a sentence-level extractive model rnn-ext to identify a sequence of salient sentences \(i^{(1)},\ldots i^{(P)}\) in each source document. Second, the system uses an abstractive model abs to rewrite each \(i^{(p)}\) into an output \(o^{(p)}\), which are then concatenated to produce the final summary. We rely on the fixed rnn-ext model to extract sentences as inputs in our pragmatic procedure, using abs as our \(S_{0}\) model and applying pragmatics to the \(i^{(p)}\to o^{(p)}\) abstractive step.

## 3 Pragmatic Models

To produce informative outputs, we consider pragmatic methods that extend the _base speaker_ models, \(S_{0}\), using _listener_ models, \(L\), which produce a distribution \(L(i~{}|~{}o)\) over possible inputs given an output. Listener models are used to derive _pragmatic speakers_, \(S_{1}(o~{}|~{}i)\), which produce output that has a high probability of making a listener model \(L\) identify the correct input. There are a large space of possible choices for designing \(L\) and deriving \(S_{1}\); we follow two lines of past work which we categorize as _reconstructor-based_ and _distractor-based_. We tailor each of these pragmatic methods to both our two tasks by developing reconstructor models and methods of choosing distractors.

### Reconstructor-Based Pragmatics

Pragmatic approaches in this category [56, 14] rely on a _reconstructor listener_ model \(L^{R}\) defined independently of the speaker. This listener model produces a distribution \(L^{R}(i~{}|~{}o)\) over all possible input contexts \(i\in\mathcal{I}\), given an output description \(o\). We use sequence-to-sequence or structured classification models for \(L^{R}\) (described below), and train these models on the same data used to supervise the \(S_{0}\) models.

The listener model and the base speaker model 

together define a _pragmatic speaker_, with output score given by:

\[S^{R}_{1}(o\mid i)=L^{R}(i\mid o)^{\lambda}\cdot S_{0}(o\mid i)^{1-\lambda} \tag{1}\]

where \(\lambda\) is a _rationality parameter_ that controls how much the model optimizes for discriminative outputs (see Monroe et al. (2017) and Fried et al. (2018) for a discussion). We select an output text sequence \(o\) for a given input \(i\) by choosing the highest scoring output under Eq. 1 from a set of candidates obtained by beam search in \(S_{0}(\cdot\mid i)\).

Meaning RepresentationsWe construct \(L^{R}\) for the meaning representation generation task as a multi-task, multi-class classifier, defining a distribution over possible values for each attribute. Each MR attribute has its own prediction layer and attention-based aggregation layer, which conditions on a basic encoding of \(o\) shared across all attributes. See Appendix A.1 for architecture details. We then define \(L^{R}(i\mid o)\) as the joint probability of predicting all input MR attributes in \(i\) from \(o\).

SummarizationTo construct \(L^{R}\) for summarization, we train an abs model (of the type we use for \(S_{0}\), Chen and Bansal (2018)) but in reverse, i.e., taking as input a sentence in the summary and producing a sentence in the source document. We train \(L^{R}\) on the same heuristically-extracted and aligned source document sentences used to train \(S_{0}\)Chen and Bansal (2018).

### Distractor-Based Pragmatics

Pragmatic approaches in this category Frank and Goodman (2012); Andreas and Klein (2016); Vedantam et al. (2017); Cohn-Gordon et al. (2018) derive pragmatic behavior by producing outputs that distinguish the input \(i\) from an alternate _distractor_ input (or inputs). We construct a distractor \(\widetilde{\imath}\) for a given input \(i\) in a task-dependent way.1

Footnote 1: In tasks such as contrastive captioning or referring expression generation, these distractors are given; for the conditional generation task, we will show that pragmatic behavior can be obtained by constructing or selecting a single distractor that contrasts with the input \(i\).

We follow the approach of Cohn-Gordon et al. (2018), outlined briefly here. The base speakers we build on produce outputs incrementally, where the probability of \(o_{t}\), the word output at time \(t\), is conditioned on the input and the previously generated words: \(S_{0}(o_{t}\mid i,o_{<t})\). Since the output is generated incrementally and there is no separate listener model that needs to condition on entire output decisions, the distractor-based approach is able to make pragmatic decisions at each word rather than choosing between entire output candidates (as in the reconstructor approaches).

The listener \(L^{D}\) and pragmatic speaker \(S^{D}_{1}\) are derived from the base speaker \(S_{0}\) and a belief distribution \(p_{t}(\cdot)\) maintained at each timestep \(t\) over the possible inputs \(\mathcal{I}^{D}\):

\[L^{D}(i\mid o_{<t}) \propto S_{0}(o_{<t}\mid i)\cdot p_{t-1}(i) \tag{2}\] \[S^{D}_{1}(o_{t}\mid i,o_{<t}) \propto L^{D}(i\mid o_{<t})^{\alpha}\cdot S_{0}(o_{t}\mid i,o_{< t})\] (3) \[p_{t}(i) \propto S_{0}(o_{t}\mid i,o_{<t})\cdot L^{D}(i\mid o_{<t}) \tag{4}\]

where \(\alpha\) is again a rationality parameter, and the initial belief distribution \(p_{0}(\cdot)\) is uniform, i.e., \(p_{0}(i)=p_{0}(\,\widetilde{\imath}\,)=0.5\). Eqs. 2 and 4 are normalized over the true input \(i\) and distractor \(\widetilde{\imath}\); Eq. 3 is normalized over the output vocabulary. We construct an output text sequence for the pragmatic speaker \(S^{D}_{1}\) incrementally using beam search to approximately maximize Eq. 3.

Meaning RepresentationsA distractor MR is automatically constructed for each input to be the most distinctive possible against the input. We construct this distractor by masking each present input attribute and replacing the value of each nonpresent attribute with the value that is most frequent for that attribute in the training data. For example, for the input MR in Figure 1, the distractor is NearBurger King].

SummarizationFor each extracted input sentence \(i^{(p)}\), we use the previous extracted sentence \(i^{(p-1)}\) from the same document as the distractor input \(\widetilde{\imath}\) (for the first sentence we do not use a distractor). This is intended to encourage outputs \(o^{(p)}\) to contain distinctive information against other summaries produced within the same document.

## 4 Experiments

For each of our two conditional generation tasks we evaluate on a standard benchmark dataset, following past work by using automatic evaluation against human-produced reference text. We choose hyperparameters for our models (beam size, and parameters \(\alpha\) and \(\lambda\)) to maximize task metrics on each dataset's development set; see Appendix A.2 for the settings used.

 



### Meaning Representations

We evaluate on the E2E task of generation from meaning representations containing restaurant attributes (Novikova et al., 2017). We report the task's five automatic metrics: bleu(Papineni et al., 2002), nist(Doddington, 2002), meteor(Lavie and Agarwal, 2007), rouge-l(Lin, 2004) and cider(Vedantam et al., 2015).

Table 1 compares the performance of our base \(S_{0}\) and pragmatic models to the baseline T-Gen system (Dusek and Jurcicek, 2016) and the _best previous_ result from the 20 primary systems evaluated in the E2E challenge (Dusek et al., 2018). The systems obtaining these results encompass a range of approaches: a template system (Puzikov and Gurevych, 2018), a neural model (Zhang et al., 2018), models trained with reinforcement learning (Gong, 2018), and systems using ensembling and reranking (Juraska et al., 2018). To ensure that the benefit of the reconstructor-based pragmatic approach, which uses two models, is not due solely to a model combination effect, we also compare to an ensemble of two base models (\(S_{0}\times 2\)). This ensemble uses a weighted combination of scores of two independently-trained \(S_{0}\) models, following Eq. 1 (with weights tuned on the development data).

Both of our pragmatic systems improve over the strong baseline \(S_{0}\) system on all five metrics, with the largest improvements (2.1 bleu, 0.2 nist, 0.8 meteor, 1.5 rouge-l, and 0.1 cider) from the \(S^{R}_{1}\) model. This \(S^{R}_{1}\) model outperforms the previous best results obtained by any system in the E2E challenge on bleu, nist, and cider, with comparable performance on meteor and rouge-l.

### Abstractive Summarization

We evaluate on the CNN/Daily Mail summarization dataset (Hermann et al., 2015; Nallapati et al., 2016), using See et al.'s (2017) non-anonymized preprocessing. As in previous work (Chen and Bansal, 2018), we evaluate using rouge and meteor.

Table 2 compares our pragmatic systems to the base \(S_{0}\) model (with scores taken from Chen and Bansal (2018); we obtained comparable performance in our reproduction3), an ensemble of two of these base models, and the _best previous_ abstractive summarization result for each metric on this dataset (Celikyilmaz et al., 2018; Paulus et al., 2018; Chen and Bansal, 2018). We also report two extractive baselines: _Lead-3_, which uses the first three sentences of the document as the summary (See et al., 2017), and _Inputs_, the concatenation of the extracted sentences used as inputs to our models (i.e., \(i^{(1)},\ldots,i^{(P)}\)).

Footnote 3: We use retrained versions of Chen and Bansal (2018)’s sentence extractor and abstractive \(S_{0}\) models in all our experiments, as well as their n-gram reranking-based inference procedure, replacing scores from the base model \(S_{0}\) with scores from \(S^{R}_{1}\) or \(S^{D}_{1}\) in the respective pragmatic procedures.

The pragmatic methods obtain improvements of 0.2-0.5 in rouge scores and 0.2-1.8 meteor over the base \(S_{0}\) model, with the distractor-based approach \(S^{D}_{1}\) outperforming the reconstructed-based approach \(S^{R}_{1}\). \(S^{D}_{1}\) is strong across all metrics, obtaining results competitive to the best previous abstractive systems.

\begin{table}
\begin{tabular}{p{99.6pt} p{34.1pt} p{34.1pt} p{34.1pt} p{34.1pt}} \hline System & bleu & nist & meteor & r-l & cider \\ \hline T-Gen & 65.93 & 8.61 & 44.83 & 68.50 & 2.23 \\ Best Prev. & 66.19\({}^{\dagger}\) & 8.61\({}^{\ddagger}\) & **45.29\({}^{\ddagger}\)****[FOOTNOTE:$\ddagger$]Footnote $\ddagger$ footnotemark: $



## 5 Analysis

The base speaker \(S_{0}\) model is often underinformative, e.g., for the E2E task failing to mention certain attributes of a MR, even though almost all the training examples incorporate all of them. To better understand the performance improvements from the pragmatic models for E2E, we compute a _coverage ratio_ as a proxy measure of how well content in the input is preserved in the generated outputs. The coverage ratio for each attribute is the fraction of times there is an exact match between the text in the generated output and the attribute's value in the source MR (for instances where the attribute is specified).4

Footnote 4: Note that this measure roughly provides a lower bound on the model’s actual informativeness for each attribute, since the measure does not assign credit for paraphrases.

Figure 2(a) shows coverage ratio by attribute category for all models. The \(S^{R}_{1}\) model increases the coverage ratio when compared to \(S_{0}\) across all attributes, showing that using the reconstruction model score to select outputs does lead to an increase in mentions for each attribute. Coverage ratios increase for \(S^{D}_{1}\) as well in four out of six categories, but the increase is typically less than that produced by \(S^{R}_{1}\).

While \(S^{D}_{1}\) optimizes less explicitly for attribute mentions than \(S^{R}_{1}\), it still provides a potential method to control generated outputs by choosing alternate distractors. Figure 2(b) shows coverage ratios for \(S^{D}_{1}\) when masking only a single attribute in the distractor. The highest coverage ratio for each attribute is usually obtained when masking that attribute in the distractor MR (entries on the main diagonal, underlined), in particular for FamilyFriendly (FF), Food, PriceRange (PR), and Area. However, masking a single attribute sometimes results in decreasing the coverage ratio, and we also observe substantial increases from masking other attributes: e.g., masking either FamilyFriendly or CustomerRating (CR) produces an equal increase in coverage ratio for the CustomerRating attribute. This may reflect underlying correlations in the training data, as these two attributes have a small number of possible values (3 and 7, respectively).

## 6 Conclusion

Our results show that \(S_{0}\) models from previous work, while strong, still imperfectly capture the behavior that people exhibit when generating text; and an explicit pragmatic modeling procedure can improve results. Both pragmatic methods evaluated in this paper encourage prediction of outputs that can be used to identify their inputs, either by reconstructing inputs in their entirety or distinguishing true inputs from distractors, so it is perhaps unsurprising that both methods produce similar improvements in performance. Future work might allow finer-grained modeling of the tradeoff between _under_- and _over_-informativity within the sequence generation pipeline (e.g., with a learned communication cost model) or explore applications of pragmatics for content selection earlier in the generation pipeline.

## Acknowledgments

Thanks to Reuben Cohn-Gordon for many helpful discussions and suggestions. This work was supported by DARPA through the XAI program. DF is supported by a Tencent AI Lab Fellowship.

Figure 2: Coverage ratios for the E2E task by attribute type, estimating how frequently the values for each attribute from the input meaning representations are mentioned in the output text.





## References

* Andreas and Klein (2016) Jacob Andreas and Dan Klein. 2016. Reasoning about pragmatics with neural listeners and speakers. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_.
* Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. _arXiv preprint arXiv:1409.0473_.
* Celikyilmaz et al. (2018) Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In _Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics_, pages 1662-1675, New Orleans, Louisiana. Association for Computational Linguistics.
* Chen and Bansal (2018) Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics_.
* Cheng and Lapata (2016) Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics_, pages 484-494, Berlin, Germany. Association for Computational Linguistics.
* Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_, pages 1724-1734, Doha, Qatar. Association for Computational Linguistics.
* Cohn-Gordon et al. (2018) Reuben Cohn-Gordon, Noah Goodman, and Chris Potts. 2018. Pragmatically informative image captioning with character-level reference. In _Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics_.
* Doddington (2002) George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In _Proceedings of the second international conference on Human Language Technology Research_, pages 138-145. Morgan Kaufmann Publishers Inc.
* Dusek and Jurcicek (2016) Ondrej Dusek and Filip Jurcicek. 2016. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. In _ACL_.
* Dusek et al. (2018) Ondrej Dusek, Jekaterina Novikova, and Verena Rieser. 2018. Findings of the E2E NLG challenge. In _Proceedings of the International Conference on Natural Language Generation_.
* Frank and Goodman (2012) Michael C Frank and Noah D Goodman. 2012. Predicting pragmatic reasoning in language games. _Science_, 336(6084):998-998.
* Frank et al. (2009) Michael C Frank, Noah D Goodman, Peter Lai, and Joshua B Tenenbaum. 2009. Informative communication in word production and word learning. In _Proceedings of the 31st annual conference of the cognitive science society_, pages 1228-1233.
* Fried et al. (2018) Daniel Fried, Jacob Andreas, and Dan Klein. 2018. Unified pragmatic models for generating and following instructions. In _Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics_.
* Golland et al. (2010) Dave Golland, Percy Liang, and Dan Klein. 2010. A game-theoretic approach to generating spatial descriptions. In _Proceedings of the 2010 conference on Empirical Methods in Natural Language Processing_, pages 410-419. Association for Computational Linguistics.
* Gong (2018) Heng Gong. 2018. Technical report for E2E NLG challenge. In _E2E NLG Challenge System Descriptions_.
* Grice (1970) Herbert P Grice. 1970. Logic and conversation.
* Hermann et al. (2015) Karl Moritz Hermann, Tomas Kocisky, Edward Grefenette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In _Advances in Neural Information Processing Systems_.
* Horn (1984) Laurence Horn. 1984. Toward a new taxonomy for pragmatic inference: Q-based and r-based implicature. _Meaning, form, and use in context: Linguistic applications_, 11:42.
* Juraska et al. (2018) Juraj Juraska, Panagiotis Karagiannis, Kevin Bowden, and Marilyn Walker. 2018. A deep ensemble model with slot alignment for sequence-to-sequence natural language generation. In _Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics_, pages 152-162, New Orleans, Louisiana. Association for Computational Linguistics.
* Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In _Proceedings of the Second Workshop on Statistical Machine Translation_, pages 228-231, Prague, Czech Republic. Association for Computational Linguistics.
* Lin (2004) Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In _Text Summarization Branches Out: Proceedings of the ACL-04 Workshop_.
* Mao et al. (2015) Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan Yuille, and Kevin Murphy. 2015. Generation and comprehension of unambiguous object descriptions. _arXiv preprint arXiv:1511.02283_.
* Monroe et al. (2017) Will Monroe, Robert XD Hawkins, Noah D Goodman, and Christopher Potts. 2017. Colors in context: A pragmatic neural model for grounded language understanding. _Transactions of the Association for Computational Linguistics_

Will Monroe, Jennifer Hu, Andrew Jong, and Christopher Potts. 2018. Generating bilingual pragmatic color references. In _Proceedings of the Annual Meeting of the North American Chapter of the Association for Computational Linguistics_.
* Monroe and Potts (2015) Will Monroe and Christopher Potts. 2015. Learning in the Rational Speech Acts model. In _Proceedings of 20th Amsterdam Colloquium_, Amsterdam. ILLC.
* Nallapati et al. (2017) Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In _Proceedings of the Meeting of the Association for the Advancement of Artificial Intelligence_.
* Nallapati et al. (2016) Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In _SIGNLL Conference on Computational Natural Language Learning_, pages 280-290, Berlin, Germany. Association for Computational Linguistics.
* Nenkova and McKeown (2011) Ani Nenkova and Kathleen McKeown. 2011. Automatic summarization. _Foundations and Trends in Information Retrieval_, 5(2-3):103-233.
* Novikova et al. (2017) Jekaterina Novikova, Ondrej Dusek, and Verena Rieser. 2017. The E2E dataset: New challenges for end-to-end generation. In _Proceedings of the Annual SIGdial Meeting on Discourse and Dialogue_.
* Orita et al. (2015) Naho Orita, Eliana Vornov, Naomi Feldman, and Hal Daume III. 2015. Why discourse affects speakers' choice of referring expressions. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, volume 1, pages 1639-1649.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics_, pages 311-318. Association for Computational Linguistics.
* Paulus et al. (2018) Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In _Proceedings of the International Conference on Learning Representations_, volume abs/1705.04304.
* Puzikov and Gurevych (2018) Yevgeniy Puzikov and Iryna Gurevych. 2018. E2E NLG challenge: Neural models vs. templates. In _Proceedings of the International Conference on Natural Language Generation_.
* Rush et al. (2015) Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_, pages 379-389, Lisbon, Portugal. Association for Computational Linguistics.
* See et al. (2017)Bigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics_, pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.
* Vedantam et al. (2017) Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik. 2017. Context-aware captions from context-agnostic supervision.
* Vedantam et al. (2015)Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. CIDEr: Consensus-based image description evaluation. In _Proceedings of the Conference on Computer Vision and Pattern Recognition_.
* Zhang et al. (2018)Biao Zhang, Jing Yang, Qian Lin, and Jinsong Su. 2018. Attention regularized sequence-to-sequence learning for E2E NLG challenge. In _E2E NLG Challenge System Descriptions_ 

Supplemental Material

### Reconstructor Model Details

For the reconstructor-based speaker in the E2E task, we first follow the same data preprocessing steps as Puzikov and Gurevych (2018), which includes a delexicalization module that deals with sparsely occurring MR attributes (name, near) by mapping such values to placeholder tokens.

MRs have only a few possible values for most attributes: six out of eight attributes have fewer than seven unique values, and the remaining two attributes (name, near) are handled by our \(S_{0}\) and \(S^{D}_{1}\) using delexicalized placeholders, following Puzikov and Gurevych (2018). In this way, the reconstructor only needs to predict the presence of these two attributes with a boolean variable, and other attributes with the corresponding categorical variable. We use a one layer bi-directional GRU (Cho et al., 2014) for the shared sentence encoder. We concatenate the latent vectors from both directions to construct a bi-directional encoded vector \(h_{i}\) for every single word vector \(d_{i}\) as:

\[\overset{\rightarrow}{h_{i}}=\overrightarrow{GRU}(d_{i},h_{i-1}),\overset{ \leftarrow}{h_{i}}=\overbrace{GRU}(d_{i},h_{i+1})\]

\[h_{i}=[\overset{\rightarrow}{h_{i}},\overset{\leftarrow}{h_{i}}],i\in[1,L]\]

Since not all words contribute equally to predicting each MR attribute, we thus use an attention mechanism Bahdanau et al. (2014) to determine the importance of every single word. The aggregated sentence vector for task \(k\) is calculated by

\[a_{i}^{(k)}=\frac{exp(W_{a}^{(k)}h_{i})}{\sum_{j=1}^{L}exp(W_{a}^{(k)}h_{j})}, v^{(k)}=\sum_{i=1}^{L}a_{i}^{(k)}h_{i},\]

The task-specific sentence representation is then used as input to \(k\) layers with softmax outputs, returning a probability vector \(Y^{(k)}\) for each of the \(k\) MR attributes.

### Hyperparameters

For structured generation, we use beam size 10, \(\lambda=0.4\), and \(\alpha=0.2\), tuned to maximize the normalized average of all five metrics on the development set.

For abstractive summarization, we use beam size 20, \(\lambda=0.9\), and \(\alpha=1.0\), tuned to maximize rouge-L on the development set