Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems . Avenues like the E2E NLG Challenge have encouraged the development of neural approaches , particularly sequence-tosequence ( Seq2Seq ) models for this problem . The semantic representations used , however , are often underspecified , which places a higher burden on the generation model for sentence planning , and also limits the extent to which generated responses can be controlled in a live system . In this paper , we ( 1 ) propose using tree-structured semantic representations , like those used in traditional rule-based NLG systems , for better discourse-level structuring and sentence-level planning ; ( 2 ) introduce a challenging dataset using this representation for the weather domain ; ( 3 ) introduce a constrained decoding approach for Seq2Seq models that leverages this representation to improve semantic correctness ; and ( 4 ) demonstrate promising results on our dataset and the E2E dataset . Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems . With their end-to-end trainability , neural approaches to natural language generation ( NNLG ) , particularly sequence-to-sequence ( Seq2Seq ) models , have been promoted with great fanfare in recent years ( Wen et al . , 2015 ( Wen et al . , , 2016 ; ; Mei et al . , 2016 ; Kiddon et al . , 2016 ; Dušek and Jurcicek , 2016 ) , and avenues like the recent E2E NLG challenge ( Dušek et al . , 2018 ( Dušek et al . , , 2019 ) ) have made available large datasets to promote the development of these models . Nevertheless , current NNLG models arguably remain inadequate for most real-world task-oriented dialogue systems , given their inability to ( i ) reliably perform common sentence planning and discourse structuring operations ( Reed et al . , 2018 ) , ( ii ) generalize to complex inputs ( Wiseman et al . , 2017 ) , and ( 3 ) avoid generating texts with semantic errors including hallucinated content ( Dušek et al . , 2018 ( Dušek et al . , , 2019 ) ) . 1In this paper , we explore the extent to which these issues can be addressed by incorporating lessons from pre-neural NLG systems into a neural framework . We begin by arguing in favor of enriching the input to neural generators to include discourse relations -long taken to be central in traditional NLG -and underscore the importance of exerting control over these relations when generating text , particularly when using user models to structure responses . In a closely related work , Reed et al . ( 2018 ) , the authors add control tokens ( to indicate contrast and sentence structure ) to a flat input MR , and show that these can be effectively used to control structure . However , their methods are only able to control the presence or absence of these relations , without more fine-grained control over their structure . We thus go beyond their approach and propose using full tree structures as inputs , and generating treestructured outputs as well . This allows us to define a novel method of constrained decoding for standard sequence-to-sequence models for generation , which helps ensure that the generated text contains all and only the specified content , as in classic approaches to surface realization . On the E2E dataset , our experiments demonstrate much better control over CONTRAST relations than using Reed et al . 's method , and also show improved diversity and expressiveness over standard baselines . We also release a new dataset of responses in the weather domain , which includes the JUSTIFY , JOIN and CONTRAST rela- tions , and where discourse-level structures come into play . On both E2E and weather datasets , we show that constrained decoding over our enriched inputs results in higher semantic correctness as well as better generalizability and data efficiency . The rest of this paper is organized as follows : Section 2 describes the motivation for using compositional inputs organized around discourse relations . Section 3 explains our data collection approach and dataset . 2 Section 4 shows how to incorporate compositional inputs into NNLG and describes our constrained decoding algorithm . Section 5 presents our experimental setup and results . 2 Towards More Expressive Meaning Representations We show that using rich tree-structured meaning representations can improve expressiveness and semantic correctness in generation . We also propose a constrained decoding technique that leverages tree-structured MRs to exert precise control over the discourse structure and semantic correctness of the generated text . We release a challenging new dataset for the weather domain and an enriched E2E dataset that include tree-structured MRs . Our experiments show that constrained decoding , together with tree-structured MRs , can greatly improve semantic correctness as well as enhance data efficiency and generalizability .

Challenge: Existing neural natural language generation models remain inadequate for real-world task-oriented dialogue systems due to inabilities of reliable planning, modeling complex inputs and avoiding hallucinations.
Approach: They propose to use tree-structure semantic representation coupled with a constrained decoding method which generates tree-structured outputs and a dataset in the weather domain.
Outcome: Evaluation of the existing and proposed datasets shows that the proposed model achieves better controllability, expressiveness and semantic correctness.