T ask2 D ial: A Novel Task and Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents (2022)

Abstract: This paper proposes a novel task on commonsense-enhanced task-based dialogue grounded in documents and describes the Task2Dial dataset, a novel dataset of document-grounded task-based dialogues, where an Information Giver (IG) provides instructions (by consulting a document) to an Information Follower (IF), so that the latter can successfully complete the task. In this unique setting, the IF can ask clarification questions which may not be grounded in the underlying document and require commonsense knowledge to be answered. The Task2Dial dataset poses new challenges: (1) its human reference texts show more lexical richness and variation than other document-grounded dialogue datasets; (2) generating from this set requires paraphrasing as instructional responses might have been modified from the underlying document; (3) requires commonsense knowledge, since questions might not necessarily be grounded in the document; (4) generating requires planning based on context, as task steps need to be provided in order. The Task2Dial dataset contains dialogues with an average 18.15 number of turns and 19.79 tokens per turn, as compared to 12.94 and 12 respectively in existing datasets. As such, learning from this dataset promises more natural, varied and less template-like system utterances.

## introduction goal and task oriented dialogue systems enable users to complete tasks, such as restaurant reservations and travel booking, through conversation @xcite . traditionally, goal-oriented dialogue is based on domain-specific database schemas @xcite , however, encoding all domain information can be prohibitive since most domain knowledge exists in some unstructured format, such as documents @xcite , grounding dialogue in documents is a promising direction for several tasks. here, we propose a new task for document-grounded dialogue, task2dial, which aims at generating instructions grounded in a document so that the receiver of the instructions can complete a task. this task requires following steps in a pre-specified order, invoking every day communication characteristics, such as asking for clarification, questions or advice, which might require the use of commonsense knowledge. the proposed task is different to existing documentgrounded tasks such as coqa @xcite in the sense that it goes beyond question answering grounded in a document, as answers might require commonsense knowledge and the underlying information might not be present in the document. at the same time, this challenging task aims to accommodate task-based dialogue, where the information follower has to comprehend (and confirm) all steps for completing the task. inspired by previous work on documentgrounded dialogue @xcite @xcite , commonsense-enhanced natural language generation (nlg) @xcite , referring expressions generation @xcite , concept acquisition @xcite , and task-based/instructional dialogue @xcite , we aim to capture two different types of knowledge: (1) document-level procedural context, i.e. what is the next step; (2) commonsense, i.e. answering questions that are not available in the document, as demonstrated in figure 1 . the task is designed as an instruction-following scenario with an information giver (ig) and an information follower (if), inspired partly by the give challenge @xcite . the ig has access to the recipe and gives instructions to the if. the ig might choose to omit irrelevant information, simplify the wording or provide it as is. the if will either 'follow' the task by providing confirmation that they have understood the instruction or ask for further information. the ig might have to rely on information outside the given document, in other words the ig will rely on their common sense to enhance understanding and success of the task. ## theoretical framework the proposed task and associated dataset have connections to several lines of research in task and goal oriented dialogue, dialogue tracking and planning, document-grounded dialogue and commonsense reasoning. we next review related work in these areas while grounding our work. ## task2dial the task2dial dataset includes (1) a set of recipe documents; and (2) conversations between an ig and an if, which are grounded in the associated recipe documents. figure 2 presents sample utterances from a dialogue along with the associated recipe. it demonstrates some important features of the dataset, such as mentioning entities not present in the recipe document; re-composition of the original text to focus on the important steps; and the break down of the recipe into manageable and appropriate steps. following recent efforts in the field to standardise nlg research @xcite , we have made the dataset freely available foot_0 . ## dataset analysis this section presents overall statistics of the task2dial dataset. we compare our dataset to the doc2dial dataset, although the latter focuses on a different domain. employing research assistants to collect and annotate data rather than using crowdsourcing platforms meant that no dialogues were discounted from the dataset. however, a preevaluation check was performed on the dataset before statistical analysis to reduce spelling and grammatical errors that may affect the results of the lexical analysis. size ## related work this research considers the development of a dgds for instruction-giving task-based dialogue. the work is inspired by previous research in dgds: doc2dial @xcite focuses on information seeking scenarios where the interaction between an assisting agent and a user is modelled as a a 3 ttr and msttr have been computed using https: //github.com/lsys/lexicalrichness . ## 6 discussion & conclusions in this paper, we introduce the task2dial dataset of task-based document-grounded conversations with everyday speech characteristics, between an ig and if during a cooking task. we further extend previous work in dgds in order to emulate the unpredictability of human-human conversations in instruction giving that do not necessarily follow a tight schema of sequential instruction giving. instead, other discourse and dialogue phenomena might take place such as clarification questions. we further considered the aforementioned challenges of modelling dialogue for instruction-giving tasks with a focus on state-tracking, task planning, and commonsense reasoning and proposed a new task and associated dataset. our proposed task aims to motivate research for modern dialogue systems that address the following challenges. firstly, modern dialogue systems should be flexible and allow for "off-script" scenarios in order to emulate real-world phenomena, such as the ones present in human-human communication. this will require new ways of encoding user indents and new approaches to dialogue management in general. secondly, as dialogue systems find different domain applications, the complexity of the dialogues might increase as well as the reliance of domain knowledge that can be encoded in structured or unstructured ways, such as documents, databases etc. many applications, might require access to different domain knowledge sources in a course of a dialogue. finally, as we design more complex dialogue systems, commonsense will play an essential part, with models required to perform reasoning with background commonsense knowledge, and generalise to tackle unseen concepts, similarly to @xcite . in the future, we aim to benchmark and evaluate a dialogue system based on the task2dial dataset and the chefbot (strathearn and gkatzia, 2021), and extend this approach to a human-robot interaction (hri) scenario.