H ybri D ialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data (2022)

Abstract: A pressing challenge in current dialogue systems is to successfully converse with users on topics with information distributed across different modalities. Previous work in multiturn dialogue systems has primarily focused on either text or table information. In more realistic scenarios, having a joint understanding of both is critical as knowledge is typically distributed over both unstructured and structured forms. We present a new dialogue dataset, HybriDialogue, which consists of crowdsourced natural conversations grounded on both Wikipedia text and tables. The conversations are created through the decomposition of complex multihop questions into simple, realistic multiturn dialogue interactions. We propose retrieval, system state tracking, and dialogue response generation tasks for our dataset and conduct baseline experiments for each. Our results show that there is still ample opportunity for improvement, demonstrating the importance of building stronger dialogue systems that can reason over the complex setting of informationseeking dialogue grounded on tables and text.

## introduction when creating dialogue systems, researchers strive to enable fluent free-text interactions with users on a number of topics. these systems can be utilized to navigate users over the vast amount of online content to answer the user's question. current systems may search for information within text passages. however, knowledge comes in many forms other than text. the ability to understand multiple knowledge forms is critical in developing more general-purpose and realistic conversational models. tables often convey information that cannot be efficiently captured via text, such as structured relational representations between multiple entities across different categories @xcite @xcite . on the other hand, text may contain more detailed information regarding a specific entity. thus, dialogue systems must be able to effectively incorporate and reason across both modalities to yield the best performance in the real world. while there are several existing datasets targeted at dialogue systems @xcite @xcite , these are limited to either table-only or text-only information sources. as a result, current dialogue systems may fail to respond correctly in situations that require combined tabular and textual knowledge. to advance the current state of dialogue systems, we create hybridialogue foot_0 . our dataset is an information-seeking dialogue dataset grounded on structured and unstructured knowledge from tables and text. hybridialogue, or hydi, is constructed by decomposing the complex and artificial multihop questions in ott-qa @xcite which may not reflect real-life queries. we transform these into a series of simple and more realistic intermediate questions regarding tables and text that lead to and eventually answer the multihop question. hybridialogue contains conversations written by crowdsourced workers in a freeflowing and natural dialogue structure that answer these simpler questions and the complex question as well. we provide an example dialogue from our dataset in figure 1 . we also propose several tasks for hybridialogue that illustrate the usage of an information-seeking dialogue system trained on the dataset. these tasks include retrieval, system state tracking, and dialogue generation. together, they demonstrate the challenges with respect to dialogue systems and the necessity for a dataset such as hybridialogue to further research in this space. our contributions are as follows: our baseline experiments demonstrate opportunities to improve current state-of-the-art models in these various tasks and the overall information-seeking dialogue setting. ## related work related work in the space of dialogue-based question-answering can be split into two areas: question-answering systems and informationgrounded dialogue. we provide a comparison of the related datasets in question-answering as question-answering (qa) is one of the long-established nlp tasks, there are numerous existing datasets related to this task. recently, qa datasets have been incorporating new modalities. the recipe-qa @xcite dataset is comprised of question-answer pairs targeted at both image and text. ott-qa @xcite qa @xcite both contain complex multihop questions with answers appearing in both text and tabular formats. several datasets are also targeted at the open-domain question-answering task such as triviaqa, hotpotqa, and natural questions @xcite @xcite . while single-turn question-answering is valuable, the dialogue setting is more interesting as it proposes many new challenges, such as requiring conversational context, reasoning, and naturalness. conversational question-answering several question-answering datasets contain question and answer pairs within a conversational structure. coqa @xcite and doqa (campos et al., 2020) both contain dialogues grounded with knowledge from wikipedia pages, faq pairs, and other domains. sharc @xcite employs a decomposition strategy where the task is to ask follow-up questions to understand the user's background when answering the original question. however, sharc is limited to rule-based reasoning and 'yes' or 'no' answer types. sqa @xcite provides a tabular-type dataset, consisting of the decomposition of wikitable questions. each decomposed answer is related to a cell or column of cells in a particular table. in these datasets, knowledge is limited to a single modality. in comparison, our dataset poses a more challenging yet realistic setting, where knowledge over structured tables and unstructured text is required to provide reasonable answers to the conversational questions. while the previous datasets contain samples written in a conversational structure, the answers are not necessarily presented in this way; they will instead formulate simple and short answers that do not emulate a human dialogue. our dataset, therefore, extends conversational questionanswering and falls into the dialogue space. hy-bridialogue contains natural dialogues with strongly related question-answer pair interactions whose answers are longer than the exact answer string. this models real-world occurrences in which a person wants to ask follow-up questions after their initial question has been answered. dialogue generation among the dialogue datasets that leverage structured (tables and knowledge graphs) knowledge, some @xcite use conversational data from twitter or reddit and contain dialogues relying on external knowledge graphs such as freebase @xcite or concept-net @xcite . on the other hand, open-dialkg @xcite , duconv @xcite , dykgchat @xcite , and kd-conv @xcite collect conversations that are explicitly related to the paired external knowledge graphs. other related work revolves around task-oriented dialogues that are grounded on tables. for example, kvret @xcite and multi-woz @xcite @xcite provide tables that require an assistant to interact with users and complete a task. dialogue datasets that are grounded on unstructured knowledge include cmu_dog @xcite , which is composed of conversations regarding popular movies and their corresponding simplified wikipedia articles. on the other hand, wizard-of-wikipedia (wow) @xcite and topical-chat @xcite simulate the human-human conversations through wizard-apprentice, in which the apprentice tries to learn information from the wizard. our proposed task shares a similar idea with wizard-of-wikipedia and topical-chat in terms of asymmetric information among participants. however, we focus more on information-seeking dialogues grounded on both structured and unstructured knowledge, which provides abundant and heterogeneous information, and requires joint reasoning capabilities using both modalities. ## tasks and baseline models we outline three different tasks in the following sections: retrieval, system state tracking, and dialogue generation. together, these tasks formulate a pipeline dialogue system grounded on both structured and unstructured knowledge from tables and text. the first step of the system is to retrieve the correct wikipedia reference given the first question in the dialogue. as the conversation continues, the system must be able to track the state of the conversation in order to obtain the correct information from the wikipedia reference for the user. finally, the system will need to generate a natural conversational response to communicate with the user at each turn. thus, following each of these tasks in order simulates the pipeline system with our dataset. we describe each of these tasks and their respective models in detail below.