CA i RE in D ial D oc21: Data Augmentation for Information Seeking Dialogue System (2021)

Abstract: Information-seeking dialogue systems, including knowledge identification and response generation, aim to respond to users with fluent, coherent, and informative responses based on users’ needs, which. To tackle this challenge, we utilize data augmentation methods and several training techniques with the pre-trained language models to learn a general pattern of the task and thus achieve promising performance. In DialDoc21 competition, our system achieved 74.95 F1 score and 60.74 Exact Match score in subtask 1, and 37.72 SacreBLEU score in subtask 2. Empirical analysis is provided to explain the effectiveness of our approaches.

## introduction recent progress in research has opened up real-life applications of dialogue systems @xcite , of which informationseeking dialogue systems are one of the major types. the goal of such dialogue systems is to provide fluent and coherent responses with sufficient information to users based on their needs, retrieving information using the dialogue history. the performance of an information-seeking dialogue system can be evaluated from three aspects: (1) user utterance understanding, (2) relevant knowledge retrieval, and (3) agent response generation @xcite . this paper presents work on the dialdoc-21 shared task, which is to teach a dialogue system to identify the most relevant knowledge in the associated document for generating agent responses in natural language. it is composed of two subtasks: knowledge identification (ki) to retrieve the knowledge from the document, and response generation (rg) to generate an agent utterance utilizing the retrieved knowledge. to tackle this problem, we leverage the pretrained language models from @xcite and @xcite and explore data augmentation methods with several training techniques so as to avoid over-fitting to the dialdoc datasets and to teach the model the general pattern of the task. ensemble and post-processing are conducted to further improve the model performance. experimental results show that data augmentation is a simple but effective approach for knowledge identification in information-seeking dialogue systems @xcite , while bringing improvement to response generation at the same time. in the dialdoc-21 competition, our system achieved 74.95 of f1 score and 60.74 of exact match in subtask 1, and 37.72 sacrebleu score @xcite in subtask 2 foot_0 . besides doc2dial, several other datasets are leveraged for augmentation, as follows: mrqa 2019 shared task dataset is a collection of multiple reading comprehension datasets for evaluating the generalization ability of qa models. six datasets are assigned to the training split, model initialization training data method roberta mrqa roberta large mrqa pt roberta mrqas roberta large mrqa small pt roberta cqa roberta large doc2dial, cqa ft roberta f(cqa) roberta cqa doc2dial ft roberta f(mrqa) roberta mrqa doc2dial ft roberta cqa(mrqa) roberta mrqa doc2dial, cqa ft roberta cqa(mrqas) roberta mrqas doc2dial, cqa ft roberta f(cqa(mrqas)) roberta cqa(mrqas) doc2dial ft roberta all roberta large doc2dial, cqa, and mrqa ft conversational qa (cqa) datasets we also introduce three cqa datasets, coqa @xcite , quac @xcite , and doqa @xcite , in the shared task because of their similar settings to the ki process. wizard-of-wikipedia (wow) is a commonlyused knowledge-grounded dialogue dataset @xcite . it aims at providing content-full responses to user utterances based on wikipedia documents. ## methodology we utilize a series of data-augmentation approaches to enable the model to obtain better representations on both dialogue context and document context and learn a general pattern of the task with less domain bias. namely, we have a two-stage training paradigm, the first step is pretraining (pt) to have a better model initialization, and the second step is fine-tuning (ft) to adapt to dialdoc task. for each step, we can apply the multi-task learning (mtl) strategy if we have multiple datasets by making the datasets format uniform and treat samples equally. as reported in @xcite , a model trained on multiple dataset under similar tasks, is supposed to provide a better initialization for further fine-tuning and is capable of generalizing to the data samples in other domains. thus, we expect a model trained with mtl in the first step to offer a better initialization and in the second step to reduce the domain bias and avoid overfitting. ## related work conversational qa is a type of reading comprehension task that requires understanding not only the question but also the previous conversation turns. various datasets have been introduced in recent years, and many of them restrict answers to be extraction of a span from the reference document, while the others allow free-form responses @xcite @xcite . in addition to the works to enrich the contents of open-domain conversations by controllable generation @xcite , the knowledge grounded dialogue task aims to offer more informative conversation by leveraging an external knowledge source @xcite . relevant knowledge selection is the key to improving the whole system, and very recently, latent variable models have been attracting more attention for this purpose @xcite @xcite @xcite . ## conclusion in this paper, we utilize data augmentation methods and several training techniques with pre-trained language models to tackle the challenge of the