Unsupervised Domain Adaptation on Question-Answering System with Conversation Data (2022)

Abstract: Machine reading comprehension (MRC) is a task for question answering that finds answers to questions from documents of knowledge. Most studies on the domain adaptation of MRC require documents describing knowledge of the target domain. However, it is sometimes difficult to prepare such documents. The goal of this study was to transfer an MRC model to another domain without documents in an unsupervised manner. Therefore, unlike previous studies, we propose a domain-adaptation framework of MRC under the assumption that the only available data in the target domain are human conversations between a user asking questions and an expert answering the questions. The framework consists of three processes: (1) training an MRC model on the source domain, (2) converting conversations into documents using document generation (DG), a task we developed for retrieving important information from several human conversations and converting it to an abstractive document text, and (3) transferring the MRC model to the target domain with unsupervised domain adaptation. To the best of our knowledge, our research is the first to use conversation data to train MRC models in an unsupervised manner. We show that the MRC model successfully obtains question-answering ability from conversations in the target domain.

## introduction conversation agents such as siri, as well as search engines, such as google, have been increasing the scope of user questions in which they can provide direct answers to questions that can be extracted from web pages. providing answers directly from a structured text is often referred to as machine reading comprehension (mrc). benefiting from deep learning technology, mrc is a question-answering (qa) task that has been extensively studied @xcite . mrc is used to find an answer position in a document to answer a given question. a number of large corpora have played a critical role in advancing mrc research @xcite @xcite . many mrc studies have focused on developing new model structures by introducing a new end-to-end neural network model to obtain state-of-the-art performance @xcite @xcite @xcite . however, these state-of-the-art models were evaluated in one domain. in fact, it has been proven that the generalization capabilities of mrc models do not perform well on different datasets @xcite . unsupervised domain adaptation is an approach to cope with transferring knowledge from a source domain to a different unlabeled target domain @xcite . to provide labels for a new domain dataset, question generation is commonly used to create synthetic data consisting of questionanswer pairs from documents of the target domain @xcite , so that an mrc model can be trained with both data from the source domain and syntactic data from the target domain @xcite @xcite @xcite . one critical issue in unsupervised domain adaptation for mrc is that previous studies assumed that the input documents must be available in the target domain. there are also many types of information (not limited to the document) in a real-word scenario. to apply mrc to such information, it is necessary to convert the information into documents. however, this conversion is not an easy task. let us take a case in customer service support. human operators in a customer service usually refer to "manual documents" containing necessary knowledge to answer customer questions. the manual usually has limited information. thus, when there is no information to answer questions, the operators pass the call to a supervisor, and the su-pervisor continues to talk with the customer to answer the question. this procedure is called "escalation". the frequency of escalation is not trivial. moreover, the number of supervisors is usually few, thus it is necessary to reduce escalations. it is obvious that conversations between supervisors and customers have plenty of information that is not included in the document. if we add new information to the document based on supervisor-customer conversations, the mrc task can answer more varied questions. in domain adaptation for mrc, in which the conversation between an expert and user is the only available data in the target domain, has become a new challenge. the user asks questions and the expert answers the questions. to address this challenge, we propose a framework of domain adaptation of mrc. this framework consists of three processes: (1) training an mrc model on the source domain, (2) converting conversations into documents using document generation (dg), which is a task we developed for retrieving important information from several human conversations and converting it to an abstractive document text, and (3) transferring the mrc model to the target domain with unsupervised domain adaptation, which consists of two stages; self-training and discriminative learning. our contributions are summarized as follows: ## proposed framework the main objective of our research is to develop an mrc technique with which the mrc model can be automatically updated on the basis of human conversations. to achieve this, we add dg to the mrc pipeline. the role of dg is to convert human conversations to documents. the generated documents are then added to the training data of the mrc model. let us assume that we have two different types of domain data: source domain that has conversations and corresponding documents and target domain that has only conversations. if we have an mrc model trained with source domain data, our goal is to update the model to cover target-domain questions. however, the target domain has no document related to the conversations. thus, the mrc model should be trained with the only available conversation data in the target domain. as shown in fig. 1 , our framework consists of the following three processes. 1. training an mrc model with answer spans for given questions and corresponding documents data in the source domain. ## conclusion we proposed a framework of unsupervised domain adaptation of mrc in which the only available data are unlabeled human conversations in the target domain. dg, which is a task in the framework, converts a given conversation into a document including conversational context. we also tackled a new challenge of conducting domain adaptation from the source domain with a structured document to a new domain with an unstructured document. we showed that only self-learning does not always improve accuracy. however, discriminative learning with self-learning successfully improved conversational-based mrc domain adaptation.