A Knowledge storage and semantic space alignment Method for Multi-documents dialogue generation (2022)

Abstract: Question Answering (QA) is a Natural Language Processing (NLP) task that can measure language and semantics understanding ability, it requires a system not only to retrieve relevant documents from a large number of articles but also to answer corresponding questions according to documents. However, various language styles and sources of human questions and evidence documents form the different embedding semantic spaces, which may bring some errors to the downstream QA task. To alleviate these problems, we propose a framework for enhancing downstream evidence retrieval by generating evidence, aiming at improving the performance of response generation. Specifically, we take the pre-training language model as a knowledge base, storing documents’ information and knowledge into model parameters. With the Child-Tuning approach being designed, the knowledge storage and evidence generation avoid catastrophic forgetting for response generation. Extensive experiments carried out on the multi-documents dataset show that the proposed method can improve the final performance, which demonstrates the effectiveness of the proposed framework.

## introduction with the rapid and vigorous development of the field of artificial intelligence and language intelligence, question answering (qa) systems has received more and more extensive attention. specifically, the qa system aims to provide precise answers in response to the user's questions in natural language. an essential task in the qa system is conversational question answering and documentgrounded dialogue modeling. the conversational question answering dialogue-like interface that enables interaction between human users and the documentation provides sufficient information. prior work typically formulates the task as a machine reading comprehension task assuming the associated document or text snippet is given, such as quac @xcite , sharc @xcite , coqa @xcite , or-quac @xcite and doc2dial @xcite . one of the difficulties of conversational qa tasks is to model the historical information in the process of system retrieval and generation. the recently released conversational question answering datasets like coqa @xcite and quac @xcite aim to lead a reader to answer the latest question by comprehending the given context passage and the conversation history. as they provide context passages in their task setting, they omit the stage of document retrieval. while on the multidoc2dial @xcite dataset, retrieval is necessary. recently, @xcite extend the quac dataset to a new or-quac dataset by adapting to an open retrieval conversational question answering system (openconvqa), it can retrieve relevant passages from a large collection before inferring the answer, taking into account the conversation qa pairs, which is similar with the mul-tidoc2dial dataset. to enhance the modeling of historical sessions and avoid the problem of weak semantic relatedness between problems and evidence in the retrieval stage. we propose a novel three-stage framework, which stores knowledge and makes alignment in semantic space. specifically, we find that it is inconsistent to search for most question-related evidence only by the inner product of the question and long text of dialogue history. as stated by @xcite about task 2: agent response generation is more difficult than task 1: grounding span prediction, because agent utterance varies in style and is not directly extracted from document content. different language styles and sources lead to different semantic spaces of question and evidence document embedding. as a result, it inspires us ## main method in this section, the overall framework is illustrated in figure 1 , where we will elaborate on the main method for the multidoc2dial task. based on the pre-trained language model, we design a threestage semantic alignment method including the knowledge storage stage, generative document retrieval, and reply generation modules, which are described in turn as follows. ## conclusion in this paper, we propose a generative evidence retrieval method, which transforms the context and problems into possible evidence for further retrieval. specifically, we first use pegasus to completely save the knowledge base into the language model and use child-tune to avoid the catastrophic forgetting problem for response generation. more precisely, it avoids the problem of weak semantic relatedness between the "question text" to be retrieved and the retrieved "answer text", and can effectively increase the accuracy of retrieval. in the future, we will study how to combine the evidence generation model with the utterance generation model to further improve the generation quality.