Enhancing Multilingual Document-Grounded Dialogue Using Cascaded Prompt-Based Post-Training Models (2023)

Abstract: The Dialdoc23 shared task presents a Multilingual Document-Grounded Dialogue Systems (MDGDS) challenge, where system responses are generated in multiple languages using user’s queries, historical dialogue records and relevant passages. A major challenge for this task is the limited training data available in low-resource languages such as French and Vietnamese. In this paper, we propose Cascaded Prompt-based Post-training Models, dividing the task into three subtasks: Retrieval, Reranking and Generation. We conduct post-training on high-resource language such as English and Chinese to enhance performance of low-resource languages by using the similarities of languages. Additionally, we utilize the prompt method to activate model’s ability on diverse languages within the dialogue domain and explore which prompt is a good prompt. Our comprehensive experiments demonstrate the effectiveness of our proposed methods, which achieved the first place on the leaderboard with a total score of 215.40 in token-level F1, SacreBleu, and Rouge-L metrics.

## introduction document-grounded dialogue systems (dgds) have emerged as a research focus in the natural language processing field. they leverage documents to provide targeted information for specialized tasks such as question answering and recommendations @xcite . these systems ensure accuracy and reliability by leveraging comprehensive knowledge bases while enhancing real-time responsiveness and information retrieval efficiency @xcite . additionally, they can accommodate the expanding scalability of new documents and knowledge sources @xcite . nonetheless, these systems encounter challenges when operating with low-resource languages, including limited training data @xcite , and significant disparities in grammar, vocabulary, and semantics across languages @xcite . to address these challenges, researchers are developing multilingual approaches to improve the performance of low-resource languages in dgds. the dialdoc23 shared task introduces training and evaluation datasets for mdgds in vietnamese and french. the training dataset comprises three distinct components: query, passage, and response. additionally, the dataset includes a set of documents for retrieval. the query combines the historical dialogue with the current inquiry. during inference, the intelligent agent retrieves the most relevant document from the document set based on the query and generates a response. notably, this task focuses on low-resource languages, setting it apart from previous tasks. in this paper, we propose cascaded prompt-based post-training models to solve mdgds challenge. as inspried by re2g @xcite framework, our approach tackles the overall task by dividing it into three subtasks: retrieval, reranking, and generation, with parallel training and sequential inference. as illustrated in figure 1 , the retrieval step identifies top k relevant passages, followed by reranking to select the most relevant passage, and in generation step the query and passage information are incorporated to generate the final response. to enhance the performance of retrieval and generation in low-resource languages such as french and vietnamese, we conduct posttraining on high-resource languages such as english and chinese to learn language similarities. additionally, we activate the models' capabilities in diverse languages within the dialogue domain by employing the prompt method. besides, we employ domain loss function to align the domain of the query and passage during retrieval training. ## method the proposed method consists of three main stages: retrieval, reranking, and generation, as depicted figure 1 . using contrast learning techniques, the retrieval step efficiently identifies top k relevant passages, followed by reranking to select the most pertinent passage. in the generation step, the query and passage information are incorporated to generate the final response. each step is described in detail in the following section. ## conclusion in this paper, we propose a cascaded promptbased post-training framework comprising retrieval, reranking, and generation three-stage, to solve the mdgd challenge. to enhance the retrieval and generation performance in low-resource languages such as french and vietnamese, we exploit the similarities between these and highresource languages such as chinese and english by applying post-training techniques. prompt method are used to activate model's ability in a specific language and dialogue domain, and in-lingual prompts show superior results. furthermore, we employ domaincls loss function in retrieval, emsemble method in rerank, and r-drop method to attain the best results in the dialdoc23 shared task.