Grounded Dialogue Generation with Cross-encoding Re-ranker, Grounding Span Prediction, and Passage Dropout (2022)

Abstract: MultiDoc2Dial presents an important challenge on modeling dialogues grounded with multiple documents. This paper proposes a pipeline system of “retrieve, re-rank, and generate”, where each component is individually optimized. This enables the passage re-ranker and response generator to fully exploit training with ground-truth data. Furthermore, we use a deep cross-encoder trained with localized hard negative passages from the retriever. For the response generator, we use grounding span prediction as an auxiliary task to be jointly trained with the main task of response generation. We also adopt a passage dropout and regularization technique to improve response generation performance. Experimental results indicate that the system clearly surpasses the competitive baseline and our team CPII-NLP ranked 1st among the public submissions on ALL four leaderboards based on the sum of F1, SacreBLEU, METEOR and RougeL scores.

## introduction the task of developing information-seeking dialogue systems has seen many recent research advancements. the goal is to answer users' questions grounded on documents in a conversational manner. multidoc2dial 1 is a realistic task proposed by @xcite to model goal-oriented information-seeking dialogues that are grounded on multiple documents and participants are required to generate appropriate responses towards users' utterances according to the documents. to facilitate this task, the authors also propose a new dataset that contains dialogues grounded in multiple documents from four domains. unlike previous work that mostly describe document-grounded dialogue modeling as a machine reading comprehension task based on one particular document or passage, the multidoc2dial involves multiple topics within a conversation, hence it is grounded on different documents. the task contains two sub-tasks: grounding span prediction aims to find the most relevant span from multiple documents for the next agent response, and agent response generation generates the next agent response. this paper focuses on our work in to the second sub-task, and presents three major findings and contributions: experimental results indicate that our proposed system achieves a performance with marked improvement over the strong baseline. ## related work open-domain question answering systems have evolved to adopt the popular "retriever-reader (generator)" architecture since drqa @xcite . previous work @xcite adopt end-to-end training strategy to jointly learn the retriever and reader with question-answer pairs. retrieval-augmented generation (rag) @xcite uses dense passage retriever (dpr) @xcite as the retriever to extract multiple documents related to the query and feed them into a bart @xcite generator for answer generation. @xcite proposed the fusion-in-decoder method which processes passages individually in the encoder but jointly in the decoder, surpassing the performance of rag. other work like quac @xcite , sharc @xcite and coqa @xcite focus on the machine reading comprehension task, which assumes that the associated document is given. in particular, @xcite proposed the doc2dial task ,which aims to extract the related span from the given documents for generating the corresponding answer. ## task description the multidoc2dial task aims to generate an appropriate response r based on an input query q (the current user turn u t and the concatenated dialogue history {u t -1 1 } := u 1 , u 2 , ..., u t -1 ) and a collection of passages {p i } m @xmath0. the passages are extracted from documents based on document structural information indicated by markup tags in the original html file. the organizer splits the multidoc2dial data into train, validation, development and test set, and results on the latter two are evaluated through the leaderboard foot_0 . the validation, development and test set contain two settings: seen and unseen, which is categorized based on whether there are dialogues grounded on the documents seen/unseen during training. we leave detailed dataset description in appendix a. ## methodology we propose a pipeline system of "retrieve, re-rank, and generate". following previous work in @xcite , we adopt dpr @xcite as the retriever ( §4.1) to efficiently filter out irrelevant passages and narrow the search space. we then refine the retrieval results with a deep cross-encoder ( §4.2) trained with localized negatives @xcite . we introduce a passage dropout and regularization technique to enhance the robustness of the generator ( §4.3) and use the grounding span prediction as an auxiliary task. further more, pipeline training is adopted where each component is individually optimized to fully utilize the supervision. experimental results ( §5.3) also indicate the effectiveness and merits of the training strategy, which we observed to be a key factor for the performance gain. ## experiments and results we evaluate the passage retrieval results with recall (r) and mean reciprocal rank (mrr). we report response generation performance based on f1, exact match (em) @xcite , sacrebleu (s-bleu; post, 2018), and rougel @xcite . ## conclusion this paper presents a pipeline system of "retrieve, re-rank, and generate" for the multidoc2dial challenge. the advantage is that each of the three components can fully exploit the ground-truth training data. we apply a deep cross-encoder architecture where we create a training set using localized hard negatives sampled from the retriever results. we adopt grounding span prediction as an auxiliary task to be jointly trained with the response generator. we also apply passage dropout and regularization to improve response generation performance. experimental results indicate that the proposed system improves over a strong, competitive baseline and our team got 1st place on all four leaderboards.