Pan More Gold from the Sand: Refining Open-domain Dialogue Training with Noisy Self-Retrieval Generation (2022)

Abstract: Real human conversation data are complicated, heterogeneous, and noisy, from which building open-domain dialogue systems remains a challenging task. In fact, such dialogue data still contains a wealth of information and knowledge, however, they are not fully explored. In this paper, we show existing open-domain dialogue generation methods that memorize context-response paired data with autoregressive or encode-decode language models underutilize the training data. Different from current approaches, using external knowledge, we explore a retrieval-generation training framework that can take advantage of the heterogeneous and noisy training data by considering them as “evidence”. In particular, we use BERTScore for retrieval, which gives better qualities of the evidence and generation. Experiments over publicly available datasets demonstrate that our method can help models generate better responses, even such training data are usually impressed as low-quality data. Such performance gain is comparable with those improved by enlarging the training set, even better. We also found that the model performance has a positive correlation with the relevance of the retrieved evidence. Moreover, our method performed well on zero-shot experiments, which indicates that our method can be more robust to real-world data.

## introduction open-domain dialogue is a long-standing problem in natural language processing and has aroused the widespread interest of researchers. many approaches have been studied, and recently, generation models trained on large-scale data have gained more attention @xcite @xcite @xcite @xcite . open-domain dialogue systems are born to deal with diverse domains, and naturally their training data, usually crawled from online resources such as reddit and twitter, are heterogeneous and contain utterances with many various topics, more freedom of topic shifting, and vague responses @xcite . as a result, directly building generation models from such data will be inefficient and usually requires "knowledge" during the training. one common solution is to introduce external knowledge, usually, in a form of unstructured knowledge passages from wikipedia @xcite or internet articles @xcite , and then, to build retrieval-augmented generation (rag) methods to improve the response quality @xcite . however, this assumes knowledge-intensive scenarios, which are not suitable for general open-domain or robust to noise. according to our preliminary study, in the reddit dataset, 43% of the dialogues are merely chitchat and cannot match "knowledge". moreover, building such a knowledge-augmented dataset is very expensive as it relies on large amounts of high-quality human annotations w.r.t. knowledge grounding. and thus, they are limited in size, making it hard for a knowledge-retrieval method to generalize on scale. motivated by the above, we would like to investigate can we have better ways of utilizing open domain data without introducing external resources? to tackle the aforementioned problem, we found that the context from the other relevant dialogue sessions can still be very useful for dialogue generation. to utilize such unstructured contexts, we take inspiration from retrieval-augmented methods @xcite . differently, we retrieve useful dialogue context as evidence, build contextevidence-response triples for each dialogue turn, and treat open-domain generation as an evidenceaware generation task. such that our model can learn to respond with useful grounding evidences. to retrieve evidences, we adopt similarity-based bertscore @xcite , which leverages pre-trained contextual embeddings from bert and matches words in two sentences by cosine similarity. it has been shown to correlate with human judgment on sentence-level and system-level evaluation. although it was proposed as an automatic evaluation metric for text generation, due to the high correlation with human judgment, we consider it as a better off-the-shelf method to pick high-relevant evidences, compared with lexicon-based bm25. by this, we show that current training methods which learn merely using context-response pairs have not fully unleashed the potential of training data and that our methods, only retrieving from the training data, can consistently improve the generation performance. we also perform zero-shot experiments, demonstrating that our method can be robust and generalized to different domains. moreover, we found that adding extra retrieval data only (without training them) can still help the model gain performance, and it can even outperform traditional methods directly trained on that part of retrieval data. this proves our method is compatible with current methods with external knowledge. our contributions are summarized as follows: • we explore a retrieval-generation training framework that can increase the usage of training data by directly considering the heterogeneous and noisy training data as the "evidence". open-domain dialogue system open-domain dialogue system aims to perform chit-chat with people without the task and domain restriction. adiwardana et al. (2020) proposed meena, a multiturn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. blender @xcite learn to provide engaging talking points and listen to their partners, as well as display knowledge, empathy and personality appropriately, while maintaining a consistent persona. adapter-bot @xcite explored prompt-based few-shot learning in dialogue tasks. plato @xcite introduced discrete latent variables to tackle the inherent oneto-many mapping problem in response generation. zhang et al. (2019b) proposed dialogpt which was trained on 147m conversation-like exchanges extracted from reddit comment chains. wang et al. (2020) introduced cdial-gpt, a pre-training dialogue model which is trained on a large-scale cleaned chinese conversation dataset. mi et al. (2022) built pangu-bot with relatively fewer data and computation costs by inheriting valuable language capabilities and knowledge from pretrained language model. ## conclusion in this paper, we propose a self-retrieval training framework for open-domain dialogue generation. different from other knowledge-intensive tasks, our framework only retrieves relevant dialogue instances from the training data (which can be extended to a retrieval set) without the need to train them in the generation model. it is significant that we demonstrate that traditional training baselines underutilize the training data and our method can utilize more potential of data. we show that our method improves the robustness and generality of generative models as well as generate proper response for complicated human conversation. we also find that bertscore can be used for better evidence retrieval. in future works, we would like to study better ways of evidence retrieval and evidence-aware training and we believe our approach can benefit to other nlp tasks, such as classification task.