RA 2 FD : Distilling Faithfulness into Efficient Dialogue Systems (2024)

Abstract: Generating faithful and fast responses is crucial in the knowledge-grounded dialogue. Retrieval Augmented Generation (RAG) strategies are effective but are inference inefficient, while previous Retrieval Free Generations (RFG) are more efficient but sacrifice faithfulness. To solve this faithfulness-efficiency trade-off dilemma, we propose a novel retrieval-free model training scheme named Retrieval Augmented to Retrieval Free Distillation (RA2FD) to build a retrieval-free model that achieves higher faithfulness than the previous RFG method while maintaining inference efficiency. The core idea of RA2FD is to use a teacher-student framework to distill the faithfulness capacity of a teacher, which is an oracle RAG model that generates multiple knowledge-infused responses. The student retrieval-free model learns how to generate faithful responses from these teacher labels through sequence-level distillation and contrastive learning. Experiment results show that RA2FD let the faithfulness performance of an RFG model surpass the previous SOTA RFG baseline on three knowledge-grounded dialogue datasets by an average of 33% and even matching an RAG model’s performance while significantly improving inference efficiency. Our code is available at https://github.com/zzysjtuiwct/RA2FD.

## introduction the faithfulness of the system response is crucial when evaluating language models (lm) powered dialogue systems @xcite . a faithful system means the system response is consistent with the appropriate knowledge. however, an unfaithful system will face the well-known 'hallucination' problem @xcite . one effective technique to improve faithfulness and reduce hallucination of the dialogue system is retrieval augmented generation (rag) @xcite ok, do they speak french? ok, do they speak french? ok, do they speak french? time cost ≈ 1s time cost ≈ 1s knowledge inject time cost ≈ 30s faithfulness efficiency retrieval augmented to retrieval free distillation (ra2fd) retrieval augmented generation (rag) retrieval free generation (rfg) yes, they offer french and english service. learn from rag faithfulness efficiency no, they offers a full english breakfast. faithfulness efficiency yes, they do speak french at the hotel. distill params params inject knowledge base figure 1 : llustration of our contribution. the rag system is faithful but is time-consuming during inference, while the rfg system offers faster reasoning speed but tends to hallucinate. our method achieved a good tradeoff between faithfulness and inference efficiency. @xcite , which retrieves passages from a knowledge base to augment the response. however, the retrieval process takes several times longer than the generation process, leading to severe inference inefficiency @xcite . a straightforward approach to improve inference efficiency is to use the retrieval free generation (rfg), which discards the retrieval process and directly utilizes the knowledge injected in its parameters @xcite to generate a response. this distinction makes it more challenging for the retrieval-free model to integrate correct knowledge into responses. to address this challenge, @xcite introduced a novel knowledge injection pretraining scheme. @xcite stored knowledge in multiple adapters, and @xcite proposed mixed contrastive learning to improve faithfulness. however, previous rfg methods exclusively use human-labeled responses as the model training target, which are natural and fluent but contain limited knowledge tokens. these labels will lead the model to imitate the response style and generate a fluent response, yet ignore learning how to fuse necessary knowledge into the output. in the wow dataset, the sota rfg's faithfulness is still 25% lower than rag's @xcite . thus, a key question arises: can we simultaneously ensure the dialogue system's efficiency and faithfulness? to address this problem, we propose a training scheme named retrieval augmented to retrieval free distillation (ra2fd) that utilizes a teacherstudent framework to build a retrieval-free model to achieve higher faithfulness and maintain inference efficiency. we distill the capacity to generate faithful responses from an oracle rag teacher's response to an rfg student model through sequencelevel distillation. the core idea is these responses are more knowledge-infused than human-labeled responses. furthermore, to fully use the teacher's generation capacity, we let the teacher generate multiple knowledge-infused responses instead of only one. we then employ contrastive objectives to let the student model focus more on learning from a more faithful knowledge-infused response. we conduct our experiments on three knowledgegrounded dialogue benchmarks. a task-oriented dialogue called dstc9 @xcite and two open-domain chatbots named wow @xcite and faithdial @xcite . our method achieves faithfulness improvements by an average of 33% to the previous sota rfg baseline. it also boosts inference speed by 50 and 2 times compared to the rag methods on the dstc9 and wow / faithdial datasets. in summary, we contribute to improving the faithfulness of the retrievalfree generation model from three aspects: ## related work the open-domain chatbot @xcite and task-oriented dialogue system @xcite that generates a response based on knowledge has received attention recently. unfaithfulness in lm generation unfaithfulness, which includes the hallucinations @xcite phenomenon, is the response generated by an lmbased dialogue system that is inconsistent or unfaithful @xcite to the appropriate knowledge. training data is essential to the unfaithfulness problem in the lm-based dialogue system. shen et al. (2021) filtered out untrustworthy samples from the training set, and dziri et al. (2022) removed hallucinations in the wizard of wikipedia (wow) dataset. retrieval augmented generation open-domain chatbots use retrieve-based methods @xcite @xcite to alleviate unfaithfulness of generation by integrating external knowledge (such as wikipedia) @xcite as the input context. meanwhile, for the task-oriented dialogue system that limits its external knowledge to a specific document or knowledge graph, the retriever @xcite @xcite slightly diverges from the chatbot. however, though retrieval enriches the response information, such methods suffer from severe inference inefficiency @xcite . retrieval free generation one way to overcome this drawback is to omit the retrieval process and use knowledge stored in parameters to generate responses. sun et al. (2023) used mixed contrastive learning to enhance the knowledge elicitation process. diao et al. (2023); bang et al. (2023); emelin et al. (2022) injected domain knowledge into the adapter while fixing the pre-trained language model (plm). instead of storing knowledge in multiple adapters, xu et al. (2023); li et al. (2022b) injected external knowledge into the plm parameters. to better probe knowledge in plms, liu et al. (2022) employed a multi-stage prompting approach in the open-domain chatbot. however, the faithfulness performance of existing retrieval-free methods is still far from satisfactory. this paper introduces a novel teacher-student framework to build a retrieval-free dialogue model with higher faithfulness and inference efficiency. previous retrieval-free methods exclusively use human-labeled responses as the training target, which are fluent but contain limited knowledge contexts. these training targets will lead the model to imitate the response style yet ignore learning how to fuse necessary knowledge context into the output. unlike previous works, we distill the capacity to generate faithful responses from the retrievalaugmented model to the retrieval-free model by training a retrieval-free model using the multiple knowledge-infused responses generated by an oracle retrieval-augmented teacher, which are more knowledge-infused than human-labeled responses. ## methodology figure 2 presents the overview of our method. we first use an oracle retrieval-augmented teacher model to generate multiple knowledge-infused responses. then, we distill the capacity to generate faithful responses from these responses to a retrieval-free student model through sequence-level distillation and contrastive learning. to keep the notation consistent with our method, let u @xmath0, • • • , u t-1 , u t } represent the dialogue history with a window size of w turns, and t is the index of each turn. u t is the current user utterance. the knowledge-based dialogue system is designed to generate an informative response u t+1 using u t and a knowledge snippet with n tokens ## conclusions in this paper, we proposed a retrieval augmented to retrieval free distillation (ra2fd) training scheme to improve the faithfulness of the retrievalfree dialogue generation model. extensive experiments conducted on the dstc9, wow, and faithdial datasets demonstrate that ra2fd outperforms existing retrieval-free generation methods and achieves a state-of-the-art result on all datasets. moreover, the faithfulness of our proposed method is comparable to retrieval-augmented generation methods while offering a faster inference speed.