Post-hoc Utterance Refining Method by Entity Mining for Faithful Knowledge Grounded Conversations (2023)

Abstract: Despite the striking advances in recent language generation performance, model-generated responses have suffered from the chronic problem of hallucinations that are either untrue or unfaithful to a given source. Especially in the task of knowledge grounded conversation, the models are required to generate informative responses, but hallucinated utterances lead to miscommunication. In particular, entity-level hallucination that causes critical misinformation and undesirable conversation is one of the major concerns. To address this issue, we propose a post-hoc refinement method called REM. It aims to enhance the quality and faithfulness of hallucinated utterances by refining them based on the source knowledge. If the generated utterance has a low source-faithfulness score with the given knowledge, REM mines the key entities in the knowledge and implicitly uses them for refining the utterances. We verify that our method reduces entity hallucination in the utterance. Also, we show the adaptability and efficacy of REM with extensive experiments and generative results. Our code is available at https://github.com/YOONNAJANG/REM.

## introduction the knowledge grounded conversation (kgc; also called knowledge grounded conversation, kgd), which is a subfield of the dialogue systems, is a task of generating human-like utterances by referring to specialized knowledge such as wikipedia 1 @xcite . the kgc task requires the ability to generate fluent and informative utterances based on source knowledge. considering this ability, there has been numerous concurrent research on the kgc to build models that can have conversations with human-like expertise @xcite figure 1: an illustrated example of the utterances with entity-level hallucination (i.e., case 1 and 2) and desirable utterances in knowledge grounded conversation. in the source-faithful utterance, entities of the given source knowledge are included, making the utterance informative. 2020; @xcite @xcite @xcite . a colossal number of training corpora and parameters have brought the recent explosion of language generation performance in pre-trained language models (plms) @xcite and large language models (llms) @xcite @xcite . however, hallucination, which has been a chronic problem in natural language generation, is still cited as one of the biggest challenges remaining unsolved @xcite . in the kgc task, even though the ground truth knowledge is given, the models make the error of generating hallucinated utterances not faithful to the source knowledge @xcite . especially, entity-level hallucination, generating names of entities that are incorrect or not present in the source document @xcite , causes critical misinformation and jeopardizes the flow of the conversation @xcite . as shown in case 1 of figure 1 , the previous kgc models generate the utterance containing information about the wrong entity, which is not given in the knowledge. further, the generated utterance is excessively general while not considering sufficient entities that align with the context of the conversation, as shown in case 2. these deficiencies can undermine the helpfulness of ai conversational models. though previous research has tried to mitigate them in general domain conversation @xcite , research to address the entity-level hallucination in kgc remains in dark @xcite . to address these entity-level hallucination problems, we propose a post-hoc utterance refining method by entity mining, called rem, for more desirable and source-faithful kgc. rem can be used to refine the unfaithful utterances generated by previous models in a plug-and-play manner. to refine utterances, we leverage the entity mining method, which extracts the named entities to implicitly utilize key information in the knowledge in a multi-tasking manner. with this simple but effective method, rem aims to mitigate entity-level hallucination and lead to a more successful conversation. in order to measure the effectiveness of rem, we conduct an extensive empirical evaluation. first, to demonstrate the post-hoc refining ability of rem, we experiment with refining the utterances generated by baseline models on three kgc datasets. we investigate the flexibility and validity of rem with cross-data experiments and adversarial data refining experiments, respectively. in addition, we conduct an ablation study and human evaluation to verify the effectiveness of our method, showing the improvement of the sourcefaithfulness score and entity coverage of refined utterances. we also demonstrate the scalability of rem by applying it to large language models. from a case study comparing the refined results of rem and baseline utterance, we demonstrate that our rem model improves the source-faithfulness in the utterances. our contributions are threefold: (1) we propose a post-hoc refining method by implicitly mining key entities in the knowledge for more source-faithful conversation; (2) we show that our simple but ef-fective method is adaptable to the existing models, including large language models, in a plug-andplay manner; (3) we substantiate that our method reduces entity-level hallucination and accomplish more desirable knowledge grounded conversation with diverse experiments. ## proposed method our proposed rem aims to refine the modelgenerated utterance to be more faithful to the source knowledge, as shown in figure 3 . the utterance is filtered by the faithfulness scoring function, whether to be refined or not. we train the pretrained language model with an encoder-decoder structure in the rem method. in the rem model, the entity miner extracts the named entities from the source knowledge and learns them implicitly. the utterance generator makes more faithful utterances based on the key information extracted by the entity miner. in this section, we formulate a kgc task and our proposed utterance refining method, and describe the rem model and its training process. ## results and analysis in this section, we report the experimental results and analysis. we first discuss the post-hoc refining ability of our method in §5.1. then, we evaluate the flexibility of rem through cross-data refining experiments in §5.2. we also conduct experiments on refining adversarial utterances in §5.3, and perform an ablation study in §5.4. furthermore, we provide the results of human evaluation in §5.5, and present a case study in §5.7. finally, we analyze the application of rem to large language models in §5.6. ## conclusion in this work, we proposed rem, a post-hoc refining method for improving the source-faithfulness of the utterances in the knowledge grounded conversation (kgc). rem enabled simple but effective refining by extracting entities from the source knowledge given in the kgc task. it makes the model learn the key information implicitly and use it for refining the utterance more faithful to the source knowledge. we presented extensive experiments applying rem in a plug-in-play method to various model-generated outputs, showing increased source-faithfulness and entity coverage after refining. also, qualitative analysis and human evaluation proved the refining efficacy of rem. we verified that our proposed method could be utilized with the prompt of the large language models.