RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding
Year: 2023

Abstract:
Dialogue systems can leverage large pretrained language models and knowledge to generate fluent and informative responses. However, these models are still prone to produce hallucinated responses not supported by the input source, which greatly hinders their application. The heterogeneity between external knowledge and dialogue context challenges representation learning and source integration, which further contributes to unfaithfulness. To handle this challenge and generate more faithful responses, this paper presents RHO (ρ) utilizing the representations of linked entities and relation predicates from a knowledge graph (KG). We propose (1) local knowledge grounding to combine textual embeddings with the corresponding KG embeddings; and (2) global knowledge grounding to equip RHO with multi-hop reasoning abilities via the attention mechanism. In addition, we devise a response re-ranking technique based on walks over KG sub-graphs for better conversational reasoning. Experimental results on OpenDialKG (Moon et al., 2019) show that our approach significantly outperforms stateof-the-art methods on both automatic and human evaluation by a large margin, especially in hallucination reduction (17.54% in FeQA (Durmus et al., 2020)).1

heading: 1 Introduction
text: An open-domain dialogue system aims to automatically interact with humans with sensible and informative responses. To produce such responses, knowledge-grounded dialogue (KGD) systems are established, which leverage external knowledge such as knowledge graphs (KGs) (Yu et al., 2022; Zhao et al., 2020). Despite impressive progress in general linguistic performance, KGD systems are still prone to the hallucination problem in which the generated response is nonsensical or unfaithful to dialogue history or external knowledge (Ji
1The source code can be obtained from https:// github.com/ziweiji/RHO
et al., 2022a; Roller et al., 2021; Mielke et al., 2022). Two types of hallucinations may exist: intrinsic hallucination (the generated response is contradicted by the dialogue history or the external knowledge) and extrinsic hallucination (the generated response is hard to be verified with the dialogue history and external knowledge) (Ji et al., 2022a; Dziri et al., 2021). As the example in Figure 1, this issue undermines dialogue systems’ performance or raises safety concerns in real-world applications. For instance, the recently emerged foundation model ChatGPT suffers from this hallucination problem (OpenAI, 2023), especially extrinsic hallucinations. OpenAI currently uses huge amounts of human feedback to fix many ChatGPT errors, which is labor-intensive. It would be beneficial to reduce such errors automatically in advance.
The heterogeneity between external knowledge and textual dialogue content makes it challenging for neural response generation models to learn the representation and correlation in the input source (Li et al., 2022; Zhang et al., 2019; Liu et al., 2020; Ji et al., 2022a). This challenge could further result in a hallucinated generation that deviates from the source. Previous studies have shown that the hallucination problem in KGD
4504
can be mitigated by retrieved knowledge augmentation (Shuster et al., 2021), control code (Rashkin et al., 2021; Wu et al., 2021), and response postprocessing (Dziri et al., 2021). However, these works do not emphasize handling the discrepancy between lexical dialogue and structured knowledge information for the harmony of their fusion. The interaction mechanism between external knowledge and dialogue context should also be clarified.
In order to address this issue and take full advantage of lexical and knowledge information, we present RHO (ρ) 2 for faithful open-domain dialogue response generation with enhanced knowledge grounding and re-ranking. A high-level framework is illustrated in Figure 2. Specifically, RHO first learns the structured embeddings of all entities and relation predicates from the KG and links their mentions in the dialogue context to the KG. In the encoder-decoder model, the representations of all the linked entities and relations are grounded by KG embeddings, both locally and globally. Here, local knowledge grounding refers to the process where an entity or relation predicate receives and fuses its KG embedding only. While in global knowledge grounding, each entity or relation predicate attentively learns the knowledge from the entire sub-graph stored in a memory bank (Vaswani et al., 2017), which assigns dynamic weights of each triple equipping RHO with multi-hop reasoning abilities. These two knowledge groundings help the model effectively encode and inject the knowledge information from context-related subgraph with proper attention. In addition, we re-rank the generated responses according to the hallucination degree. This technique utilizes conversational reasoning to enforce the whole conversation to follow the knowledge traverses throughout KG.
In the experiments, we show that RHO outperforms state-of-the-art (SOTA) (Dziri et al., 2021) on the OpenDialKG (Moon et al., 2019) dataset by a large margin: improving 17.54% in FeQA (Durmus et al., 2020), and reducing 32.93% hallucinations according to human evaluation. In particular, the responses have a broader coverage of entities and relations in the KG, demonstrating higher faithfulness of responses. The quantitative and qualitative analysis further shows its effectiveness in reducing hallucination while not sacrificing conversational abilities. In summary, the major contribu-
2RHO is short for Reducing Hallucination in Opendomain dialogue systems.
tions of this work are threefold: • We propose the RHO model, which lever-
ages the structured knowledge in KGs to mitigate the hallucination problem in dialogue response generation. • To improve faithfulness, we introduce local and global knowledge grounding techniques (from a context-related knowledge sub-graph) into dialogue generation and further utilize a conversational reasoning model to re-rank the generated responses. • We conduct a thorough faithfulness analysis via automatic and human evaluation, and empirically demonstrate that RHO substantially reduces intrinsic and extrinsic hallucinations in the KGD generation task.

heading: 2 Related Work

heading: 2.1 Hallucination Reduction in KGD
text: Researchers have been devoted to reducing hallucination in open-domain dialogue systems incorporating external knowledge. Neural Path Hunter (NPH) (Dziri et al., 2021) leverages a hallucination critic and retrieves faithful entities by a query signal propagated over a sub-graph in the refinement stage. Shuster et al. (2021) explore various neuralretrieval-in-the-loop architectures where a retriever is introduced for knowledge selection. Rashkin et al. (2021) propose an faithfulness control code in decoding using re-sampling techniques. Wu et al. (2021) define a control mechanism with lexical control phrases and inductive attention where potentially uninformative attention links are removed. Our work improves the fusion and interaction between external knowledge and dialogue context via various knowledge groundings and reasoning techniques, further reducing hallucination.

heading: 2.2 KG Enhanced Dialogue Generation
text: KGs convey large amounts of structured knowledge, which can help to improve dialogue systems’ performance in informativeness (Tuan et al., 2019) and empathy (Li et al., 2020). For opendomain dialogue generation, Liu et al. (2019) unify knowledge triples and texts as a graph, and conduct multi-hop reasoning for explainability. Xu et al. (2020a) propose a proactive dialogue generation method based on agnostic meta-learning considering the limited number of KGs. Kumar et al. (2020) learn unified representations by training syntactic graph convolution networks, knowledge, and

heading: C <assistant> I do! It's directed by Bryan Singer. Have you
text: memory module with triplet loss. Xu et al. (2022); Zhou et al. (2018); Zhang et al. (2020) explore and demonstrate how commonsense KG facilitates language generation in dialogue systems. Besides, Yang et al. (2020); Rony et al. (2022); Chaudhuri et al. (2021) are committed to incorporating KG into task-oriented dialogue models. Different from the above literature, our work employs the factoid knowledge paths from KG to improve the faithfulness of open-domain dialogue systems.

heading: 3 Methodology
text: In this section, we begin with a brief introduction to our KGD task. Then, the detailed techniques of RHO are presented. Please refer to Figure 2 for an overview of our approach. RHO incorporates both textual and structured information from external KG into dialogue system via knowledgeable task guidance (§ 3.2) and enhanced knowledge groundings (§ 3.3, § 3.4). Specifically, we introduce local token-level knowledge grounding in § 3.3 and global grounding to provide a comprehensive view and the multi-hop reasoning ability in § 3.4. To effectively encode the heterogeneous sources, we
sum textual embedding, local, and global grounded embedding. Figure 3 is a more detailed diagram of the encoder with proposed knowledge groundings. In addition, during the post-processing stage, we incorporate a re-ranking technique that rewards generation candidates with low hallucination levels; this technique is proposed based on the hypothesis that the faithful response can be reasoned backward to the source (§ 3.5). We implement our model based on BART (Lewis et al., 2020) architecture, and please refer to Appendix A for details.

heading: 3.1 Task Definition
text: In the response generation task in dialogue systems, each data sample consists of a dialogue history H , including a set of utterances U conducted by humans and agents interactively. The goal of the response generation model in the dialogue system is to learn to generate a proper response R based on the dialogue history H .
The response generation for KGD task is a special case of the above task that takes a multirelational KG as an additional input; Multirelational KG is a directed graph G formulated by a collection of triples, denoted as T = ⟨[SBJ],[PRE],[OBJ]⟩, as additional input. Here, [SBJ],[OBJ] are subject and object entities, and [PRE] is the relation predicate r indicating the relationship between the subject and object entities e. The goal of KGD is to generate a faithful response R based on the history H and a knowledge sub-graph GH , which is the subset of the entire KG G with triples semantically related to the dialogue history H . Our task is also in line with the previous works (Zhou et al., 2021; Dziri
et al., 2021). Figure 2 has an illustrated example.

heading: 3.2 Input with Knowledgeable Task Guidance
text: A naive approach of input construction from raw data samples for language models is simply concatenating all triples in GH with the dialogue history H3. However, there is a lack of guidance to specifically excavate the model’s innate ability to handle the KGD task (Brown et al., 2020). Inspired by Raffel et al. (2020), we design a prompt to guide the PLM for KGD and convert the structured GH into textual information. Here, we linearize the triples in GH into texts (treated as meta-knowledge) and cooperate the dialogue history utterances U with the following template: “Given the knowledge: [SBJ]1 <sep> [PRE]1 <sep> [OBJ]1 <triple> [SBJ]2 <sep> [PRE]2 <sep> [OBJ]2 <triple> · · · <user> U1 <assistant> U2 · · · ”, where <sep>, <triple>, <user>, and <assistant> are special markers.

heading: 3.3 Local Knowledge Grounding
text: Although GH is converted and injected as additional input (§ 3.2), the model using textual information only cannot effectively handle the semantics of KGs which are typically sparse and complex in form (Petroni et al., 2019; Logan et al., 2019). Therefore, we ground the language representations with KG to take full advantage of lexical and structured knowledge information simultaneously.
During pre-processing, we obtain the collections of linked mentions of entities (EH ) and relations
3Similar to the baseline approach in Dziri et al. (2021).
(RH ) from the dialogue history H and their KG embeddings as follows:
1. Identify entity mention em that appears in dialogue history H that can be linked to an entity e in the sub-graph GH. We utilise an open-source linking tool named FLAIR (Akbik et al., 2019). 2. Since relations connecting entities are crucial in knowledge reasoning for PLMs (Labutov et al., 2018; Feng et al., 2020), we also link the relation mention rm in H to the relation predicate r in GH . 3. We employ TransE (Bordes et al., 2013) to learn the KG embeddings of entities (e⃗G) and relation predicates (r⃗G) from the entire G 4.
Then, we obtain a locally grounded token embedding w⃗local for an arbitrary non-special token w in H as follows:
w⃗local =    M(e⃗G) substr(w, EH) M(r⃗G) substr(w,RH)
0⃗ otherwise (1)
where M(·) transforms the space from the KG embeddings to the PLM token embeddings. A typical way of implementing M(·) is through a mapping matrix. Specifically, if dim(w⃗) = dim(e⃗G), M(·) can be further simplified as an identity mapping. substr(w, EH) is a Boolean indicator that returns true if the current token w is a sub-string of any em in EH . This way, tokens related to specific entities or relation predicates can be grounded by their respective KG embeddings by fusing w⃗local into the vanilla token embedding w⃗. We regard this approach to be local as w⃗local is only related to the KG embedding of the corresponding node.
As in Figure 2 and Figure 3, the tokens "X", "-", "Men", "2" in dialogue history are linked to the entity "X-Men 2" in KG. Then, we take the corresponding KG embedding from the database gained by TransE as the local knowledge grounding which will fuse into these tokens’ textual embeddings.

heading: 3.4 Global Knowledge Grounding
text: Focusing only on a single token in the context and a single node in the graph is insufficient to enhance the multi-hop reasoning abilities of the dialogue
4We employ TransE via OpenKE (Han et al., 2018) as our underlying KG representation learning algorithm due to its effectiveness and simplicity. We have also experimented with some recent algorithms and observed no improvement in the performance (refer to Appendix C for details).
system. In addition to local grounding, we further propose global knowledge grounding which enriches the semantics considering the entire subgraph GH and hence offers the model a comprehensive view of the background knowledge.
Following our observation, we adopt the attention mechanism (Vaswani et al., 2017) to draw global dependencies between the dialogue history H and a memory bank storing the representations of all knowledge triples in GH . Let TH be the collection of all triples in GH . The memory bank stores |TH | embedding vectors where the i-th vector v⃗i corresponds to the KG embedding of the i-th relation triple Ti = ⟨[SBJ],[PRE],[OBJ]⟩:
v⃗i = M([SBJ ])⊕M([PRE])⊕M([OBJ ]) (2)
where ⊕ is the concatenation operator for vectors. We gather all vectors and further project them to a global knowledge embedding space by:
KH = Wproj · [v⃗1 ⊕ · · · ⊕ v⃗|TH |] (3) where Wproj is a learnable projection matrix.
Based on the formulation of KH , we compute how much attention the current token w in H pays to each relation triple according to the semantic relevance and obtain the globally grounded token embedding w⃗global as follows:
w⃗global =    softmax( w⃗·K T H√ dim(w⃗) ) ·KH substr(w, EH),
substr(w,RH) 0⃗ otherwise
(4) As in Figure 2 and Figure 3, for “X-Men 2” in the dialogue history, Triple 1 should have more influence on the tokens’ representation than Triple 2 due to its higher relevance.
Finally, the encoder of RHO sums the vanilla token embedding w⃗, the locally grounded embedding w⃗local, and the globally grounded embedding w⃗global as: w̃ = w⃗ + w⃗local + w⃗global. During training, while w⃗ is rapidly updated via back propagation, w⃗local and w⃗global are relatively fixed with few parameters trainable (e.g., Wproj).

heading: 3.5 Response Re-ranking
text: With the above approaches, our knowledgegrounded model generates N candidate responses by beam search. Yet, the grounding process mainly applies to the embedding level, lacking output constraints. To enhance our RHO’s ability to reduce hallucination, we extend KG-CRUSE (Sarkar et al., 2022) and train a conversational reasoning model ϕ
for response re-ranking, with emphasis on the KG. If the generated response can be reasoned backward to the source, we can assume it is faithful.
In our approach, we obtain the semantic embeddings of the dialogue history H and a possible response R via a contextual sentence encoder, i.e., Sentence-BERT (Reimers and Gurevych, 2019). The model ϕ is an LSTM-based decoder that learns the probability pt,ϕ of an action a⃗t given the state s⃗t at step t. Here, the action refers to a walking step on the graph GH , represented as a⃗t, which is the concatenation of the relation and entity embeddings derived from the KG, together with their semantic embeddings based on Sentence-BERT, i.e., 5
a⃗t = (e⃗G + e⃗S)⊕ (r⃗G + r⃗S) (5) where e⃗S and r⃗S are the semantic sentence embeddings of an entity e and a relation predicate r, respectively. The state s⃗t contains the representations of the dialogue history, together with entities and relations already traversed by ϕ (action history). It is defined as a tuple (H, (⃗a1, a⃗2, · · · , a⃗t-1)). Hence, the model ϕ explicitly models the process of a traversal upon GH conditioned on the dialogue history H and a possible response R. During training, each action a⃗t made by ϕ is combined into a path, and the target path is the given context-related subgraph GH .
After our encoder-decoder model generates N candidate responses {R1, · · · , RN}, we select the best response R∗ with the highest probability pϕ =∏
t pt,ϕ over all the generated responses, i.e.,
R∗ = argmax n∈{1,··· ,N} pϕ(A = GH|H,Rn) (6)
where A is a collection of actions a⃗t (i.e. knowledge path) that ϕ has already traversed conditioned on the dialogue history H and each response Rn.
For a more intuitive understanding, refer to the example in Figure 2 where the model generates three candidate responses: A, B, and C. It selects Response C as the final output with the traversal path “X-Men 2, directed by, Bryan Singer, ∼directed by, Superman Returns”. 6 As seen, there is a higher matching degree between the sub-graph in Figure 2 and the Response C, compared to other candidate
5We have also investigated the impact of the KG embeddings (e⃗G and r⃗G) for action modeling in Appendix E.1 by comparing the performance of the re-ranker under two settings: i) a⃗t = (e⃗G + e⃗S)⊕ (r⃗G + r⃗S) and ii) a⃗t = e⃗S ⊕ r⃗S (the vanilla model in KG-CRUSE).
6∼directed by refers to the opposite direction of the relation directed by.
responses (i.e., A and B).

heading: 4 Experiments

heading: 4.1 Dataset
text: OpenDialKG (Moon et al., 2019) contains openended dialogues between two speakers, initiated by talking about a given entity and grounded on the relevant facts from a structured KG. Thus, the sequential turn-based dialogues can be regarded as traversing the paths in the KG. To our knowledge, OpenDialKG is currently the only publicly available corpus for English open-ended dialogues with KG path annotations (Yu et al., 2022; Ni et al., 2022), and previous works (Dziri et al., 2021; Zhou et al., 2021) evaluate their effectiveness on this corpus. Hence, we also conduct our experiments on OpenDialKG. Consistent with previous works (Dziri et al., 2021; Liu et al., 2019; Zhou et al., 2021), we filter OpenDialKG by keeping only the dialogue samples that are annotated with a KG path. The dataset is divided into training, validation, and testing sets in the ratio of 8:1:1.

heading: 4.2 Baselines
text: The following strong baselines are employed to show the efficiency of our method. We fine-tune pre-trained language models GPT2 (Radford et al., 2019) and BART (Lewis et al., 2020) on our task. NPH (Dziri et al., 2021) refines the generated responses by retrieved entities from the KG. To our knowledge, the integration of GPT2 and NPH, called GPT2+NPH, reaches the SOTA performance on OpenDialKG. Since this post-processing technique is agnostic to the generation model, we apply it to BART, named BART+NPH as our baseline. In addition, EARL (Zhou et al., 2021) utilizes external KGs for conversation generation without parameterizing specific entity representations. KGBART (Liu et al., 2021), a KG-augmented pretrained language generation model based on BART, introduces the information of the relations among concepts for generative commonsense reasoning. We are the first to adapt this model to the KGD generation. Furthermore, we explore ChatGPT on this task in Appendix A. Please refer to it for the details of baseline implementations.

heading: 4.3 Evaluation Protocols

heading: 4.3.1 Automatic Evaluation
text: To evaluate the generation quality, we use the classical word overlap-based NLG metrics: BLEU and
ROUGE-L (Lin, 2004). Due to the possible presence of hallucinations in the dataset (especially extrinsic ones) the metrics based on the n-gram overlap between the golden answer and generated texts are not sufficient (Ji et al., 2022a). Therefore, we also use source-dependent metrics, i.e., FeQA (Durmus et al., 2020), QuestEval (Scialom et al., 2021), and Entity Coverage, to estimate the hallucination degree. FeQA and QuestEval are both questionanswering (QA)-based metrics for evaluating the faithfulness of the output in generation tasks 7. QuestEval has two modes: reference-dependent mode assesses a text with one or multiple groundtruth references; and reference-free mode assesses a text without any references. In addition to the metrics used in previous works (Dziri et al., 2021; Zhou et al., 2021), we assume that entities in generated responses should be covered by those in the given knowledge triples and dialogue history. The higher Entity Coverage is, the lower hallucination degree can be to some extent. Specifically, we utilize a named entity recognition (NER) model to extract named entities in generated responses and the dialogue history. We compute Entity Precision, Recall and F1 scores between generated entities and entities in KG and dialogue history to evaluate the faithfulness of generated responses.

heading: 4.3.2 Human Evaluation
text: To further assess the quality of generated responses from different systems, we conduct human evaluations using Amazon Mechanical Turk8. For hallucination level assessment, we first ask annotators
7Please refer to Appendix B.1 for details. 8https://www.mturk.com/
to identify whether each response is Faithful, or Hallucinated given the dialogue history and KG triples. The judgment criteria are as described in § 1. “Faithful” means that the response is supported by the knowledge triples and dialogue context, while “hallucinated” means that the response contradicts or cannot be verified by the source. If the response is hallucinated, we further ask annotators to identify whether the hallucination is Extrinsic, Intrinsic or Both (Dziri et al., 2021).
We also conduct an A/B test of our framework against the baselines GPT2+NPH and BART+NPH to evaluate generated responses on Fluency (Ji et al., 2022b; Dathathri et al., 2020). The annotators are asked whether the writing is smooth and grammatically correct and given four choices: Neither, Both, Sample A, or Sample B. Please refer to Appendix B.2 for details.

heading: 5 Results and Analysis

heading: 5.1 Overall Evaluation Results
text: Automatic Evaluation. The first eight rows of Table 1 shows the experimental results on automatic metrics over the OpenDialKG test set. Our model outperforms all baselines on both classic overlap metrics and hallucination metrics, indicating the high quality of the generated utterances. Specifically, compared to SOTA (GPT2+NPH), RHO gives a significant rise of 17.54% in FeQA, 9.33% in QuestEval (RF), and 33.97% in Entity Coverage (F1). RHO also achieves better results compared to the stronger baseline BART+NPH. The results indicate the faithfulness of KGD systems can be improved by knowledge grounding and re-ranking
techniques. Human Evaluation. As in Table 2, the faithfulness of RHO is higher than GPT2+NPH and BART+NPH. The results are statistically significant with p-value<0.05. Specifically, we see a 12.38% increase in faithfulness compared to SOTA (GPT2+NPH). As shown in Table 4, the information “Judy Davis starred in My Brilliant Career” in golden answer, is not supported by the input, although it is factual according to the world knowledge. The baseline model hallucinates unfactual information, i.e., “Judy Davis starred in both The Referendum and The Golden Compass”, while our model is better grounded on the input source.
At a more granular level, the extrinsic hallucination problem is more frequent than the intrinsic one in all models. This phenomenon is also observed in other works (Dziri et al., 2021; Nan et al., 2021). Specifically, compared to SOTA (GPT2+NPH), RHO reduces extrinsic hallucination by 42.85%. Compared to BART+NPH, RHO reduces intrinsic hallucination by 13.66% and extrinsic hallucination by 46.74%. According to the A/B test results for fluency in Table 3, RHO is slightly more fluent than SOTA methods. Overall, human evaluation results are in line with automatic evaluation. RHO mitigates both intrinsic and extrinsic hallucination issues without sacrificing fluency.

heading: 5.2 Ablation Study
text: We conduct an ablation analysis to assess the contribution of each component of our method: Local Knowledge Grounding (LKG), Global Knowledge Grounding (GKG), and Response Re-ranking (RR). As shown in the last four rows of Table 1, fully-implemented RHO performs best in au-
tomatic hallucination metrics with a slight sacrifice of classical overlap metrics. Specifically, compared to models equipped with only local/global knowledge grounding, the model equipped with both two (LKG+GKG) gains higher scores in FeQA, QuestEval, and Entity Coverage. The same trend is observed when comparing the fullyimplemented model with the model without reranking (LKG+GKG) and please refer to Appendix E.2 for an example. The trade-off between hallucination metrics and the others is due to the fact that some reference responses in the dataset diverge from the sources (Dziri et al., 2022). Hence, improving responses’ consistency with the source will inevitably reduce that with the references. Overall, the above three mechanisms synergy to improve the generated responses’ faithfulness without significant reduction in quality.

heading: 5.3 Error Analysis
text: Although RHO achieves better performance than the baselines in the KDG task, it still generates a few cases of failure. To gain more insight into our model, we present failure examples from RHO in Appendix D and conduct an error analysis.
As shown in Table 2, the extrinsic hallucination problem is more frequent. Because the corpus itself has more extrinsic hallucinations and the model is more likely to produce unverified rather than contradictory information with the limitation of the source. To solve the problem, we suggest that the quality of the corpus used for model training, such as OpenDialKG, can be further improved by filtering out irrelevant and contradictory information. Fact-checking can also be potentially used to find evidence from world knowledge and verify the generated responses (Ji et al., 2022a).
In addition to intrinsic and extrinsic hallucination issues, we discover two other types of negative cases, namely No Full Coverage, and Unnatural Connection. No Full Coverage refers to the situation where the output does not cover the full answer as expressed by all triples mentioned in the given sub-graph. We believe that further research on the reasoning ability and interpretability of the model can help address this issue. Unnatural Connection denotes that a response is not connected to the dialogue history especially the last utterance naturally. We find that in a few cases, the dialogue system delivers an irrelevant answer or poorly replies to the dialogue history. This issue occurs because
our model focuses more on hallucination reduction and sometimes pays less attention to flexibility and diversity. In the future, we can explore the controllability of the grounding degree on knowledge and strike a dynamic balance between faithfulness and flexibility in response generation.

heading: 6 Conclusion and Future Work
text: In this work, we investigate the hallucination in KGD systems and propose the RHO framework to tackle this issue with three mechanisms: Local Knowledge Grounding, Global Knowledge Grounding, and Response Re-ranking. Our method is empirically proven effective in reducing hallucinations with automatic and human evaluation. We also conduct deep error analysis on the generated responses. For future work, the re-ranking module can be combined with foundation models such as ChatGPT to reduce manual labor in the human feedback process. We also call for research to strike a better balance between response faithfulness and diversity.
Limitations
The deep neural networks in RHO uses feature extraction and vectorization to represent the texts. The model only detects the statistical regularities and quantitative relationships among the variables but can not see qualitative relationships, such as causality, hierarchy, and other abstractions (Tsimenidis, 2020). Although we leverage the response re-ranking technique, which improves the explainability of RHO, the neural networks are undoubtedly still “black boxes” to humans. Therefore, the faithfulness of generated responses can not be fully guaranteed.
Ethical Considerations
In addition to the hallucination problem, another critical challenge, the offensive language, is also
introduced with the evolutionary progress toward building reliable dialogue systems. The data-driven models are susceptible to delivering offensive responses while mimicking human conversations (Xu et al., 2020b). It has been shown that racial and gender biases are encoded in the PLMs (Blodgett et al., 2020), and these biases are present in the training corpus. Since RHO leverages PLMs and the training corpus, it is possible to generate offensive languages. We suggest that in real-world dialogue systems, it is necessary to employ some postprocessing steps to alleviate this problem when it is deployed online.

heading: Acknowledgement
text: This work has been supported by the China NSFC Project (No. NSFC21EG14), SAAIR Project (No. Z1286), and HKJCCT21EG01 (RG192).

heading: B Evaluation
text: B.1 Automatic Metrics
FeQA FeQA, a QA-based metric for evaluating the faithfulness of the generated output, has been
applied in summary (Chen et al., 2021) and dialogue (Dziri et al., 2021) tasks. As a reference-free metric, it takes the source (such as a document) and the corresponding output to be evaluated (such as a summary) as input. Given the source, a question generation (QG) model generates a question based on the source and then a QA model generates an answer A. The QA model generates another answer B based on the question and output to be evaluated. The average F1 score between answers A and B reflects the hallucination level of the output. Following the setting in (Dziri et al., 2021), we concatenate all the knowledge triples in GH with the dialogue history H as the source, and the generated response is the output. We calculate the FeQA score using the code and models from the official library12 with their default hyper-parameters.
QuestEval QuestEval (Scialom et al., 2021) is also a QA-based metric that has referencedependent and reference-free modes. The reference-dependent mode depends both on the input source and golden answers, while the referencefree mode does not require any ground-truth references. The input source is built in the same way as in FeQA. We calculate the QuestEval score using the code and models from the official library13 with their default hyper-parameters.
B.2 Human Evaluation
We conduct the human evaluation to assess RHO’s performance in response generation, especially the ability to reduce hallucination. In detail, we randomly select 100 samples generated by each model. Each sample is then evaluated by three different annotators to rule out potential bias. We specify that annotators must meet the following qualifications: Their Human Intelligence Task (HIT) approval rates are greater than or equal to 95%, and the numbers of HITs approved are greater than or equal to 5000. The annotators are located in Australia, the United Kingdom, and the United States. Figure 4 and 5 are the user interfaces (UIs) on Amazon Mechanical Turk for human evaluation of Hallucination and Fluency, respectively. The instructions, questions, and examples for annotators are shown.

heading: C KG Representation Learning
text: For the proposed knowledge grounding techniques in § 3.3 and § 3.4, we employ several KG representation learning algorithms including TransE (Bordes et al., 2013), TransH (Wang et al., 2014), RotaE (Sun et al., 2019), SimplE (Kazemi and Poole, 2018), DistMult (Yang et al., 2015), and ComplEx (Trouillon et al., 2016) via OpenKE 14. The link prediction results are shown in Table 5, where MR is the mean rank and Hits@10 is the proportion of correct entities ranked in the top 10. “Filter” means removing all corrupted triplets in the dataset (Han et al., 2018). Due to the effectiveness and simplicity, we finally choose TransE to gain the KG embeddings of all entities and relation predicates in G.

heading: D Error Analysis
text: As discussed, we characterize the negative cases of KDG systems into four types: Extrinsic Hallucination, Intrinsic Hallucination, No Full Coverage, and Unnatural Connection. To gain more insights into our model, we present more failure examples from RHO in Table 6.
In the example of Extrinsic Hallucination, we find that in some cases, both the golden answers and our responses suffer from this issue. Based only on the given dialogue context, we cannot know or infer the genre of Windtalkers. The referenced answer is “thriller”, while RHO guesses it is “sci-fi”. They are both extrinsic hallucinations since the source cannot confirm them without other information. For Intrinsic Hallucination, the opinion in the reference answer is “Todd Walker drags the team down”, while RHO generates “Minnesota
12https://github.com/esdurmus/feqa 13https://github.com/ThomasScialom/
QuestEval 14https://github.com/thunlp/OpenKE
Twins drafted Todd Walker”. However, according to the conversation, the relationship “Drafted” in the KG is wrong and should be “Dragged”. In the example of No Full Coverage, the first triple “<Batman, Romantically involved with, Catwoman>” is not mentioned in the conversation. In the Unnatural Connection’s example, our response grounds the response on the KG, but “The Rise of Theodore Roosevelt has genre Autobiography” is expressed mechanically and rigidly.

heading: E Response Re-ranking
text: E.1 Performance of Re-rankers
For entities and relations in the KG, the original settings in Sarkar et al. (2022) only utilize the semantic sentence embeddings via a contextual sentence encoder, i.e., Sentence-BERT. In contrast, we sum their KG embeddings obtained by TransE with the above sentence embeddings. The evaluation results are shown in Table 8, where Hits@k is the proportion of the ground-truth path ranked in the top-k highest probability paths searched by the model. Since our settings achieve higher recall, we then use the model leveraging both sentence and KG embeddings as our re-ranker.
E.2 Case Study of Re-ranking As shown in Table 7, before re-ranking, there is a No Full Coverage issue mentioned in § 5.3. The response does not include the information from the given triple “<No Strings Attached, has genre, Romantic comedy>”, which deviates from the established flow. After reranking, the response includes the missing information and becomes more similar to the golden answer.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 7

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 8

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3, Section 4.2, 4.3

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 3, Section 4, Appendix A
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? We only use the publicly available pretrained models and datasets.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 3, Section 4
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? We only use the publicly available corpus (OpenDialKG).

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 5, Appendix D, E
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4.1
C 3 Did you run computational experiments? Section 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Appendix A

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4.3, Section 5

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 3, 4, Appendix A, B
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 4.3.2, Appendix B.2

heading: 3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Appendix B.2

heading: 3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Section 4.3.2
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Appendix B.2

heading: Future Work
text: For future work, the re-ranking module can be combined with foundation models such as ChatGPT to reduce manual labor in the human feedback process. We also call for research to strike a better balance between response faithfulness and diversity. Limitations
The deep neural networks in RHO uses feature extraction and vectorization to represent the texts. The model only detects the statistical regularities and quantitative relationships among the variables but can not see qualitative relationships, such as causality, hierarchy, and other abstractions (Tsimenidis, 2020). Although we leverage the response re-ranking technique, which improves the explainability of RHO, the neural networks are undoubtedly still “black boxes” to humans. Therefore, the faithfulness of generated responses can not be fully guaranteed. Ethical Considerations
In addition to the hallucination problem, another critical challenge, the offensive language, is also
introduced with the evolutionary progress toward building reliable dialogue systems. The data-driven models are susceptible to delivering offensive responses while mimicking human conversations (Xu et al. , 2020b). It has been shown that racial and gender biases are encoded in the PLMs (Blodgett et al. , 2020), and these biases are present in the training corpus. Since RHO leverages PLMs and the training corpus, it is possible to generate offensive languages. We suggest that in real-world dialogue systems, it is necessary to employ some postprocessing steps to alleviate this problem when it is deployed online