Diving Deep into Modes of Fact Hallucinations in Dialogue Systems (2022)

Abstract: Knowledge Graph(KG) grounded conversations often use large pre-trained models and usually suffer from fact hallucination. Frequently entities with no references in knowledge sources and conversation history are introduced into responses, thus hindering the flow of the conversation—existing work attempt to overcome this issue by tweaking the training procedure or using a multi-step refining method. However, minimal effort is put into constructing an entity-level hallucination detection system, which would provide fine-grained signals that control fallacious content while generating responses. As a first step to address this issue, we dive deep to identify various modes of hallucination in KG-grounded chatbots through human feedback analysis. Secondly, we propose a series of perturbation strategies to create a synthetic dataset named FADE (FActual Dialogue Hallucination DEtection Dataset). Finally, we conduct comprehensive data analyses and create multiple baseline models for hallucination detection to compare against human-verified data and already established benchmarks.

## introduction knowledge-grounded conversational models often use large pre-trained models @xcite . these models are notorious for producing responses that do not comply with the provided knowledge; this phenomenon is known as hallucination @xcite . faithfulness to the supplementary knowledge is one of the prime designing factors in these knowledge-grounded chatbots. if a response is unfaithful to some given knowledge, it becomes uninformative and risks jeopardizing the flow of the conversation. despite retaining strong linguistics abilities, these large language models(lm) inadequately comprehend and present facts during conversations. lms are trained to emulate distributional properties of data that intensify its hallucinatory attributes during test time. figure 1 : hallucination manifested by generated responses using gpt2 @xcite trained on kg triples can be more nuanced. on the one hand, many prior works @xcite @xcite have suggested training these models on external data to ensure faithfulness may lead to a sourcereference divergence problem, where the reference contains additional factual information. to address this problem holistically, dziri et al. has proposed a two-step generate-then-refine approach by augmenting conventional dialogue generation with a different refinement stage enabling the dialogue system to correct potential hallucinations by querying the kg. also, this work employs a token-level hallucination classifier trained on a synthetic dataset constructed using two perturbation strategies foot_1 . though this method has clear benefits, the hallucination perturbation strategies proposed in this work might fail to capture some of the subtle attributions of a factual generative model. as illustrated in figure 1 , neural models can inject hallucinated entities into responses that are present in the k-hop kg and are deceptively similar to what is expected. also, if we cannot detect these elusive hallucinations beforehand, it will cause a cascading effect and amplify hallucinations in subsequent turns @xcite . on the other hand, relying on human annotations is challenging due to error-prone collection protocols and human ignorance to complete the tasks with care @xcite . prior research @xcite shows that knowledge-grounded conversational benchmarks contain hallucinations promoted by a design framework that encourages informativeness over faithfulness. as studied by @xcite , when the annotators are asked to identify hallucination in a response, there is a high chance of error due to lack of incentive, personal bias, or poor attention to the provided knowledge. see and manning have studied different shortcomings in a real-time neural model. in this work, based on some of the findings of see and manning, like repetitive and unclear utterances promoting hallucination, we extend the already defined modes of hallucinations @xcite . our contributions to this work are threefold: our study extends the work of @xcite where they specifically explore two broad circumstances -extrinsic and intrinsic to the provided kg, under which lms are likely to exhibit unfaithful behavior. though this categorization is beneficial for detecting hallucinations, these categories can be further subdivided into subcategories, which are described in §2.3. ## dataset creation hallucination type index type selection criteria soft same as original entity ei with max document score hard same as original entity ei with min document score grouped same as one predefined type, selected randomly same as soft ## task to identify utterances that contain hallucinations and to locate the entities of concern. we create two tasks: 1. utterance classification: given the dialog history d, knowledge triples k n and the current utterance x n+1 we classify x n+1 is hallucinated or not. 2. token classification: given d, k n and x n+1 , we need to perform sequence labelling on x n+1 and identify the hallucinated spans. ## baseline models as an initial effort toward tackling the suggested hallucination detection task, we create several baseline detection models based on pre-trained transformer models, including bert, xlnet, and roberta. these transformer-based models represent the state-of-the-art and can potentially better leverage context or embedded world knowledge to detect self-contradictory or anti-commonsense content. for training the utterance classifier, given d, k n and x n+1 , we fine tune a pre-trained model m to predict binary hallucinated label y for x n+1 . here, d and k n are considered as sequence a with token type ids as 0 and x n+1 is considered as sequence b with token type ids as 1. during inference, from the last hidden states @xmath0, l are hidden size and sequence length, respectively), then we obtain the representation @xmath1.e., @xmath2. we then pass w through a mlp layer with a tanh activation to get the binary label @xmath3, 1}. during training time, we fine-tune the model using cross entropy objective between the predicted labels and the actual labels. similarly, for training the sequence classifier, we fine-tune a pre-trained model m s . at first, we encode d, k n and x n+1 using m s to get the last hidden states @xmath4, (h, l are hidden size and sequence length, respectively). instead of doing a binary classification of each token, we adopt a bilou encoding scheme. the hidden states are passed through an mlp layer with a tanh activation to get the 5-way label @xmath5, i, l, o, u }. during training time, we fine-tune the model using a cross-entropy objective between the predicted and actual labels. ## experimental setup baseline configurations we experiment with a variety of pre-trained models via hugging face transformers, including bert-baseuncased(110m), roberta-base(125m) and xl-net-base-cased(110m). though using large or medium versions of these models will produce better results, we refrain from using those models as scaling large models in production is costly. more details about training parameters can be found in §e we also experimented with model architecture as follows: (i) varied the length of the history (ii) experimented with max/ mean pooling. (iii) whether to concatenate the hidden states corresponding to k n with the hidden states corresponding to x n+1 before passing them through the mlp layer. (iv) using a crf layer instead of mlp for predicting labels in the sequence tagger. the best configuration uses 4 turns of conversational history, max pooling, it does not concatenate hidden states of k n with hidden states of x n+1 and uses a 2-layer mlp. evaluation metrics we evaluate the baselines with formal classification metrics, including precision, recall, and f1 for the hallucination sequence tagger. for the utterance-level hallucination classifier, we report accuracy, precision, recall, f1, and auc (area under curve) for roc. we also use the g-mean metric @xcite , which measures the geographic mean of sensitivity and specificity. we also employ the brier skill score (bss) metric @xcite , which computes the mean squared error between the reference distribution and the hypothesis probabilities. ## results and discussion baseline performance in §3.4. using the existing benchmark and baseline models, we also perform a zero-shot inference on the human-verified data. from generalisability we make zero-shot inference on begin and faithdial datasets' test splits. to make a fair comparison with the benchmark models, we further fine-tune roberta-large model on our datasets. ## conclusion in this work, we have analyzed the modes of entitylevel fact hallucination, which is an open problem in kg-grounded dialogue systems. through a human feedback analysis, we demonstrate that these kg-grounded neural generators manifest more nuanced hallucinations than straightforward studied approaches. we have proposed fine-grained perturbation strategies to create a dataset that mimics the real-world observations and create a series of datasets collectively known as fade. our entitylevel hallucination detection model can predict hal-lucinated entities with an f1 score of 75.59% and classify whether an utterance is hallucinated or not with an f1 score of 90.75%. our models can generalize well when zero-shot predictions are made on benchmarks like begin and faithdial, indicating our perturbation strategies' robustness. this work can be extended by devising more sophisticated perturbation mechanisms, which can simulate other types of hallucinations.