Eyes Show the Way: Modelling Gaze Behaviour for Hallucination Detection (2023)

Abstract: Detecting hallucinations in natural language processing (NLP) is a critical undertaking that demands a deep understanding of both the semantic and pragmatic aspects of languages. Cognitive approaches that leverage users’ behavioural signals, such as gaze, have demonstrated effectiveness in addressing NLP tasks with similar linguistic complexities. However, their potential in the context of hallucination detection remains largely unexplored. In this paper, we propose a novel cognitive approach for hallucination detection that leverages gaze signals from humans. We first collect and introduce an eye tracking corpus (IITB-HGC: IITB-Hallucination Gaze corpus) consisting of 500 instances, annotated by five annotators for hallucination detection. Our analysis reveals that humans selectively attend to relevant parts of the text based on distributional similarity, similar to the attention bias phenomenon in psychology. We identify two attention strategies employed by humans: global attention, which focuses on the most informative sentence, and local attention, which focuses on important words within a sentence. Leveraging these insights, we propose a novel cognitive framework for hallucination detection that incorporates these attention biases. Experimental evaluations on the FactCC dataset demonstrate the efficacy of our approach, obtaining a balanced accuracy of 87.1%. Our study highlights the potential of gaze-based approaches in addressing the task of hallucination detection and sheds light on the cognitive processes employed by humans in identifying inconsistencies.

## introduction hallucination detection in text refers to the task of identifying and validating information that is inaccurately or falsely represented within textual content. detection of hallucination involves examining the claims made in the text and assessing * these authors contributed equally to this work their alignment with the surrounding context and external knowledge. addressing hallucinations has become paramount, particularly in the context of automatically generated text utilizing powerful language models (llms), which often exhibit humanlike fluency but are prone to hallucinatory outputs @xcite . many existing methods for hallucination detection depend on knowledge sources that are explicit such as wikipedia or knowledge graphs @xcite @xcite or ingrained in language encoders such as bert or roberta @xcite . while these traditional approaches can reasonably detect hallucinations in a text when supplemented with knowledge sources, they face sustainability challenges due to the constant need for up-to-date knowledge. obtaining the latest information for hallucination detectors is often impractical, as it requires readily available and current sources of knowledge. to address this issue, we propose an alternative approach that leverages cognitive and behavioural information from humans in the form of gaze patterns while they analyze text for potential hallucinations. our work is motivated by the notion that humans, while reading text for hallucination identification would naturally employ their cognitive faculties to navigate the intricate relationship between language and real-world knowledge. linguistically, this involves scrutinizing whether (a) entities in the text are adequately placed (e.g., is canada a right choice of entity) (b) the semantic roles played by the entities are valid w.r.t the context (e.g, pluto is a planet). this critical examination would often manifest as prolonged fixations 1 on specific sections of text (e.g., longer fixations on entities and phrases such as canada, pluto and a planet) that require closer evaluation, resulting in denser and more extensive fixation activities. in essence, fixations may serve as invaluable indicators, acting as a reliable surrogate for knowledge-based validation of contextual information pertaining to potential hallucinations. this may open up possibilities for the development of a hallucination detector that can leverage gaze data as a primary input, alleviating the reliance on supplementary external knowledge. to validate this, we first collect a first-of-its-kind eye-tracking data of 5 annotators annotating 500 instances of claim-context pairs, carefully derived from the factcc dataset @xcite . during the annotation process, we capture the fixation patterns of annotators on both the claim and context texts, along with their corresponding labels. notably, our inter-annotator agreement (iaa) kappa score reaches 0.60, indicating substantial agreement. behavioural analysis of the annotated data reveals a recurrent pattern where annotators tend to skim through somewhat irrelevant context while selectively focusing on information crucial for establishing or refuting hallucinations. building upon insights gained from this behavioural analysis, we term this selective reading phenomenon as "attention bias". furthermore, our observations indicate that attention bias can manifest as either a "global" approach, involving the extraction of sentences containing relevant information about hallucinations, or a "local" approach, focusing on specific phrases within sentences to evaluate the alignment of semantic roles between the claim and the selected phrases. building upon these insights, we propose a modular architecture that incorporates global and local attention bias using transformer-based deep learning techniques @xcite and a gaze-based attention saliency module @xcite . experimental evaluations on the factcc dataset demonstrate the efficacy of our approach, outperforming baseline models while attaining better interpretability. we have open-sourced our code, data and results for academic usage 2 . our contributions are summarised as follows: ## collection of iitb-hgc (iitb-hallucination gaze corpus) we collect eye-tracking data from annotators while they perform the task of hallucination detection. for the data collection process, we use 500 claimcontext pairs from the factcc dataset @xcite as stimuli. the annotators are given a claim-context pair and are asked to assess the faithfulness of the claim in relation to the context. to the best of our knowledge, no eye-tracking corpus is available for hallucination detection. 300 pairs are chosen from the training set, 100 from the validation set, and 100 from the test set of the factcc dataset and are used in similar splits during the training of the local attention bias model to avoid data leaks. the acquisition of the iitb-hallucination gaze corpus is then carried out in three phases, in which we presented annotators with 180, 160, and 160 instances respectively. the average number of words in the claimcontext pair is 105, the shortest claim having 6 words and the most extended claim having 32. on average, each claim consists of 13 words. the con-text length ranges from a minimum of 56 words to 132 words, with 92 words per context on average. ## methodology building upon our theory of attention bias, our proposed cognitive framework addresses the hallucination task by integrating this behaviour. figure 2 provides an overview of the framework, featuring the modelling of global and local attention bias. ## analysis we perform analysis on the fixation data from iitb-hgc and present it in section 7.1. we also show that including complete gaze data increases inter-pretability in section 7.2. we perform an error analysis (section d) to understand the scenarios in which our model gives incorrect responses. our comparative study with the summac-conv model (section 7.4) shows that our model performs better in 92% of the cases in which summac-conv makes erroneous predictions. error category no. of instances (%) instances where our model is correct (%) exact claim 34 (45.9%) 34/34 (100%) paraphrased claim 19 (25.7%) 18/19 (94.73%) ambiguous claim 7 (8.1%) 7/7 (100%) coreference in claim 8 (9.5%) 8/8 (100%) incorrect true label 7 (10.8%) 2/7 (37.5%) ## conclusion and future work hallucination detection is a pertinent and unresolved task in computational linguistics, often necessitating the incorporation of extralinguistic information into models. while traditional approaches benefit from leveraging diverse knowledge sources, we recognize the limitations of relying on outdated information and propose a rather orthogonal and cognitive-inspired approach that harnesses human eye-gaze patterns for hallucination detection. to support our approach, we created and analyzed an eye-tracking corpus, which revealed various attention biases exhibited by humans during hallucination searches. building upon this insight, we designed a deep learning architecture that combines gaze data and hallucination classification data for training. our experiments demonstrate promising results, indicating that a gaze-driven model exhibits superior performance and improved interpretability. moving forward, we intend to explore deeper cognitive features, such as progressive and regressive saccades, to further enhance hallucination detection. additionally, extending this work to multilingual and multimodal settings is a key objective on our agenda.