Maha Bhaashya at S em E val-2024 Task 6: Zero-Shot Multi-task Hallucination Detection (2024)

Abstract: In recent studies, the extensive utilization oflarge language models has underscored the importance of robust evaluation methodologiesfor assessing text generation quality and relevance to specific tasks. This has revealeda prevalent issue known as hallucination, anemergent condition in the model where generated text lacks faithfulness to the source anddeviates from the evaluation criteria. In thisstudy, we formally define hallucination and propose a framework for its quantitative detectionin a zero-shot setting, leveraging our definitionand the assumption that model outputs entailtask and sample specific inputs. In detectinghallucinations, our solution achieves an accuracy of 0.78 in a model-aware setting and 0.61in a model-agnostic setting. Notably, our solution maintains computational efficiency, requiring far less computational resources than other SOTA approaches, aligning with the trendtowards lightweight and compressed models.

## introduction the contemporary landscape of natural language generation (nlg) is marked by a confluence of complexities, wherein two primary challenges emerge as focal points of concern. firstly, the prevalent neural models within nlg frameworks consistently produce outputs that exhibit linguistic fluency yet suffer from inaccuracies @xcite . secondly, the current evaluation metrics, vital for evaluating the effectiveness of nlg systems, demonstrate a significant inclination towards fluency measures while neglecting to prioritize accuracy. so, this highlights the need to consider the "truthfulness" of the model's output, i.e its alignment with the source to ensure a comprehensive assessment. (dale et al., 2022) † the authors contributed equally to this work. in the realm of nlg applications, the criticality of output accuracy cannot be overstated. a divergence between the fluency and factual correctness of generated content not only undermines the utility of nlg systems but also engenders substantial risks across various domains. consider, for instance, the domain of machine translation,the production of seemingly plausible yet inaccurate translations not only compromises the integrity of the translated content but also defeats the purpose of facilitating correct translations. likewise, in tasks like definition modeling and paraphrase generation, where accurately conveying semantic meaning is crucial, the presence of incorrect outputs presents notable challenges in upholding the integrity and dependability of the generated content. ## shroom dataset shroom (a shared-task on hallucinations and related observable overgeneration mistakes) dataset is a task-based hallucination detection dataset which is divided into two major categories: ## definitions as described earlier, the shroom shared task encompasses of three different taks, definition modelling (dm), paraphrase generation (pg), and machine translation. we define the hallucination in the context of the specific task at hand. defining hallucinations individually in the context of a specific task enables detecting hallucinations quantitatively and qualitatively. we offer distinct definitions of hallucinations and methodologies for detecting hallucinations within the context of each of the aforementioned task. in the context of definition modelling, the model is expected to generate the definition of a word which has been used in the provided context by making use of distributional semantics. definition modelling models such as flan-t5-definition-enbase @xcite are not fully capable of making use of distributional semantics to define a word as used in a context. in a sample where the word w has been used in a setting contrasting to the definition the model has learnt during its training process, the models fails to provide a contextual definition of the word w. examples for the same have been demonstrated in with this assumption, we define "hallucination to be an instance where the output generated by the definition modelling model does not entail the target output." thereby reducing the hallucination detection task to a natural language inference task in the context of definition modelling. in the context of paraphrase generation and machine translation, the model's inputs and outputs are anticipated to exhibit semantic equivalence. if the generated paraphrase or translation diverges from semantic equivalence with the source text, they are deemed imperfect paraphrases or translations. therefore, in the context of paraphrase generation and machine translation, we define "hallucination to be an instance where the paraphrase or translation generated by the model is not semantically equivalent to the source." this reduces the hallucination detection in the context of given tasks to a semantic equivalence detection task, which could also be framed as bidirectional entailment detection, a variation of the natural language inference task. the aforementioned definitions of hallucination allow us to simplify the hallucination detection task to a natural language inference task, thereby enabling us to qualitatively and quantitatively detect hallucinations. in a more generic setting, we provide a definition of hallucinations that can be adapted to any task to effectively detect them. we define "hallucinations as instances where the output generated by the model is not faithful to the input or the training data of the model. if the model generates information that is contradictory to the model's training data or the input to the model, it can be termed as a hallucination." 4 methodology grounding to the above definitions, the experimental setup we designed goes on to quantify the alignment of the model's output (hyp) with either the source (src) or the target (tgt) based on the task (task) the data sample corresponds to. we propose that examining the entailment relationship between the model's output (hyp) and either the source (src) or the target (tgt) (which is example 1: definition modeling (dm) model input: i went into the water bottle to withdraw cash. what is the definition of water bottle? model output: a container for holding liquids. expected output: a financial institution such as a bank or atm to withdraw cash model: flan-t5-definition-en-base recent research heavily relies on large language models (llms) to benchmark various natural language understanding and generation (nlg) tasks. however, this practice extends to hallucination detection as well, which we find ironic and counterproductive, considering llms' inherent tendency to hallucinate. using llms for hallucination detection presents two major drawbacks. firstly, their computational demands are significant, making them an expensive solution @xcite . secondly, the lack of complete interpretability in llms renders them unreliable for this task @xcite . ## results after several experiments with the above methodology, leveraging the accuracy and the spearman correlation (ρ) metrics, we have bench-marked the hallucination detection task on the shroom validation and test sets to achieve an accuracy of 0.78 in model-aware and 0.61 on model-agnostic test sets respectively. for our analysis let us take only the accuracy metric into account. the bench-marking saw a utilisation of opensource pre-trained natural language inference (nli) models available on hugging face. several experiments brought out interesting observations which are worthy discussing. we evaluated the following models: ## conclusion our work makes two significant contributions to the study of hallucinations in language models. first, we provide a concrete definition of the term "hallucination," enabling both qualitative and quantitative study and detection of such phenomena. second, we offer a computationally efficient approach to detect hallucinations in tasks such as definition modeling, machine translation, and paraphrase generation. we frame the hallucination detection task as a function of the input to the generation model and the data used to train it. our definitions and approaches also provide a framework that can be utilized for hallucination detection in various natural language generation tasks across the spectrum.