Halu- NLP at S em E val-2024 Task 6: M eta C heck GPT - A Multi-task Hallucination Detection using LLM uncertainty and meta-models (2024)

Abstract: Hallucinations in large language models(LLMs) have recently become a significantproblem. A recent effort in this directionis a shared task at Semeval 2024 Task 6,SHROOM, a Shared-task on Hallucinationsand Related Observable Overgeneration Mis-takes. This paper describes our winning so-lution ranked 1st and 2nd in the 2 sub-tasksof model agnostic and model aware tracks re-spectively. We propose a meta-regressor basedensemble of LLMs based on a random forestalgorithm that achieves the highest scores onthe leader board. We also experiment with var-ious transformer based models and black boxmethods like ChatGPT, Vectara, and others. Inaddition, we perform an error analysis com-paring ChatGPT against our best model whichshows the limitations of the former

## introduction the recent rapid deployment of large language models (llms) has led to a hallucination proliferation which poses a barrier to the reliability and trustworthiness of llms @xcite . one of the widely agreed upon definition of hallucinations @xcite is output text containing information not relevant to the input or a desired output. hallucinations should not be thought of as an occasional nuisance, but rather as a systemic issue inherent to these models and their web-sourced training data which can be rife with bias and misinformation. this can directly cause user discontent when these systems are implemented in production or real-world scenarios. these type of hallucinations have been widely studied in the context of text related tasks like machine translation @xcite , summarization @xcite and dialogue generation * *equal contribution @xcite . gaps in hallucination detection methods in llm outputs persist across many such tasks. despite some progress in hallucination detection, existing methods may rely upon comparisons to reference texts, overly simplified statistical measures, reliance upon individual models, or annotated datasets which can limit their extensibility. our approach leverages the uncertainty signals present in a diverse basket of llms to detect hallucinations more robustly. in this paper, we present a meta-regressor framework for llm model selection, evaluation, and integration. 1 the overall approach is depicted in figure 1 . for the first step, each llm-generated sentence is compared against stochastically generated responses with no external database as with selfcheckgpt @xcite . a metamodel that leverages input from a diverse panel of expert evaluators evaluates and integrates the output of multiple iterations of the process. our framework focuses on creating a metamodel for identifying hallucinations, with the idea that the meta-model's prediction power is linked to the performance of the underlying base models. this model achieves the highest scores in the semeval-2024 task 6 competition across three subtasks: machine translation, paraphrase generation, and definition modeling. ## related work in this section, we describe prior work on hallucination detection methods. we will examine two potential streams for hallucination detection: modelaware detection and black-box detection. modelaware techniques have access to the model's internals, such as weights and logits while black-box methods do not has access to such model internals. ## task description and datasets in the shroom task-6, the organizers propose a binary classification task to predict a machine generated sentence is a hallucination or not. the organizers considered 3 types of text generation tasks -definition modelling, machine translation and paraphrase generation. ## conclusion our meta-model strategy represents a step forward in addressing the challenges of mitigating hallucinations and the importance of a nuanced approach to model selection, evaluation, and integration. the work also acknowledges the need for additional research into more transparent, interpretable, and multilingual models, as well as the integration of external knowledge sources and feedback mechanisms to refine and improve hallucination detection methods. in the future, some areas we would like to work on include utilizing additional multilingual datasets, expand the scope of our work to more set of text generation task, and focus more on white box hallucination detection systems. while the current system was tested on some machine translation tasks, we believe it could benefit from more work on multilingual datasets. the current system could improve by integrating with external knowledge bases via retrieval augmented generation. the system could also be made more usable by distilling its knowledge into a portable fine-tuned model widely available to others. another area for potential improvement includes integration of human or agent feedback through reinforcement learning.