N oot N oot At S em E val-2024 Task 6: Hallucinations and Related Observable Overgeneration Mistakes Detection (2024)

Abstract: Semantic hallucinations in neural language gen-eration systems pose a significant challenge tothe reliability and accuracy of natural languageprocessing applications. Current neural mod-els often produce fluent but incorrect outputs,undermining the usefulness of generated text.In this study, we address the task of detectingsemantic hallucinations through the SHROOM(Semantic Hallucinations Real Or Mistakes)dataset, encompassing data from diverse NLGtasks such as definition modeling, machinetranslation, and paraphrase generation. We in-vestigate three methodologies: fine-tuning onlabelled training data, fine-tuning on labelledvalidation data, and a zero-shot approach usingthe Mixtral 8x7b instruct model. Our resultsdemonstrate the effectiveness of these method-ologies in identifying semantic hallucinations,with the zero-shot approach showing compet-itive performance without additional training.Our findings highlight the importance of robustdetection mechanisms for ensuring the accu-racy and reliability of neural language genera-tion systems.

## introduction the modern nlg landscape is plagued by two interlinked problems: on the one hand, our current neural models have a propensity to produce inaccurate but fluent outputs; on the other hand, our metrics are most apt at describing fluency, rather than correctness. this leads neural networks to "hallucinate", i.e., produce fluent but incorrect outputs that we currently struggle to detect automatically. for many nlg applications, the correctness of an output is however mission critical. for instance, producing a plausible-sounding translation that is inconsistent with the source text puts in jeopardy the usefulness of a machine translation pipeline. with our shared task, we hope to foster the growing interest in this topic in the community. with shroom @xcite we adopt a post hoc setting, where models have already been trained and outputs already produced: participants will be asked to perform binary classification to identify cases of fluent overgeneration hallucinations in two different setups: model-aware and model-agnostic tracks. that is, participants must detect grammatically sound outputs which contain incorrect or unsupported semantic information, inconsistent with the source input, with or without having access to the model that produced the output. to that end, we will provide participants with a collection of checkpoints, inputs, references and outputs of systems covering three different nlg tasks: definition modeling (dm), machine translation (mt) and paraphrase generation (pg), trained with varying degrees of accuracy. the development set will provide binary annotations from at least five different annotators and a majority vote gold label. ## methodology our methodology involved first labelling the training data, fine tune a model on the test data then evaluating the model on test data. we chose roberta-base as our base model for fine tuning as xlm-roberta is a multilingual language model optimized for classification tasks. it is pretrained on massive multilingual data, and has a robust architecture and performance enable efficient fine-tuning across diverse text classification problems with state-of-the-art accuracy. for labelling the training data, we used mixtral 8x7b @xcite , specifically mixtral-8x7b-instruct-v0.1.q5_k_m.gguf ## results we present the results of our experiments using three different methodologies for detecting semantic hallucinations in neural language generation systems.