DUT h at S em E val-2024 Task 6: Comparing Pre-trained Models on Sentence Similarity Evaluation for Detecting of Hallucinations and Related Observable Overgeneration Mistakes (2024)

Abstract: In this paper, we present our approach toSemEval-2024 Task 6: SHROOM, a Sharedtask on Hallucinations and Related ObservableOvergeneration Mistakes, which aims to determine weather AI generated text is semanticallycorrect or incorrect. This work is a comparative study of Large Language Models (LLMs)in the context of the task, shedding light ontheir effectiveness and nuances. We present asystem that leverages pre-trained LLMs, suchas LaBSE, T5, and DistilUSE, for binary classification of given sentences into ‘Hallucination’or ‘Not Hallucination’ classes by evaluatingthe model’s output against the reference correct text. Moreover, beyond utilizing labeleddatasets, our methodology integrates syntheticlabel creation in unlabeled datasets, followedby the prediction of test labels.

## introduction hallucinations in machine generated text are cases when the model generates output that is partially or fully unrelated to the source sentence. while being a non-frequent phenomenon, it can dramatically impact the user experience and the trust toward the system. hallucination rates in multiple models vary from 2.8 to 16.2 percent, according to the hallucination leaderboard created by vectara hosted on hf and github. 1 while the problem of hallucinations is known, it remains challenging, and one aspect of that is the absence of proper datasets. as a result, previous studies relied on scenarios where models are encouraged to hallucinate @xcite @xcite @xcite . however, it is uncertain if these approaches are effective in more natural, undisturbed environments @xcite . recent research conducted in relatively clean settings @xcite demonstrates that existing hallucinations detection methods fall short. the authors create a natural setting dataset, annotate it for various nmt (neural machine translation) pathologies, and evaluate detection methods. they find most existing detection methods inadequate, with sequence log-probability performing best. although they demonstrated interesting results, they were limited on detecting hallucinations on machine translation (mt) generated text. semeval-2024 task 6 @xcite goes a step further by providing a human annotated dataset of hallucinated text regarding three different scenarios. along with machine translation (mt), also definition modeling (dm) and paraphrase generation (pg) cases are considered. this paper describes the system developed by the duth team for semeval-2024 task 6. our strategy is based on utilizing embeddings to evaluate the similarity between context and hypothesis sentences in order to detect hallucinated text. in our case context sentence is the 'gold' output expected from to the models for generation and hypothesis sentence is the actual model production. for generating the embeddings of the context and hypothesis we are utilizing a pretrained t5 tokenizer @xcite . then we measure their similarity by taking the dot product of their corresponding embeddings, followed by summation along axis 1. finally, using that similarity score, we train an ensembler machine learning model to distinguish hallucinated form non-hallucinated text. we provide our code publicly foot_1 2 background ## results ## conclusion through these experiments, we found that the pretrained t5 model exhibits optimal performance in the detection of hallucinated text in the domain of artificial intelligence. furthermore, we successfully employed an ensemble of multiple popular top-tier classifiers to augment the predictive capabilities of our system and investigated the implications of synthetically labeling unlabeled data, presenting it as a novel approach to hallucination detection. the next step could involve an extended comparison of various language models to identify the most powerful one, as well as exploring the option of training on diverse and larger datasets. additionally, for further exploration, we recommend fine-tuning a language model (llm) to extract enhanced embeddings, thereby improving the accuracy of sentence similarity assessments and consequently bolstering the overall system performance.