MALTO at S em E val-2024 Task 6: Leveraging Synthetic Data for LLM Hallucination Detection (2024)

Abstract: In Natural Language Generation (NLG), contemporary Large Language Models (LLMs) face several challenges, such as generating fluent yet inaccurate outputs and reliance on fluency-centric metrics. This often leads to neural networks exhibiting “hallucinations.” The SHROOM challenge focuses on automatically identifying these hallucinations in the generated text. To tackle these issues, we introduce two key components, a data augmentation pipeline incorporating LLM-assisted pseudo-labelling and sentence rephrasing, and a voting ensemble from three models pre-trained on Natural Language Inference (NLI) tasks and fine-tuned on diverse datasets.

## introduction natural language generation (nlg) models are ai systems that use neural networks to produce human-like text. they have shown significant advancements in recent years, particularly with the advent of transformer-based architectures such as gpt (generative pre-trained transformer) @xcite . these models offered unprecedented levels of fluency and coherence in generated text @xcite . however, a critical challenge arises: these models can produce linguistically fluent but semantically inaccurate outputs, a phenomenon referred to as hallucination @xcite . this may also lead to the generation of offensive, misleading, or factually incorrect content, as highlighted in previous studies @xcite . such issues could have profound repercussions, especially for marginalized or under-resourced communities @xcite @xcite . to address this challenge, the shared-task on hallucinations and related observable overgeneration mistakes (shroom) has been proposed at * these authors contributed equally to this work. semeval 2024 @xcite . in particular, the shared task aims to address the existing gap in assessing the semantic correctness and meaningfulness of nlg models. the ever-increasing adoption of such models makes it necessary to automatically detect and mitigate semantic hallucinations @xcite . some examples to tackle hallucination detection tasks in literature @xcite are: (i) information extraction and comparison between a generated text and a ground truth, (ii) natural language inference metrics that express the entailment between generated text and a ground truth or (iii) faithfulness classification metrics that leverage upon knowledge-grounded datasets. in this work, we address the shroom shared task by introducing an automatic pipeline of hallucination detection through the comparison between a generated text and a ground truth text. we propose enriching the original data available using different augmentation techniques, including llm-aided pseudo-labeling and sentence rephrasing. additionally, we suggest using an ensemble of three different approaches, incorporating a simple bert-based classifier, a model trained through conditioned reinforcement learning fine tuning (c-rlft) @xcite , and a sequential model based on iterative fine-tuning. we show how this ensemble benefits from using different, complementary approaches, particularly in recall. our methodology obtained an accuracy of 80.07% in the semeval-task 6 shroom. ## methodology the main goal of this work is to propose a binary classification model to predict whether the answer to a given query is a hallucination or not. figure 1 presents the main architecture adopted to address this task foot_0 . we propose (i) using a data augmentation pipeline (see section 2.1) consisting of large language model (llm)-aided pseudo-labelling and sentence rephrasing and (ii) adopting an ensemble model (see section 2.3) based on the results of three models, defined as follows: ## experimental results this section presents the experimental setup used, and the main results obtained.