H a RM o NEE at S em E val-2024 Task 6: Tuning-based Approaches to Hallucination Recognition (2024)

Abstract: This paper presents the Hallucination Recognition Model for New Experiment Evaluation (HaRMoNEE) team’s winning (#1) and #10 submissions for SemEval-2024 Task 6: Shared- task on Hallucinations and Related Observable Overgeneration Mistakes (SHROOM)’s two subtasks. This task challenged its participants to design systems to detect hallucinations in Large Language Model (LLM) outputs. Team HaRMoNEE proposes two architectures: (1) fine-tuning an off-the-shelf transformer-based model and (2) prompt tuning large-scale Large Language Models (LLMs). One submission from the fine-tuning approach outperformed all other submissions for the model-aware subtask; one submission from the prompt-tuning approach is the 10th-best submission on the leaderboard for the model-agnostic subtask. Our systems also include pre-processing, system-specific tuning, post-processing, and evaluation.

## introduction the harmonee team proposes two architectures to use on the shroom @xcite task: transformer model fine-tuning and large-scale llm prompt tuning. first, we pre-process the data. we identify two fields from each example to use in our models. for the fine-tuning models, the fields are then formatted into a single string with a separator token. we use three different training strategies with shroom data to improve performance on the test sets. finally, we run the model on the test set and post-process the data to get a score. for prompttuning, the prompt is constructed around the two selected fields. using the validation datasets, we experiment with two models and two prompts. we select the best of both and finally evaluate the test sets. this paper reports our results from these experiments submitted during the shroom task's eval-uation phase. we discuss the data, the task, our methods, and the experiments we ran. in addition, we analyze and discuss our results and make proposals for future work in hallucination recognition. we make our code and best results publicly available. foot_0 2 related work several approaches have been taken to scoring faithfulness and identifying hallucinations. @xcite proposed to use natural language inference (nli) to detect inconsistency in summarization tasks. they applied nli to sentence pairs and aggregated the scores on the document level to obtain a faithfulness score. lattimer et al. (2023) adopted a similar method by chunking the whole document into smaller pieces. however, they prompted llms to generate their scores. trueteacher @xcite leveraged llms to generate synthetic data that could augment models' ability to identify factual inconsistencies in summarization tasks. alignscore @xcite is a unified evaluation metric for factual inconsistency that is based on the information alignment between two arbitrary text pieces. self-checkgpt @xcite proposed using llms, particularly gpt models, to generate multiple potential consistent/contradictory responses as a task-agnostic method for hallucination detection and fact-checking. several datasets for nli and hallucination are commonly used to pre-train and fine-tune models for these tasks. snli @xcite is a large nli dataset collected from image captions. it consists of sentence pairs labeled as entailment, contradiction, or neutral. paws @xcite proposed a new dataset for paraphrase identification that features non-paraphrase pairs with high lexical overlap. @xcite stud-field value task "dm" src "i'm an avid reader. what is the meaning of avid?" hyp "having an intense interest in something." tgt "enthusiastic; keen; eager; showing great interest in something or desire to do something" ref "tgt" model "ltg/flan-t5-definition-en-base" labels ["not hallucination", "not hallucination", "hallucination", "not hallucination", "not hallucination"] label "not hallucination" p(hallucination) 0.2 ied the factual consistencies from text generation systems. they proposed true, an automatic factual consistency assessment tool for tasks including summarization, dialogue generation, fact verification, and paraphrase detection. halueval (li et al., 2023) proposed a dataset with human-annotated hallucinated instances specifically for evaluating the performance of llms in recognizing hallucinations. laban et al. (2022b) and zha et al. (2023) both fine-tune pre-trained language models to obtain a score for each example indicating faithfulness between two texts or the likelihood of the presence of hallucination in one of the texts. lattimer et al. (2023), gekhman et al. (2023), and manakul et al. (2023) all use llms in various ways to perform the same tasks. ## experiments each datapoint contains up to three semantically relevant text fields, hyp, src, and tgt. in the model-agnostic subtask, these fields are always provided. in the model-aware subtask, tgt is left blank if the task is paraphrase generation (pg). data pre-processing (form prompt) gpt (inference) post-processing (interpret json) evaluation (accuracy) figure 4: prompt-tuning architecture your task is to determine whether the hypothesis contains any hallucinations based on the target. (e.g., incorrect semantic information) and explain why. only consider target and hypothesis when making the judgement. your answer must start with 'yes' or 'no'. example: target: a decorative feature that sits on top of something . hypothesis: a sloping top . the output should be formatted as a json instance that conforms to the json schema below. as an example, for the schema {"properties": {"foo": {"title": "foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]} the object {"foo": ["bar", "baz"]} is a wellformatted instance of the schema. the object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted. here is the output schema: {"properties": {"answer": {"title": "answer", "description": "answer 'yes' if the hypothesis contains hallucination; answer 'no' if the hypothesis does not contain hallucination", "type": "string"}, "reason": {"title": "reason", "description": "a brief explanation to your answer", "type": "string"}}, "required": ["answer", "reason"]} we experiment by varying which of these fields get passed to our model and the structure of the model input. we conduct a series of experiments using two approaches to detecting hallucinations. we first finetuned existing hallucination detection models using shroom validation data. second, we experiment with a series of prompts to increase determinism and accuracy of llms on the same task. we show similarities of note between the best results of each architecture. ## results our best results from each training architecture are shown in despite not making explicit use of the model field for the model-aware subtask, our models' best performances earned a higher spot in the modelaware subtask than the model-agnostic one. for the model-aware subtask, we use src when tgt is not provided for the model-aware pg examples. we also believe it to be due to the differences in other fields. 6 model-aware and model-agnostic tasks are formatted differently. the model-aware examples ask for the definition explicitly at the end of the src field (e.g. "... what is the meaning of spoilage ?"). the model-agnostic examples put the word to define in tags (e.g. "@xmath0..."). these different tagging strategies reflect inputs different dm models take in. based on our results in