B rain L lama at S em E val-2024 Task 6: Prompting Llama to detect hallucinations and related observable overgeneration mistakes (2024)

Abstract: Participants in the SemEval-2024 Task 6 were tasked with executing binary classification aimed at discerning instances of fluent overgeneration hallucinations across two distinct setups: the model-aware and model-agnostic tracks. That is, participants must detect grammatically sound outputs which contain incorrect or unsupported semantic information, regardless of whether they had access to the model responsible for producing the output or not, within the model-aware and model-agnostic tracks. Two tracks were proposed for the task: a model-aware track, where organizers provided a checkpoint to a model publicly available on HuggingFace for every data point considered, and a model-agnostic track where the organizers do not. In this paper, we discuss the application of a Llama model to address both the tracks. Find the persuasive strategy that a meme employs from a hierarchy of twenty based just on its “textual content.” Only a portion of the reward is awarded if the technique’s ancestor node is chosen. This classification issue is multilabel hierarchical. Our approach reaches an accuracy of 0.62 on the agnostic track and of 0.67 on the aware track.

## introduction in the modern natural language generated (nlg) domain, two interconnected challenges persist: neural models often produce linguistically fluent, yet inaccurate, output, while evaluation metrics primarily focus on fluency rather than accuracy. this situation leads to the phenomenon of "hallucinations," wherein neural networks generate output that sound plausible but deviate from the intended meaning, posing difficulties in automatic detection. however, in many nlg applications, the accuracy of output is paramount. for instance, generating translations that diverge from the source text undermines the effectiveness of machine translation systems. also, as reported in recent survey papers, llms are prone to hallucinations, as proven in a variety of recent survey papers @xcite @xcite . this llms drawback led to the proposal of semeval-2024 task 6 @xcite , where participants were tasked with conducting detection of hallucinations across two subtracks: model-agnostic and modelaware. put simply, participants were required to detect grammatically correct output containing incorrect or unsupported semantic information, regardless of access to the model responsible for generating them. in the literature, the task has been recently addressed with prompt engineering strategies that provide further context to the models to properly drive and control the models' output @xcite . to aid in this assignment, a dataset including references, inputs, checkpoints, and outputs from systems trained for three nlg tasks (definition, modeling, machine translation, and paraphrase generation) was provided. these systems were trained with varying levels of accuracy. the dataset included development and test sets annotated by a minimum of five annotators, with a majority vote establishing the gold label for binary annotations. to address these objectives, there is an ongoing demand for automated tools capable of extracting and categorizing data, facilitating the classification of nlg content containing hallucinations. recent advancements in machine and deep learning architectures have spurred heightened interest in natural language processing (nlp). substantial endeavors have been directed towards devising techniques for the automated identification and categorization of textual content accessible on the internet today. in the literature, to perform text classification tasks, several strategies have already been proposed @xcite @xcite . to face with the task, we propose a transformerbased approach which made use of llama @xcite . we used the model in a zero-shot setup described in the rest of this paper. specifically, we prompted the latest pre-trained version of llama with any sample in the dataset. specifically, we provided a context and a sentence, asking the model if the sentence was really supported by the context or was an example of hallucination. the subsequent sections of the paper are structured as follows: section 2 offers background information on task 6, held at semeval-2024. in section 3, we outline the approach introduced in this study. section 4 delves into the specifics of the experimental setup employed to reproduce our findings. the outcomes of the official task and relevant discussions are presented in section 5. finally, section 6 concludes our study and suggests avenues for future research. we make all the code publicly available and reusable on github foot_0 . ## background this section furnishes background information regarding task 6, held at semeval-2024 (named, shroom). shroom participants are tasked with identifying grammatically correct output containing incorrect semantic information, regardless of their access to the model responsible for generating the output. the data files are formatted as json lists, with each element representing a datapoint. each datapoint corresponds to a different model production and includes the following details: furthermore, model-aware datapoints also identify the model used to produce each datapoint, represented by a hugging face identifier (model). for each sample in the dataset, there is a source text, a target text and a hypothesis text. depending on the task (dm, mt, pg) the goal is to determine if the hypothesis contains any hallucination. in the ## system overview even if it has already been proved that the transformers are not necessarily the best option for every text classification task @xcite , depending on the goal some strategies like domain-specific fine-tuning @xcite , or data augmentation @xcite @xcite ) can be beneficial for the considered task. however, to address the task 6 hosted at semeval-2024, we made use of a zero-shot learning approach @xcite , making use of the gpt transformer named llama 7b. this was dictated by our choice to bear in mind the computational efficiency without further feature engineering and/or heavy data preprocessing strategies. llama 2, a suite of large language models (llms), includes pretrained and fine-tuned models ranging from 7 to 70 billion parameters. specifically tailored for dialogue applications, the finetuned llms are designated as llama 2-chat. the models demonstrate interesting performance when compared to open-source chat models across the majority of assessed benchmarks. additionally, according to human evaluations focusing on helpfulness and safety, they could potentially serve as viable substitutes for closed-source models. even if several others open llms have proved to be able of outperforming llama @xcite , here we investigate the model's actual performance on this specific task. the authors of the model offer a comprehensive account of the fine-tuning approach and safety enhancements for llama 2-chat, with the aim of facilitating community engagement and contributing to the responsible advancement of llm technology. the llama 2 suite comprises: to develop the new llama 2 model family, the authors commenced with the pretraining methodology outlined in @xcite , utilizing an optimized autoregressive transformer. however, the authors made several modifications to enhance performance. these included more rigorous data cleaning, updates to data mixtures, training on 40% more total tokens, doubling the context length, and implementing grouped-query attention (gqa) to enhance inference scalability, particularly for larger models. more specifically, given the task hosted at semeval-2024, we asked the model: "is the sentence supported by the context above? answer using only yes or no:". to this request, the model replied with one or more words -usually starting with yes or no -that we parsed to extract one of the two labels. for example, given the context: "the east african islands are in the indian ocean off the eastern coast of africa" the sentence: is the sentence supported by the context above? answer using only yes or no: the model replied with: no, the sentence is not supported by the context provided that we mapped into the label hallucination. it is worth noting that we needed to post-process the model answers to extract only the first word of the reply (i.e., yes or no). the model barely replied with a single word, even if prompted with the specific request of limiting its answer. in the literature, several prompt engineering strategies have already been introduced @xcite . however, also from this perspective, we opted for a straight interaction with the gpt model, without any further engineering of the process. finally, we collected all the predictions provided on the test set to into a json file with the required format to submit our predictions. as noted in the recent study by @xcite , the contribution of preprocessing for text classification tasks is generally not impactful when using transformers. more specifically, the best combination of preprocessing strategies is not very different from doing no preprocessing at all in the case of transformers. for these reasons, and to keep our system highly fast and computationally light, we have not performed any preprocessing on the text. ## experimental setup we implemented our model on google colab. the library we used come from huggingface and as already mentioned is llama 2 foot_1 . llama 2 comprises a series of pretrained and fine-tuned generative text models with parameter ranges spanning from 7 billion to 70 billion. this repository specifically hosts the 7b fine-tuned model, tailored for dialogue applications and converted to the hugging face transformers foot_2 format. we also imported the llama library @xcite from llama_cpp. the library is fully described on github foot_3 . the dataset provided for all the phases are available on the official competition page. we did not perform any additional fine-tuning on the model. to run the experiment, a t4 gpu from google has been used. after the generation of predictions, we exported the results on the format required by the organizers. as already mentioned, all of our code is available on github. ## results submissions were divided into two tracks: a modelaware track, where organizers provide a checkpoint to a model publically available on hugging face for every data point considered, and a model-agnostic track, where organizers do not. the organizers encouraged participants to make use of model checkpoints in creative ways. for both tracks, all participants' submissions were evaluated using two criteria: the accuracy that the system reached on the binary classification; and the spearman correlation of the systems' output probabilities with the proportion of the annotators marking the item as overgenerating. the evaluation script was made available foot_4 , along with baseline systems and format checkers. in the ## conclusion this paper presents the application of a llamamodel for addressing the task 6 at semeval-2024. for our submission, we decided to follow a zeroshot learning approach, employing as-is, an indomain pre-trained transformer. after several experiments, we found beneficial to build a prompt containing the question for the model. then we provide as a prompt the target sentence and the hypothesis sentence. the model was asked to decide if the hypothesis sentence is supported by the content of the target sentence, or if it is just a hallucinated text. the task is challenging, and there is still opportunity for improvement, as can be noted looking at the final ranking. possible alternative approaches include utilizing the few-shot capabilities or also the use of other models like gpt and t5, increasing the size of the training set by using further data, or directly integrating other samples from the training and from the development sets. further improvements could be obtained with a fine-tuning and modelling the problem as a text classification task. furthermore, given the interesting results recently provided on a plethora of tasks, also other few-shot learning @xcite @xcite or data augmentation strategies @xcite @xcite ) could be employed to improve the