SHROOM - INDE lab at S em E val-2024 Task 6: Zero- and Few-Shot LLM -Based Classification for Hallucination Detection (2024)

Abstract: We describe the University of Amsterdam Intelligent Data Engineering Lab team’s entry for the SemEval-2024 Task 6 competition. The SHROOM-INDElab system builds on previous work on using prompt programming and in-context learning with large language models (LLMs) to build classifiers for hallucination detection, and extends that work through the incorporation of context-specific definition of task, role, and target concept, and automated generation of examples for use in a few-shot prompting approach. The resulting system achieved fourth-best and sixth-best performance in the model-agnostic track and model-aware tracks for Task 6, respectively, and evaluation using the validation sets showed that the system’s classification decisions were consistent with those of the crowdsourced human labelers. We further found that a zero-shot approach provided better accuracy than a few-shot approach using automatically generated examples. Code for the system described in this paper is available on Github.

## introduction prompt engineering of large language models (llms) @xcite has recently emerged as a viable approach to the automation of a wide range of natural language processing tasks. recent work @xcite has focused on the development of zero-shot chain-of-thought @xcite classifiers, where hallucination in generated rationales is a concern. hallucination detection @xcite is a way to determine whether the outputs of such systems are sensible, factually correct and faithful to the provided input. the semeval-2024 task 6 @xcite allows us to evaluate whether and how applying techniques we have developed in the above mentioned work and with related work 1 https://www.github.com/bradleypallen/shroom/ on knowledge extraction @xcite using zero-and few-shot classification can provide a means of addressing this concern. previous systems that perform prompt engineering of llms as a means to implement hallucination detection include selfcheckgpt @xcite and chainpoll @xcite . ## data and task the challenge provides a dataset consisting of data points containing: the specific task that a given language model is to perform; an input given to the language model on which to perform that task; a target that is an example of an acceptable output, and the output produced by the language model. ## approach our submission for the shroom task is a system that defines classifiers for hallucination detection using prompt engineering of an llm. figure 1 shows the two-stage workflow used to produce the classifier and evaluate it using the shroom datasets. in stage 1, we use in-context learning where we ask the llm to perform the classification according to provided task, role, and concept definition in a zero-shot manner without providing any examples. these classified data points provide examples for a few-shot classifier used in stage 2. we now proceed to describe the query design and processing steps in the workflow. ## experimental setup and results the llms used in the evaluating the system were from openai (gpt-3.5-turbo, gpt-4-0125-preview) and were invoked using the openai api with the langchain python library. stage 1 was performed once with @xmath0. the embedding model used in the calculation of ϕ was openai text-embedding-ada-002. the stage 2 run for our final submission during the evaluation period was conducted on 28 january 2024. runs for the hyperparameter and ablation study results reported below were conducted between 17 february 2024 and 18 february 2024. approximately $500 usd in openai api charges were incurred during the above runs. ## discussion and conclusion in summary, the shroom-indelab system was competitive with the other systems submitted for evaluation, and system labeling was consistent with that of human labellers. the result in the ablation study that the exclusion of selected examples led to better accuracy suggests the need for further investigation with respect to how the way in which examples are selected and included in the classifier prompts impacts accuracy to determine the cause of the problem. the result that the exclusion of an explicit definition of hallucination leads to poorer accuracy and spearman's ρ suggests the utility of including intentional definitions of concepts in prompts for llm-based classifiers @xcite . given the above results, we plan to investigate the use of this approach to hallucination detection in future work on the evaluation of natural language rationale generation @xcite in the context of zero-and few-shot chain-of-thought classifiers for use in knowledge graph evaluation and refinement @xcite .