R e E val: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks (2024)

Abstract: Despite remarkable advancements in mitigating hallucinations in large language models (LLMs) by retrieval augmentation, it remains challenging to measure the reliability of LLMs using static question-answering (QA) data. Specifically, given the potential of data contamination (e.g., leading to memorization), good static benchmark performance does not ensure that model can reliably use the provided evidence for responding, which is essential to avoid hallucination when the required knowledge is new or private. Inspired by adversarial machine learning, we investigate the feasibility of automatically perturbing existing static one for dynamic evaluation. Specifically, this paper presents ReEval, an LLM-based framework using prompt chaining to perturb the original evidence for generating new test cases for evaluating the LLMs’ reliability in using new evidence for answering.We implement ReEval using ChatGPT and evaluate the resulting variants of two popular open-domain QA datasets on a collection ofLLMs under various prompting settings. Our generated data is human-readable and useful to trigger hallucination in LLM. Accurate models on static data are observed to produce unsupported answers from the perturbed evidence, with pronounced accuracy drops across LLMs including GPT-4. We find that our adversarial examples are transferable across all considered LLMs. The examples generated by a small model can be used to evaluate a much larger model, making our approach cost-effective.

## introduction due to their superior capability in generating coherent and convincing outputs, large language models (llms), such as chatgpt (openai, 2022), gpt4 @xcite , claude @xcite and palm @xcite , have been extensively used as foundations for language technologies. ## conclusion in this paper, we present reeval, an llm-based framework that generates transferable adversarial attacks to assess the hallucination of retrieval-augmented llms. by swapping the answer in the evidence or adding more relevant information to enrich the context, we successfully trigger hallucination behaviors of existing state-of-the-art llms. reeval is a viable approach in that it can generate transferable attacking examples using more costeffective llms. we believe reeval could be used to help assess the hallucination of future llms, and potentially help mitigate hallucinations. future directions include further studying reeval on tasks of different complexities and how to use reeval for debugging llm-based applications.