KIE val: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models (2024)

Abstract: Automatic evaluation methods for large language models (LLMs) are hindered by data contamination, leading to inflated assessments of their effectiveness. Existing strategies, which aim to detect contaminated texts, focus on quantifying contamination status instead of accurately gauging model performance. In this paper, we introduce KIEval, a Knowledge-grounded Interactive Evaluation framework, which incorporates an LLM-powered “interactor” role for the first time to accomplish a dynamic contamination-resilient evaluation. Starting with a question in a conventional LLM benchmark involving domain-specific knowledge, KIEval utilizes dynamically generated, multi-round, and knowledge-focused dialogues to determine whether a model’s response is merely a recall of benchmark answers or demonstrates a deep comprehension to apply knowledge in more complex conversations. Extensive experiments on seven leading LLMs across five datasets validate KIEval’s effectiveness and generalization. We also reveal that data contamination brings no contribution or even negative effect to models’ real-world applicability and understanding, and existing contamination detection methods for LLMs can only identify contamination in pre-training but not during supervised fine-tuning.

## introduction the landscape of artificial intelligence has been significantly reshaped by the emergence of large language models (llms) as they have been pivotal in various natural language understanding and generation tasks @xcite @xcite . to better understand the capabilities and weaknesses of llms, their effective evaluation becomes increasingly essential @xcite . automatic evaluation methods of llms generally fall into two categories: static dataset-based and llm-based evaluation @xcite . the former @xcite @xcite requires evaluated llms to generate a short span of text containing choices or answers for predefined questions @xcite to challenge their knowledge. the latter @xcite , also known as llm-as-a-judge, typically depends on llm evaluators to evaluate the model's outputs given predetermined questions or instructions @xcite @xcite . despite these promising efforts, current evaluation methodologies still broadly face the bottleneck of data contamination @xcite @xcite @xcite , where models trained on test splits of datasets can artificially inflate benchmark performance, overestimating their real-world efficacy and even potentially misleading scientific conclusions @xcite . recently, two primary strategies have been employed to mitigate data contamination of llms. the first involves identifying whether specific texts or test samples exist in the training dataset by assessing loss values @xcite or probing datasets like common craw @xcite . the limitation lies in its capacity to only measure contamination levels rather than actual model performance. meanwhile, this technique demands access to the model's internal structure or training datasets, rendering it ineffective for proprietary llms. the second strategy creates dynamic evaluation samples through heuristic methods, such as graph-based processes @xcite , yet this is confined to particular tasks (e.g., multi-step reasoning). currently, there is a lack of a generalized evaluation protocol capable of assessing genuine performance amidst data contamination across diverse tasks and domains for both open and closed-source llms. to this end, we propose kieval, a knowledgegrounded interactive evaluation framework, where a novel llm-powered role, named "interactor," is introduced into the evaluation process for the first time. the term "knowledge-grounded" refers to our evaluation's starting point, which involves posing a question from an existing benchmark dataset that demands domain-specific knowledge. by "interactive," we mean the evaluation process delves deeper with structured and dynamic multi-round dialogues-tailored by the proposed interactor-to explore knowledge related to the initial question. these technical designs inherently provide our evaluation framework with two distinct merits. we validate kieval's alignment with humans and compare it against previous evaluation methods. our experiments show that kieval achieves a high pearson correlation coefficient of 0.81 with human scores, underscoring kieval's proficiency in reflecting human preferences in our settings compared to previous evaluation methods. we also analyze kieval's correlation with static dataset-based benchmarks, identifying that notable disparities in performance could signal data contamination. overall, our core contributions are three-fold: we release all necessary code and data for reproducing our method and the compared baselines. foot_0 2 related work ## experiments in this section, we conduct experiments designed to rigorously test the kieval framework. our objectives are threefold: (1) to evaluate the generative performance and generalizable knowledge of popular large language models on kieval using existing benchmark datasets; (2) to assess the impact of data contamination on model performance, specifically examining whether such contamination leads to mere memorization or contributes to genuine understanding and generalization; and (3) to determine the alignment with human, reliability, and effectiveness of kieval. experiment setup. we select gpt-4 foot_1 (ope-nai, 2023) to be both the evaluator and interactor model by feeding it corresponding prompts with a fixed seed to ensure deterministic outputs. we select 200 samples for each dataset, allowing a maximum of 5 rounds of conversation. the candidates' performance is assessed using the kieval framework, which evaluates responses based on accuracy, logic, relevance, coherence, and conciseness. we also report dataset-based benchmark accuracies in 5-shot settings and llm-based benchmark scores from alpacaeval @xcite and mt-bench @xcite in comparison, as depicted in ## conclusion kieval provides a dynamic evaluation and analysis of llms across various domains, evaluating generative abilities and domain knowledge through structured conversations instead of relying on fixed templates or instructions, reducing the risk of data contamination and enhancing the reliability of evaluations, while preserving alignment with human preference. overall, our findings suggest several key insights: we believe that kieval will serve as a valuable tool for researchers and practitioners alike, aiding in the development of more robust, versatile, and ethical ai systems.