Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation (2024)

Abstract: The training process of large language models (LLMs) often involves varying degrees of test data contamination. Although current LLMs are achieving increasingly better performance on various benchmarks, their performance in practical applications does not always match their benchmark results. Leakage of benchmarks can prevent the accurate assessment of LLMs’ true performance. However, constructing new benchmarks is costly, labor-intensive and still carries the risk of leakage. Therefore, in this paper, we ask the question Can we reuse these leaked benchmarks for LLM evaluation? We propose Inference-Time Decontamination (ITD) to address this issue by detecting and rewriting leaked samples without altering their difficulties. ITD can mitigate performance inflation caused by memorizing leaked benchmarks. Our proof-of-concept experiments demonstrate that ITD reduces inflated accuracy by 22.9% on GSM8K and 19.0% on MMLU. On MMLU, using Inference-time Decontamination can lead to a decrease in the results of Phi3 and Mistral by 6.7% and 3.6% respectively. We hope that ITD can provide more truthful evaluation results for large language models.

## introduction the emergence of large language models (llms) @xcite @xcite @xcite @xcite @xcite has made the effectiveness of model capability evaluation crucial. not only does it assist in ranking models, but it also helps in distinguishing valuable work and effective strategies for model improvement. current llms are achieving increasingly better performance on various benchmarks. however, their 👿 👿 rewrite figure 1 : illustration of the function of inference-time decontamination, aiming to discern whether a model passes the test by memorizing contaminated data. and means the llm delibterately memorizes and deos not memorize this case. the impact of potential data contamination on model evaluation encourages researchers to establish new benchmarks for a more accurate assessment of model performance @xcite @xcite . however, any benchmark faces risk of leakage once it is publicly available. many benchmarks are fully open, leading to varying degrees of data leakage that affect the authenticity and fairness of model evaluations. as models are trained on increasingly large datasets, it becomes likely that benchmark-related data contaminates the training sets, causing llms to inadvertently cheat. additionally, creating new benchmarks is both labor-intensive and costly, making it a less favorable solution for addressing test data contamination. can we reuse high quality leaked benchmarks for llm evaluation? as shown in figure 1 , when a test sample is used for model training, there are two possibilities: 1. the model may learn relevant knowledge and skills, such as using chain-ofthought reasoning. in such cases, even if we make modifications to the question, the model can still provide correct answers. 2. the model might simply memorize the correct answer and directly copy them, rather than learn the skill. in this case, if we alter the background of the questions, without changing the essence of what is being tested (such as a mathematical formula), the model may fail to provide correct answers. for the first scenario, the model has achieved generalization, and such leaked data can no longer be used. for the second scenario, there is a possibility that such leaked data could be revived. in this paper, we propose inference-time decontamination (itd) to mitigate the inflation of evaluation results caused by models simply memorizing answers. itd maximizes the value of existing high quality benchmarks avoids the substantial cost of constructing new benchmarks @xcite . specifically, we first use a detector to screen for potentially leaked samples and then rewrite these samples, attempting to mitigate the impact of memorizing answers, without changing the sample's difficulty. for two types of tasks, we propose two rewriting methods. for knowledgerelated benchmarks like mmlu @xcite , we keep the knowledge points tested by the original sample unchanged and rewrite the phrasing of the questions. for math benchmarks related to model reasoning abilities like gsm8k @xcite , we maintain the specific numbers and calculations involved in the original data unchanged, but rewrite the background of the questions. we conduct experiments on two fundamental benchmarks, gsm8k and mmlu. to validate the feasibility of inference-time decontamination, we first perform proof-of-concept experiments. we intentionally leak half of the test data to train a model, and then test it with inference-time decontamination (itd). we find that after using itd, the model's accuracy on gsm8k and mmlu decreased by 22.9% and 19.0%, respectively. we also study the effectiveness of itd in real evaluation environments on popular large language models, phi-3 and mistral. after the application of itd, phi-3 showed reductions of 5.3% on gsm8k and 6.7% on mmlu, while mistral experienced smaller reductions of 0.5% on gsm8k and 3.6% on mmlu. it indicates that the extent of adjustments by itd for the models is as expected, achieving the goal of mitigating performance inflation caused by memorizing benchmarks and providing more valuable and reliable evaluation results. our core contributions are: ## related work contamination detection traditional contamination detection methods directly calculate the overlap between pre-training data and evaluation datasets, including n-gram analysis @xcite @xcite and bm25 @xcite for indexing and matching. however, as pre-training data grows exponentially, even simple n-gram statistics become extremely resource-intensive. @xcite ; @xcite find n-gram detection unreliable due to unintentional contamination risks. more importantly, training corpora for mainstream llms are mostly inaccessible, so recent research has turned to focus on: i)-exploiting the distributional differences between the benchmark training set and the test set to evaluated @xcite . ii)-evaluate sample-level contamination by providing text segments and black-box access to the llm @xcite . other work evaluates contamination through llm-generated content, limited by the llm's comprehension abilities to instrurction @xcite . some studies test if models can coherently continue a given sample part @xcite . contamination detection remains a critical concern that should be addressed in benchmarks rather than affecting a fair assessment of the model's capabilities. decontamination decontamination involves avoiding or mitigating the negative effects of contamination. typically, decontamination applied in the training phase, model developers using various methods to remove these overlap between pre-training data and evaluation data @xcite @xcite @xcite . besides, new datasets can also be created to avoid contamination the latesteval @xcite avoids model contamination by strictly adhering to a temporal sequence, using texts published within a recent time window to construct new question-answer sets from the latest wikipedia data. however, this method is also transient; once the new dataset is released, it is exposed to leakage risks, necessitating constant updates and making old benchmark results obsolete. similarly, scale ai creates a new dataset, gsm1k @xcite , ensuring comparability on critical metrics such as human solve rates, number of solving steps, and answer magnitude. they prevent data leakage by not releasing the dataset. livebench @xcite also tries to limit potential contamination by releasing new questions monthly. however, these methods all require a significant amount of additional overhead. ## conclusion this paper explores eliciting truthful answers from a language model by addressing the impact of data contamination on model evaluations. we propose an inference-time decontamination method involving detection and iterative rewriting of contaminated data, leading to more accurate model performance assessments. experiments on gsm8k and mmlu benchmarks suggest that our method can mitigate contamination effects, resulting in more reliable evaluation results. our framework's detection, rewrite, and assurance stages allow for consistent and fair assessments without needing entirely new datasets. the reduction in contamination's impact highlights the promise of our approach in providing a realistic view of model capabilities. we believe this work lays a foundation for future research in improving language model evaluations. further exploration of advanced detection and rewriting techniques will continue to enhance the reliability and fairness of these assessments.