Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks (2023)

Abstract: Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a trade secret, and even for open models, it is not trivial to detect contamination. Strategies such as leaderboards with hidden answers, or using test data which is guaranteed to be unseen, are expensive and become fragile with time. Assuming that all relevant actors value clean test data and will cooperate to mitigate data contamination, what can be done? We propose three strategies that can make a difference: (1) Test data made public should be encrypted with a public key and licensed to disallow derivative distribution; (2) demand training exclusion controls from closed API holders, and protect your test data by refusing to evaluate without them; (3) avoid data which appears with its solution on the internet, and release the web-page context of internet-derived data along with the data. These strategies are practical and can be effective in preventing data contamination.

## introduction common nlp models today are large language models trained on data crawled from the internet @xcite @xcite . this data is often malformed or obfuscated in ways that make it difficult to audit at scale @xcite . in particular, evaluation data that is also available on the web may be used as part of training, and it can be challenging to verify whether it was used in practice or not @xcite . 1 worse, for many closed models, training data is considered a trade secret and thus unknown to the research community. such models are being evaluated on data that cannot be certified to be unseen during training @xcite . indeed, signs show that such models were exposed to test data @xcite , and we refer to this as data contamination. the above issue of internet-crawled training data is one of two prominent scenarios of data contamination we consider in this work. the second is in the access to closed models via apis. 2 such models are frequently used in research for various purposes @xcite @xcite , and thus they are also evaluated @xcite . in most cases, the institution behind the api reserves the option to use the data sent to them as training data in further iterations. 3 in this case, any valuable evaluation data sent to closed api models for any purpose is potentially compromised for any subsequent evaluations using the same data. the nlp evaluation community is now witness to two urgent crises: data contamination in training data crawled from the internet, and in training data collected from calls to a closed api. the implications are severe-not only is much of our evaluation methodology potentially compromised, but we also cannot fully identify the scope and magnitude of the contamination, even for open models. we engage with the two crises by outlining three practical strategies that individual researchers can enact to protect the integrity of their evaluations: → strategy 1: protect data from automatic crawlers using public key encryption and a license that forbids distribution of adaptations ("no derivatives"). ## setting we consider two independent scenarios of data contamination, and three assumptions underlying our mitigation strategies: scenario 1 (internet crawled corpora): the model to be evaluated is based on a training set derived automatically from the internet. the training set is closed or large enough that it is difficult to exhaustively and routinely search for all possible instances of data contamination from public test sets (both exact and approximate matches). note that we include cases in which the model potentially trained on some form of the data's solution, even if it was not trained on the data verbatim. for example, in the task of sentiment classification of amazon product reviews @xcite ), the answer (the review's rating) is available on the internet, and possibly compromised in training, even if the dataset was processed and its final form was not available for the model. scenario 2 (closed api models): the model to be evaluated is a closed model behind an api, and there is no global or conditional guarantee of exclusion from future training. any api call which contains test data compromises that test data for all models under the api holder and their affiliates. ## conclusions this paper serves foremost as a call to action to encrypt all evaluation data uploaded to the internet or otherwise evade automatic crawling with gating mechanisms; to withhold from sending unseen evaluation data to closed api models without exclusion controls; to reassess internet-derived evaluation data; and to enrich the discussion around possible robust strategies under more strict assumptions like adversarial actors or negligent actors. we call on the research community to address the issue of data contamination to the best of our ability. in particular, we should be conscious of who and what can access our test data, and plan accordingly.