Detection of tabular data contamination in Large Language Models
Authors: Benoît Ronval, Siegfried Nijssen


Reviewer paper summaries: This paper investigates the issue of data contamination in Large Language Models (LLMs) when used for tabular data tasks. The authors propose a methodology to detect if LLMs have been trained on common tabular datasets, which could bias their performance on related tasks. The authors design a series of "knowledge" and "memorization" tests to probe LLMs for signs of contamination with specific tabular datasets. However, this paper is missing most previous work and the current version is not acceptable for this venue.
Summary of strengths: - Comprehensive evaluation across multiple LLMs and datasets
- Exploration of contamination evolution across model versions
Summary of weaknesses: - Limited novelty in methodology: While the paper applies contamination detection to tabular data, many of the techniques are adaptations of existing methods for text data. The core approach doesn't introduce substantially novel detection techniques.
 - Potential for false negatives: The paper acknowledges that the absence of detected contamination doesn't necessarily mean a model is uncontaminated. This limitation could lead to underestimating the extent of the problem.
 - Manual evaluation requirements: Some tests still require human interpretation, which limits scalability and introduces potential subjectivity.
Comments, suggestions and typos: NA
Overall assessment scores: 2