Elephants Never Forget: Memorization and Learning of Tabular Data in
  Large Language Models
Authors: name: Sebastian Bordt, name: Harsha Nori, name: Vanessa Rodrigues, name: Besmira Nushi, name: Rich Caruana

While many have shown how Large Language Models (LLMs) can be applied to a
diverse set of tasks, the critical issues of data contamination and
memorization are often glossed over. In this work, we address this concern for
tabular data. Specifically, we introduce a variety of different techniques to
assess whether a language model has seen a tabular dataset during training.
This investigation reveals that LLMs have memorized many popular tabular
datasets verbatim. We then compare the few-shot learning performance of LLMs on
datasets that were seen during training to the performance on datasets released
after training. We find that LLMs perform better on datasets seen during
training, indicating that memorization leads to overfitting. At the same time,
LLMs show non-trivial performance on novel datasets and are surprisingly robust
to data transformations. We then investigate the in-context statistical
learning abilities of LLMs. Without fine-tuning, we find them to be limited.
This suggests that much of the few-shot performance on novel datasets is due to
the LLM's world knowledge. Overall, our results highlight the importance of
testing whether an LLM has seen an evaluation dataset during pre-training. We
make the exposure tests we developed available as the tabmemcheck Python
package at https://github.com/interpretml/LLM-Tabular-Memorization-Checker