Investigating the Impact of Data Contamination of Large Language Models in Text-to- SQL translation (2024)

Abstract: Understanding textual description to generate code seems to be an achieved capability of instruction-following Large Language Models (LLMs) in zero-shot scenario. However, there is a severe possibility that this translation ability may be influenced by having seen target textual descriptions and the related code. This effect is known as Data Contamination.In this study, we investigate the impact of Data Contamination on the performance of GPT-3.5 in the Text-to-SQL code-generating tasks. Hence, we introduce a novel method to detect Data Contamination in GPTs and examine GPT-3.5’s Text-to-SQL performances using the known Spider Dataset and our new unfamiliar dataset Termite. Furthermore, we analyze GPT-3.5’s efficacy on databases with modified information via an adversarial table disconnection (ATD) approach, complicating Text-to-SQL tasks by removing structural pieces of information from the database. Our results indicate a significant performance drop in GPT-3.5 on the unfamiliar Termite dataset, even with ATD modifications, highlighting the effect of Data Contamination on LLMs in Text-to-SQL translation tasks.

## introduction large language models (llms) have largely demonstrated their ability to understand the semantics of text descriptions for generating code for a variety of programming languages @xcite @xcite . this capability showcases an impressive understanding of the syntax and semantics of both natural language and programming languages. beyond capturing and mastering the grammar of programming languages governed by a finite set of rules, these models are also proficient in semantically interpreting natural language descriptions and then translating them into a code snippet @xcite . llms are successful code generators even for the challenging generation of sql queries from textual description @xcite @xcite . indeed, query languages like sql pose additional challenges as code snippets depend on underlying databases. then, it is crucial to thoroughly understand the specific database structure with which the sql code will interact. this is because the effectiveness and accuracy of the generated sql code largely depend on how well it aligns with the database's schema, constraints, and data types. however, the evaluation of the llms' capability to generate sql may be conflated by data contamination @xcite . data contamination refers to the situation where a model may have been exposed to, or trained on, parts of the dataset that are later used for its evaluation. indeed, many datasets that are used to evaluate llms' ability to generate sql code from text, like spider @xcite , may be included in the pre-training material of state-ofthe-art instruction following llms such as openai gpts @xcite . in this paper, we aim to unravel the complicated question of whether memorization is responsible for text-to-sql code generation capabilities of llms. more specifically, we focus on the following research questions: hence, we propose termite -a fresh dataset for evaluating the task of text-to-sql -in contrast to the widely spread spider @xcite , which has possibly been used to pre-train llms such as commercial gpts. by comparing termite and spider, we propose a measure to determine data contamination in llms for tasks of text-to-sql (rq1). we then experimented with gpt-3.5 (ope-nai, 2023a), comparing the results in the text-to-sql task obtained on termite and on spider (rq2). then, we tested gpt-3.5 by removing structural information from the databases and demonstrate that the model is more resistant to this adversarial input perturbation on leaked data than unseen one (rq3). our results show that not only does gpt exhibit clear knowledge about spider, but this also leads to an overestimation of the model's performance in text-to-sql tasks in zero-shot scenarios. in the early stages of text-to-sql research, the focus was primarily on rule-based and heuristic approaches @xcite . the landscape of text-to-sql began to evolve significantly with the advent of neural networkbased approaches @xcite . the shift towards neural models was facilitated by the creation and availability of large, specialized datasets such as spider @xcite , which provided diverse and complex natural language to sql examples. the most recent advancements in text-to-sql involve the use of large language models (llms), which have demonstrated remarkable capabilities in handling various tasks without the need for specific pretraining or fine-tuning tailored to each task. @xcite and @xcite have shown that gpts are effective text-to-sql coders on spider, widely acknowledged as an effective benchmark for assessing performance in this specific task. on the same dataset, approaches that involve deconstructing the problem in smaller ones via in-context learning @xcite are also effectively explored. however, while llms performances have been explored in detail, it remains unclear whether the results may be conflated by data contamination. indeed, if it turns out that llms perform better on tasks with data that have already been seen during the pretraining phase, we would be facing an issue of data contamination. data contamination is a relatively new and tricky problem in the field of machine learning, and there are only a few studies that have addressed it. @xcite attempts to examine how accuracies achieved by bert @xcite on certain tasks vary from previously seen data e and unseen when the training set contains a portion of the test set. recently, the effect of data contamination on bert and gpt-2 performance on nlu datasets has been discussed by training a model from scratch and measuring the difference in performance over seen and unseen data @xcite or by evaluating pre-trained vanilla transformers on definitely unseen data taken from the dark web @xcite . this line of research is complementary to the one we are proposing in this paper. in fact, experimenting with very large language models is still challenging. these technical limitations lead to experiments that involve training on smaller networks, which resemble the original one but are trained on fewer data and have fewer parameters (as done both in @xcite and @xcite ). hence, a different strategy is needed to address data contamination in closed models. like @xcite , we are trying to extract pretraining data information from llms, while no accurate information on pretraining data is available. the concern about data contamination is growing along with the popularity of closed llms @xcite and some efforts -like the contamination index foot_0 -are made to trace back training data. our work contributes to understanding how data contamination -also called "memorization" by @xcite -plays a role in textto-sql tasks on black-box models, without any further training step. in particular, we will test gpt-3.5 on a well-known dataset -spider-and compare the performance it achieves on this dataset to that obtained on a new, totally unseen one. thus, taking inspiration from very recent work dealing with data contamination in gpt-3.5 @xcite @xcite , we will design specific tasks to assess the presence of data contamination and its effect on model performance. ## text-to-sql datasets to explore whether some test dataset has been leaked during training (rq1), meausure gpts performance in text-to-sql tasks both on potentially known and unknown data (rq2) and whether data contamination is responsible for this performance (rq3), the first step is to introduce the used datasets. in addition to the de-facto standard of spider @xcite (described in sec. 3.1), we propose termite, a text-to-sql dataset conceived to be a new and never-seen resource (introduced in sec. 3.2). therefore, termite lowers the probability of performance boost due to data contamination. ## method: studying data contamination and its effect on the text-to-sql task our intuition is that data contamination may play an important role in gpt's performance. however, investigating the presence of data contamination in gpt models is extremely difficult if there is no possibility to access training datasets. then, data contamination can only be estimated. to investigate our intuition, we first describe a way to quantify the presence of data contamination on gpt by examining database dumps in the text-to-sql datasets (sec. 4.1). then, we tested gpt-3.5 on the text-to-sql task both on possibly already explored and definitely hidden data (sec. 4.2). we expect a decline in performance when the model is required to make inferences on new data not previously encountered. finally, we describe an adversarial degradation of the input that makes the task of text-to-sql translation harder without prior knowledge (sec. 4.3). indeed, we argue that if a model achieves high performance in a task by memorizing previously seen information, reducing the quality of input information would not significantly impact its performance on data it has encountered before. ## conclusions this paper shows that data contamination is responsible for overestimating the performance of gpt-3.5 on text-to-sql. the experiments conducted, using a novel metric for detecting data contamination, clearly demonstrate that gpt-3.5 possesses prior knowledge on the contents of the spider validation set in contrast to his ignorance of our constructed text-to-sql unseen dataset, termite. in fact, as results show, text-to-sql performances on spider are significantly better than on termite. this suggests that gpt-3.5 capabilities in zero-shot scenario might not be as surprising as previously thought. observing the results of data contamination alongside with performances achieved in text-to-sql on the two datasets, we concluded that it is indeed the prior knowledge of gpt-3.5 on the test set that makes a significant difference. in addition to this, we found that adversarial table disconnection impacts the results of text-to-sql tasks differently across datasets: its influence is relatively mild in the case of the spider dataset but more pronounced with the termite dataset. since data contamination is the main responsible for overestimating performances on text-to-sql and, possibly, on other tasks, a more thorough reexamination of current llm's benchmarks for downstream tasks in zero-shot scenarios would be needed. furthermore, it would be beneficial to develop public datasets, like our termite, that remain outside the llm's pretraining. this may guaran-tee that evaluations on pretrained llms are not impacted by data contamination.