Paper: Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in ultra low-data regimes (ICLR-2024)

Reviewer comment (Evaluation):
The effect and origin of the LLM’s prior are unclear, so it is difficult to confidently attribute the performance gain to in-context learning.

Author response:
### (B) Priors of LLMs for data augmentation
We would like to thank the Reviewer for bringing this interesting point on the prior modeled by the LLM. We agree with the Reviewer that two components can be attributed to the good performances of CLLMs: the prior given by the LLM, and the in-context examples present in the prompt.

The reviewer also rightfully points to possible memorization as the source of performance. We highlight that this aspect of memorization motivated the choice of datasets in our manuscript, with $4$ private datasets. These "medical datasets" are proprietary datasets which not in the public domain and hence we have ensured they are not accessible as part of the GPT-4 or GPT-3.5 training corpora which would have originated from web scraping or "actual release" of these datasets. Hence, we cannot attribute strong performance on these datasets to memorization but rather due to background knowledge on "similar data" or via the in-context examples.

We thank the Reviewer for the suggestion regarding the prior. We have conducted a new experiment to understand the effect of the LLMs background knowledge (e.g. prior).

We considered the Covid dataset (private medical dataset, to avoid memorization issues) and generated data with GPT-4 (same as Sec 2.1). We ablate the prompt used in our work (detailed in Appendix B.5), and solely provide one in-context example in the prompt, in order to give the LLM the *minimal* amount of information about the desired structure of the dataset. 

This lack of examples forces the LLM to rely on its own prior (background knowledge), and removes the effect of in-context examples which could be used to build a data model. 

We report the results for CLLM in the following table:
| In-context samples |  Downstream accuracy  |
|--------------------|----------------|
| n=1 (Prior)        | 70.20+-1.60    |
| n=20               | 73.87+-0.50    |
| n=40               | 73.95+-0.67    |
| n=100              | 74.71+-0.34    |
| $D_{\mathrm{oracle}}$| 74.6 +- 0.15 |



From these results, we conclude the following:

(1) The LLM prior permits to obtain good downstream performance, but it is outperformed by $\mathcal{D_{oracle}}$ by a margin of $4.4\%$. Hence, we cannot solely rely on the prior.

(2) Downstream performance increases as the number of in-context samples increases. This shows it is indeed important to include the in-context examples if we wish to obtain downstream performance close to $\mathcal{D_{oracle}}$, as the LLM can build a good data model.

This implies that while the LLM does use background knowledge of similar datasets, it still requires in-context samples to refine its prior by creating a good data model.

**UPDATE**: We included these results in Appendix C.3 of our updated manuscript

Reviewer comment (Theory):
The curation method seems to be model-dependent and may intensify the weaknesses of the ML model.

Author response:
### (C) Details on data curation

We provide answers to three aspects of data curation: (i) Curation model, (ii) Relation to active learning and (iii) Clarifying difference of criteria/metrics


**(i) Curation model**

We thank the reviewer for the point on curation and the model used to perform it. The curation indeed requires training a model to compute the metrics of confidence and aleatoric uncertainty.

The suggestion of the reviewer ("curating based on an ML model and evaluating on many models") is an approach that we have adopted in our submitted manuscript, and the detailed results are given in **Appendix C.1**. We apologize if this was unclear.

The curation, on the one hand, was performed by training an XGBoost model, leading to a curated dataset.

On the other hand, we evaluated the downstream performance with $4$ different types of classifiers namely: XGBoost, Random forest, Logistic regression and Decision Tree, which were trained on the curated dataset.

As we can see, the conclusion that curation helps improve downstream performance holds for each these various downstream models, as is indicated by the green arrows in Tables 6,7,8,9.

**Update:** We have made this point clearer by adding this discussion in Appendix C.

**(ii) Relation to Active Learning**
We also thank the reviewer for bringing up this interesting point about active learning and our curation step. While we have already discussed some differences to active learning in the Extended related work section (Appendix A), we now provide a detailed discussion on how they differ on the uncertainty metrics they use. 

Active learning primarily focuses on the iterative process of selecting data samples that, when labeled, are expected to most significantly improve the model's performance. This selection is typically based on criteria such as uncertainty sampling which focuses on **epistemic uncertainty** [R1-R4]. The primary objective is to minimize labeling effort while maximizing the model's learning efficiency. Additionally, active learning would aims to label instances based on epistemic uncertainty where the model struggles to make accurate predictions, yet the samples themselves are _correct_.

In contrast, CLLMs leverage training dynamics based on **aleatoric uncertainty** and confidence and is designed to discard samples that might jeopardize the downstream accuracy. These samples can be considered to have inherent issues or are erroneous, such as being "mislabeled".

To summarize, in active learning, epistemic uncertainty is used to identify data points that, if labeled, would yield the most significant insights for model training. In our approach, they serve to identify and exclude/filter data points that could potentially deteriorate the model's performance.

**UPDATE:** We have included this discussion on uncertainty metrics in Appendix A in our revised manuscript.

**(iii) Clarifying difference of criteria/metrics**

Finally, let us explain why confidence and the aleatoric uncertainty do not have a 1-to-1 relationship. 

 Let us consider a setting with $e \in$ {1,2}, and  

$[f_{1}(x)]_{y}=0.1$ and  

$[f_{2}(x)]_{y}=0.9$ 

Then the confidence is equal to $\frac{0.1 + 0.9}{2} = 0.5$ and the aleatoric uncertainty is equal to $\frac{0.1 * 0.9 + 0.1 * 0.9}{2} = 0.09$. 

Consider an alternative case where:

$[f_{1}(x)]_{y}= 0.5$ and

$[f_{2}(x)]_{y}=0.5$

The confidence is also equal to $0.5$ but the aleatoric uncertainty is equal to $\frac{0.5 * 0.5+0.5 * 0.5}{2} = 0.25$. Hence these two examples share the same confidence but differ in their aleatoric uncertainty, which demonstrates why these two notions do not capture the same aspect of the learning dynamics.

Reviewer comment (Evaluation):
Because the method proposed is quite simple to use, we need to know when the methods will break down in order to understand its pitfalls, but there is not really anything on that in the paper.

Author response:
### (A) Identifying possible pitfalls of CLLM
Thank you for raising the point about the possible pitfalls of LLMs. We agree that there are risks when using large language models (LLMs), as explained in Sec. 1 and Sec. 2.2 in our manuscript, especially for sensitive applications like healthcare and finance. This motivated our contribution of post-generation curation as a step towards addressing this issue. That said, we agree more should be done to discuss these challenges of using LLMs in the paper and possible "pitfalls".

We wish to highlight our *'Hardness' Proxy Signal* introduced in **Section 3.2**, which is a way to identify failure cases of data augmentation (i.e. proactively flagging the "pitfall"). 

In this part of the paper, we highlight that, given the black-box nature of an LLM, we might not trust generated datasets and specifically those that will lead to poor downstream performance.  The inclusion of our 'hardness' proxy serves as a proactive measure to flag potentially problematic datasets reflecting our commitment to ensuring the quality and reliability of the data used in our models. To build this proxy signal, we directly leverage the output of the curation module, with the rationale stated in _Section 3.2_: $D_{syn}$ should intuitively be considered imperfect if curation discards many of its samples since the number of discarded samples measures the quality of samples with respect to the small but goldstandard $D_{train}$.

This proxy signal aligns with the results in Table 3, where we show that CLLM struggles with the Adult dataset before curation. **Figure 6** shows that the proportion of discarded samples is the highest for this dataset.

Consequently, this shows a practitioner _could_ have used this proxy signal to flag this failure case of the LLM before using it to train a downstream predictive model.

We hope this clarifies how we proactively identify pitfalls and failure cases from the augmentation.


**Limitations and warning readers.**
Of course, we note that CLLM is only a first step towards better synthetic data. In practice, we thus agree that the output of generative models, including CLLM, should always be exhaustively evaluated on quality, fairness, and other forms of bias, before application to sensitive areas like healthcare, to avoid any potentially harmful applications. 

We have consulted an academic ethicist to discuss how these ethical concerns should be best communicated. To ensure readers may not inadvertently cause bias and harm in the real world through applying CLLM, we have included the following warning at the end of the introduction:

*LLMs may make errors and may reflect or exacerbate societal biases that are present in their data [R1]. Though the curation in CLLM improves synthetic data quality, it does not directly aim to remove biases. The quality and fairness of generated data should always be evaluated. More research into LLM bias is required before methods like CLLM should be applied to real-world sensitive settings like healthcare and finance.*

Furthermore, we extended the Ethics statement to include this warning and will include the same warning in the code repository.

Thank you again for bringing up this important issue and we hope that our clarifications have addressed your concerns. 

**UPDATE:** we have updated Section 1 and our Ethics statement in our revised manuscript with a warning.

Reviewer comment (Evaluation):
Section 2.1 suggests that the LLM has a strong prior, as it is able to extrapolate synthesis to regions without any training data. It is unclear whether in-context learning is flexible enough to learn the nuances in the ultra-small dataset, or if the LLM’s prior will overwhelm those nuances. Table 3 shows that performance does not improve much as n increases for LLM models relative to non-LLM models, which is consistent with either the prior being too strong or in-context learning being very good. The authors are encouraged to tease these two scenarios apart.

Author response:
Dear Reviewer 4AgE


Thank you for your response!

As you suggested, we have **updated our manuscript** with the discussion on the choice of the curation model, both in Appendix B.3 and in Section 2.2 in our revised manuscript. 

We agree that the prior encoded in the LLM can be a double-edged sword, but we also have reasons to be optimistic. First, a realistic prior is better than no prior at all, which is the status quo in deep generative modeling for tabular data. Second, the LLM’s prior may not be as strong as you suggest. We have conducted a new experiment to quantify and visualize the strength of the prior, by studying how much the LLM's output distribution adapts to the in-context samples we provide.  More specifically, we show t-SNE plots for data generated by the prior of the LLM ($n=1$), and for added in-context examples of $n=20,40,100$ on the Covid dataset.

The plots are included in the updated manuscript (Appendix C.3) and at the following link:

**https://i.imgur.com/x00toXC.png**


**Interpretation:** 
In particular, we observe a region in the oracle data that is _not_ captured by the LLM’s prior output (the left part of the leftmost blob, circled in blue in our plot). However, as the number of in-context examples (real data) increases in the prompt of the LLM, we observe that the LLM is steered to generate data that covers this region. 

We investigated this region and noticed that it is associated with the subgroup of people older than $87$ years old, and having many severe comorbidities (e.g. Diabetes, Cardiovascular diseases) and respiratory symptoms. This subgroup, in the Oracle dataset, represents less than $3.5\%$ of the data, and is completely ignored by the GPT-4 prior. In particular, the prior defaults to more typical old patients in the range of $70$-$80$ years old.

On the contrary,  as $n$ increases, the LLM is guided by the in-context samples and generates samples from this subgroup, which as you mention are "rarer" or different from the general population. 

This demonstrates that the LLM captures the distinct features of this particular region, and hence is _not overwhelmed by the prior_, but instead, the data in the form of in-context samples adapts it, hence aligning the augmented dataset with the ground-truth distribution.


Additionally, similar to the main paper, we assess the synthetic data using the Precision (Quality) and recall (Diversity) metrics [R1] with respect to $D_{\mathrm{oracle}}$:

| $n_{\mathrm{samples}}$ in $\mathcal{D_{train}}$ | Precision |   Recall | 
| ------------------ | ------------------- | ------------------- |
| Prior ($n_{\mathrm{samples}}=1$) |   0.29 +-  0.026   |   0.64 +- 0.09        |
| 20                                 | 0.41+-0.04      | 0.87+-0.03      | 0.74+-0.01     | 0.13+-0.0          | 0.82+-0.01         | 0.66+-0.01         | 0.33+-0.07 | 0.50+-0.03 | 0.59+-0.02 |
| 40                                 | 0.40+-0.01      | 0.91+-0.01      | 
| 100                                 | 0.42+-0.01      | 0.86+-0.02          |

These results further validate the adaptation of the LLM to capture the nuances of the data, as more in-context samples are provided.


**UPDATE:** We have updated Appendix C.3 which now dissects the role of the prior along two dimensions:

(1) The prior alone cannot match the oracle downstream performance and needs in-context samples [``last update``]

(2) The in-context samples guide the LLM generation and is not overwhelmed by the prior --- allowing us to generate rarer subgroups. [``new update``]

----

_We thank the reviewer for encouraging us to disentangle this important aspect, which has helped us improve the paper!_ 

_We hope that our clarifications and new experiment have addressed your concerns, please let us know if there is anything else we could do before the discussion period ends._


Paper 7282 Authors

----

[R1] Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. Advances in neural information processing systems, 31,2018.

Reviewer comment (Experiments):
For scenarios of “truly” scarce data—data not in the training data of the LLM—it is important to quantify the likelihood that datasets were seen during training. Although four datasets are unlikely to be in the LLM’s training due to being closed-source, confirmation is needed. Table 3 shows that for n=20, most non-LLM Uncur methods have worse utility than D_train, whereas most LLM Uncur methods have higher utility than D_train, which could indicate either memorization or effective in-context few-shot learning. Distinguishing between the two is important for the truly scarce scenario.

Author response:
### (B) Priors of LLMs for data augmentation
We would like to thank the Reviewer for bringing this interesting point on the prior modeled by the LLM. We agree with the Reviewer that two components can be attributed to the good performances of CLLMs: the prior given by the LLM, and the in-context examples present in the prompt.

The reviewer also rightfully points to possible memorization as the source of performance. We highlight that this aspect of memorization motivated the choice of datasets in our manuscript, with $4$ private datasets. These "medical datasets" are proprietary datasets which not in the public domain and hence we have ensured they are not accessible as part of the GPT-4 or GPT-3.5 training corpora which would have originated from web scraping or "actual release" of these datasets. Hence, we cannot attribute strong performance on these datasets to memorization but rather due to background knowledge on "similar data" or via the in-context examples.

We thank the Reviewer for the suggestion regarding the prior. We have conducted a new experiment to understand the effect of the LLMs background knowledge (e.g. prior).

We considered the Covid dataset (private medical dataset, to avoid memorization issues) and generated data with GPT-4 (same as Sec 2.1). We ablate the prompt used in our work (detailed in Appendix B.5), and solely provide one in-context example in the prompt, in order to give the LLM the *minimal* amount of information about the desired structure of the dataset. 

This lack of examples forces the LLM to rely on its own prior (background knowledge), and removes the effect of in-context examples which could be used to build a data model. 

We report the results for CLLM in the following table:
| In-context samples |  Downstream accuracy  |
|--------------------|----------------|
| n=1 (Prior)        | 70.20+-1.60    |
| n=20               | 73.87+-0.50    |
| n=40               | 73.95+-0.67    |
| n=100              | 74.71+-0.34    |
| $D_{\mathrm{oracle}}$| 74.6 +- 0.15 |



From these results, we conclude the following:

(1) The LLM prior permits to obtain good downstream performance, but it is outperformed by $\mathcal{D_{oracle}}$ by a margin of $4.4\%$. Hence, we cannot solely rely on the prior.

(2) Downstream performance increases as the number of in-context samples increases. This shows it is indeed important to include the in-context examples if we wish to obtain downstream performance close to $\mathcal{D_{oracle}}$, as the LLM can build a good data model.

This implies that while the LLM does use background knowledge of similar datasets, it still requires in-context samples to refine its prior by creating a good data model.

**UPDATE**: We included these results in Appendix C.3 of our updated manuscript

Reviewer comment (Experiments):
The curation depends on the model via f_e(x) (see equations 1 and 2), so the augmentation is optimized for a model and may make the model’s inductive biases more pronounced. Possible ways to address this include: (1) curating based on a ML model and evaluating on many models, and (2) including more metrics such as resemblance. A clear rationale for addressing this issue would be helpful.

Author response:
### (C) Details on data curation

We provide answers to three aspects of data curation: (i) Curation model, (ii) Relation to active learning and (iii) Clarifying difference of criteria/metrics


**(i) Curation model**

We thank the reviewer for the point on curation and the model used to perform it. The curation indeed requires training a model to compute the metrics of confidence and aleatoric uncertainty.

The suggestion of the reviewer ("curating based on an ML model and evaluating on many models") is an approach that we have adopted in our submitted manuscript, and the detailed results are given in **Appendix C.1**. We apologize if this was unclear.

The curation, on the one hand, was performed by training an XGBoost model, leading to a curated dataset.

On the other hand, we evaluated the downstream performance with $4$ different types of classifiers namely: XGBoost, Random forest, Logistic regression and Decision Tree, which were trained on the curated dataset.

As we can see, the conclusion that curation helps improve downstream performance holds for each these various downstream models, as is indicated by the green arrows in Tables 6,7,8,9.

**Update:** We have made this point clearer by adding this discussion in Appendix C.

**(ii) Relation to Active Learning**
We also thank the reviewer for bringing up this interesting point about active learning and our curation step. While we have already discussed some differences to active learning in the Extended related work section (Appendix A), we now provide a detailed discussion on how they differ on the uncertainty metrics they use. 

Active learning primarily focuses on the iterative process of selecting data samples that, when labeled, are expected to most significantly improve the model's performance. This selection is typically based on criteria such as uncertainty sampling which focuses on **epistemic uncertainty** [R1-R4]. The primary objective is to minimize labeling effort while maximizing the model's learning efficiency. Additionally, active learning would aims to label instances based on epistemic uncertainty where the model struggles to make accurate predictions, yet the samples themselves are _correct_.

In contrast, CLLMs leverage training dynamics based on **aleatoric uncertainty** and confidence and is designed to discard samples that might jeopardize the downstream accuracy. These samples can be considered to have inherent issues or are erroneous, such as being "mislabeled".

To summarize, in active learning, epistemic uncertainty is used to identify data points that, if labeled, would yield the most significant insights for model training. In our approach, they serve to identify and exclude/filter data points that could potentially deteriorate the model's performance.

**UPDATE:** We have included this discussion on uncertainty metrics in Appendix A in our revised manuscript.

**(iii) Clarifying difference of criteria/metrics**

Finally, let us explain why confidence and the aleatoric uncertainty do not have a 1-to-1 relationship. 

 Let us consider a setting with $e \in$ {1,2}, and  

$[f_{1}(x)]_{y}=0.1$ and  

$[f_{2}(x)]_{y}=0.9$ 

Then the confidence is equal to $\frac{0.1 + 0.9}{2} = 0.5$ and the aleatoric uncertainty is equal to $\frac{0.1 * 0.9 + 0.1 * 0.9}{2} = 0.09$. 

Consider an alternative case where:

$[f_{1}(x)]_{y}= 0.5$ and

$[f_{2}(x)]_{y}=0.5$

The confidence is also equal to $0.5$ but the aleatoric uncertainty is equal to $\frac{0.5 * 0.5+0.5 * 0.5}{2} = 0.25$. Hence these two examples share the same confidence but differ in their aleatoric uncertainty, which demonstrates why these two notions do not capture the same aspect of the learning dynamics.

Reviewer comment (Novelty):
The selection criteria based on high aleatoric uncertainty make the approach similar to active learning. This similarity means curation will improve model performance even with random sampling, as shown by curation increasing performance for all methods. It may be worth making this connection explicit.

Author response:
### (C) Details on data curation

We provide answers to three aspects of data curation: (i) Curation model, (ii) Relation to active learning and (iii) Clarifying difference of criteria/metrics


**(i) Curation model**

We thank the reviewer for the point on curation and the model used to perform it. The curation indeed requires training a model to compute the metrics of confidence and aleatoric uncertainty.

The suggestion of the reviewer ("curating based on an ML model and evaluating on many models") is an approach that we have adopted in our submitted manuscript, and the detailed results are given in **Appendix C.1**. We apologize if this was unclear.

The curation, on the one hand, was performed by training an XGBoost model, leading to a curated dataset.

On the other hand, we evaluated the downstream performance with $4$ different types of classifiers namely: XGBoost, Random forest, Logistic regression and Decision Tree, which were trained on the curated dataset.

As we can see, the conclusion that curation helps improve downstream performance holds for each these various downstream models, as is indicated by the green arrows in Tables 6,7,8,9.

**Update:** We have made this point clearer by adding this discussion in Appendix C.

**(ii) Relation to Active Learning**
We also thank the reviewer for bringing up this interesting point about active learning and our curation step. While we have already discussed some differences to active learning in the Extended related work section (Appendix A), we now provide a detailed discussion on how they differ on the uncertainty metrics they use. 

Active learning primarily focuses on the iterative process of selecting data samples that, when labeled, are expected to most significantly improve the model's performance. This selection is typically based on criteria such as uncertainty sampling which focuses on **epistemic uncertainty** [R1-R4]. The primary objective is to minimize labeling effort while maximizing the model's learning efficiency. Additionally, active learning would aims to label instances based on epistemic uncertainty where the model struggles to make accurate predictions, yet the samples themselves are _correct_.

In contrast, CLLMs leverage training dynamics based on **aleatoric uncertainty** and confidence and is designed to discard samples that might jeopardize the downstream accuracy. These samples can be considered to have inherent issues or are erroneous, such as being "mislabeled".

To summarize, in active learning, epistemic uncertainty is used to identify data points that, if labeled, would yield the most significant insights for model training. In our approach, they serve to identify and exclude/filter data points that could potentially deteriorate the model's performance.

**UPDATE:** We have included this discussion on uncertainty metrics in Appendix A in our revised manuscript.

**(iii) Clarifying difference of criteria/metrics**

Finally, let us explain why confidence and the aleatoric uncertainty do not have a 1-to-1 relationship. 

 Let us consider a setting with $e \in$ {1,2}, and  

$[f_{1}(x)]_{y}=0.1$ and  

$[f_{2}(x)]_{y}=0.9$ 

Then the confidence is equal to $\frac{0.1 + 0.9}{2} = 0.5$ and the aleatoric uncertainty is equal to $\frac{0.1 * 0.9 + 0.1 * 0.9}{2} = 0.09$. 

Consider an alternative case where:

$[f_{1}(x)]_{y}= 0.5$ and

$[f_{2}(x)]_{y}=0.5$

The confidence is also equal to $0.5$ but the aleatoric uncertainty is equal to $\frac{0.5 * 0.5+0.5 * 0.5}{2} = 0.25$. Hence these two examples share the same confidence but differ in their aleatoric uncertainty, which demonstrates why these two notions do not capture the same aspect of the learning dynamics.

Reviewer comment (Theory):
It is unclear whether the two selection criteria have a one-to-one relationship, since [f_e(x)]_y is the only variable in both quantities. This raises the question of whether the criteria are equivalent to selecting a range of confidence or a range of uncertainty, which may be worth clarifying.

Author response:
### (C) Details on data curation

We provide answers to three aspects of data curation: (i) Curation model, (ii) Relation to active learning and (iii) Clarifying difference of criteria/metrics


**(i) Curation model**

We thank the reviewer for the point on curation and the model used to perform it. The curation indeed requires training a model to compute the metrics of confidence and aleatoric uncertainty.

The suggestion of the reviewer ("curating based on an ML model and evaluating on many models") is an approach that we have adopted in our submitted manuscript, and the detailed results are given in **Appendix C.1**. We apologize if this was unclear.

The curation, on the one hand, was performed by training an XGBoost model, leading to a curated dataset.

On the other hand, we evaluated the downstream performance with $4$ different types of classifiers namely: XGBoost, Random forest, Logistic regression and Decision Tree, which were trained on the curated dataset.

As we can see, the conclusion that curation helps improve downstream performance holds for each these various downstream models, as is indicated by the green arrows in Tables 6,7,8,9.

**Update:** We have made this point clearer by adding this discussion in Appendix C.

**(ii) Relation to Active Learning**
We also thank the reviewer for bringing up this interesting point about active learning and our curation step. While we have already discussed some differences to active learning in the Extended related work section (Appendix A), we now provide a detailed discussion on how they differ on the uncertainty metrics they use. 

Active learning primarily focuses on the iterative process of selecting data samples that, when labeled, are expected to most significantly improve the model's performance. This selection is typically based on criteria such as uncertainty sampling which focuses on **epistemic uncertainty** [R1-R4]. The primary objective is to minimize labeling effort while maximizing the model's learning efficiency. Additionally, active learning would aims to label instances based on epistemic uncertainty where the model struggles to make accurate predictions, yet the samples themselves are _correct_.

In contrast, CLLMs leverage training dynamics based on **aleatoric uncertainty** and confidence and is designed to discard samples that might jeopardize the downstream accuracy. These samples can be considered to have inherent issues or are erroneous, such as being "mislabeled".

To summarize, in active learning, epistemic uncertainty is used to identify data points that, if labeled, would yield the most significant insights for model training. In our approach, they serve to identify and exclude/filter data points that could potentially deteriorate the model's performance.

**UPDATE:** We have included this discussion on uncertainty metrics in Appendix A in our revised manuscript.

**(iii) Clarifying difference of criteria/metrics**

Finally, let us explain why confidence and the aleatoric uncertainty do not have a 1-to-1 relationship. 

 Let us consider a setting with $e \in$ {1,2}, and  

$[f_{1}(x)]_{y}=0.1$ and  

$[f_{2}(x)]_{y}=0.9$ 

Then the confidence is equal to $\frac{0.1 + 0.9}{2} = 0.5$ and the aleatoric uncertainty is equal to $\frac{0.1 * 0.9 + 0.1 * 0.9}{2} = 0.09$. 

Consider an alternative case where:

$[f_{1}(x)]_{y}= 0.5$ and

$[f_{2}(x)]_{y}=0.5$

The confidence is also equal to $0.5$ but the aleatoric uncertainty is equal to $\frac{0.5 * 0.5+0.5 * 0.5}{2} = 0.25$. Hence these two examples share the same confidence but differ in their aleatoric uncertainty, which demonstrates why these two notions do not capture the same aspect of the learning dynamics.

Reviewer comment (Experiments):
Demonstration of pitfalls is especially important because the proposed method is easy to use. Such demonstrations could also shed light on the issues regarding the LLM prior and scarce data scenarios.

Author response:
### (A) Identifying possible pitfalls of CLLM
Thank you for raising the point about the possible pitfalls of LLMs. We agree that there are risks when using large language models (LLMs), as explained in Sec. 1 and Sec. 2.2 in our manuscript, especially for sensitive applications like healthcare and finance. This motivated our contribution of post-generation curation as a step towards addressing this issue. That said, we agree more should be done to discuss these challenges of using LLMs in the paper and possible "pitfalls".

We wish to highlight our *'Hardness' Proxy Signal* introduced in **Section 3.2**, which is a way to identify failure cases of data augmentation (i.e. proactively flagging the "pitfall"). 

In this part of the paper, we highlight that, given the black-box nature of an LLM, we might not trust generated datasets and specifically those that will lead to poor downstream performance.  The inclusion of our 'hardness' proxy serves as a proactive measure to flag potentially problematic datasets reflecting our commitment to ensuring the quality and reliability of the data used in our models. To build this proxy signal, we directly leverage the output of the curation module, with the rationale stated in _Section 3.2_: $D_{syn}$ should intuitively be considered imperfect if curation discards many of its samples since the number of discarded samples measures the quality of samples with respect to the small but goldstandard $D_{train}$.

This proxy signal aligns with the results in Table 3, where we show that CLLM struggles with the Adult dataset before curation. **Figure 6** shows that the proportion of discarded samples is the highest for this dataset.

Consequently, this shows a practitioner _could_ have used this proxy signal to flag this failure case of the LLM before using it to train a downstream predictive model.

We hope this clarifies how we proactively identify pitfalls and failure cases from the augmentation.


**Limitations and warning readers.**
Of course, we note that CLLM is only a first step towards better synthetic data. In practice, we thus agree that the output of generative models, including CLLM, should always be exhaustively evaluated on quality, fairness, and other forms of bias, before application to sensitive areas like healthcare, to avoid any potentially harmful applications. 

We have consulted an academic ethicist to discuss how these ethical concerns should be best communicated. To ensure readers may not inadvertently cause bias and harm in the real world through applying CLLM, we have included the following warning at the end of the introduction:

*LLMs may make errors and may reflect or exacerbate societal biases that are present in their data [R1]. Though the curation in CLLM improves synthetic data quality, it does not directly aim to remove biases. The quality and fairness of generated data should always be evaluated. More research into LLM bias is required before methods like CLLM should be applied to real-world sensitive settings like healthcare and finance.*

Furthermore, we extended the Ethics statement to include this warning and will include the same warning in the code repository.

Thank you again for bringing up this important issue and we hope that our clarifications have addressed your concerns. 

**UPDATE:** we have updated Section 1 and our Ethics statement in our revised manuscript with a warning.

Reviewer comment (Experiments):
Does the LLM handle continuous variables well?

Author response:
### (D) Answers to the other questions
- LLMs handling continuous variables

Thanks for the question. Yes, the LLMs do handle continuous variables. Our datasets contain a mixture of continuous and categorical variables. For example, variables such as age or biomarkers which are continuous. The LLMs capability to generate samples resulting in good downstream model performance is hence indicative of this ability to handle such continuous variables.

- Figure 2 markers

The reason for putting the markers over one another is to show how each of the different augmentation approaches matches the oracle (in turquoise). We apologize if this was unclear that the Oracle fills all the space of the regions shown and that we plot the other markers (GPT-4, $\mathcal{D_train}$, TVAE) and on top of it.  We have updated the caption to clarify this

**UPDATE**: We updated the caption of Figure 2 to clarify that the markers should ideally match the underlying oracle. 

- Table 1: performance measure

We report in Table 1 the average gain in downstream accuracy compared to the downstream accuracy of a model trained on $\mathcal{D_train}$.

**UPDATE**: We updated the column in Table 1 to clarify this in our revised manuscript.


- Figure 3 Clarification and gain.

We clarify how the curve in Figure 3 is obtained. We stratified the list of subgroups depending on their number of samples and then took the average of the performance gain using a model trained on CLLM data (i.e. augmentation) versus a model trained on $\mathcal{D_train}$. Naturally, not all points on the x-axis have the same number of subgroups, and this variability explains the slight dip at $6$ samples. The goal of the figure is to highlight that the performance gains from augmentation are largest for subgroups that originally had the fewest samples in $\mathcal{D_train}$. Hence, the result highlights that augmentation improves downstream performance on small subgroups. Finally, as suggested by the Reviewer, we clarify the actual performance gain (overall) after augmentation: augmentation gives an AUC gain of 5.28%.

- ICL acronym definition

We thank the reviewer and have defined the acronym ICL (in-context learning) in our revised manuscript.

**UPDATE**: Updated the manuscript to define the acronym in Section 2.


We hope this answers your points, please let us know if there are any remaining concerns.

----
-----

Reviewer comment (Novelty):
I don't see the technical contribution of this work.

Author response:
### (B) Technical contributions of CLLM

On technical contributions, we discuss the two dimensions mentioned by the reviewer: (i) Contrast to NLP augmentation and (ii) Comparison to (random) noise perturbation.

**(i) Contrast to NLP augmentation**
We would like to thank the Reviewer for bringing up the point about data augmentations in modalities such as NLP. While similar in objectives, we want to stress that data augmentation for tabular data and data augmentation in "traditional NLP" are two very different problems that have traditionally been tackled with solutions of different natures. While language models can generate coherent and contextually relevant text, tabular data augmentation is a challenging problem where the difficulty lies in preserving and extrapolating the relational dependencies and statistical characteristics within the data. This is all the more arduous as we posit for the ultra-low data regime ($n_{lab}<100$, see answer below). 

Furthermore, these inherent challenges entailed by tabular augmentation in the ultra-low data regime motivate the key technical contribution of CLLM which is **post-generation curation**. We demonstrate in Table 3 across 7 real-world datasets that our curation mechanism plays a key role in improving data augmentation quality and utility and is an aspect overlooked by previous works. In particular, we show that with very few samples, the augmented and curated data generated by CLLM leads to downstream performance close to that of the Oracle baseline. Furthermore, we highlight our curation mechanism tackles an overlooked aspect that not only benefits LLM tabular generation but is also applicable and beneficial to other baseline tabular generation methods. 


**(ii) Comparison to (random) noise perturbation**
We appreciate the suggestion of the Reviewer for an experiment with a *random noise perturbation* baseline. We clarify that we considered this point in our manuscript, but realized that it has not been made clear enough. In our experiments in Sec.  3.1, we use the baseline **SMOTE** [R2], which consists in augmenting the dataset with random interpolations between samples in $D_{\mathrm{train}}$, and is thus noise injection around a given sample for data augmentation. As we show in Table 3, CLLM outperforms SMOTE by a large margin on almost all the datasets, both with and without the curation. This performance gap highlights the capacities of the LLMs to extrapolate and perform data augmentation, as we explain in Sec. 2.1, and shows that CLLM should be prefered over this baseline.

Reviewer comment (Novelty):
This essentially comes down to the few-shot generation problem, where there is much more existing work beyond tabular data. If it is a traditional NLP task such as generating data for sentiment analysis, there are already abundant success cases. Whether the data supports training a model depends on what model you are training and what you want to achieve.

Author response:
### (B) Technical contributions of CLLM

On technical contributions, we discuss the two dimensions mentioned by the reviewer: (i) Contrast to NLP augmentation and (ii) Comparison to (random) noise perturbation.

**(i) Contrast to NLP augmentation**
We would like to thank the Reviewer for bringing up the point about data augmentations in modalities such as NLP. While similar in objectives, we want to stress that data augmentation for tabular data and data augmentation in "traditional NLP" are two very different problems that have traditionally been tackled with solutions of different natures. While language models can generate coherent and contextually relevant text, tabular data augmentation is a challenging problem where the difficulty lies in preserving and extrapolating the relational dependencies and statistical characteristics within the data. This is all the more arduous as we posit for the ultra-low data regime ($n_{lab}<100$, see answer below). 

Furthermore, these inherent challenges entailed by tabular augmentation in the ultra-low data regime motivate the key technical contribution of CLLM which is **post-generation curation**. We demonstrate in Table 3 across 7 real-world datasets that our curation mechanism plays a key role in improving data augmentation quality and utility and is an aspect overlooked by previous works. In particular, we show that with very few samples, the augmented and curated data generated by CLLM leads to downstream performance close to that of the Oracle baseline. Furthermore, we highlight our curation mechanism tackles an overlooked aspect that not only benefits LLM tabular generation but is also applicable and beneficial to other baseline tabular generation methods. 


**(ii) Comparison to (random) noise perturbation**
We appreciate the suggestion of the Reviewer for an experiment with a *random noise perturbation* baseline. We clarify that we considered this point in our manuscript, but realized that it has not been made clear enough. In our experiments in Sec.  3.1, we use the baseline **SMOTE** [R2], which consists in augmenting the dataset with random interpolations between samples in $D_{\mathrm{train}}$, and is thus noise injection around a given sample for data augmentation. As we show in Table 3, CLLM outperforms SMOTE by a large margin on almost all the datasets, both with and without the curation. This performance gap highlights the capacities of the LLMs to extrapolate and perform data augmentation, as we explain in Sec. 2.1, and shows that CLLM should be prefered over this baseline.

Reviewer comment (Experiments):
With less than 100 data but 30 attributes, it is impossible for LLMs to effectively infer the pattern of data. And I doubt the generation is trivial. I wonder how the proposed generation process compares to just adding random noise to augment existing samples before curation. D_train in many cases is very high, while the baseline performance is significantly worse. This raises doubt on whether the setup is correct. I suspect using small random noise to augment the dataset would perform better than many baselines.

Author response:
### Choice of baselines and random noise 

**(i) Choice of baselines:** We first clarify our existing baselines, which encompass a broad and representative spectrum of different classes of generative modeling techniques used for tabular data augmentation (Qian et al, 2023): GAN, VAE, Normalizing Flow, LLM, and Score-based modeling, as well as classical augmentation (e.g. SMOTE). As the Reviewer rightfully mentions, the baselines struggle in the ultra-low data regime. This illustrates a key research gap: traditional data augmentation approaches struggle in such ultra-low sample settings, which CLLM addresses with the LLM and its generative capabilities. 

**(ii) Random noise baseline:** We thank the Reviewer for the suggestion of the random noise baseline. We performed a new experiment, where we augment the dataset with random additive Gaussian noise. In order to capture the correlations between the different features, we fit a Kernel Density Estimator with a Gaussian kernel and bandwidth given by Scott's rule. We then sample $1000$ points to create an augmented dataset $D_{\mathrm{syn}}$.

We report the performance gap between CLLM and this baseline (with and without curation) for the Covid and Compas datasets. The result can be found at the following link: **https://i.imgur.com/iUkTZ5w.png**


We observe that the random noise baseline does not match the performance of CLLM, although the baseline naturally improves as the dataset $D_{\mathrm{train}}$ grows in size.

**UPDATE**: We include this result in a new Appendix C.5 in our revised manuscript.

---

_We thank you for your time and energy in the reviewing process, please let us know if you have any additional questions._

---

### References
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9), 1-35.

Qian, Zhaozhi, Bogdan-Constantin Cebere, and Mihaela van der Schaar. "Synthcity: facilitating innovative use cases of synthetic data in different data modalities." arXiv preprint arXiv:2301.07573 (2023).

Reviewer comment (Writing):
There are existing notions such as "low-resource" refers to less than 5k annotated samples and "strict few-shot" refers to <=16 labeled samples per class. I'm not aware of the definition of "ultra-low-resource". Based on the illustration of this paper, it seems to be < 100 samples. The title suggests ultra-low, but without a definition for it, the abstract and introduction talk about "low-resources". **What is ultra-low? Is this an existing notion or is it proposed by this work?** This range of samples seems to be the case that is often considered "few-shot".

Author response:
* "What is the ultra-low data regime?
 
 We apologize if this point has not been made clear enough in our submitted manuscript. We used the words "ultra-low" for settings where $\vert D_{\mathrm{train}}\vert = n_{lab} < 100$, and updated our manuscript accordingly. 
 
 We also want to emphasize that our setting differs from the typical few-shot learning literature, as we are not using the LLMs to issue predictions on given test samples. Instead, we prompt the LLM to generate new samples for data augmentation given a small $D_{\mathrm{train}}$, where the output is higher-dimensional than for few-shot prediction which only outputs labels and can typically operate with very few samples.

Reviewer comment (Experiments):
To have diversity in the generated samples, I would expect the researchers to look into decoding strategies (such as temperature or sampling). It usually does not work by just "asking the model to generate diverse samples".

Author response:
* Diversity in generated samples & decoding strategies (temperature or sampling)    
    
We desire diversity in the generated samples, defined by the distribution that the LLM is sampling from. This is why we include the instruction of generating diverse samples in the prompt. Indeed, this information is part of the context window of the LLM and hence conditions the distribution that the LLM models, as explained in **Section 2** in our manuscript: the prompt and the in-context examples can be compactly grouped in a vector $\Phi$, defining a distribution $P_{\Phi}$.

We appreciate the Reviewer's suggestion on decoding strategies. We notice that these strategies are not orthogonal to our prompting choices. Indeed, sampling assumes that a given distribution is already specified. Temperature scaling also assumes that a distribution is given, and is a way to modify its entropy in a post-hoc manner. In these two cases, we need to define an initial distribution, which we do via our prompting choices.

Reviewer comment (Evaluation):
It is better to provide average performance (with standard deviation) for each method for easy comparison between methods.

Author response:
* Update to include standard deviation

Since we have many results across many baselines, for clarity we reported average performance. However, we agree with the reviewer and have added the detailed results with standard deviations in Table 10 in our revised manuscript.

**UPDATE:** We have added these results in the updated manuscript Table 10, in Appendix C.2.

Reviewer comment (Experiments):
The paper only tests generation with GPT-4. It would be interesting to see if the findings in the paper are consistent across LLMs, for example, including Claude 2 or LLAMAv2 would be helpful.

Author response:
### (C) LLM backbone
We acknowledge the reviewer's suggestion to test our approach with other large language models such as Claude 2 or LLAMA2.  We agree that exploring the consistency of our findings across various LLMs such as Claude and LLAMA2 is an interesting avenue for future research. We want to highlight that CLLM aims to underscore the broader applicability of using an LLM with post-generation curation for data augmentation and used both GPT 3.5 and GPT-4 as examples. Extending our experiments to include Claude 2 or LLAMA2 could indeed provide additional insights, though we want to clarify that our paper's primary contribution lies in demonstrating and validating the general concept of leveraging LLMs for data augmentation in ultra-low data regimes, as well as highlighting the value of our post-generation curation mechanism of the LLM output, rather than conducting a comparative analysis between different LLMs.

Reviewer comment (Experiments):
What is your dataset selection logic? It would be helpful to include more datasets from the papers of other tabular data generation baselines.

Author response:
* What is your dataset selection logic?
 
We wish to clarify our selection of the 7 datasets we use --- both open-source and private datasets.

(1) **Open-source:** Adult, Drug and Compas are widely used open-source datasets used in the tabular data literature. Adult and Drug are both UCI datasets that have been used in many papers, while Compas is part of OpenML [R1]. Our reason for selecting them is that, despite them being open-source, they are highly reflective of domains in which we might be unable to collect many samples --- hence in reality would often be in an ultra-low data regime. 

(2) **Private datasets:** We wanted to disentangle the possible role of memorization in the strong performance of the LLM. To ensure the datasets are not in the LLMs training corpus, we selected 4 private medical datasets that need an authorization process to access. Hence, these datasets would not be part of the LLMs training corpus given their proprietary nature and hence would be unseen to the LLM. While the private and unseen aspect was the main motivation, we also wish to highlight that these are real-world medical datasets. Consequently, this allows us to test a highly realistic problem setting. 

We apologize if this was unclear and have attempted to better clarify this in our description of dataset selection in Appendix B.1.

**UPDATE**: We have added this discussion in Appendix B.1 of our revised manuscript.

Reviewer comment (Experiments):
Do the performance gains come solely from the background knowledge of GPT-4 or is GPT-4 also able to build a strong data model? Have you tested CLLM on any datasets where the background knowledge of GPT-4 would not be as useful, for example, on datasets with anonymized features? Such an experiment would help further understand the source of performance gains.

Author response:
### (A) Background knowledge of LLMs

We would like to thank the Reviewer for the astute point on disentangling the background knowledge of LLMs. As the Reviewer rightfully points out, two components can be attributed to the good performances of CLLMs: the background knowledge of the LLM, and its capacity to build a strong data model. Additionally, the Reviewer asks how would anonymized features affect the results.

Let us tackle in what follows (i) anonymized features and (ii) background knowledge.

(i) **Anonymized features:** Firstly, in terms of anonymized features, let us clarify that our ablation study in **Section 2.1 (page 5)** tackles this point. We assess the importance of contextual information in the prompt, where we test a variant called *prompt w/ no context* which anonymizes the features as feat1, feat2 etc and only provides the numerical in-context examples, without explanation of what these features are. The results in Table 2 (on the Covid dataset) show that the lack of feature information reduces precision and recall and also reduces the performance of a downstream model (utility). This shows the importance of providing contextual information to exploit the potential of the LLM, and hence motivates the prompt that we use in our work.

We have also run an additional experiment on the Compas dataset, similarly ablating with anonymized features and similarly demonstrating the importance of contextual information in the prompt itself. We report the results in our response below (in **D) Answers to the other questions**) in reference to one of the other Reviewer's questions.

**UPDATE**: We have updated Appendix C.4 to show the result on the Compas dataset.

(ii) **Background information vs ICL to build a data model**:
We followed the suggestion of the Reviewer and conducted a new experiment to understand the effect of the LLMs background knowledge (e.g. prior).

We considered the Covid dataset (private medical dataset, to avoid memorization issues) and generated data with GPT-4 (same as in Sec 2.1). We ablate the prompt used in our work (detailed in Appendix B.5), and solely provide one in-context example in the prompt, in order to give the LLM the *minimal* amount of information about the desired structure of the dataset. 

This lack of examples forces the LLM to rely on its own prior (background knowledge), and removes the effect of in-context examples which could be used to build a data model. 

We report the results for CLLM in the following table:
| In-context samples |  Downstream accuracy  |
|--------------------|----------------|
| n=1 (Prior)        | 70.20+-1.60    |
| n=20               | 73.87+-0.50    |
| n=40               | 73.95+-0.67    |
| n=100              | 74.71+-0.34    |
| $D_{\mathrm{oracle}}$| 74.6 +- 0.15 |



From these results, we conclude the following:

(1) The LLM prior permits to obtain good downstream performance, but it is outperformed by $\mathcal{D_{oracle}}$ by a margin of $4.4\%$. Hence, we cannot solely rely on the prior.

(2) Downstream performance increases as the number of in-context samples increases. This shows it is indeed important to include the in-context examples if we wish to obtain downstream performance close to $\mathcal{D_{oracle}}$, as the LLM can build a good data model.

This implies that while the LLM does use background knowledge of similar datasets, it still requires in-context samples to refine its prior by creating a good data model.

**UPDATE**: We included these results in Appendix C.3 of our updated manuscript 

We thank the reviewer for their suggestion, where this examination of different aspects of the LLM's background knowledge has strengthened the paper.

Reviewer comment (Evaluation):
If the results in Table 2 are indeed volatile, it would be very helpful to include error bars and bold all statistically significant winners. The error bars could be constructed by, for example, running the simulation for multiple seeds or resampling D_train. Right now, for example Recall of 0.89 is bolded for GPT-4 w/context for 40 samples and not bolded for GPT-no-context for 40 samples. This makes the results in Table 2 unconvincing in their current form. Although it is reasonable to hypothesize that including the background information about a dataset is helpful, as I mentioned above, a better validation of that would be helpful.

Author response:
* Error bars on Table 2



We thank the reviewer for the suggestion and have reported the results with error bars across multiple seeds with the data resampled. Please see the table below, which shows results for the Covid dataset.


| $n_{\mathrm{samples}}$ in $\mathcal{D_{train}}$ | GPT-4 w/ context Precision | GPT-4 w/ context Recall | GPT-4 w/ context Utility | GPT-4 no context Precision | GPT-4 no context Recall | GPT-4 no context Utility | TVAE Precision | TVAE Recall | TVAE Utility |
| ---------------------------------- | ------------------- | ------------------- | ------------------ | ------------------ | ------------------ | ------------------ | ------ | ------ | ------ |
| 10                                 | **0.41+-0.04**      | **0.87+-0.03**      | **0.74+-0.01**     | 0.13+-0.0          | 0.82+-0.01         | 0.66+-0.01         | 0.33+-0.07 | 0.50+-0.03 | 0.59+-0.02 |
| 20                                 | **0.40+-0.01**      | **0.91+-0.01**      | **0.76+-0.0**      | 0.11+-0.0          | 0.89+-0.0          | 0.69+-0.0          | 0.27+-0.01 | 0.68+-0.01 | 0.62+-0.03 |
| 50                                 | **0.42+-0.01**      | 0.86+-0.02          | **0.75+-0.01**     | 0.11+-0.01         | **0.90+-0.01**     | 0.74+-0.01         | 0.39+-0.02 | 0.67+-0.03 | 0.64+-0.06 |
| 100                                | 0.44+-0.02          | 0.85+-0.02          | **0.75+-0.0**      | 0.08+-0.01         | **0.90+-0.0**      | 0.60+-0.01         | **0.47+-0.0** | 0.73+-0.01 | 0.65+-0.02 |

**UPDATE:** We have updated Table 2 in the revised manuscript with the standard deviations.

Reviewer comment (Novelty):
Related work is currently limited and would benefit from including other related papers. While a few examples of tabular data generation methods are included and used as baselines, at least citing other prominent tabular data generation methods such as STaSy[1] would be helpful. Additionally, even though the introduction mentions that the low-data problem is undervalued, a few tabular deep learning works in fact address extreme low-data regimes, some of them also include experiments in the medical domain [2,3,4,5]. For example, [2,3] are tabular transfer learning approaches with experiments in extreme low-data regimes in the medical domain, while [4] and [5] are knowledge-graph-augmented tabular approaches enabling performance improvements in low-data regimes of wide and short datasets. The idea of leveraging a knowledge graph is similar to the idea of using an LLM in that they both rely on prior knowledge. It would definitely be helpful to cite these works, including them in experiments might be tricky because of the knowledge graph construction.

Author response:
### (B) Related work


We would like to thank the Reviewer for pointing these five references out. We agree with the Reviewer that using a knowledge graph is a relevant way to inject prior knowledge in the low-data regime, even though constructing them is not always straightforward. As such, we thank the Reviewer for rightfully mentioning that the LLM could serve a similar purpose (while being flexible with regard to prior knowledge).  

**UPDATE**: We have included the references in our Related Work section as well as covered them further in the Extended related work section in Appendix A.