EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models
Authors: Jinhee Kim
Taesung Kim
Jaegul Choo
Venue: NeurIPS 2024 poster (2024)
Keywords: Large language model
In-context learning
Few-shot learning
Class imbalance
Tabular data
Synthetic data generation
Primary area: generative_models
TL;DR: Can LLMs effectively generate synthetic tabular data to address class imbalance for classification tasks via in-context learning? How should prompts be structured to achieve this goal?

Abstract: Large language models (LLMs) have demonstrated remarkable in-context learning capabilities across diverse applications. In this work, we explore the effectiveness of LLMs for generating realistic synthetic tabular data, identifying key prompt design elements to optimize performance. We introduce EPIC, a novel approach that leverages balanced, grouped data samples and consistent formatting with unique variable mapping to guide LLMs in generating accurate synthetic data across all classes, even for imbalanced datasets. Evaluations on real-world datasets show that EPIC achieves state-of-the-art machine learning classification performance, significantly improving generation efficiency. These findings highlight the effectiveness of EPIC for synthetic tabular data generation, particularly in addressing class imbalance.

EPIC:
Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models
Jinhee Kim KAIST Daejeon, South Korea seharanul170kaise ac
Taesung Kim KAIST
Jaegul Choo KAIST Daejeon, South Korea jchoo@kaist.c ac
Daejeon, South Korea 2km1989kkaist..o ac kr
Original data
csv-balanced- grouping unique
Ablation csy balanneee grouping
cav-balanced
Ablation
csv-original
Ablation
Ablation cav-single
Ablation sen-balanced
GReaT
Figure I: Generation results on an imbalanced toy dataset with majority and minority classes Our approach, leveraging in-context learning with LLMs, achieves (1) distinct class boundaries, (2) accurate feature correlations, (3) well-matched value ranges, (4) robust numerical-categorion relationships (last row), and (5) comprehensive data distribution coverage, with improvements over its ablated versions and the fine-tuned GReaT model [4]. Complete results available in Fig.
Abstract
Large language models (LLMs) have demonstrated remarkable in-context learning capabilities across diverse applications. In this work, we explore the effectiveness of L.LMs for generating realistic synthetic tabular data, identifying key prompt de- sign elements to optimize performance. We introduce EPIC, novel approach that leverages balanced, grouped data samples and consistent formatting with unique variable mapping to guide LLMs in generating accurate synthetic data across all classes, even for imbalanced datasets. Evaluations real-world datasets show that EPIC achieves state-of-the- art machine learning classification performance, significantly improving generation efficiency. These findings highlight the effec- tiveness of EPIC for synthetic tabular data generation, particularly addressing class imbalance. Our source code for work is available here.
Both authors contributed equally.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

1.0
Specificity
Sensitivity
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Original data +TVAE +CopulaGAN +CTGAN
+GReaT +CTABGAN +CTABGAN+ +TabDDPM +Ours
Figure 2: Comparison of MI. classification performance with synthetic data on the Travel dataset. Results are averaged across four classifiers: XGBoost, CatBoost, LightGBM, and gradient boosting classifier, with each classifier run five times. Our method uses the GPT-3.5-turbo model.
1 Introduction
Tabular data, consisting of mixed variable types such as numerical and categorical variables, represents widely applicable and essential data format [30). It plays crucial role in enhancing decision- making and efficiency in various real-world applications, such as finance, healthcare, manufacturing, and natural sciences [ 3 26 However, challenges such data scarcity and class imbalance, particularly for rare yet crucial events, often significantly degrade the performance of machine learning (ML) models, often resulting in reduced accuracy for underrepresented classes. To address these challenges, efficient synthetic data generation methods have been developed to augment tabular datasets. Traditional methods, such as synthetic minority oversampling technique (SMOTE) and its variants [111] focus on generating minority class samples to alleviate class imbalance, Recently, advanced generative models, such TVAE (55), CTAB-GAN [39], and TabDDPM (19), have shown promising results in producing high-quality synthetic tabular data.
Large language models (LLMs) have also demonstrated significant potential for generating realistic tabular data, effectively handling both numerical and categorical variables 4 37]. However, these models often require extensive, data-specific fine-tuning, which increase the risk of overfitting to majority classes or dominant feature values, especially in small and highly imbalanced tabular datasets. An alternative approach involves leveraging the in-context learning capabilities of LLMs. Existing work has shown that LLMs can solve complex reasoning tasks, learn and mimic patterns
from input data, and augment textual datasets without parameter updates [13 20 34
However, designing effective prompts to optimize this capability for tabular data generation is challenging, especially for imbalanced datasets. This is because tabular data not naturally expressed in textual form and involves unique challenges, such as identifying feature correlations and accurately representing underrepresented attributes. Therefore, carefully crafted prompt design tailored to synthetic tabular data generation essential to fully leverage the capabilities LLMs. While several studies have employed in-context learning with LLMs for synthetic tabular data generation (7). few have conducted comprehensive investigations into optimal prompt designs that significantly impact data quality and generation efficiency.
In response, this study examines key components of prompt design and identifies an effective method for generating high-quality synthetic tabular data, particularly addressing class imbalance. We introduce a novel approach, EPIC, which leverages the in-context learning capabilitics of LLMs to produce synthetic tabular data with balanced class representation. EPIC incorporates prompt design strategies such as CSV style formatting, balanced class grouping, and unique variable mapper, which together contribute to generating synthetic data that accurately represents class- class-specific distributions and feature correlations, as illustrated in Fig.
Extensive evaluations across six real-world datasets demonstrate that EPIC significantly improves the data quality and generation efficiency, achieving state-of-the-ard performance. As shown in Fig. 2 baselines exhibit low sensitivity, struggling to accurately generate minority class samples due to inherent class imbalances. In contrast, EPIC achieves high sensitivity and balanced performance for all classes, underscoring its robustness and practicality for real-world applications.

---

Group-wise CSV-style Prompting
Unique variable mapping
Model Input
inoome, www.aeddcccoooccoo hours per-week pou.r-per-eeeeke
Variable
Random word mapper
Variable
Bachelors,40
JYO
GGN
>50K
M5sters,55
<=50K
11th,15
Varlable
Random word mapper
Variable
<00
HS HS--grad.37
E3R
L2J
Income:
Varlable
Variable 3
>50K
11h,57
Random word mapper
>50K
Masters,40
CLC
ZWI
<=50K
HS--grad.h
(a)
50K
Masters,28
income, www.ceatooonoouuursosoooooo
JYO
EsR
ZWI
Model Output
2
2
JYO
>50K
Masters,25
>0K
11h.40
3
CLC
L2J
ZWI
50K
HS-grad.33
50K
Masters,
(b)
Figure 3: Overview of our approach. Our prompt includes repeated data example sets consisting of feature names and class-balanced groups, with the feature name at the end serving trigger for the LLM to generate realistic synthetic tabular data. The proposed unique variable remaps categorical values distinct alphanumeric strings, ensuring clear distinction and variability among variables.
In summary, our key contributions are as follows:
Our study explores the effectiveness of LLMs in generating realistic synthetic tabular data through in-context learning, providing prompt design guidelines to efficiently generate high-quality data while addressing class imbalance.
We propose EPIC, simple yet effective prompting method that uses balanced, grouped data samples with unique variable mapping to generate tabular data that accurately represents both minority and majority classes, preserving feature correlations and overall data distribution.
The proposed approach is model-agnostic, generally applicable to various LLMs, and easy to implement for any tabular data with minimal preprocessing requirements.
Extensive experiments six real-world public tabular datasets and toy dataset demon- strate the effectiveness of our approach, significantly improving ML classification perfor- mance and data generation efficiency.
By addressing class imbalance and enhancing classification outcomes, our work contributes to the advancement the crucial field of tabular data research, significantly impacting various domains.
Method
In this study, we investigate various prompt design components to maximize the in-context learning capabilities of LLMs for generating high-quality tabular data. Our objective to develop an optimized approach that reliably and effectively produces realistic tabular data, accurately representing both minority and majority classes to improve ML classification performance, especially addressing class imbalance. Formally, given a tabular dataset I' of dimensions variables, aim to generate synthetic dataset π of dimensions n' x that accurately reflects key characteristics of including class-specific attributes, feature correlations and, data distributions.
m, with samples and m
To achieve this, we explore the following key prompt design elements: data format (Section 2.1). class presentation methods (Section 2.2). variable mapping (Section 2.3), and task specification (Section) 2.4 These design choices are evaluated through extensive analyses on the public datasets

---

from diverse domains, focusing on ML classification performance and generation efficiency. Detailed analysis is provided in Section with Fig. 12 Appendix [c.] illlustrating the ablated versions.
Building on these analyses, we introduce EPIC, a structured prompting method designed to guide LLMs in synthetic data generation while effectively addressing class imbalance, as depicted in Fig. D EPIC begins with optional variable descriptions, followed by series of structured data sample sets, each containing balanced samples for each class. Each set consists of feature names and samples organized by target class. To prompt the LLM to generate corresponding set of synthetic samples, feature names are appended the end of the prompt to serve as trigger, leveraging the pattern recognition capabilities of LLMs. The following sections detail each design component of EPIC.
2.1 Data format: Sentence vs CSV style
We investigate effective data formatting methods to represent tabular data within prompts, ensuring that LLMs can interpret the demonstrations and generate new data accurately. Tabular data can be formatted as plain text or as comma-separatee values (CSV style). Let the feature name of the k-th variable be uk. and the i-th observation of this variable be o, According to previous work 4. this value can be transformed into a sentence-style representation as [Uk, "is", Ok,i ". In contrast, the CSV style can be expressed as [ok,e, "; "], presenting only values, with variable names for all "
variables specified as v = [01, 12, : "."", Umm] at the beginning of the data sample presentation. Here, the sentence style redundantly uses variable names and "is" tokens for every value, leading to higher token usage and computational cost than the CSV style. The CSV-style format enables higher volume of in-context learning examples within the same token constraints. Thus, our method utilizes a CSV-style format to maximize the number of examples, as providing a large number of data samples is crucial for ensuring sufficient representation of the original dataset in in-context learning.
2.2 Class presentation
Class presentation methods aim to ensure adequate representation of all target classes within prompt, particularly addressing underrepresented classes in imbalanced datasets. The construction of data samples within the prompt is crucial, as it can either worsen or mitigate existing imbalances. To address this, explore three prompt design options and propose class balancing and grouping approach that enables accurate and representative generation of data samples across all target classes.
Single-class vs. Multi-class generation When generating data under class conditions, primary consideration is whether to generate data for one class time or for all classes simultaneously. Single-class generation allows the model to focus on the unique characteristics of specific class, while multi-class generation enables direct comparison across classes. Our experiments indicate that generating data for multiple classes simultaneously in structured manner (discussed in the following sections) yields samples that more accurately capture the distinctive features of each class. This finding suggests that generating data with contextual awareness of other class characteristics enhances the representativeness of synthetic samples.
Original vs. Balanced class ratio When constructing prompts to generate multiple classes simultaneously, simple random sampling from the original dataset T often results in an imbalanced distribution of samples across target classes. This imbalance may cause the LLM to overfit to the majority class, limiting its ability to learn the characteristics of the minority class. To mitigate this, we employ balanced sampling approach, equalizing sample numbers for all classes rather than strictly following the original class distribution. Our findings reveal that balancing sample sizes substantially improves the quality of generated data for minority classes, indicating that providing balanced data cnables LLMs to learn more effectively and replicate the accurate characteristics of all target classes.
Listing vs. Class grouping Within a prompt, data samples can either be presented sequentially, in the order they are sampled, or grouped by class. Grouping emphasizes contrasts between different classes and reinforces similarities within each group, enabling the LLM to generate more distinct and coherent samples for each group. Empirical evidence suggests that grouping yields a better representation of class-specific feature correlations in the generated data. Class grouping also contributes to constraining the LLM to generate data with specific attributes more efficiently. Without grouping, guiding the LLM to produce samples with targeted attributes can be challenging, often requiring numerous iterations to achieve the desired outcome. contrast, grouping samples by specific features facilitates conditioning the LLM to generate data in groups with particular attributes or conditions. Although

---

this work primarily focuses on class conditions, this approach can also be applied to feature grouping. enabling controlled generation based on selected criteria.
2.3 Variable mapping: Original values vs. Unique variable mapping
Unlike existing methods that often require extensive preprocessing and handling of noisy or missing data [T9 40], ; approach minimizes these steps, preserving the integrity of the raw data, including original feature names and values, similar to other LLM-based approaches [4. However, a critical challenge arises when dataset contains numerous categorical variables with identical values (e.g.. extensive boolean variables). In such cases, data examples in the prompt may become monotonous and repetitive, making difficult for the model distinguish between variables, which can significantly degrade the quality of the generated data. Moreover, when faced with such repetitive input, LLMs tend to struggle generate valid samples, leading to a notable decline in generation efficiency.
To address this, we propose a unique variable mapping method, which remaps the values of each categorical variable to distinct random alphanumeric strings, as illustrated in Fig. 3 Uniform substitu- tion is applied across the dataset, maintaining the integrity of the data structure while introducing necessary variation. For example, consider categorical variable 11 with values of 't' and 'f" Our approach remaps these values to distinct three-character strings, such as 'JYO' and 'GGN,' respec- tively, and applies them consistently across all data points the dataset. Other categorical variables are similarly remapped to unique values. The transformed dataset is then used as in-context learning demonstrations. Although these new values do not hold inherent semantic meaning, they effectively represent categorical distinctions as symbols, allowing the LLM to identify and utilize patterns within them. This approach ensures that each variable has a unique representation, making the variables clearly distinguishable and leading to more accurate and efficient data generation.
2.4 Task specification
A crucial aspect of the prompt design is clearly guiding the LLM to generate synthetic data as intended. To achieve this, we consider two options and ultimately find that prompting the LLM to learn the pattern from structured, consistent format with optional descriptions for variables is more effective for reliable generation than providing explicit instructions.
Explicit instruction vs. Completion triggering To clarify the task for the LLM, a prompt might include explicit instructions, such as "generate new data samples." However, crafting specific instructions can be challenging, as LLMs are highly sensitive to subtle prompt variations, making it inefficient to determine an optimal phrasing for tabular data generation. A simpler and more effective approach is to provide patterns within the prompt for the LLM to mimic by formatting data samples without explicit instructions, leveraging its advanced pattern recognition capabilities, Formally, let the i-th set of data samples across all classes be structured S [e, $1,1," si.e). where each Si,k contains n samples for the k-th class. Here, u represents feature names and serves as a header to indicate the start of a set. This approach enables the LLM to recognize the pattern in which each set of data samples is structured and consistently begins with v. We then input & such sets in prompt, [S], S2,..., S.], to establish a recognizable pattern, expecting the LLM to generate the next set, S = (v. 8,t1.1, st-1,e]- prompt by including the new header [v] for S signaling the LLM complete the sequence with [st-1,1.".. ... s..1.e). By structuring the input prompt in this way, we guide the LLM to generate a consistent number of samples, n, for each target class.
e]] To reinforce this, we place a trigger at the end of the
Providing contextual information Optionally, adding contextual information to the prompt can help the LLM understand the characteristics of the original dataset and generate accurate samples. Therefore, when available, we include line-by-line variable descriptions at the beginning of the prompt. Additionally, we treat the target class variable as of the attributes and position it the first variable to clarify that each group is organized by the target class.
2.5 Overview of tabular data generation process
The synthetic data generation process begins by creating prompt template and determining whether the unique variable mapping is needed. The prompt template specifies parameters such as the target class, sample size per group, the number of groups, and the number of sets to construct demonstrations.
5

---

We then randomly sample data examples from the dataset to populate the template, ensuring that examples do not overlap within each prompt to maintain diversity. Using the provided examples. the I.I. generates new data in CSV format, which is subsequently converted into structured tabular format. Instances containing categorical variable values not present the original dataset are discarded to maintain authenticity. This generation process is repeated until the desired sample size is achieved, with new examples sampled with replacement in each iteration. This approach enables the LLM to encounter diverse combinations of real samples and produce rich synthetic samples (see Appendix [B.] for analysis). Although each individual generation reflects the distribution of the provided subset, successive iterations enable the LLM to cover broader range of the original data cunulatively. This iterative method ensures that the output generated by the LLM comprehensively reflects the characteristics of the original data. Final prompting examples are available in Appendix C.5
3 Experiments
This section describes the experimental setup and provides comprehensive evaluation of our method, assessing its effectiveness across diverse real-world datasets.
Summary We conduct three unique experiments to analyze the utility of our method in enhancing ML classification performance: augmenting the original dataset with generated data (Tables [ [9 Figs. 2 [4), augmenting only the minority class similar to SMOTE (Table and using only generated data (Table 7 Additionally, we evaluate our approach using three LLMs: Mistral, Llama2, and
gpt3.5-turbo [Table2 2 Figs. 5}. We perform ablations on prompt elements to compare classification performance (Tables 3 5 and conduct unique analyses of token usage and LLM generation efficiency
(Tables 4 8 99. To examine feature correlations, we separately analyze minority and majority classes,
comparing results across all prompt variations (Figs. 4 5). We further investigate how varying the number of generated samples affects classification performance (Figs. 6 77 Using toy dataset, we analyze how different prompts influence the accuracy of generated data distributions (Figs. D 9
Lastly, we explore the sampling of input examples and their corresponding outputs in L.LMs using the toy dataset, providing insights into the variability and reliability of generated data (Figs. 10
3.1 Experimental setup
Datasets We evaluate our method using six real-world public tabular classification datasets from diverse domains: Travel (Marketing), Sick (Healthcare). HELOC (Finance), Income (Social science), Diabetes (Healthcare), and Thyroid (Healthcare). The Thyroid dataset, released after the training cut- off date for GPT-3.5-turbo-0661 provides more rigorous validation of our approach on completely unseen data. For binary classification datasets, the minority class is designated as one. All duplicate data samples are removed from the datasets. Each dataset is randomly split into 80% training and 20% test sets. We retain the original data, including missing or noisy features. The exception is the Sick dataset, where we follow the source's method. Further details are provided in Appendix C.2
Evaluation measure We evaluate our method based on ML classification performance, generation efficiency, feature correlation, and analysis of generated data distribution. For classification, we report F1 score, sensitivity, specificity, and balanced accuracy (BAL ACC). Feature correlation is measured using Pearson correlation for numerical variables and Cramér's V correlation for categorical variables.
Baseline models In our study, we compare our method with various generative models for tabular data, including SMOTE (5) SMOTENC (5), TVAE (35), CopulaGAN (22), CTGAN K5). CTAB-
GAN (39, CTAB-GAN+ [40]. GReaT 4 and TabDDPM [19]. To compare prompting methods, we
also use the prompts from CuratedLLM 27 and LITO 37..
and GPT- 15-turbo-16k-06111 Mistral-7b-vO. [3]. and Llama-2-7b 29). Unless stated otherwise, we use the GPT-3.5 model for our method. The number of synthetic data samples is based on the size of the original datasets. Across all experiments, unless stated otherwise, we report results from four top-performing ML classifiers: XGBoost 77, CatBoost (24), LightGBM [17, and Gradient boosting classifier [10]. known for their strong performance, often surpassing recent deep learning models on tabular datasets. Each classifier executed five independent runs, with results averaged over a total of 20 runs to ensure robustness. Further details are available in Appendix C
Experimental details Our method utilizes the GPT-3.5 models

---

Table 1: Comparison of MI classification performance when synthetic data are added to the original dataset. Results are averaged across four classifiers, with each model run five times. Complete results, including all baselines and standard deviation values, are provided in Table [9
Dataset
Method
#syn
FI score
BAL ACC f
Sensitivity
Specificity
Original
58.12 (0.00)
71.00 (0.00)
57.00 (0.00)
85.00 (0.00)
+TVAE
35
+IK
59.78 (+1.66)
72.35 (+1.35)
62,00 (+5.00)
82.69 (-2.31)
+CopulaGAN
22
+IK
21.76 (-36.36)
55.52 (-15.48)
12.80 (-44.20)
98.23 (+13.23)
Travel
+CTAB-GAN+
+IK
54.66 (-3.46)
68.62 (-2.38)
53.00 (-4.00)
84.23 (-0.77)
+GReaT
4
+IK 60.95
(+2.83)
72.86 (+1.86)
58,80 (+1.80)
86, 16,92 (+1.92)
+TabDDPM
19
+1K 53.20 (-4.22)
67.70 (-3,30)
50,40 (-6.60)
85.00 (0.00)
+Ours
+1K
66.65 (+8.53)
78.23 (+7.23)
78.00 (+21.00)
78.46 (-6.54)
Original
87.81
(0.00)
91.22 (0.00)
82,83 (0.00)
99,61 (0.00)
+TVAE
35
+IK
87.77
(40.04)
91.47 (+0.25)
83.37 (+0.54)
99.56 (-0.05)
+CopulaGAN
+IK
83.60 (4421)
86.61 (-4.61)
73.37 (-9.46)
99. 86 (+0.25)
Sick
+CTAB-GAN+
+IK
82.35
(-5.46)
86.28 (-4.44)
72.83 (-10.00)
99.74 (+0.13)
+GReaT
4
+IK
87.23
(-0.58)
90.83 (-0.39)
82.07 (-0,76)
99 60 (-0.01)
+TabDDPM
19
+IK
85.17 (-2.64)
89.30 (-1.22)
79.02 (:3.81)
99.57 (-0.04)
+Ours
+IK
88.71
(+0.90)
92.93
86.41 (+3.58)
99, 44 (:0.17)
Original
71.01 (0.00)
73.21 (0.00)
67.89 (0.00)
78.52 52 (0.00)
+TVAE 35
+IK
71.12 (+0.11)
73.25 (40.04)
68.15 (+0.26)
78.34 18.34 (-0.18)
+CopulaGAN
22
+1K
71.23
(+0.22)
73.32 (+0.11)
68,37 (-0.48)
78. 28.26 (-0.26)
HELOC
+CTAB-GAN+
+1K
71.03
(+0.02)
73.15 (.0.66)
68, 13 (-0.24)
78.17 17 (-0.35)
+GReaT
4
+1K 70.35
(-0.66)
72.96 (.0.25)
66,22 (-1.67)
79.70 (+1.18)
+TabDDPM
19
+1K 70.65
(-0.36)
72.89 (-0.32)
67.51 (-0.38)
78.26 (-0.26)
+Ours
+IK
71.92
(+0.91)
73.66 (+0.45)
69.96 (+2.07)
77.35 (-1.77)
Original
66.90 (0.00)
76.45 (0.00)
57.28 (0.00)
95.61 (0.00)
+TVAE
35
+20K
66.96 (+0.06)
76.80 (40.35)
59.13 (+1.85)
94.48 (-1.13)
+CopulaGAN
22
+20K
66.75 (-0.15)
76.73 (40.28)
59.16 (+1.88)
94,29 (-1.32)
Income
+CTAB-GAN+
40
+20K
66.49 (-0.41)
76.42 (-0.33)
58.14 (+0.86)
94.70 (-091)
+GReaT
4
+20K
67.95 (+1.05)
77.51 (+1.06)
60.69 (+3.41)
94.33 (-1.28)
+TabDDPM
19
+20K
66.85 (-0.05)
76.50 (40.05)
57.70 (+0.42)
95.30 (:0.31)
+Ours
+20K
69.16 (+2.26)
79.15 (+2.70)
66.45 (+9.17)
91.85 (-3.76)
Original
54.87 (0.00)
42.07 (0.00)
60.00 (0.00)
60.73 (0.00)
+TVAE
35
+10K
54.79
(-0.08)
41.96 (.0.11)
59.96 (.0.04)
60.71 (.0.02)
+CopulaGAN
22
+10K
54.27
(4.60)
41.59 (.0.8))
59.73 (.0.27)
59.97 (-0.76)
Diabetes
+CTAB-GAN+
40
+10K
54.24
(-0.33)
41.52 (.0.5))
59.63 (-0.37)
60.01 (-0.72)
+GReaT
4
+10K
54.78
(-0.09)
41.98 (-0.09)
59,98 (-0.02)
(+0.12)
+TabDDPM
19
+10K
54.64
(-0.23)
41.83 (-0.24)
59,91 (-0.09)
60,55 (-0.18)
+Ours
+10K
54.94
(+0.07)
42.14 (+0.07)
60.04 (+0.04)
60.82 (+0.09)
Original
94.23 (0.00)
95.08 (0.00)
91.14 (0.00)
99.02 (0.00)
+TVAE
35
+IK
90.45
(-3.78)
92.20 (-2.88)
86.36 (-4.78)
98.04 (-0.98)
+CopulaGAN
22
+1K
86.73
(-7.50)
88.71 (-6.77)
78.41 (-12.73)
99.02 (0.00)
Thyroid
+CTAB-GAN+
+IK
27.46
(-66.77)
58.07 (:37.01)
16.14 (-75.00)
100,0 (+0.98)
+GReaT 4
+IK
91.31
(-2.92)
92.46 (-2.62)
85,91 (-5.23)
99.02 (0.00)
+TabDDPM
19
+IK 94.39
(+0.16)
96.26 (+1.18)
95,45 (+4.31)
97.06 (-1.9)
+Ours
+IK
94.80 (+0.57)
96.39 (+1.1)
95.23 (+4.09)
97.55 (-1.47)
3.2 Machine learning classification performance using the synthetic data
We evaluate the quality of the synthetic data samples by assessing the machine learning classification performance when the synthetic data are added the original dataset. Here, our method utilizes the GPT-3.5-turbo model. As shown in Table [ our method achieves state-of-the-art F1 scores and balanced accuracy across all six datasets, surpassing the baselines that require model training. Notably, our method is the only one that consistently outperforms the original data in both F1 score and balanced accuracy across all six datasets. in classification, the correlation between input data and labels in the training data must be precise. Thus, the consistent improvement across six datasets demonstrates that the data generated by our method aligns well with the class labels. Other methods often result in worse performance than the
a machine learning model to perform well

---

Table 2: Comparison of classification performance using synthetic data generated by different LLMs with our method. We report the average performances of five runs of gradient boosting classifier. #syn denotes the number of synthetic samples added to the original dataset.
Dataset
Method
Model
#syn
Fl score
BAL ACC 1
Sensitivity f
Specificity t
Original
60.00.10.00
72.31 1.0.00
60.00....00
84.62.0.00
+Ours
Mistral
+1K
66.67.10.00
78.00
76.00.0..00
80.001...0
Travel
+Ours
Llama2
+1K
67.80=+0.00
79.23=+0.00
80.00=0.00
78.46=0.00
+Ours
GPT3.5
+IK
70.18+0000
80.77:+0.00
80.00=0.00
81.54+0.00
Original
84.09
89.86.10.00
80.43
99.28
+ Ours
Mistral
+1K
88.42.10.00
95.15.10.00
91.30
99.00 00.00000.00
Sick
+ Ours
Llama2
+1K
85.53-+0.95
93.14
87.39
08.88
+Ours
GPT3.5
+IK
91.95:100000
93.41
86.96=00.00
99.86=00.00
Original
71.32:+0004
73.39
68.47 =0006
78.31
+Ours
Mistral
+1K
71.48:+0.00
73.74::0.00
68.00=0.00
79.47=10.00
HELOC
+Ours
Llama2
+1K
70.77+0.00
73.08:+0.00
67.37
78.79+0.00
+Ours
GPT3.5
+1K
71.93110.02
73.68
69.90.10.00
77.45
original data, indicating that the data generated by baselines disrupts the original data distribution. These findings underscore the robustness and effectiveness of our method, affirming its superiority over baselines, even for challenging datasets.
Moreover, baselines such as GReaT and TabDDPM exhibit significantly lower sensitivity compared specificity, particularly for Travel and Income. Their high balanced accuracy is due to high specificity. but their severely low sensitivity indicates a failure generate appropriate minority class data. In contrast, our method significantly improves sensitivity and achieves balance between sensitivity and specificity while also attaining the best- balancee accuracy and F1 score compared to the baselines. For example, in Travel, our method achieves sensitivity of 78%, which is 19%p higher than the second-bes method, GReaT, at 58.8% In Income, our method shows 66.45% sensitivity. surpassing the second-bes best score of 60.69%. These results highlight the effectiveness of our method accurately representing minority classes tabular datasets.
Overall, our findings demonstrate that our approach is generally applicable across various imbalanced tabular datasets and excels generating high-quality samples that improve ML classification perfor- mance, Further analyses exploring the impact of augmenting the original dataset for the minority class (Table 6) and replacing the original dataset with synthetic data (Table 7 are available Appendix [
3.3 Open-source LLMs
We also apply our prompting method to open-source LLMs, including Llama2 (29 and Mistral [5] As shown in Table our method exhibits robust performance when used with open-source LLMs across various datasets. Notably, in the Sick dataset, Mistral demonstrates the highest balanced accuracy, indicating its effectiveness in generating high-quality tabular data. These results validate the broader applicability and general effectiveness of our approach with open-source LLMs.
3.4 Exploring optimal prompt design through ablation studies
We investigate key prompt design choices and validate method through extensive ablation studies across multiple datasets from diverse domains, focusing on MI performacce and generation efficiency, as detailed in Tables and respectively. Given a fixed number of input samples, we evaluated (1) the number of input tokens required, (2) the number of valid generated samples, and (3) the generation success rate. Our results indicate that CSV-style prompting generally outperforms sentence-style prompting in ML performance with the same number of input samples. While generating data for one class at a time yields good results for the Sick and Income datasets, it results in a low Fl score of 58.67% on the Travel dataset. However, generating data for multiple classes simultaneously with balanced samples improves this score to 70.37%. % Furthermore, applying unique variable mapping consistently enhances Fl scores on the Sick dataset. The combined approach of random mapping and grouping achieves higher score of 91.99 91.95% compared to 86.60% with random mapping but without grouping, underscoring the positive impact of this combined approach. These findings indicate
8

---

Table 3: Comparison of F1 scores with ablated methods using the gradient boosting classifier. Asterisk (*) in Income indicates where only +1K synthetic samples are used due low success rate in sample generation. The '-' symbol denotes where our unique variable mapping is not needed.
Format
Class
Balance
Group
Unique
Sick (+1K)
Travel (+IK)
Income (+20K)
Sentence
single single
X
X
81.52
43.24.10.00
69.62
CSV-style
x
X
X
86.60=0.00
58.67=0.00
70.49=0.00
Sentence
multi
X
X
X
77.95=0.33
38.33=+0000
67.96=0.00
CSV-style
multi
X
X
X
87.74
50.98110.00
68.55611.00
Sentence
multi
X
X
81.68=0.49
55,56=:0.00
69.26
CSV-style
multi
X
X
84.68
70.37:
70.64:0..0
CSV-style
multi
86.60=0000 /
CSV-style multi
V
X
81.555:0.00 70.18+0.00
70.17:0.00
VSV-style
multi
/
91.95.10.00
Table 4: Comparison of token usage and generation efficiency across ablated methods the Income dataset. Results are based on 100 inferences, each with 20 random input samples. Input tokens indicates the number of tokens required in the LLM prompt for fixed number of input samples. Output samples shows the average number of synthetic samples generated per iteration. Success rate measures the ratio of inferences that generate at least one valid data sample.
Input tokens
Output samples
Success rate
Format
Class
Balance
Group
Unique
#set
+
Sentence
single
x
X
1832.9
1.26
96%
CSV-style
single
x
X
1024.6
1.11
16%
Sentence
multi
x
X
1827.4
0.97
97%
CSy-style
multi
x
912.8
0.38
38%
Sentence multi
1938.7
1.73
97%
VSyystyle
multi
/
X
920.3
0.88
31%
CSV-style
multi v
965.6
8.6
48%
CSy-style
multi
X 2 1014.6
10.52
94%
Table 5: Comparison of classification performance the Sick dataset for task specification elements in prompt design. Results are averaged across XGBoost, CatBoost, LightGBM, and gradient boosting classifier. Methods marked with an asterisk (*) use the prompt designs proposed in the respective papers. #syn denotes the number of synthetic samples added to the original dataset.
Method
#syn
Fl score f
BAL ACC f
Sensitivity f
Specificity
Instrutton-CCrrteeLLLLM
+1K
15.86.15.13
56.02.13544
17.17111.80
94.86.14.9
Instruction-LITO"
+1K
1..3
72.06:12.49
84.46...110
59.66.1..50
Ours w/ class distinction
+1K
74.06.12.64
96.23
96.74.1.99
95.72.10.60
Ours w/o description
+1K
76.06:44.56
96.29
96.41
16.
Ours
+IK
88.71
92.93
86.41=1.8
99.44:10.27
that prompt design significantly influences the quality of generated data, emphasizing the need for carefully tailored prompts to generate high-quality synthetic tabular data.
For the Travel and Income datasets, performance without grouping slightly better than with grouping, but the differences are within 0.5%p, making their performance on par. However, these show significant differences LLM generation efficiency. Without grouping. an average of 0.88 samples are generated per iteration, whereas grouping increases this to an average of 8.6 samples per iteration. Repeating the set twice within the prompt further improves generation efficiency, boosting the average output to over 10 samples and the success rate to 94%, by creating input patterns that M.LMs can mimic to generate data. Unique variable mapping also significantly enhances L.LM generation efficiency and success rates, detailed in Table 8 of Appendix A.3

---

str wurai
vie balana
viri swodts
can Ealanad
Abl
Abl
Abi
Tab0OPM
CTGAN
Abl
TahO0PM
CTGAN
Abl hodeeeas
Abllowoongee
Abl
CTAB-GAN+
TVAE
Abl halemee
Abl
Abl
CTAB-GAN
TVVE
Minorily class
Majority
Figure Difference between Cramér's correlation matrices of real and synthetic datasets for categorical variables in the Travel dataset. More intense colors indicate larger differences, with positive differences shown in red and negative differences shown in blue. Black indicates where the correlation is not measured since only unique value is generated for those variables.
Furthermore, we conduct an ablation study on the task specification elements, as shown in Table Our approach of triggering completion significantly enhances overall ML performance compared to the instruction-bast achieves the best sensitivity, indicating that the distinction facilitates better generation for minority classes. Providing variable descriptions improves balanced accuracy but reduces the FI score, sug- gesting that this element may be optional rather than a definitive improvement. A comparison of generation efficiency using these methods is provided in Table of Appendix A.3
prompting methods. Interestingly, adding class distinctions within the prompt
3.5 Analysis of feature correlation
We compare the feature correlations of the categorical variables between original and synthetic data, as illustrated Fig. Our synthetic data generated with Mistral and Llama2 exhibit the closest feature correlation with the original data for both minority and majority classes, outperforming the baselines and ablated versions. Additionally, our grouping method significantly contributes to preserving feature correlation for the minority class. These findings indicate that method generates data that more accurately matches each class, particularly for underrepresented classes, compared to other methods. Additional results the Sick dataset are available in Fig. 5 of Appendix A.3
4 Limitations and future work
When the training dataset is large and cannot be fully included in the LLM prompt due to token size limitations, only subset of the data can be used as examples for generating samples. If these prompt samples do not fully represent the original data distribution, the generated data may be incomplete and of low quality, as LLMs are limited to producing data based only on the patterns present in the input. To overcome this, our method employs multiple rounds of random sampling with replacement create a synthetic dataset that more comprehensively represents the original distribution, resulting in improved machine learning classification performance (a comprehensive analysis is provided in Appendix I. However, this approach still carries the risk that samples may not fully capture the original data distribution. Future research will focus on developing methods to identify key examples that more accurately represent the entire dataset.
Conclusion
This study demonstrates the effectiveness of using LLMs for synthetic tabular data generation. We introduce EPIC, simple yet effective solution for generating realistic, high- high-quality data without additional training, specifically designed to address class imbalance. Our method demonstrates significant improvements over state-of-the-an generation models across six real-world public datasets, generating data with highly accurate feature correlations and significantly improving MI classification accuracy for minority classes. These results underscore the substantial impact of our approach in real-world applications, making a significant contribution to the field of tabular data research.
10

---

Acknowledgements
This work was supported by Institute for Information communications Technology Promotion(IITP) grant funded by the Korea (MVI) NN..RS-0019-1110077 Artificial Intelligence Graduate School Program(KAIST), Electronics and Telecommunications Research InstituteETTRI) grant funded by the Korean government (24ZB 1220, Fundamental Technology Research for Human-Centric Autonomous Intelligent Systems), and the National Supercomputing Center with supercomputing resources including technical support IKSC-2024-CRE-0028)
References
[1] Mohammad Alauthman, Amjad Aldweesh, Ahmad Al-qerem, Faisal Aburub, Yazan Al-Smadi, Awad M Abaker, Omar Radhi Alzubi, and Bilal Alzubi. Tabular data generation to improve classification of liver disease diagnosis. Applied Sciences, 13(4):2678, 2023.
[2] Rok Blagus and Lara Lusa. Smote for high digeesiimnni class-imbalanced data. BMC bioinformatics, 14: 1-16, 2013.
[3] Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug. Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[4] Vadim Borisov, Kathrin Seßler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Language models realistic tabular data generators. Proc. the International Conference Learning Representations (ICLR), 2022.
[5] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
[6] Zhengping Che, Yu Cheng, Shuangfei Zhai, Zhaonan Sun, and Yan Liu. Boosting deep learning risk prediction with generative adversarial networks for electronic health records. In 2017 IEEE International Conference on Data Mining (ICDM), pp. 787-792. IEEE, 2017.
[7] Tianqi Chen and Carlos Guestrin. Xghoost: A scalable tree boosting system. In Proc. the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 785-794, 2016.
[8] Edward Choi, Siddharth Biswal, Bradley Malin, Jon Duke, Walter F Stewart, and Jimeng Sun. Generating multi-lahel discrete patient records using generative adversarial networks. In Machine learning for healthcare conference, pp. 286 305. PMLR, 2017.
[9] Aakanksha Chowdhery, Sharan Narang, Jacoh Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Schastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
[10] Jerome H Friedman. Greedy function approximation: gradient boosting machine. Annals of statistics, pp. 1189 1189-1232, 2001.
[11] Elif Ceren Gök and Mehmet Onur Olgun. Smote-ne and gradient boosting imputation based random forest. classifier for predicting severity level of covid-19 patients with blood samples. Neural Computing and Applications, (5122):15693-15777 2021.
[12] Yury Gorishniy, Ivan Rubachey, Nikolay Kartashev, Daniil Shlenskii, Akim Kotelnikov, and Artem Babenko. Tabr: Tabular deep learning meets nearest neighbors. Proc. the International Conference on Learning Representations (ICLR), 2024.
[13] Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 36, 2024.
[14] Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A transformer that solves small tabular classification problems in second. In Proc. the International Conference on Learning Representations (ICLR), 2023.
[15] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.

---

[16] James Jordon, Jinsung Yoon, and Mihaela Van Der Schaar. Pate-gan: Generating synthetic data with differential privacy guarantees. In Proc. the International Conference on Learning Representations (ICLR), 2018.
[17] Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Yan Liu. Lightghm: A highly efficient gradient boosting decision tree. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 30, 2017.
[18] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zcro-shot reasoners. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 35:22199-22213, 2022.
[19] Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. Tabddpm: Modelling tabular data with diffusion models. In Proc. the International Conference on Machine Learning (ICML), pp. 17564-1777 17579. PMLR, 2023.
[20] Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao. Dorsa Sadigh, and Andy Zeng. Large language models as general pattern machines. In Conference on Robot Learning, pp. 2498-2292 2518. PMLR, 2023.
[21] Mimi Mukherjee and Matloob Khushi. Smote-enc: A novel smote-based method to generate synthetic data for nominal and continuous features. Applied System Innoaation, 4(1):18, 2021.
[22] Neha Patki, Roy Wedge, and Kalyan Veeramachaneni. The synthetic data vault. In 2016 JEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 399-410. IRFE, 2016.
[23] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg. J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-lcamg Machine learning in Python. Journal of Machine Learning Research, 12:2825 2830, 2011.
[24] Liudmila Prokhorcnkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: unbiased boosting with categorical features. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 31, 2018.
[25] Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.0792 2024.
[26] Joydeb Kumar Sana, Mohammad Zoynul Abedin, M Sohel Rahman, and M Saifur Rahman. A novel customer churn prediction model for the telecommunication industry using data transformation methods and feature selection. Plos one, 17(12):c0288095 2022.
[27] Naheel Seedat, Nicolas Huynh, Boris van Breugel, and Mihacla van der Schaar. Curated Um: Synergy of I1ms and data curation for tabular augmentation in ultra low-data regimes. arXiv preprint 2rXiv:2312.12112 2023.
[28] Kashun Shum, Shizhe Diao, and Tong Zhang. Automatic prompt augmentation and selection with chain- of-thought from labeled data. Findings of the Association for Computational Linguistics: EMNLP 2023, 2023.
[29] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307. 230.000887 2023,
[30] Boris van Breugel and Mihaela van der Schaar. Position: Why tabular foundation models should be a research priority. In Proc. the International Conference Machine Learning (ICML). 2024.
[31] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quac V Le. Ed H Chi, Sharan Narang, Aakanksha Chowdhery. and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In Proc. the International Conference on Learning Representations (ICLR), 2023.
[32] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thouuhh prompting elicits reasoning in large language models. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2022.
[33] Xumeng Wen, Han Zhang, Shun Zheng, Wei Xu, and Jiang Bian. From supervised to generative: A novel paradigm for tabular deep learning with large language models. In Proc. the ACM SIGKDD International Conference Knowledge Discovery and Data Mining (KDD), 2024.
12

---

[34] Junjielong Xu, Ziang Cui, Yuan Zhao. Xu Zhang, Shilin He, Pinjia He, Liqun Li, Yu Kang, Qingwei Lin, Yingnong Dang, et al. Unilog: Automatic logging via Ilm and in-context learning. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, pp. 1-12, 2024.
[35] Lei Xu, Maria Skoularidou, Alfredo Intesta-nnfante and Kalyan Veeramachaneni. Modeling tabular data using conditional gan. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2019.
[36] Chengrun Yang. Xuezhi Wang. Yifeng Lu, Hanxiao Liu, Quoe V Le, Denny Zhou, and Xinyun Chen. Large language models optimizers. Proc. the International Conference Learning Representations (ICLR), 2024.
[37] June Yong Yang. Geondo Park. Joowon Kim, Hyeongwon Jang, and Bunho Yang. Language-interfaee tabular oversampling via progressive imputation and self-authenticttonn. In Proc. the International Conference on Learning Representations (ICLR), 2024.
[38] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2023.
[39] Zilong Zhao, Aditya Kunar, Robert Birke, and Lydia Y Chen. Ctab-gan: Effective table data synthesizing. In Asian Conference Machine Learning, pp. 97 112. PMLR, 2021.
[40] Zilong Zhao, Aditya Kunar, Robert Birke, Hiek Van der Scheer, and Lydia Y Chen. Ctab-gan+: Enhancing tabular data synthesis. Frontiers in big Data, 6: 6:1296508, 2024.
[41] Pei Zhou, Jay Pujara, Xiang Ren, Xinyun Chen, Heng- Tze Cheng, Quoc V Le. Ed H Chi, Denny Zhou, Swaroop Mishra, and Huaixiu Steven Zheng, Self-disoove:: Large language models self compose reasoning structures, arXiv preprint arXiv:2402.0380 036200 2024.
Rieg-Tzee
13

---

Appendix
This supplementary material enhances the main manuscript by providing detailed experimental results and additional visualizations.
Appendix provides additional experimental results on the machine learning classification for imbalanced data, ablation study, and feature correlation analysis.
Appendix B studies the impact of sample size on the machine learning performance and comprehensively analyzes the experiments conducted on the toy dataset.
Appendix provides comprehensive information on the datasets, baselines, implementation details, and prompt design and examples.
Appendix D provides related work.
Appendix discusses the broader impacts of our research.
Appendix presents confusion matrix results and the complete results Table
A
Additional experimental results
Our experiments with synthetic tabular data for imbalanced classes follow three main approaches: (1) Adding synthetic data to both minority and majority classes of the existing dataset and evaluating machine learning classification performance. The results of this approach are presented in Table of the main manuscript. (2) Adding synthetic data only to the minority class, with detailed results provided Appendix(A.]] (3) Evaluating classification performance using only synthetic data without the original dataset, as discussed in Appendix A.2 Moreover, Appendix A..3 presents further ablation study results, and Appendix A.4 provides feature correlation analysis for the sick dataset.
A.1 Comparative analysis of augmenting the original dataset for the minority class
One of the major advantages of our method is its ability to generate high-qualit synthetic data for minority classes, even with limited samples. Similar to oversampling techniques like SMOTE, we add synthetic dataset only to the minority class and evaluate machine learning classification performance. To balance the ratio of majority to minority classes, we add only minority class samples to the original data and then measure the classification performance of the machine learning model. For this task, we additionally investigate the effectiveness of our method compared to SMOTE and SMOTENC (5). As shown in Table 6 adding minority class samples generated from method leads to higher
Table Comparison binary classification performance when augmenting the minority class with synthetic data to balance class sizes. Average performance of the gradient boosting classifier is reported over five runs. #syn denotes the number of synthetic samples added the original dataset.
Dataset
Method
#syn
Fl score f
BAL ACC 1
Sensitivity 1
Specificity
Original
60.00
72.3111000
60.00.10.00
84.62-10.00
+SMOTE
+163
13.45::1.05
75.08:10.99
68.00.10.00
12.15
+SMOTENC
+163
02.6142.54
74.74442.00
70.40
79.08
+TVAE
+163
54.55+00.00
68.46:20.00
60.00++0.00
76.92:10.00
Travel
CCopulaGAN
+163
52.46::0.00
66.62-10.00
64.00.10.00
69.23-1-.00
+CTGAN
+163
57.63
70.92+00.0
68.00.10.00
73.85-10.00
+GReaT
+163
62.07+00.00
74,46:00.0
72.00:+0.00
76.92+0.00
+Ours
+163
72.13+0.00
83,23+0000
88,00+0.00
78.46=0000
Original
69.05
78.02-10.00
61.03
95.00.10.00
+SMOTE
+13,487
69.62.10.25
10.23
18.86
:1.11
ASMOTENC
+13,487
69.7910.11
80.51
71.80.10.23
89.22.10.19
+TVAE
+13,487
68. 08.26++0.00
79, 79.12+0000
68.43-+0.00
89,82++0.0
Income
+CopulaGAN
+13,487
67.76+00.2
81.12+00001
80.15++0.03
82.09440.00
+CTGAN
+13,487
68.51++0.00
78.82+00.00
66.26++0.00
91.38++0.00
+GReaT
+13,487
70.0440.00
83.36110.00
85.08
81.64-10.00
+Ours
+13,487
71.09
83.60-10.01
83.53.10.03
83.66410.00
14

---

Table 7: Comparison of classification performance using only synthetic data for training. Average performance of the gradient boosting classifier is reported over five runs.
Dataset
Method
#syn
Fl
BAL ACC1 f
Sensitivity t
Specificity t
Original
60.00.10.00
1..00
60.00.10.00
84.62.10.00
TVAE
IK
46.51+00.0
63.85=+0..00
40.00::0.00
87.69:00.00
CopulaGAN
IK
0,00#00.00
49.23=+0.00
0,00#0.00
98.46::00.0
Travel
CTGAN
IK
12.50++0.00
50.15+00.00
8,00+0.00
=0000
GReaT
IK
62.50.10.00
73.85:10.00
60.001:0.00
87.69
Ours
IK
70.00
81.233+0.00
84.00
78.46::0.00
Original
71.32:10:04
13.39
68.47
78.31
TVAE
IK
68.91
71.50
65.28:10.12
77.72+1007
CopulaGAN
IK
69.50:+0.00
70.82:0..00
69.17:+0.00
72.47:+0.00
HELOC
CTGAN
IK
68.71+0002
71.98+00.3
63.36++0.00
80,60+00.05
GReaT
IK
64.144+0.10
69.544+0.07
55.76++0.15
83.33++0.11
Ours
IK
71.48
70.17100.0
78.90.10.16
61.44.0.16
Fl scores and balanced aceuracy on both Travel and Income datasets, compared to when samples generated by other baselines are added. the Travel dataset, where the number of minority class samples is substantially low, baselines, especially CopulaGAN and CTGAN, demonstrate markedly low sensitivity, indicating their failure to accurately learn the minority class data distribution. In contrast, our method demonstrates its efficacy in producing high-quality data for minority classes on the Travel dataset. Compared to the Original, significantly improves the F1 score by 12.13%% 13%%, underscoring its superior performance of our approach.
A.2 Comparative analysis of replacing the original dataset with synthetic data
We compare the classification performance using only synthetic data for training, shown in Table 7 For both the Travel and HELOC datasets, the classification model trained on synthetic datasets generated by our model outperforms the one trained on original data in terms of F1 score and sensitivity. This improvement in performance suggests that the data generated by our method accurately represents the distribution of the original data. Furthermore, it highlights the effectiveness of our class-balanced generation approach in improving the performance of classification models.
A.3 Additional ablation study results
This section presents additional ablation study results on the Sick datasets. We evaluate the token efficiency and generation stability of ablated versions based on 100 inferences, each with 20 random input samples. As shown in Table g for the Sick dataset, due to its repetitive, monotonous values, the number of generated samples is low, even with grouping. Using a unique variable mapper in such cases significantly increases the number of samples generated per attempt from 6.93 to 17.68. This improvement demonstrates that our approach effectively controls the quantity of generated data samples, ensuring the stable production of synthetic tabular data. Moreover, the ability to generate multiple data samples a single inference can significantly lower generation time and costs.
We also compare the efficiency and stability of generation when ablating task specification elements in our prompt design, as shown Table The prompt design from Curated LLM [27], despite requiring the highest number of input tokens, achieves high performance in terms of output samples and success rate. However, the generated data shows relatively low quality, resulting a low classification performance with an Fl score and balanced accuracy, discussed in Section 3.4 In contrast, the prompt design of LITO [3] is the most efficient in terms of input tokens but produces the fewest output samples and has the lowest success rate, indicating instability in synthetic data generation.
Our proposed method balances efficiency and performance, using approximately half the input tokens compared to CuratedLLM'ss prompt while generating a high number of output samples and maintaining high success rate. Additionally, the synthetic data generated by our method achieves the highest classification performance in terms of F1 score, as shown in Section 3.4 When class distinctions are provided, the average number of output samples decreased from 17.68 to 9.70. This result indicates that it is more robust distinguish groups with simple numbering, such as A and
15

---

Table 8: Comparison of token usage and generation efficiency across ablated methods on the Sick dataset. Results are based on 100 inferences, each with 20 random input samples. Input tokens indicates the number of tokens required in the LLM prompt for fixed number of input samples. Dutput samples shows the average number of synthetic samples generated per iteration.
Success rate measures the ratio of inferences that generate at least one valid data sample.
Input tokens
Output samples
Success
Format
Class
Balance
Group
Unique
#set
+
rate
Sentence
single single
x
X
3867 2
0.52
52%
CSV-style
X
X
1223.6
2.42
38%
Sentence
multi multi
X
X
3879.9
0.52
52%
CSV-style
X
1226.3
3.05
48%
Sentence
multi
X
X
3870.0
0.63
63%
VSyystyle
multi
X
1117.3
0.70
16%
V-ysteele
multi
X
1985.4
3.00
15%
VSy-style
multi
1331.3
4.73
64%
VSyystyle
multi
X
2
1439.9
6.93
99%
CSV-style
multi
2
2060.8
17.68
95%
Table Comparison token usage and generation efficiency for task specification ablation on the Sick dataset. Results are based on 100 inferences, each with 20 random input samples.
Method
Input tokens +
Output samples 1
Success rate f
Instruction CuratedLLM Instruction-LLTPT
4677.4
21.67
95 %
1277.4
4.06
57%
Ours w/ class distinction
2222.7
9.7
91 %
Ours w/o description
2066.9
15.96
88%
Ours
2060.8
17.68
%
Ours-orsssets
Oursvara
Ours-
CopulGGAN
Minority class
(b) Majority class
Figure 5: Difference between Pearson correlation matrices of real and synthetic data for merical variables in the Sick dataset. More intense colors indicate larger differences, with positive differences shown in red and negative differences shown in blue. Black indicates where the correlation not measured since only one unique value is generated for those variables.
B, rather than explicitly specifying class information each group. Including variable descriptions slightly reduces the number of output samples and success rate compared not including them, but the difference is not significant. This suggests that the grouping strategy plays a more crucial role in robust data generation than the presence of variable descriptions.
A.4 Analysis of feature correlation between numerical variables
In the main manuscript, we visualize the feature correlation between categorical variables. Here, we analyze the Pearson correlation between numerical variables in the Sick dataset and compare differences feature correlation values to the original data, as shown Fig. 5 As result, our methodology consistently shows feature correlations similar to the original data for both minority and majority classes. In contrast, other baselines either show significant deviations in feature correlation for all classes perform well for majority classes but not for minority ones. These results demonstrate
16

---

Specificity (w/ rep.)
Specificity (w/o rep.)
(w/ rep.)
BAL ACC (wlo rep.)
Sensitivity (w/ rep.)
Sensilivity (wlo rep.)
F1 score (w/ rep.)
score (w/o rep.)
0
20k
40k
60k
80k
100k
Number of generated samples
Figure 6: Classification performance on the Income dataset when synthetic data generated by proposed method added the original dataset. We experiment with varying synthetic sample sizes, comparing data sampling methods: with replacement (w/ rep.) and without replacement (w/o rep.). EPIC samples with replacement. GPT-3.5-turds is used for the experiment.
0.50
3.00
0.50m
score
0.62
0.5
0.40
0.38
0.40
10k
2
10k
Number of synthetic samples
Number of synthetic samples
(a) Income
(b) Diabetes
Figure 7: Classification performance on the Income and Diabetes datasets using only synthetic data generated by our proposed method for training. Black dashed lines denote the F1 score using only the original data. GPT-3.5-turbe is used for our method.
that our methodology shows superior performance compared to the baselines in generating high- quality data that accurately represents both minority and majority classes within the original dataset.
B
Additional analysis of EPIC
This section provides an in-depth analysis of the proposed method, EPIC. Appendix B. analyzes the effect of sample size classification performance. Appendix (B.2 discusses the significance of the observed performance improvements. Appendix B.3 provides complete qualitative results on the toy dataset. Finally, Appendix B.4 explores tabular data classification with in-context learning methods.
B.1 Impacts of sample size on the classification performance
Adding synthetic data the original dataset Data samples in prompts can be selected either with or without replacement. Our method uses sampling with replacement to generate large and diverse dataset, as sampling without replacement limits the amount of synthetic data to the number of actual data points. While ensure that there maintain diversity, each prompt is constructed using sampling with replacement.
overlapping examples within each prompt to
We conduct experiments to analyze (1) how much datasets can be enlarged and (2) how this impacts classification performance by comparing sampling with and without replacement. As shown in Fig. 6 performance improves steadily in both scenarios as the volume of generated data increases. However, improvement constrained when sampling without replacement due to the limited number of possible samples. When sampling with replacement, as the dataset size expands, there is noticeable improvement in the balance between sensitivity and specificity, which contributes to enhanced overall performance, including gains in balanced accuracy and Fl score. Generating up to 40K synthetic
17

---

90
85
80
75
70
65
Original data
+GeeaT
TabDDDPM
+Ours
score
BAL ACC
Sensitivity
B Specificity
Figure 8: Average classification results across six datasets compared with the closest baselines.
Table 10: Performance improvement after adding synthetic data to the original dataset. Results reorganized from Table of our main manuscript.
Improvement from original (% p)
Method
Dataset
spore
BAL ACC
Sensitivity
Specificity
Traell
2.83
L86
1.8
1.92
Sick
-0.58
0.39
-0.76
40.01
HELOC
-0.66
-0.25
-1.67
1.18
Income
1.05
1.06
3.41
-0.31
Diabetes
-0.09
-0.09
-0.02
-0.12
Thyroid
-2.92
-2.62
-5.23
0.00
Travel
-4.2
.3.3
0.00
Sick
-2.64
-1.92
-3.81
-0.04
HELOC
-0.36
-0.32
+0.38
-0.26
TRDDPPM
Income
-0.05
0.05
0.42
-0.31
Diahetes
Thymid
0.16
1.18
4.31
-1.96
Travll
8.53
7.23
21
-6.54
Sick
0.9
1.71
3.58
-0.17
HELOC
Ours
Income
2.26
2.7
9.17
-3.76
Diabetes
0.07
0.07
0.04
0.09
Thymid
0.57
1.31
4.09
-1.47
data points resulted in even better performance than the 20K synthetic data points reported in Table of our main manuscript. We also observed that as the volume of generated data continues to increase, the gains in balanced accuracy and F1 score eventually plateau, indicating diminishing returns and suggesting that further data generation beyond certain point offers limited additional benefit.
Using only synthetic data We investigate how varying the number of synthetic samples affects classification performance. This analysis is conducted on the Income and Diabetes datasets, and the results are shown in Fig- 7 For the Income dataset, sample sizes range from 500 10,000, while for Diabetes, the range was 500 to 20,000. For both datasets, a consistent increase in the Fl score is observed as the number of samples grows. This result demonstrates that our method can adequately represent the original data distribution by generating sufficient number of synthetic data samples.
B.2 Significance of performance improvements
We analyze the significance of performance improvements achieved by EPIC. As shown in Fig. 6 while the gains in Fl score and balanced accuracy between 0 and 40K samples may seem modest, the sensitivity actually increases significantly from around 50% to 70%. This improvement balances performance across all classes, leading to more meaningful outcome and greatly enhancing model usability. Baselines often learn biases the training data, resulting in abnormally high specificity at the expense of sensitivity. This imbalance can inflate balanced accuracy or Fl scores, but such performance is ineffective for real-world tasks. In contrast, our method significantly enhances sensitivity with only slight reduction in specificity. As result, method achieves the most balanced performance across all four metrics, as shown in Fig.
Our method is the only approach among the seven baselines that consistently improves both Fl score and balanced accuracy compared using only the original data (Table Extensive experiments across six real-world datasets from diverse domains (finance, healthcare, marketing, and social
18

---

Original data
Ours
CSt ee--talancedd
Ablation balanced
Ablation Givoorginal
Ablation Giniingle
Ablation
Ablation sonn original
Ablation Son single
GReaT
Figure 9: Complete generation results on an imbalanced toy dataset with majority and minority classes. Our approach, leveraging in-context learning with LLMs, achieves (I) distinct class bound- aries, (2) accurate feature correlations, (3) well-matched value ranges, (4) robust numerical-categorica relationships (last row), and (5) comprehensive data distribution coverage, with improvements its ablated versions and the fine-tuned GReaTT model 4
science) using four classifiers, each tested five times, demonstrate the state-of-the-art performance of our method, with significant gains in the highly imbalanced Travel, Sick, Income, and Thyroid datasets (Fig. [3). While GReaT and TabDDPM
the closest baselines, they exhibit inconsistent performance. As highlighted in Table I0 they reduce original data performance in more than half of the cases (red). For example, both underperform on HELOC and Diabetes.
In stark contrast, our method consistently outperforms the original data across all datasets (blue). In the challenging Diabetes dataset, only method improves over the original data. While specificity decreases in all methods, our method still achieves a superior balance of metrics, leading greater practicality. These results underscore that method offers the greatest practical utility and meaningful performance advantage among the models tested.
B.3 Complete results on toy dataset
In Fig. we illustrate the complete qualitative results on the toy dataset. We assume challenging multivariate data scenario an imbalanced binary classification situation, where variables with and without correlations These variables have strong correlations in pairs. There are 1 numerical variables and one categorical variable, and the categorical variable is shown on the x-axis in the last row of Fig. 9 In the figure, the majority class is shown in blue, and the minority class is shown in red.
mixed. Using the scikit-learn library (23) we created total of 12 variables.
The generated results are evaluated from four aspects: (1) assessing whether generated data samples correctly belong to their respective classes, ensuring clear class boundaries: (2) examining whether shapes formed by variable correlations, such circles or moon patterns, are accurately represented; (3) evaluating whether the generated values fall within an acceptable range for each variable; and (4) analyzing whether relationships between categorical and numerical variables accurately captured.
19

---

SEORK
Illustration of the input samples (majority and minority) and the corresponding generated samples (majority and minority) for each inference of EPIC on the imbalanced toy dataset. In this example, with 180 input samples per inference, EPIC requires 23 inferences to generate 1,000 samples. Empty boxes represent cases where the model fails to generate valid samples.
Figure 10:
The baseline method, GReaT [4, has successfully learned from the training data generate class- separated samples. However, upon closer inspection, there are samples belonging to different classes. Ours, despite generating samples without training, shows similar or better quality.
Overall, CSV-style generation produces better class separation boundaries compared to sentence- style generation. Regarding class presentation, giving single class can sometimes result in good generation outcomes, but fails to accurately capture the correlation with the majority class, resulting in shapes like squares instead of circles, and it also fails accurately generate the categorical variable. On the other hand, when generating multi-class samples, if the sample numbers are not balanced, the generation results for the minority class can be compromised. Even when balanced, providing grouping leads to clearer class distinctions. Additionally, adding unique variable mapping significantly improves the performance, resulting in well vell-distinguished categorical variable.
In addition, we visualize the inputs and outputs both our method and the ablated model without grouping, as shown in Fig. 10 and Fig. [1] For each inference, 180 bals-cal samples are input into the LLM prompt. To generate total of 1,000 samples, our method required 23 inferences, whereas the ablated model needed 37 inferences, demonstrating the stability of method in generating synthetic data samples. In terms of data generation quality, our method successfully generates both majority and minority classes in most inferences, with the distribution of the generated samples closely mirroring that of the actual input data. In contrast, the ablated model exhibits a higher number of inferences where the generated data deviates from the input data distribution.
B.4
Comparison advanced tabular classification models
The ability of LLMs to generate high-quality synthetic data for imbalanced classes in our approach highlights their potential to directly address classification tasks, offering a promising avenue for future research. However, the challenge lies in designing effective prompts fully leverage this potential. The performance of LLMs is highly dependent on prompt design, and numerous studies have shown that small variations prompt crafting can lead to significant differences outcomes [8 25 Our work shares this motivation, focusing on designing prompts that enable LLMs effectively generate high-quality tabular data, particularly to address class imbalance. While exploring LLMS for direct classification could enhance performance, this direction is beyond the scope of our current study.
20

---

FEEE
Illustration of the input samples (majority and minority) and the corresponding generated samples (majority and minority) for each inference of the ablated version of our method the imbalanced toy dataset. In this example, with 180 input samples per inference, the ablated method without grouping requires 37 inferences to generate 1,000 samples. Empty boxes represent cases where the model fails generate valid samples.
Figure 11:
We conduct experiments using recent in in-context learning based tabular classification methods, such as TabPFN [1], which employs pretrained transformer, and T-Table [33, which leverages LLMs. We also test. TabR [12], an advanced deep-learning tabular classification model, using its official code. As detailed Table IT we evaluate classification performance on the Travel dataset using the original and synthetic data. For T-Table, we use the GPT-3.5-turde model, incorporating all original data but limiting synthetic samples to 200 per input prompt due to token constraints. To further enhance performance, employ voting across five inferences (3I.
Table 11: Comparison of Fl scores using robust tabular classification models on the Travel dataset. GB refers to gradient boosting classifier.
Model
Original
+Ours
+TabDDPM
+GReaT
XGBoost
55.32
67.74
51.06
64.00
LightGBM
60.00
64,29
55.32
62.50
CatBoost
57.14
64.41
48.09
57.73
GB
60.00
70.18
58.33
59.57
TabPEN
56.00
59.26
55.32
53,06
T-Table
21.62
30.43
10.53
17.65
TabR
46.41
60.78
44.88
32.41
21

---

CSVVBalaanadd-gruuuingggmuu
CSV-balanced-muttt
CSV-original-muitt
Model Inpott
Model Input
Model Inputt
Incorm,, education houre-per-wwook
Incomo www.ccacccccccuur.ooom.o
inoome, www.acion..cuuree.ccoo per-wook
>50K
Bachelors.40
111h
Masters,55
505
HS gradd7
<=50K
HSS grad,37
<=50K
.110,15
>50K
Bachelons.40
>50K
Bachelors,
<=50K
HH8-grd,40
HS-grad,40
50K
Masters,29
<>50K
Masters,2e
education wwwp:aaliiiooooeeeee-reeeeee
>50K
Masters,55
50K
HB-grad22
>S0K
HS grad,60
>50K MMsttrrs,e 40
>50K
Mastors,40
500
Masserrs
<=50K HHS-grad, 40
<-50K
Income, aducation hours-ooeeee wwurs-perr-wox
Model Output
Model Output
Model Output
>50K
Masters.25 25
Masters 25
<<50K 110h220
>50K
,10h,40
<=50K [MS-grad,37
<=50K grad.37
<-50K
Masters,20 20
CSV-single
Sentenoe-orfginal-mutt
Model Input
Model Input
income,
Income <=50K, Education HS-
<>50K
,119,15
40
HS.grad.37
<=50K
HS grad,40
Masters, Sourr-per-wook
Masters_20 20
Income <=50K. Education 11th, Hours wer- ls
<=50K
Bachelors,,1 15
Income =50K Education is
Rachelors,21
Masters, Hoursper-west
SOK
.10R,00
Income =S0K Education is
.91h.40
Bachelors, Hours-perweek <-50K, Education HS-
Hours-per-eekk
Education Rturs-par-waskii
Income is <-50K, Education is 10th Hours per waok is 60
Model Output
S0K
JHS grad,37
Model Output
Income c=:0K, Education is Mastors, Hours par-weok
Figure 12: Illustration of ablated versions of our proposed method.
The results indicate that, while the newly introduced classifiers do not outperform traditional ones, adding synthetic data generated by method the original data consistently leads to significant performance improvements across all classifier. In contrast, using TabDDPM and GReaT often results in decreased performance. Our method uniquely and consistently enhances the performance of different classifiers, demonstrating superior label-matching quality. These findings underscore the value of high-quality synthetic data in enhancing classifier performance across diverse models, highlighting the importance of data generation research as distinct area from classifier development.
Tabular data generation is critical area of research with significant implications 4
37 This
task serves two primary purposes: (1) enhancing classification performance in model -agnosic manner through data augmentation, similar SMOTE, as shown in Tables generating synthetic data to replace original data in security-critical or srivacy-sensitie contexts, demonstrated in Table 7 In fields such as healthcare, where collecting samples achieving balanced class labels is challenging and data may be noisy or incomplete, generating high-quality synthetic data is crucial.
and 6 and (2)
In conclusion, even as more advanced classification models for tabular data are developed in the future, proposed method could continue to play crucial role in enhancing performance
22

---

Table 12: Dataset details used in this study.
#Categorical features
#Numerical features
Dataset
#Class
#Samples
Domain
Travel
447
Marketing
Sick
2
21
6
3,711
Medical
HELOC
9,872
Financial
Income
2
8
6
32,561
Social
Diabetes
36
101,766
Medical
Thyroid
2
16
1
364
Medical
by generating high-quality synthetic data, thereby potentially making a significant impact on the tabular data classification community and related tasks.
C Additional experimental details
This section provides the illustration of ablated versions (Appendix comprehensive information on the datasets (Appendix C.2 and comparison baselines (Appendix C.3
implementation details
(Appendix C.4 and final prompt design and examples (Appendix
C.1 Ablation prompt examples
This section visualizes five ablated versions of our proposed prompt design, shown in Fig, 12 The ablations involve removing or altering four key components: data format, balanced class ratio, multi-class generation, and class grouping.
C.2 Dataset
detailed description of the datasets is shown in Table 12 The variable descriptions are directly sourced from dataset platforms like Kaggle or the UCI repository. In our approach. we designate the class with fewer instances as the positive class for binary classification tasks. We also provide the class distribution information for the datasets used in this work, as shown in Fig. [3] Among the six datasets, the Travel, Income, Sick, and Thyroid datasets exhibit substantial class imbalances.
100%
80%
60%
40%
20%
0%
Travel Marketing
Sick Medical
HELOC Finance
Income Social
Diabetes Medical
Thyroid Medical
Class
Class
Class 2
Figure 13: Class distribution for the real-world public tabular datasets used in this study.
C.3 Baseline reproducibility
In our research, we utilize implementations from the Synthetic Data Vault 22 for TVAE [5] CTGAN [35. and CopulaGAN, while for the GReaT [4] model, we employ code from its offi- cial repository. In our experiment, the GReaT model fails to accurately generate column names "max_gl Leax_glu_seeeee serumm'm'' and 'AlCresult" in the Diabetes dataset, even after training for 85 epochs, as re- ported in the original paper. Extending the training to 105 epochs does not rectify this issue, with "max_ saaegggu_uu_ serumm and 'AlCresult' being correctly generated only approximately 2.8% and 12. % of instances, respectively. For these columns, failure cases are treated as NaN values.
23

---

We utilize the official open-soure code from the official TabDDPM GitHub repository to reproduce CTAB-GAN [9), + 440, and TabDDPM [9] For the Adult Income and California datasets, the hyperparameters provided are used. For other datasets lacking provided hyperparameters, we apply the hyperparameters from the Wilt dataset, which, according to the corresponding paper, demonstrated the largest performance improvement over the original data.
Due to the lack of official code provided by the authors of LITO [37 we adopt the prompt design described in their original paper. Specifically, we use the prompt for LITO-C to generate minority class data. To generate samples for both majority and minority classes, we also use the modified prompt to generate majority class data. Similarly, for CuratedLLM (27], we follow the prompt design detailed in their paper. Although LITO and Cu propose multi-stage techniques for data generation, we focus solely on evaluating their prompt designs.
C.4 Experimental setup
For classification, we utilize the target class of each dataset to divide groups. Specifically, the Diabetes dataset has three groups, and the Travel, Sick, HELOO,, Income, and Thyroid datasets have two groups. For the process of unique variable mapping, each discrete value is transformed into combination of three characters, including uppercase letters and digits. Regarding hyperparameter settings of downstream machine learning models: For the gradient boosting classifier, we use the default hyperparameters provided by scikit-learn library. For XGBoost 77] CatBoost [24], and LightGBM [17], we conduct 5-fold Pross-validatee two key hyperparameters: learning rate and max depth. These optimized settings are then consistently applied across all experiments, aligning the model's performance closely with the characteristics of the original data. Where possible, we train the classification models using a single NVIDIA GeForce Ubuntu 18.04 system.
the training set of the original data, optimizing
RTX 3090 GPU
C.5 More details on the prompt design
This section outlines the construction of the proposed prompt. As shown in the prompt examples in the final prompt template for synthetic data generation consists of four parts.
Tables 13 18
Descriptions (Optional). The prompt may start with descriptions of the variables in the dataset, provided line-by-line, if precise and available. These descriptions define what each column in the dataset represents, offering contextual understanding to the LLM about the data. It will be located at the top the prompt (but not repeated to maintain token efficiency) for additional context.
Set-level examples. Then, a set-level examples are present. This section begins with data entry format header, specifying the order and names of each feature. This header acts as a guide for the LLM, indicating the start of an example set, which is crucial for interpreting the structure of the tabular data. Following the header, there's line specifying a group name, signaling the start of a specific group. Instances belonging to this group are then listed, one per line, in a CSV style. This process is repeated for each group within the set.
Repetition of set-Level template. To reinforce the data structure and patterns, this set-level template repeated several times within the prompt, each time with distinct data samples. This repetition is key in enabling the LLM to recognize, learn, and subsequently replicate the set format in its synthetic data generation process.
Trigger for LLM data generation. After all the example sets are presented, the same header that indicates the start of example set is included the end of the prompt. This header serves signal for the LLM to commence generating synthetic data, maintaining the structure and patterns established in the prompt.
By following this structured approach, the LLM is guided to effectively generate synthetic data that matches the structure and characteristics of the original dataset.
D Related work
LLM in-context learning prompting With the remarkable advancements in LLMs, extensive research has been conducted in prompt engineering maximize their potential for various natural
24

---

language processing (NLP) tasks, including text summarization and question answering, yielding significant value (9 28 36 Optimizing prompts for specific tasks inherently a combinatorial challenge, and in the absence of established optimization principles, progress has often been driven by heuristic methods validated through rigorous empirical evaluations 225 32 38 Our study adopts and structured formatting and unique variable mapping.
this empirical approach, conducting extensive experiments to develop effective prompts for synthetic tabular data generation, which holds unique challenges that differ from typical NLP tasks. Through these experiments, we introduce a tailored prompt design, EPIC, that employs class-balanced, grouped,
Tabular data generation In the field of tabular data synthesis, traditional methods have predomi- nantly focused on interpolation techniques, such as SMOTE [5]. Although useful, these methods often struggle to capture complex relationships between features
2 Recently, generative models have and diffusion models have also achieved promising results in this area [19]. Building these advances, GRea'T [4] achieved notable success by transforming tabular data into natural text format and fine-tuning LLMs for synthetic tabular data generation. However, this fine-tuning process is resource-intensive. requir- ing extensive training of large models for each dataset. To address this, LITO [37] and Curated LLM (27) have explored using LLMs through in-context learning for tabular data generation, yet these approaches require additional filtering steps to ensure data quality. contrast, we demonstrate that with comprehensive prompt exploration, high-quality tabular data generation can be achieved through prompt design alone, without the need for additional steps. Our work is distinguished by the thorough and comprehensive evaluations across six real-world datasets and synthetic toy data, aiming to provide deeper insights into the prompting process. We addressed experiments and analyses often overlooked in prior research on tabular data generation.
shown promise in generating realistic tabular data 66. 8
35
39
E
Broader Impact
Our method is generally applicable to tabular data generation and classification tasks with minimal preprocessing, making it accessible for diverse research and industrial applications. Potential positive impacts include enhanced data accessibility and privacy preservation, which can support decision- making processes in fields such as healthcare and finance by reducing reliance on real data and protecting individual privacy. However, potential negative impacts include the risk of data misuse. Synthetic data generation could be exploited maliciously, such as to create deceptive datasets intended to mislead systems or individuals. Additionally, using closed-source LLMs poses potential risks, including data leakage through API calls, particularly concerning when handling sensitive data, and limitations direct access to verify the models being used. Ensuring quality, security, and ethical use of generated data under these conditions is essential.
F
Complete quantitative results
This section presents the complete confusion matrix results in Appendix F.1 and compares the classification performance with the haselines in Appendix F.2
F1 Confusion matrix results
We present normalized confusion matrix results binary classification datasets, shown in Fig. [4] The results demonstrate that adding the synthetic data generated by our method to the original dataset significantly enhances classification performance, outperforming baselines by large margin. Additionally, using only the synthetic data generated by our method achieves better performance than augmenting the original dataset with synthetic data from baselines.
F.2 Complete results comparison with baselines
Table 19 offers the complete performance comparison results with standard deviation values of 20 independent experiments on all six datasets.
25

---

Original data
+Ors
Ours only
+TaDDDPM
+GReaT
84.6%
15.4%
81.5%
18.5%
78.5%
21.5%
86.2%
13.8%
77.%%
12.3%
40.0%
60.0%
20.0%
80.0%
16.0%
84.0%
44.0%
56.0%
44.0%
56.0%
+CTAB-GAN+
+CTAB-GAN
+CTGAN
+CopulaGAN
+TVAE
87.7%
12.3%
92.3%
7.7%
95.4%
4.6%
96.9%
3.1%
81.5%
18.5%
88.0%
(a) Travel
Original data
+Ours
Ours only
+TaDDDPM
+GReaT
0.1%
19.6%
80.4%
13.0%
87.0%
6.5%
93.5%
21.7%
78.3%
19.6%
80.4%
+CTAB-GAN+
+CTAB-GAN
+CTGAN
+CopulaGAN
+TVAE
99.7%
0 3%
99.6%
0.4%
99.60
0.4%
99.9%
0.1%
99.4%
0.6%
26.1%
19.6%
80.4%
26.1%
19.6%
(b) Sick
Original data
+Ours
Ours only
+ +TbDDDMM
+GeeaT
95.0%
0%
91.9%
8.1%
82. 3%
17.7%
94.9%
5.1%
94.0%
6.0%
True
39.0%
61.0%
32.2%
67.8%
21.6%
78.4%
39.8%
60.2%
37.2
62.8%
+CTAB-GAN+
+CTAB-GAN
+CTGAN
+CopulaGAN
+TVAE
94.3%
94.6%
5.4%
94.1%
5.9%
93.8%
6.2%
93.8%
6.2%
39.5%
60.5%
39.0%
61.0%
40.2%
59.8%
37.8%
62.2%
37.9%
62.1%
Prediction
Prediction
Prediction
Prediction
Prediction
(c) Income
Figure 14: Normalized confusion matrix results on binary classification datasets using the gradient boosting classifier. Ours only denotes cases where only our synthetic data are used.
26

---

Table 13: Example of an EPIC prompt for the Travel dataset. This prompt illustrates the structure of sets and groups. Each iteration utilizes randomly selected samples from the training data.
Template
Prompt sample
Descriptions
Churn: whether customer churns or doesnt churn for tour and travels company,
age: the age of customer,
FrequentFlyer: whether customer takes frequent flights, Annual class of annual income of user, ServicesOpted: number times services opted during years,
IccouttSycceddsSociaalllll whether company account of user
synchronised to their social media,
BookedHoteslrroo:: whether the customer book lodgings/Hotele using company services. \n'n
Set
Header
Group
Churn, 28, Yes, High Income,6,No. Yes Churn, 37, Yes, Income, 4, Yes, Yes Churn, Low Income, 1, Yee, Yesln B. Doeant churn, churn,38,10 Low Income, 1, Yes, No Doeant churn, 28,No Record, Low Income, Yes Doesnt churn,34, Ye, Income, No No
Group
Yes
Set
Header
Group
A-
Churn, 30, Yes High Income, No, Churn, 28, No, Low Income, 6,, Yes Churn, 28, No Record, Kiddle Income, 2,, No B.
Group
Doesnt churn, 37, Yow Low Income, Income,1,No No, Doesnt churn, No,No, Middle Income, No, Yes Doesnt churn, 28, Middle Income, 3, An
Set
Header
Churn, Bau.n..g...rruuun...g.g.....aaae Sewpeeeeeeecceeeesseeeeeeeeeeeee
Group
A.
Churn, 27, Yes, High Income, N,,ome,,,NNo, N,No,, Churn, 37, Now Income, 5, Yes ,No Churn, 33, Low Income, 5, Yes, Yes \n
Group
B.
Doeant churn, ,7, Yea, Income, 1, 1,,No, Doeant (.nur,,36,1. Ni,dMiddee Kiddle Income, No, Yes Doesnt churn, Middle Income, 3, No \n\n
Set
Header (trigger)
Churn, han.e/rreeeenttyyyeemmmmmaaalaae
27

---

Table 14: Example of EPIC prompt for the Sick dataset. This prompt illustrates the structure of sets and groups. Each iteration utilizes randomly selected samples from the training data. In this scenario, unique variable mapping is employed, whereby each unique value of variable is consistently substituted with unique three-character alphanumeric string. This approach ensures diversity and robustness in the synthesized data.
Template
Prompt sample
Descriptions
Class: hypothyroidism condition which thyroid gland is underperforming or producing too little thyroid hormone, age: the age of an patient,
sex: the biological sex of an patient,
TSH: thyroid stimulating hormone,
T3: triiodothyronine hormone,
TT4: total levathyroxine hormone,
T4U:
levothyroxine hormone uptake,
FTI:
free levothyroxine hormone index,
referral_eource. institution that supplied the thyroid disease record. \n'n
Set
Header
Claas, age, sex, thyroxioxeeee query_on_thyroxinn
on on_ant allannithh ww..antttrridd..om celidddmemiiiaiiiiiommmm sick, preegagnnttt.,rrnttttt snyyroid_uurgerry
hell__trateetttccmm query htt.:/aeet..rrryy.ppphoororooooo query. hyperthyroid, lithium
goitre, tumor, hypapituitaary psych, TSH.. measured, TSH, T3 measured, T3,
TT4_measured, TT4, TrU_uueasure, T4U, FTI, "TI__meaured, FTI weT.,rrferra_suure
Group
A.
JY0,64.00 270,64.0,E33 E38,, ZIQ, AAA,,,6, K6Y, RUS, RUS, SGD Q6D, I9R, Z9P 250, Y9J,SYD, IID,,,MI,DD, U28,XWH, XWH, 8,85, PIX, 1.1, 1, ,9S, 99..,99.00.00.... D,,, 1. 11, SD4, SD,,,,D,,,,1 90.0,x66. YY0,72.0,12J 1,,,,, TU1, , K6Y,,US RUS, 006,000,,000800. IER, TFC, 250, STD,SYYDCCCCCC X28, 0.2.H,,0.2..000 0.,, P.X,0. 795,,9.000000. 7,44 112.0,11.0 X5Z\n
Group
CCN,56.0,100 T22,,TU1,AGA.E... XE,,XXYYRUUS, RUS, SQ6, TFG, 250, Y9J,SYD,
S.N, 100,,,04000,111,, 11,,,,,,111,0000 103. 103.0, GGX, 0,00,, T6,,,,1,,,,A,, RU5, SQ6.060
TFG, YYJ,SYD,ZWI,Pa
U28, XWH, 000,000,,2,, ASS, 138.0,8.0000 1. SD4, 88, 2.B.0,,7B\. W7WBB)
Set
Header
Claas, age, sex, on_ ww__thyroxine,. quertyueerree. query_on_tttaaeeeeeeee wtw.oonttttonnee..ooo
on ant. all.nnithhyy tantithyroid.cmm ctciidddmeiiiiatiaaa
pregnant, thyroidd_urgerry
www.trrttettuuurr..yyyyt.. [rst_ttratment,,. query hww.httuuuuryyy.oyyyooi.... www.:hhyoohhhiidd.omm query wwwrrluuurrrrrrraeeee hwperthyroid, lithium
goitre, tumor, hww.phtuitarry...oooomm psych, TSH measured, TSH, T3, st_.aauured, T3,
T74_measured, TT4, 774__measured, T4U, FTI. hTT__meauueddd11 FTI, wwwerrall.eroooooeoeerrreeeeeeee source
Group
A.
Y9J,SYD,2N 2W,,,PD,, PDL, 028,XWHH PIX, ASS, 97 o, DOT, 0.65,,66...... SD4, 150. 150.0,,x52 JY0,60.0,, 1,..6,,,,,2,,..11.1. TU1, K6Y, RU5, SQ6, IER, TPG, 250, Y9J,SYD,.MM FDI, UZ8, 1.I,1,,,..XXX..... PI., ASS, 00,. 02.0,,
700.7,.0,L2J,1 46A, K6Y, KUU,RU,,,,,,AAAACCC SQS,00D, Q6D, IER, TFG, 250 250,
Group
GGN 23,.23.0,E33 TG,, X,,XXYY,UUSSSS RUS, SQ6, 06D IER, TFG, 250, SBJ,SYD,ZWI,PP
3,8,XWW,,..6,IIX, P.X,,,,.,,7.0,44 ASS, 1,110,,00T,1111 1.7,1544444,, SD4,80 80.,80000,Y77. YFF GGX 22.. E3R, TU1, AGA, RU5, SQ6 Q6D. IER, TFG, 250, ZW,,SYY,,W,,,,, U28,
ASS, 102.0,2200000 D,T 0..6..7....... S04,134,.0 YF8AnAn
Set
Header (Irigger)
Class age sex, oe_,,en_ttteeeeeeeeee ww__thyoxxine..o wtw.raeo_eaeyyaeeeeeeeeeee www.oontttooiiiee.m
on ant on__ntithyyln httpooddddeiiccattioooom
pregnant, www.paattt.yyooll.euuueeeeee thyroid_eurgeryy
www.:ttreaeetttque hww.:tuuuuuuyyyyoyyyooiiiie www.thyyooooidd.oooomm query wwwrtuuuerrrrrreeee hyperthyroid,lithee
goitre, tumor, www.pituitaar....mmmmpo. psych, TSH_ measured, TSH, T0SH,,,3__aeeeeeeea measured, T3,
TT4 measured, TT4, Tru_ueeaueee, T4U, FTI measured, PTI, referral source
28

---

Table 15: Example of an EPIC prompt for the HELOC dataset. This prompt illustrates the structure of sets and groups. Each iteration utilizes randomly selected samples from the training data.
Template
Prompt sample
Descriptions
Set
Header
Group
A.
Bad, 90, 90,,,9,22111 [at.::1111.2111000..111111111111
Bad,56, 16,,,,,,,66,,1111 hat.11115...7700011....4441111..
Group
B.
Good, ,7, |o0a::/..8881.80000......1111110 Good,81, 11..
Set
Header
Group
A.
Bad, 63, oto._......03.111333337777333113
73,,73,269 B.
Group
Good,78 [.c.::/...44222.220.............
Good, 38, |11.0
Header (trigger)

---

Table 16: Example of EPIC prompt for the Income dataset. This prompt illustrates the structure of sets and groups. Each iteration utilizes randomly selected samples from the training data.
Template
Prompt sample
Descriptions
Set
Header
income, , workeaewr.aass, ortaa:/ccc....iaa.aa.a..aa.aaaae
patiaaal.sskuuuuu..oppa.r.r.rrra race, -ggin,, -gain,
capital-lsss hours hour,
Group
A.
52,,,5,, Privatt, Prrvatt,,99225, 2ee.222,,.cheeeeee Bachhelrrs,,.... Mar civ-spouse, Seaftrrepair, Husband, White, vale,0,0,6 50, United-States >50K, 350x,30,Selffeete -en i2c,,,,,,,.00 321990, Mastera, 14, Married-civ--pouse, exec-managerral, Husband, White, 15024,0,66 >50K, 28, Local-gov, 33662, Mastera 14, Married--ii-ssoueee Prof-specialty Wife, White, 0.0
Group
B.
Private, Preeea,cc Mirr:eed.....rrrr-iiiii------
Craft-ree ceatttreeairr...e repair,Husbande Husband, White, Male, Male, 0,0, 40,,Nicaaauu
<=50K, 25, 2elifff,,,eee--ee seelf-emp-not-eo cmp-not-inc, 159909, Assoc-voc, Mor.iio..,....edeeeeeeeo
Farming- Farming-fishing, fieeliiihiingneeee Husband, Husbang White, Male,0,0,,000 40, 100,,40,,iiee--ee United-Statee
<=50K, , ?, 182117, comeeco, Never -earriee--rreed,.... ?, 1.000-hhhldd White,
0ale,0,0, 40 United-States'
Set
Header
income hugpaaae...cccaaaa.......i.a.aar
marital stw.sssttsssssss m stt.uuu..ccccttttooorrraaaaisiis race, sex, capital gain,
capital hww.soooruu-o whw.oo--ss--eeeee eet
Group
A. >50K, 30, Srrivate,58108 Bachelors, oor -1,,Married-civ-apoo
exec-managerial Husbaar White, Male,0,,,,000,, 0,0,50, United-States >50K, 3,0k,31,State, 21,tttte-oov Assoo
12, Married civ-spouse,
Tech-support, Husband, White, White,Male,,, 0, 4040, United-States Privvtte,, 130126, pco.-scoolo 15, Married-civ-spouse Prof Prof-specialty Husband, Kale,,
United-Statest
Group
B.
<=50k, 5-50K,51,Prrvateee Private, 138514,, 11, Rnmarried, Black,Female,, O, 0,0,, 48,48,188 United-States
Tech-support.
<-50k, 18, Private, 205218, 11th, Never- Sales, Own-child,Whit
Female, 0 0, 20, United-ttates
<=50k, 21, Private, 185948 2ar
Sales,
Oun-child, White, 0 Mate
Set
Header (trigger)
marital-statte wtt.ssttus..pppptttoooooaaaaaaaa rexe,acceeeeeeeeeeeeeee cepicaitaal-
capital sapital-loss. wook, native-atttiteeeee oeu
30

---

Table 17: Example of an EPIC prompt for the Diabetes dataset. This prompt illustrates the structure of sets and groups. Each iteration utilizes randomly selected samples from the training data. In this scenario, our unique variable mapping is employed, whereby each unique value of variable consistently substituted with unique three-character alphanumeric string. This approach ensures diversity and robustness in the synthesized data.
Template
Prompt sample
Descriptions
Set
readmitted, prter//eeeo........o......ooooor age, weight,
Header
htwp://www...................aaa number
gl algpaeeeeeettttnnnngnggggpgppepe
met orterraiheeesssilaaaaaoooooooonn change,
haapaaaeeeccaggii.deattatttt diabetesMed
Group
A. AD6, 39850434, 428733, HX7, 10F H7H, 2,1,1, MTD, ,S2,, 24,,,44,44 44,4 4, 1,,,,,,,, 6,QDY, HR7, 8K9, JAT NOH,, K06,,,06,22000 2TV, 65H, A7C, NCCC,MK76p JWD, HXX, SSQ,,MY, FFV, ZRU, IDJ, 8C0, 78A, NXL, CWD, 6TQ, VDM, HMH, ODU, HNM, NGY NCA4
AD6, 1117571422 66907593, 7MT, LL7, 10F, BES, 2, 1, 1, 1,, 1,V,FF, CF,,,,Y,, ,Y,0,50,3, 3,16,,,0, 3, GRA, BSI, DQg, JAT, NOHH, K06, 2TV, 65H, 65C,,7C,,, BQS,,
JWD, JWD, DDV,DQV DDV,DDVVV,,YY,,.EE BNV,V, ZRU, ZDJ, 800, 78A, NNL, 6TQ, VDM, HMH, ODU, HNM, UXM,NCANN B.
Group
YRB, 16573200,, 91571517 7NT LL7, HMO, 37H, 3,,,,,,,1,,,,,11 QTD,,, STTD,SKI,7,,6,91
1,6RA, 4UR, OTR, JAT, NOH, K06, 2TV, 65H, A7C, ,K77, JWO, , BVQ,, FFV, ZRU, IDJ, 8C0, 78A, PDO, 6TQ, VDM, HMH, OOU, HNM, ,X51
YRB, 108763158, 24232068, 7NT, UWV, TVU, H7H, 1,6,7,7 7,7,QBR, Q9B,799,46, 00,0,000, 0, I3K, O0H, 68C, 9, JAT, NOH, K06, 2TV, 65H, A7C, A7C,GJH GJH, JWD, HXX,SVQ, B,,, NNV, ZRU, IDJ, 8CO, 78A, NXL, 05I, 6TQ, VDM, HMH, ODU, HNM, NGY An C.
Group
PS6, 21663542935,, 68737500 7MT LL7, 10F HHH, 1.3,7 S,R, 1,1,60,,,,8,,0,0
0,DL8, AL8, AL8,P2T, JAT, ZAX, , K06, 2TV, A5C, A7C, A7C,MX7, ,K7,JWO, HXX, HXX,HVQ, BMY, FCV, 6TQ, VDM, HMH, ODU, HNM, HIMM,NGY NGY NC44
ZRU, IDJ, 8C0, 78A,
PS6, 156572340, 114902370, 7HT, UWV BMO HHHH, 1,1,7, 2SQ,S, S,QSB,SKI S,,,, ,,,,,,8,,,,... 0,0,0,00000 8,0,0, 0, 24 FKD, DZJ, 9, NOH, K06, 2TV, 65H, A7C, MK7, JWD, HXX, HXX,SVX, SVQ, BVQ, BMY,FCV IDJ, 8C0, 78A,,8A, NXL, 8GG, 6TQ, VDM, HMH, ODU, HNM, NGY, MCA An'n
ZRU, IRJ,
Set
readmitted, org.re.........................r ieh
Header (trigger)
time ttt.apa...aaaaaaa..........aaoaa
number number_inpatii hateaeae....iiiiicaa...........e
gl a|
met ottermammaaroollgllaooooooootton chang, diabetesMed
31

---

Table 18: Example of an EPIC prompt for the Thyroid dataset. This prompt illustrates the structure of sets and groups. Each iteration utilizes randomly selected samples from the training data.
Template
Prompt sample
Descriptions
Set
Header
Recurred, Age, Gender, Smoking, Hx Smoking, äx Radiothreapy,
Thyroid Function, Phyeical Examination, (eamination,,Adeopptthy Adenopathy Pathology, Focality, Riak,,,,,,M,,, Stage, Reaaage,Respee Response
Group
A. Yes, 46, M, Yes, Billteral,Follicullaa coli Uni-Focal, T4b, Incomplete
Buthyroid,Singi nodular goiter-left,
N1,, M1, H4g
Structural
Yes, No, No,No,NoNNO M.N.o...,...Buutyrrodd........ Muthrroid,, goiter, Bilateral, Itte
Sogillary,Multiiie
Structural
Incomplete
3ea, F, :NN,No,, No,No, Buthyrooid
RRihht,
Papillary Multrr,,Mltiiooooooo Interrmmdiate, Incompleteln B.
Structural
Group
Mo,31,M,No: No,NNN,,,,tuhoooiii Single nodular goiter-right,No, Papillary,Um Uni-Focall Low, T3a,N0, NO,, Indetrminate Indeontnt No No, NNN, No,, Multinodular goiter,lo No,
Fniilaar,,ui-Foaal,,a. T2,NO, NO, N,,,,,,,,,, Indettrnneeeee Euthyroid, Single nodular golter-right,No, I,,,M,,,,II.Ieleeeee In
No, Fo, Nu,thyyoo
Bimillary,uni-ocal.a
Set
Header
Recurred, Age, Gender, Smoking, Hx Smoking, Hx Radiothreapy,
Thyroid Function, Physical Examination, Exemination,,deeopptty pathhppttyy,,tthloggg Pathology,
Focality Risk, M, Stage Stagee,eeeeeeee
Group
A.
Yea, 37, No, Euthhyo, Multinncuu goiter, Bilateral, Papillary, Sepillaay,Multi-ooooo Intermeiittt N3a,Nib,MM, M,,t Structural Incomplete
Yes, 63, Yes, No, No, Euthyroid, Single nodular goiter-right, Intermediate, N3a, X0,, II, Structural
Right, Paritaaapyllrry...m Multi Mat..pllayy,Miiiii-ooo..... Incomplete
Yes, 80, Yes, No NN, Butbyyoiydii nodular geiter-left, Hurthel cell, Mal1,MultiFFoas Intermediteeee N4,,NO,,M,, II,,Mu,,t,t Incomplete\n
Group
B. No, No,55, F,No, No,NN, No..ooo.ttrroidd... Papillaay,,uii-Foaal] MO, Exccelent No,31, No, No, No, Euthyroid, Micchyroid,Miitinouulaaaa Multinodular goiter,Right,
nodular goiter-left,No,
Mapillary,Multi-PPonn Folti
No. N0.29,F, No, Euthyroid, Single nodular goiter-right,
Focalaal Low, Tlb, M,
Set
Header (trigger)
Recurred, Gender
Smoking Aend
Smoking. Radiothreapy,
Thyroid Function, Phyeical Examination. Examinatioon,Adenqaathh Addeoppathy,Pathhloggg Pathology,
Focal italityy. Risk, M,M, Stage, Response
32

---

Table 19: Complete results comparing ML classification performance when synthetic data are added to the original dataset. Results are averaged across four classifiers: XGBoost, CatBoost, LightGBM, and the gradient boosting classifier, with each model run five times. #syn denotes the number of synthetic samples added to the original dataset.
Dutaset
Method
#syn
Fl score
BAL ACC 1
Sensitivity 1
Specificity
Original
58. 22:204 (0.00)
71.00 71.00 (000)
57.00 (0.00)
85.00 (0.00)
+TVAE
351
+IK
59.78 (-1.66)
72.35 1382 (+1.55)
62. 62.00 (+5.00)
82.69 (231)
+CopulaGAN
+1K
21.76:2200 (33.36)
55.52 (:15.88)
12. 12.80 (44.20)
98.233:1.59 (+13.23)
+CTGAN
35
+1K
29.84+361 (-28.28)
57.79 (-13211
19.20 (37780)
96.38+1.15 (411.38)
Travel
+CTAB-GAN
39
+1K
56.07=889 (-2.05)
69.58=55.14 (-1,42)
51 7..00 (4600)
88. 18.331
+CTAB-GAN+
40
+1K
54.662364 (63.46)
68, 28.662:2240 40 (-2,88)
53. 00 14.000
84.23-+2.06 (-0.77)
+GReaT
[4]
+1K
60.95
(+2,8))
72.86:1180 (+1.86)
58.80 3:0 (+1 (00)
86. 86.92.1.:79 (+1.99)
+TabDDPM
+1K
53.2014410 ( (4992)
67.70.2.69 (3,30)
50. 10.40 14.19 0.80)
85 00.1331 (0.00)
+Ours
+1K
66.65 (+8.5)
78.23::210 (+7.23)
78.00:4459 :4.59 (+21.000
78 18.46622.50
Original
87.81 1.2.4 (0.00)
91. 222.1.995 (0.00)
82.83 $2.83 (0.00)
61 1::22 (0.00)
+TVAE 35
+1K
87.77 12.88 (0.04)
91.47 (+0.25)
83.37 (+0.54)
99. .9.12..3.233 0.05)
+CopulaGAN
22
+1K
83.60=1.33 (:4.21)
86. 36.61 #:.53 (4.60)
73.37
(9946)
99,86=0010
+CTGAN
35
+1K
87.52++166
(-0.29)
89, 86+032 (+1.36)
79.89=0097 (-2.94)
99.82 99.822+0.19 (40.21)
Sick
+CTAB-GAN
39
+1K
86. 36.12.11:40 (-1.69)
89.5 +1.24 (-1.71)
79.35=2249 (-3.48)
99 60.668220066 (40.07)
+CTAB-GAN+
40
+1K
82.35
(-5466)
86. 28:2282220 (4..44)
72.83
(-10.0)
99.
(40119)
+RReaT
4
+1K
87.23 (-058)
90.83 6-0.9)
82.07 1:2.10 (9,26)
99 60.01 (000))
*TabDDPM
19
+1K
85.17
89,30
(3.80)
99.57 57. 50.10 0.03)
+Ours
+1K
88.71:1955 (+0,90)
92.93 (+1.71)
86.41 11885 (+3.50)
99 +0.27 (-1.17)
Original
71.01 10.47 (0.00)
73,21 1:331 (0.00)
67.89 77.89...88 1o.82 (0.00)
78.52 1:034
+TVAE
35
+1K
71.12 (+0,11)
73.
1:033 (+0.04)
68. 15 15.2221 (+0.26)
78. 2:.48 (.018)
+CopulaGAN
+1K
71.23++027 (v0.22)
73.32 (v0.11)
68.37 -6.33 (40.48)
78. 26:00.36 (-1.26)
+CTGAN
15
+1K
70.82+024
(-0.19)
73.06+327 (4015)
67. 67.60=025 (4020)
78. 18.552 (0.00)
HELOC
+CTAB-GAN
39
+1K
70.60 (-044))
72.87:2221 (40 94)
67.39 (49.0)
78. 18.36625744
+CTAB-GAN+
40
+1K
71.03
(-0.02)
73. 15 (-006)
68. 13::2:1 (+0244)
78.1 2:006 (-035)
+RReaT
[
+1K
70.35
10:3 ( (0066)
72, 16.24 (0.25)
66 66.22 2..2..9 (1,67)
79.701.0. (+1.18)
+TabDDPM
+1K
20.15 (.036)
:2.89: (0052)
67.51 (0.58)
78. 28:26:100.30 (0.20)
+Ours
+1K
71.92 (+0.91)
73.66 (40.45)
69.96=0021 (v2.07)
77. +0.55 (-1.77)
Original
66.90 (0.00)
76.45::1445
(0.00)
57.28 (0.00)
95.61 :0.46 (8.00)
+TVAE 35
+20K
66.96
16.80
(+0.35)
$9.13:22.9
94.48 94.481071 (-1.18)
+CupulaGAN
2
+20K
66.75 (-015)
76.73
(+0,28)
59.16.3341 (+1.88)
94.291031 (-1.2)
+CTGAN
15
+20K
66.22 (-0.68)
76.27=a065
(-0.18)
$7.9 1.795 (40.67)
94.59 04.59+0-44 (+1.02
Income
*CTAB-GAN
39
+20K
66.48:2.11 (-0-42)
76.31 1: (+0
$7.45 17)
95. 17 20.58 (-0-49)
+CTAB-GAN+
+20K
16.49.1114
(-041)
76.42.1900
(-0.0)
58. 58.14:22.41 (+0.36)
70..60 (0.9))
GReeTT 4
+20K
67.95
(-1.05)
77.5
(+1.0)
60.69 (+3.41)
94.3310.11 (1.28)
+TabDDPM
19
+20K
66.85 (-0.5)
76.50
57.70=3326 (60.42)
95. =0.30000.00
+Ours
+20K
69.16:1101 (+2.26)
79.15 = (+2.700
66.45 #1.8 (49.17)
91.85 (:3.76)
Original
14.87
(0.00)
42.07=1123 (0.00)
60.00 (0.00)
60.73 63 (0.00)
+TVAE
K5
+10K
54.79.1140 (-0.8)
41, 11,96111.24 (-010)
59, 96.666 (4000)
60.7 71 68 (-002)
+CopulaGAN
+10K
54.27.1148 (..60)
41.59.1.20 (0.9))
59.73 (0.27)
99,97.1 1.65 (.70)
+CTGAN 35
+10K
54.72 (0.15)
41.92 (0.15)
59.86 (0.14)
60.63 (0.10)
Diabctes
+CTAB-GAN
39
+10K
54,22=1128 (-0.55)
41.53 (40.4)
$9.73 (4027)
59.91 9:146
+CTAB-GAN+
40
+10K
$4,24 (+0.63)
4 41.52++101 (-0.5)
59.63=a057 (40.5)
60.01 +:.31 (-0.72)
+GRea'T
[
+10K
54.78 (-0.0%)
41.98-21.17
(+0.0%)
59.98
(40.00)
60.61 20.611:1 1:555 (-0.12)
+PabDDPM
+10K
54. 14.64 (-0.2))
1. (-0.4)
59.9 1:0.5 (4009)
60.55 (-0.18)
+Ours
+10K
54,94
42.14.1.29 (+0,07)
60.04...66
60.82 (+8.09)
Original
94. 23: (0.00)
95. 08 (0.00)
91.14 (0.00)
99. (0.002:::1 (0.00)
+TVAE 33
+1K
90.45
(:378)
92.20 (:2.88)
86.36 1.3.0.30 (4478)
98. (04-00000000
+CopulaGAN
22
+1K
86.73 63.79
88.7 1:21 (6:57)
78.41 15.05 (12.3)
99.00 19.002.1.01
+CTGAN
35
+1K
76.40
(:17.83)
81.14 14.14444555 (-13.94)
62.27==9.0 (:2887)
100.0::0.0 (+0.95)
Thyroid
+CTAB-GAN
39
+1K
53.49#371
(-40.74)
68. 68.30 (-26.78)
36.59 23.4 (-54.55)
100.0=00.00 (+0.95)
-CTAB-GAN+
40
+1K
27.46=780
(666777)
58.07+22.0 (637.1))
16. 14-45.522 (-75.00)
100.0 (+8.98)
+GReaT
4
+1K
91.31
92.46:390 (-2,6h
85.91 1:100
99.9 99.02::11
+ TPbDDPM
19
+1K
94.391100 (-0.16)
96,26
(+1.18)
95.45 14,00 (+4.31)
97. 17.06.1.01 (-196)
+ +Ours
+1K
94.80 1102 (+0,57)
96.39 1862 (+1.31)
95.23 1.00 (+4.099
97.55
(147)
33

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The Abstract and Introduction (Section accurately reflect the contributions and scope of our paper. Refer to the contribution summary at the end of the Introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are provided in Section 4
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken low lighting. Or speech-to-tex system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
34

---

Justification:
This work does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer:| Yes]
Justification:
The paper includes comprehensive experimental details in Section and and provides supplementary code to facilitate reproducibility.
Appendix C
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
35

---

Answer: [Yes]
Justification: We publicly release the source code, along with instructions to reproduce the experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https
/nsps.cccpublic//ine guwdes/ddee/// CodeSubmeeeeeeeee wwweesubmissiooPoee ssionyolccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The paper specifies the training and test details, including data splits, hyperpa- rameters, and their selection process, in Section 3 and Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports standard deviations in Fig. 2 and ML classification perfor- mance tables in main manuscript, and the experimental results in Section and Section F.2
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
36

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer:
[Yes]
Justification: We provide information about the GPU resources used for the experiments in Appendix C.4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed and adhered to the NeurIPS Code of Ethics throughout the research process.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
We discuss broader impacts in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
37

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No data or models are being released in this paper, so there is no risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All existing assets used in the paper are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.ppaersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
38

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The source code includes README file that provides comprehensive docu- mentation.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "7BatgIDGL8", "reviewer": "Reviewer_zoZq", "summary": "The paper investigates how to use LLMs to generate synthetic tabular data for mitigating class imbalance in machine learning tasks. By exploring various prompting methods, the authors aim to identify key design elements that optimize the generation performance. The paper shows that using GPT3.5/Mistral/LLaMA to balancing classes, and employing unique variable mapping produces realistic and reliable data, enhancing classification performance (XGBoost etc) for minor classes in imbalanced datasets.", "questions": "1. Could you provide the experiment results of in-context learning LLM-based classifiers on these tasks?", "limitations": "Yes in Appendix G", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a detailed exploration of various prompt design elements, such as data format, class presentation, and variable mapping, offering valuable insights for future research in this direction.\n2. The proposed methods are easy to implement and require minimal preprocessing, making them accessible for a wide range of applications in tabular data classification tasks.\n3. The experimental results show that the proposed approach consistently improves machine learning classification performance, particularly for minor classes.", "weaknesses": "1. Why not use the LLM itself to perform classification: The designed method uses LLMs to generate data examples for imbalanced classes, which means the LLMs must already have a good ability in modeling the data distribution even for the imbalanced classes. If that's the case, why not use the LLM itself to perform classification directly with the help of in-context learning? I expect that it will be a strong baseline compared to the XGBoost classifiers. The only downside of LLM-based classifiers may be the inference cost, but we should include this baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1. Why not use the LLM itself to perform classification: The designed method uses LLMs to generate data examples for imbalanced classes, which means the LLMs must already have a good ability in modeling the data distribution even for the imbalanced classes. If that's the case, why not use the LLM itself to perform classification directly with the help of in-context learning? I expect that it will be a strong baseline compared to the XGBoost classifiers. The only downside of LLM-based classifiers may be the inference cost, but we should include this baseline.\n\n> Q2. Could you provide the experiment results of in-context learning LLM-based classifiers on these tasks?\n\nA1. Thank you for your insightful suggestion. We concur that the demonstrated ability of LLMs to generate high-quality synthetic data for imbalanced classes suggests their potential for directly addressing classification tasks. Indeed, this represents a promising avenue for future exploration.\n\nHowever, the challenge lies in **designing prompts to fully harness this potential.** The effectiveness of LLMs is heavily influenced by how the prompts are crafted. Numerous studies have shown that performance can vary significantly depending on prompt design, leading to ongoing research aimed at optimizing prompts for specific tasks [1,2]. Our work focuses on designing prompts that enable LLMs to efficiently generate high-quality tabular data, particularly to address class imbalance. While leveraging LLMs for direct classification is an intriguing direction to improve classification performance, it is beyond the scope of our present study.\n\nWe conducted **preliminary experiments using existing in-context learning-based tabular data classification methods**, such as TabPFN [3], which utilizes a pretrained transformer, and T-Table [4], which employs LLMs. Specifically, as detailed in **Table B of the attached PDF**, we evaluated classification performance on the Travel dataset using the original data and synthetic data generated by tabular data generation methods. For T-Table, we employed the GPT3.5-turbo model and, due to token limits, included all the original data but only 200 synthetic samples within each input prompt. Additionally, we used voting across five inferences to enhance performance [5]. \n\nThe results indicate that while the newly introduced models did not outperform traditional classifiers, adding synthetic data generated by our method to the original data consistently led to the greatest performance improvements in both TabPFN and T-Table models. These findings underscore **the value of high-quality synthetic data in enhancing classifier performance across diverse models**, highlighting the importance of research on data generation as a distinct research area, separate from classifier development.\n\nTabular data generation is a critical area of research with significant implications [6,7,8]. This task serves two primary purposes: \n* Enhancing classification performance in a model-agnostic manner through data augmentation, similar to SMOTE, as the results demonstrated in Tables 1, 2, and 6.\n* Generating synthetic data to replace original data in security or privacy-sensitive contexts, as the results demonstrated in Table 7. \n\nIn fields such as healthcare, where obtaining new samples or achieving balanced class labels is challenging and where data may be noisy or incomplete, generating high-quality synthetic data is crucial. \n\nIn conclusion, even as more advanced classification models for tabular data are developed in the future, **our proposed method could continue to play a crucial role in enhancing performance by generating high-quality synthetic data**, thereby potentially making a significant impact on the tabular data classification community and related tasks.\n\n***\n\n\n[1] Kojima, et al., Large language models are zero-shot reasoners., NeurIPS’22  \n[2] Sahoo, et al., A systematic survey of prompt engineering in large language models: Techniques and applications., arXiv’24  \n[3] Hollmann, et al., TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second., ICLR’23  \n[4] Wen, et al., From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models., KDD’24  \n[5] Wang, et al., Self-consistency improves chain of thought reasoning in language models., ICLR’23  \n[6] Yang, et al., Language-Interfaced Tabular Oversampling via Progressive Imputation and Self-Authentication. ICLR’24  \n[7] Seedat, et al., Curated llm: Synergy of llms and data curation for tabular augmentation in ultra low-data regimes., ICML’24  \n[8] Borisov, et al., Language models are realistic tabular data generators., ICLR’23", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O5OxfU9iSI", "reviewer": "Reviewer_p2sf", "summary": "The paper explores the domain of tabular data generation using in-context learning with LLM, in order to improve performance of an ML classifier, especially in imbalanced classes scenarios. The paper explores different prompting techniques, with detailed results on 6 datasets as well as a visualized study on a toy dataset. The paper also compares the result performance boost on several ML classifiers, which is indeed significant. There's a further discussion of efficiency and stability of the generation itself, and how it is affected given the different prompting methods proposed. Results using three different LLMs are presented.", "questions": "I'd add which model was used in table 1 (I'm assuming it's ChatGPT). \nI would stress better the counts of additional synthetic data and why do you sometimes use 1K and sometimes 10K.", "limitations": "Limitations are discussed in the appendix, which is okay. I would perhaps add further discussion on social implications, clarifying the meaning of fig 10 - this method might create more bias in already biased data.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is very clear and discusses almost all important points soundly. It can be used as a guide both for researches facing the problem of generating data for imbalanced class learning, and for researches who are planning to study on LLM prompting techniques in a methodical way. It also presents clear benefits to using the proposed method. I enjoyed reading this paper very much.", "weaknesses": "The main discussion that I missed in the paper was regarding - how much extra data ends up being generated, and does generating more and more data using this method make any sense? I understand that the method uses actual data points, and samples from the original dataset without replacement. This probably creates a limitation to data creation - a point which is not discussed in the main paper body, and should be added. Furthermore, it might make sense to scramble and re-sample the dataset to create even more samples. I would have liked to see a discussion of this point, both to how much the datasets are actually enlarged using the no-replacement sampling, and also if it makes sense to generate more data with replacement, with the limits of this method discussed - when do we reach repetition and duplication, without any further improvement to the classifiers. \nI'm also missing the original dataset sizes and minority vs majority class distributions - to show more strongly the class imbalance. In fact, no dataset descriptions are given at all, which in my opinion takes away from the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1. The main discussion that I missed in the paper was regarding - how much extra data ends up being generated, and does generating more and more data using this method make any sense? I understand that the method uses actual data points, and samples from the original dataset without replacement. This probably creates a limitation to data creation - a point which is not discussed in the main paper body, and should be added. Furthermore, it might make sense to scramble and re-sample the dataset to create even more samples. I would have liked to see a discussion of this point, both to how much the datasets are actually enlarged using the no-replacement sampling, and also if it makes sense to generate more data with replacement, with the limits of this method discussed - when do we reach repetition and duplication, without any further improvement to the classifiers.\n\nA1. Thank you for your insightful comment. You are correct that sampling without replacement limits the amount of synthetic data that can be generated to the number of actual data points. However, we would like to clarify that our method uses sampling with replacement to generate synthetic data. While we ensure that there are no overlapping examples within each prompt to maintain diversity, each prompt is constructed using sampling with replacement. We will clearly state this in our main manuscript.\n\nIn response to your suggestion, we conducted **experiments to evaluate how much the datasets can be enlarged and how this impacts classification performance, comparing sampling with and without replacement**. As shown in **Fig. C of the attached PDF**, performance improves steadily in both scenarios as the volume of generated data increases. However, the improvement is constrained when sampling without replacement due to the limited number of possible samples.\n\nWhen sampling with replacement, as the dataset size expands, there is a noticeable improvement in the balance between sensitivity and specificity, which contributes to enhanced overall performance, including gains in balanced accuracy (BAL ACC) and F1 score. Generating up to 40K synthetic data points resulted in even better performance than the 20K synthetic data points reported in Table 1 of our main manuscript. We also observed that as the volume of generated data continues to increase, the gains in balanced accuracy and F1 score eventually plateau, indicating diminishing returns and suggesting that further data generation beyond a certain point offers limited additional benefit.\n\n***\n> Q2. I'm also missing the original dataset sizes and minority vs majority class distributions - to show more strongly the class imbalance. In fact, no dataset descriptions are given at all, which in my opinion takes away from the paper.\n\nA2. We appreciate your observation. The dataset descriptions, including the original dataset sizes, are provided in Table 10 of Appendix I.2. However, we recognize that we omitted the class distribution information, which is now included in **Fig. D of the attached PDF**. Notably, the datasets where our method showed the most significant performance improvements (Travel, Income, and Sick) exhibit substantial class imbalances. We will incorporate this information into the main paper to enhance the clarity and impact of our findings.\n\n***\n> Q3. I'd add which model was used in table 1 (I'm assuming it's ChatGPT).\n\nA3. We appreciate the reviewer's attention to detail. The results labeled 'Ours' in Table 1 were generated using the GPT3.5-turbo model. To enhance clarity and avoid any potential confusion, we will explicitly specify the model used in Table 1 in our paper.\n\n***\n> Q4. I would stress better the counts of additional synthetic data and why do you sometimes use 1K and sometimes 10K.\n\nA4. The number of synthetic data points we generated was based on the size of the original datasets. For datasets with fewer than 10K samples, we generated 1K synthetic data points; for larger datasets, we generated 10K.\n\n***\n> Q5. Limitations are discussed in the appendix, which is okay. I would perhaps add further discussion on social implications, clarifying the meaning of fig 10 - this method might create more bias in already biased data.\n\nA5. Our initial explanation may have been unclear. When the training dataset is large and cannot be fully included in the LLM prompt due to token size limitations, only a subset can be used as examples for generating samples. If these prompt samples do not fully represent the original data distribution, the generated data may be incomplete and of low quality. As illustrated in Fig. 10, an LLM can only generate a half-circle when prompted with one, highlighting its inability to produce data beyond what is presented in the input.\n\nTo address this limitation, our method employs multiple rounds of random sampling with replacement to create a combined dataset that more accurately represents the original distribution, resulting in improved machine learning classification performance. However, this approach still carries the risk that the samples may not fully capture the original data distribution. Future research could focus on developing techniques to better identify and sample key examples that more accurately reflect the entire dataset.\n\nThe selection of input examples can either reinforce or mitigate existing data biases. We proposed methods such as balancing and grouping to provide examples aimed at reducing these biases. While our primary focus was on addressing class imbalance, this approach could be extended to feature grouping, allowing for the generation of less biased data across specific features.\n\nAdditionally, in Appendix F, we discuss the social implications, particularly the potential misuse of generated data to deceive systems or individuals. This highlights the importance of ethical considerations in applying our methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BtWvZsvlyp", "reviewer": "Reviewer_8bof", "summary": "This paper proposed an LLM-driven tabular data class balancing approach, adapting the in-context learning paradigm and trying various formats and templates to explore optimal prompts to mitigate imbalanced tabular data. Experimental results showed that the proposed method alleviated the CSV-format data imbalance.", "questions": "1. This paper proposed to mitigate the label imbalance issue for tabular data by generating scarce label data, but how to guarantee the generated data and labels were matched?\n2. The presented method is basically an instruction prompting approach, and why didn’t the authors analyze the LLM during the prompting process to make it more traceable?\n3. How did the proposed method handle missing data or noisy features in the original dataset?\n4. Empirical results on the Travel data outperform other data, does that mean that such a prompt is more suitable for it and there exists better prompts for other datasets?\n5. The overall performance of the proposed method achieved limited improvement, and have the authors conducted the improvement-computing cost ratio to make the profit more intuitive?", "limitations": "The authors have adequately discussed the limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The authors probed CSV-format data generation from different perspectives, including data format, class presentation, variable mapping, and task specification.\n2. Experiments and data analysis of this paper were adequate for understanding the proposed data balancing approach, and data performance has been improved to some degree.", "weaknesses": "1. The research purpose of this paper covers merely the CSV-format data, limiting the further adaptivity and impact of the proposed method.\n2. This paper merely tried to find plausible optimal prompts by handcraft, lacking theory analysis, making the prompting even a black box.\n3. Empirical results are not consistent on all test data, especially in the specificity aspect, indicating the instability of the prompting approach.\n4. According to the prompting design, it seems that the prompting process is a bit sophisticated, however, there are no computing cost comparison experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Due to space constraints, we denote weaknesses and questions as W and Q, respectively.\n> W1. Research scope \n\nA. Focusing on tabular data does not limit the adaptivity or impact of our method. Tabular data, composed of mixed variable types such as numerical and categorical variables, represents a widely applicable and essential data format. It is the backbone of many datasets in various fields, including finance, healthcare, manufacturing, and natural sciences [1]. Enhancing performance on tabular data has a significant impact on real-world applications.\n\nSubstantial research has been dedicated to tabular data, such as data generation and classification [2]. By addressing class imbalance and enhancing classification outcomes, our work contributes to the advancement of this crucial field, significantly impacting various domains.\n***\n>W2, Q2. LLM analysis\n\nA. With the remarkable advancements in LLMs, extensive research has been conducted in prompt engineering to maximize their potential, yielding significant value. Optimizing prompts for specific tasks is inherently a combinatorial challenge, and in the absence of established optimization principles, progress has often been driven by heuristic methods validated through rigorous empirical evaluations [3].\n\nOur study adopts this empirical approach, conducting extensive experiments to develop effective prompts for synthetic tabular data generation. **Our work is distinguished by the thorough and comprehensive evaluations we performed across six real-world datasets from various domains and synthetic toy data, aiming to provide deeper insights into the prompting process.** We addressed experiments and analyses often overlooked in prior research on tabular data generation:\n* We used a synthetic dataset to observe how different prompts influence the accuracy of generated data distributions (Figs. 1, 7). For example, multi-class prompts yielded more accurate distributions than single-class prompts, as they allow LLMs to contrast different classes (Fig. 1, 4th row).\n* We explored the sampling of input examples and their corresponding outputs in LLMs using the toy set, providing insights into the variability and reliability of generated data (Figs. 8, 9, 10).\n* We conducted three unique experiments to analyze the utility of our method in enhancing ML classification performance: augmenting the original dataset with generated data (Tables 1, 17), augmenting only the minority class similar to SMOTE (Table 6), and using only generated data (Table 7).\n* We distinctively analyzed feature correlations by separately examining minor and major classes, comparing them across all prompt variations (Figs. 4, 5).\n* We investigated how varying numbers of generated samples affect classification performance (Figs. 6, C).\n* We ablated prompt elements to compare classification performance (Tables 3, 5) and conducted unique analyses of token usage and LLM generation efficiency (Tables 4, 8, 9).\n* We conducted experiments using three LLMs: Mistral, Llama2, and gpt3.5-turbo (Table 2, Figs. 4, 5).\n\nThese analyses demonstrate how specific prompt design components impact the quality of generated data, addressing concerns about prompt optimization being a black box. While our study provides substantial empirical insights, we acknowledge that some areas may need further exploration and welcome detailed feedback, with additional analyses available.\n***\n>W3. Performance\n\nA. Please see General Response 2.\n***\n>W4. Cost\n\nA. We compared the computational cost of our method and its ablated versions in Tables 4, 8, and 9. Given a fixed number of input samples, we evaluated (1) the number of input tokens required, (2) the number of valid generated samples, and (3) the generation success rate. Our proposed prompt demonstrates superior efficiency compared to other prompt designs.\n***\n>Q1. Label matching\n\nA. We demonstrated that the data generated by our method aligns well with the labels through detailed experiments:\n* Our method successfully produces data samples that match the classes, as denoted by color in Fig. 1, even when compared to fine-tuned GReaT.\n* For a machine learning model to perform well in classification, the correlation between input data and labels in the training data must be precise. Adding our synthetic data to the original data consistently improved the F1 score and balanced accuracy across six datasets  (Table 1).\n* Our synthetic data exhibits the closest feature correlation with the original data for each class, outperforming the baselines. (Figs 4, 5).\n\nThese results indicate that our method generates data that is more accurately matched to the classes than the baselines.\n***\n>Q3. Preprocessing\n\nA. Similar to GReaT, we retained the original data, including missing or noisy features, leveraging the capabilities of LLMs. The exception was the Sick dataset, where we followed the source’s method, replacing categorical missing values with \"?\" and imputing numerical values with the mean.\n***\n>Q4. Optimal prompt\n\nA. We carefully selected key prompt design choices and validated our method through extensive ablation studies across **multiple** datasets from diverse domains, including social, healthcare, and marketing:\n* ML classification performance on three datasets (Tables 3, 5) \n* Feature correlations on two datasets (Figs. 4, 5),\n* Generated data distribution on the toy set (Figs. 1, 7) \n\nWhile it is possible that more advanced prompts could be developed in future work, our approach currently represents the state-of-the-art method for tabular data generation.\n***\n>Q5. Cost-benefit analysis\n\nA. Please see General Response 1.\n***\n[1] Breugel et al., Why Tabular Foundation Models Should Be a Research Priority, ICML’24  \n[2] Fang et al., Large Language Models (LLMs) on Tabular Data: Prediction, Generation, and Understanding - A Survey, TMLR’24  \n[3] Sahoo, et al., A systematic survey of prompt engineering in large language models: Techniques and applications., arXiv’24", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QlA241tryP", "reviewer": "Reviewer_kZFF", "summary": "The paper proposes a method by which large language models are utilized to generate synthetic tabular data in order to mitigate class imbalances in existing datasets. The authors provide some tips which they discovered to result in more reliable data being generated, such as enforcing a CSV format. Their methods are validated on 6 real world datasets and one toy dataset.", "questions": "1. In Table 1, for the row \"Ours\" --- which LLM was used to generate the synthetic data? If it one of the gpt-3.5 models, then that should be stated up front so as to not increase confusion.", "limitations": "The limitations are addressed in Appendix G. However, they should be moved up to the main paper for full transparency. Additionally, the provided limitations are quite sparse. It would be helpful if the authors could think a bit more about the real-world usability of their method, as well as the drawbacks associated with using potentially closed-source large language models for their synthetic data generation purposes.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. the justifications for CSV formatting, class presentation, variable mapping, and task specification are all clear and make sense\n2. the choice of experimental results is motivated sufficiently well\n3. the provided results are extensive, including the plethora of ablation studies in the appendix", "weaknesses": "1. In Table 1, outside of the Travel dataset, the other 5 datasets do not show a significant increase in F1 score or balanced accuracy between the proposed method and the next closest baseline. Given this, it is not clear how to interpret the utility of the proposed method.\n2. The motivation for why a large language model would be used to generate synthetic data for tabular settings, at scale, is a bit lacking. In particular, there needs to be discussion about the reasons for why the use of an LLM (with all of the computational and prompt-related overhead) would be preferable to existing data augmentation techniques for tabular data. In particular, it is not clear that the stated results provide enough evidence to persuasively demonstrate the utility of using LLMs for this purpose", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q1. In Table 1, outside of the Travel dataset, the other 5 datasets do not show a significant increase in F1 score or balanced accuracy between the proposed method and the next closest baseline. Given this, it is not clear how to interpret the utility of the proposed method.\n\nA1. Please see the general response, titled “**General Response: 2**.”\n***\n\n> Q2. The motivation for why a large language model would be used to generate synthetic data for tabular settings, at scale, is a bit lacking. In particular, there needs to be discussion about the reasons for why the use of an LLM (with all of the computational and prompt-related overhead) would be preferable to existing data augmentation techniques for tabular data. In particular, it is not clear that the stated results provide enough evidence to persuasively demonstrate the utility of using LLMs for this purpose\n\nA2. Please see the general response, titled “**General Response: 1**.”\n\n***\n> Q3. In Table 1, for the row \"Ours\" --- which LLM was used to generate the synthetic data? If it one of the gpt-3.5 models, then that should be stated up front so as to not increase confusion.\n\nA3. We appreciate the reviewer's attention to detail. The results labeled 'Ours' in Table 1 were generated using the GPT3.5-turbo model. To enhance clarity and avoid any potential confusion, we will explicitly specify the model used in Table 1 in our paper.\n\n***\n> Q4. The limitations are addressed in Appendix G. However, they should be moved up to the main paper for full transparency. Additionally, the provided limitations are quite sparse. It would be helpful if the authors could think a bit more about the real-world usability of their method, as well as the drawbacks associated with using potentially closed-source large language models for their synthetic data generation purposes.\n\nA4. We appreciate your suggestion and will address the limitations in the main manuscript for full transparency.\n \nRegarding the real-world usability of our method, our initial explanation may have been unclear. When the training dataset is large and cannot be fully included in the LLM prompt due to token size limitations, only a subset can be used as examples for generating samples. If these prompt samples do not fully represent the original data distribution, the generated data may be incomplete and of low quality. As illustrated in Fig. 10, an LLM can only generate a half-circle when prompted with one, highlighting its inability to produce data beyond what is presented in the input.\n\nTo address this limitation, our method employs multiple rounds of random sampling with replacement to create a combined dataset that more accurately represents the original distribution, resulting in improved machine learning classification performance. However, this approach still carries the risk that the samples may not fully capture the original data distribution. Future research could focus on developing techniques to better identify and sample key examples that more accurately reflect the entire dataset.\n\nThe selection of input examples can either reinforce or mitigate existing data biases. We proposed methods such as balancing and grouping to provide examples aimed at reducing these biases. While our primary focus was on addressing class imbalance, this approach could be extended to feature grouping, allowing for the generation of less biased data across specific features.\n\nAdditionally, we would like to clarify that our method has been tested with both open-source LLMs (Llama2 and Mistral) and closed-source LLMs (GPT3.5-turbo). The underlying code, data, and algorithms of the open-source LLMs are publicly available and can be accessed, reviewed, and modified by the research community. Notably, Mistral generated data with feature correlations more closely aligned with the original data than GPT3.5-turbo, particularly for minor classes, as shown in Figs. 4 and 5. \n\n**Our method is model-agnostic and can be effectively used with open-source LLMs, not just closed-source ones, thereby not being limited by the drawbacks associated with closed-source LLMs.** However, when using closed-source LLMs, it is important to consider potential risks such as data leakage via API calls when handling privacy-sensitive data and the inability to directly access and verify the model being used. Ensuring the quality and security of the generated data in these circumstances is crucial.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate the reviewers' valuable feedback and positive support, recognizing our method as 
* Providing valuable guidance for researchers addressing class imbalance (zoZq, p2sf).
* Clear, well-articulated, and comprehensive in covering the method (zoZq, p2sf, 8bof, kZFF).
* Supported by extensive, well-motivated experiments (8bof, kZFF).
* Demonstrating clear benefits and consistent improvements, especially for minor classes (zoZq, p2sf). 

**We will thoroughly incorporate their feedback into the camera-ready version.**

Please refer to **the attached PDF** for the updated figures and tables.

We hope our responses address all reviewers’ concerns and would greatly appreciate any further comments or clarifications.
***
# General response 1: Cost-benefit considerations of using LLMs (kZFF, 8bof)

A.  The motivation for using LLMs for synthetic tabular data generation stems from the limitations of existing methods, particularly in addressing underrepresented classes within imbalanced datasets. Traditional approaches often replicate existing biases, which can degrade ML classification performance, sometimes resulting in worse outcomes than using the original data. Our approach overcomes these by using in-context learning to present balanced, grouped examples to LLMs, effectively leveraging their advanced pattern recognition to mitigate data imbalance.

While the costs associated with LLMs are a valid concern, our findings show that these costs are manageable. For example, **generating 1,000 new samples costs less than $1 and takes under 100 seconds** with GPT API calls:
* Sick (27 features): $0.71, 85.0 seconds
* Income (14 features): $0.45, 95.6 seconds

Additionally, using open-source LLMs like Mistral or Llama2 requires approximately 25GB of GPU memory. 

**Our approach also offers significant reductions in various overheads**:
* **Minimal preprocessing**: Unlike existing methods that often require extensive data preprocessing and handling of noisy or missing data, our LLM-based approach minimizes these steps.
* **No training**: Our method eliminates the need for extensive model training and hyperparameter tuning.
* **Single model versatility**: Our approach uses a single model across multiple datasets, unlike traditional methods that require separate models for each dataset.
* **Optimized prompting**: We provide a prompt specifically designed for synthetic tabular data generation, adaptable across various datasets with minimal adjustments, significantly reducing prompt engineering efforts.
* **Open access**: Our provided code allows users to easily generate new samples using open-source LLMs via Hugging Face or the GPT API.

**Despite the reduced costs, our method consistently outperforms existing baselines, producing the highest quality synthetic data**:
* **Consistent performance improvement**: To our knowledge, our method is the first to consistently outperform using only original data or SMOTE among the baselines, achieving state-of-the-art results. This was validated across six real-world datasets from diverse domains, including marketing, medical, finance, and social sciences (Tables 1, 6, 17, **A**).
* **Effective class imbalance mitigation**: Our method excels in generating balanced and representative data for minor classes in imbalanced datasets, preserving feature correlations (Figs. 4, 5) and improving overall model performance. As shown in **Fig. A**, our method achieves balanced improvements across key metrics, underscoring its robustness and practicality for real-world applications.

Our LLM-based approach offers a cost-effective and high-quality solution for synthetic tabular data generation, demonstrating the significant advantages of using LLMs for a wide range of real-world applications.
***
# General Response 2: Significance of performance improvements (kZFF, 8bof)

A. The relatively modest improvements in F1 score and balanced accuracy are due to the challenges of imbalanced datasets. As shown in **Fig. C**, while the gains in F1 score and balanced accuracy between 0 and 40K samples may seem modest, the sensitivity actually increases significantly from around 50% to 70%. This improvement balances performance across all classes, leading to a more meaningful outcome and greatly enhancing model usability.

Baselines often learn biases in the training data, resulting in abnormally high specificity at the expense of sensitivity. This imbalance can inflate balanced accuracy or F1 scores, but such performance is ineffective for real-world tasks. In contrast, our method significantly enhances sensitivity with only a slight reduction in specificity. As a result, **our method achieves the most balanced performance across all four metrics**, as shown in **Fig. A**.

Our method is the only approach among the 7 baselines that consistently improves both F1 score and balanced accuracy compared to using only the original data (Table 17). Extensive experiments across six real-world datasets from diverse domains (finance, medical, marketing, and social) using four classifiers, each tested five times, demonstrate the state-of-the-art performance of our method, with significant gains in the highly imbalanced Travel, Sick, and Income datasets (**Figs. B, D**). **While GReaT and TabDDPM are the closest baselines, they exhibit inconsistent performance.** As highlighted in **Table A**, they reduce original data performance in more than half of the cases (red). For example, both underperform on HELOC and Diabetes.

In stark contrast, **our method consistently outperforms the original data across all datasets** (blue). In the challenging Diabetes dataset, only our method improves over the original data. While specificity decreases in all methods, our method still achieves a superior balance of metrics, leading to greater practicality. 

These results underscore that our method offers the greatest practical utility and a meaningful performance advantage among the models tested.

Meta-review: The paper explores the effectiveness of LLMs in generating synthetic tabular data to address class imbalance in machine learning tasks. It introduces an approach that leverages in-context learning with LLMs to generate realistic and balanced synthetic data, enhancing the performance of machine learning models, particularly for minority classes in imbalanced datasets. The study investigates various prompt design strategies, including data formatting, class presentation, and variable mapping, finding that a CSV-style format and class-balanced grouping significantly improve data generation quality. The approach is validated across multiple real-world datasets, demonstrating its ability to improve classification performance and generate high-quality data without extensive model fine-tuning.

The paper is well-written and easy to follow. The reviewers had some questions and concerns about the experiments and analysis of the paper. During the rebuttal, the authors addressed these issues, and the reviewers were satisfied with the responses and additional experiments. Therefore, I recommend accepting the paper.
Decision: Accept (poster)