<figure><img src="image_1.png" /></figure>

HAL
open science

# Under the Hood of Tabular Data Generation Models: the Strong Impact of Hyperparameter Tuning

G. Charbel N. Kindji, Lina Maria Rojas-Barahona, Elisa Fromont, Tanguy Urvoy

<figure><img src="image_2.png" /></figure>

## To cite this version:

G. Charbel N. Kindji, Lina Maria Rojas-Barahona, Elisa Fromont, Tanguy Urvoy. Under the Hood of Tabular Data Generation Models: the Strong Impact of Hyperparameter Tuning. 2024. hal-04612244

HAL Id: hal-04612244

https://hal.science/hal-04612244

Preprint submitted on 17 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Tabular Data Generation Models: An In-Depth Survey and Performance Benchmarks with Extensive Tuning

G.Charbel Kindji, Lina Maria Rojas-Barahona, Elisa Fromont, Tanguy Urvoy

► To cite this version:

G.Charbel Kindji, Lina Maria Rojas-Barahona, Elisa Fromont, Tanguy Urvoy. Tabular Data Generation Models: An In-Depth Survey and Performance Benchmarks with Extensive Tuning. Neurocomputing, 2025, 658, pp.131655. 10.1016/j.neucom.2025.131655. hal-04612244v4

HAL Id: hal-04612244

https://hal.science/hal-04612244v4

Submitted on 30 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Under the Hood of Tabular Data Generation Models: the Strong Impact of Hyperparameter Tuning

G. Charbel N. Kindji<sup>a,b</sup>, Lina M. Rojas-Barahona<sup>b</sup>, Elisa Fromont<sup>a</sup>, Tanguy Urvoy<sup>b</sup>,

<sup>a</sup>Univ Rennes, IUF, Inria, CNRS, IRISA, Rennes, 35000, France

<sup>b</sup>Orange Labs, Lannion, 22300, France

## Abstract

We investigate the impact of dataset-specific hyperparameter, feature encoding, and architecture tuning on five recent model families for tabular data generation through an extensive benchmark on 16 datasets. This study addresses the practical need for a unified evaluation of models that fully considers hyperparameter optimization. Additionally, we propose a reduced search space for each model that allows for quick optimization, achieving nearly equivalent performance at a significantly lower cost.

Our benchmark demonstrates that, for most models, large-scale dataset-specific tuning substantially improves performance compared to the original configurations. Furthermore, we confirm that diffusion-based models generally outperform other models on tabular data. However, this advantage is not significant when the entire tuning and training process is restricted to the same GPU budget for all models.

**Keywords:** Tabular data generation, Generative Models, Evaluation Metrics, Deep Learning, Hyperparameter Tuning, Neural Architecture Search

## 1. Introduction

The capability to develop generative models that generate realistic, safe, and useful tabular data is crucial for industries where this type of data is most prevalent. Among the direct applications of tabular data generation we can cite data privacy, imputation, oversampling, explainability or simulation [1, 2]. Another significant application of generative models is their ability to learn data representations that are valuable for pre-training and
# Tabular Data Generation Models: An In-Depth Survey and Performance Benchmarks with Extensive Tuning

G. Charbel N. Kindji<sup>a,b,*</sup>, Lina M. Rojas-Barahona<sup>b</sup>, Elisa Fromont<sup>a</sup>, Tanguy Urvoy<sup>b</sup>,

<sup>a</sup>Univ Rennes, IUF, Inria, CNRS, IRISA, Rennes, 35000, France

<sup>b</sup>Orange Labs, Lannion, 22300, France

## Abstract

Generating realistic, safe, and useful tabular data is important for downstream tasks such as privacy preserving, imputation, oversampling, explainability, and simulation. However, the structure of tabular data, marked by heterogeneous types, non-smooth distributions, complex feature dependencies, and categorical imbalance, poses significant challenges. Although many generative approaches have been proposed, a fair and unified evaluation across datasets remains missing. This work benchmarks five recent model families on 16 diverse datasets (average 80K rows), with careful optimization of hyperparameters, feature encodings, and architectures. We show that dataset-specific tuning leads to substantial performance gains, particularly for diffusion-based models. We further introduce constrained hyperparameter spaces that retain competitive performance while significantly reducing tuning cost, enabling efficient model selection under fixed GPU budgets. As future perspectives, we can study cross-domain and cross-table generation.

**Keywords:** Tabular data generation, Generative Models, Evaluation Metrics, Deep Learning, Hyperparameter Tuning, Neural Architecture Search

## 1. Introduction

The demand for generative models is rapidly increasing. Particularly, generating realistic, safe, and useful tabular data is crucial for industries,

<footnote>*Corresponding author. Email: charbel.kindji.orange@gmail.com</footnote>
fine-tuning for various downstream tasks [3, 4, 5]. However, generating high-quality tabular data presents several specific technical challenges that are not encountered with text or images [6]. First, tabular columns are encoded through heterogeneous data types with distributions that are often non-smooth with mixed (continuous/discrete) behaviours and various modalities. There can also be complex dependencies between columns, and the categorical features are often highly imbalanced. Finally, the wide variety of problems represented in tabular form makes it challenging to establish a universal data encoding and architecture suitable for pre-training across all scenarios.

To handle these challenges, many models have been proposed in the literature, spanning over very diverse approaches that can be probabilistic, GAN-based, diffusion-based, or LLM-based. These models are often evaluated on different datasets with inconsistent metrics, tuning and training budgets. Yet, the performance of the allegedly best tabular generation models published seem very unstable from one dataset to another: these models seem quite sensitive to the feature-encodings and hyperparameter-choices made by the authors.

The main purpose of this work is to study the impact of dataset-specific preprocessing and hyperparameter tuning on tabular data generation models. For each model we want to answer the following questions: (i) is it worth optimizing the hyperparameters/preprocessing specifically for each dataset? (ii) can we propose a reduced search space that fits well for all datasets? (iii) is there a clear trade-off between training/sampling costs, and synthetic data quality? Another goal of this work is to address the practical need for a unified model evaluation that explicitly incorporates hyperparameter optimization

We hence benchmarked 5 model families that are representative of the recent literature on 16 datasets with a strict 3-fold cross-validation procedure. The datasets were chosen based on their size, purpose and diversity. For each fold and dataset, we optimized the model's hyperparameters, feature encoding, and architecture through hundreds of trials. We conducted two benchmarks: the first involved a large-scale, nearly unconstrained optimization, while the second employed a rapid optimization within a reduced search space, all within an equal compute budget. This work extends surveys like [2, 7] by covering recent diffusion-based models [8, 9] and by providing a larger scale benchmark.

As detailed in section 3.1, we consider multiple facets of the tabular data
wherein this type of data is most common. Among the direct applications of tabular data generation we can cite data privacy, imputation, oversampling, explainability or simulation [1, 2]. Because of their ability to learn valuable data representations, another significant application of these models is pre-training and fine-tuning for various downstream tasks [3, 4, 5]. However, generating high-quality tabular data presents several technical challenges that are not encountered with text or images [6]. First, tabular columns are encoded through heterogeneous data types with distributions that are often non-smooth with mixed (continuous/discrete) behaviours and various modalities. There can also be complex dependencies between columns, and the categorical features are often highly imbalanced. Finally, the wide variety of problems represented in tabular form makes it challenging to establish a universal data encoding and architecture suitable for pre-training across all scenarios.

To handle these challenges, many models have been proposed in the literature, which are often evaluated on different datasets with inconsistent metrics, tuning and training budgets. Indeed, the performance of the allegedly best tabular generation models seem very unstable from one dataset to another. Moreover, these models seem quite sensitive to the feature-encodings and hyperparameter-choices made by the authors.

In this work we propose a unified evaluation benchmark for tabular data generation. Our main contributions are:

* We present the state-of-the-art and propose the following typology of the existing models: *non-iterative* and *iterative* neural, which groups *auto-regressive* and *diffusion* models; as well as *non-neural* models.

* We study the impact of dataset-specific feature encoding, hyperparameter and architecture tuning on tabular data generation models. For each model we answer the following questions: (i) is it worth optimizing the hyperparameters/preprocessing specifically for each dataset? (ii) can we propose a reduced search space that fits well for all datasets? (iii) is there a clear trade-off between training/sampling costs, and synthetic data quality?

* We benchmark 5 models that are representative of the recent literature on 16 datasets with a strict 3-fold cross-validation procedure. The datasets were chosen based on their size, purpose and diversity.
generation problem. The key target of our study is the *realism of data* that we mainly assess through the *Classifier two-Sample Test (C2ST)*, a metric which quantifies the ability of a classifier to discriminate real from synthetic data [10, 11]. We also consider other metrics such as, the *Machine Learning Efficacy (ML-efficacy)* for utility, and the *Distance to Closest Record Rate (DCR-rate)* for anonymity. We also measure or estimate the cost and carbon footprint at three stages: training (i.e. gradient descent), tuning, and sampling.

In Section 4, we analyze the results of these benchmarks and derive some intriguing insights about the models. For example, our experiments indicate that while diffusion-based models like TabSyn and TabDDPM [9] generally outperform other models when left unconstrained, they do not significantly surpass their simpler counterparts when tuning and training budgets are limited. This is because models without Transformers have a smaller memory footprint, allowing for more thorough optimization within the same GPU budget.

In Section 2.1, we quickly review the state-of-the-art with a particular focus on the competitors that were compared in our benchmarks. This review categorizes approaches as non-neural (e.g. SMOTE [12]), non-iterative neural models [11] (e.g. VAE and GANS), and finally, the most recent iterative generative models: diffusion [9] (e.g. TabDDPM and TabSyn) and auto-regressive ones (e.g. GReat [13]). Section 3 presents the datasets and the evaluation metrics. Section 4 and Section 5 present respectively the large-scale and limited-budget benchmark results. We conclude in Section 6.

## 2. Benchmark Challengers

Tabular data generation is a booming research field which gives birth every month to a host of new data synthesis algorithms. In this section we quickly survey the existing families of tabular data generation methods with a specific focus on the models that we selected for our study. This includes models already covered in [2] and [7] as well as more recent diffusion-based models and LLM-based ones [14]. We chose models known for their strong performance, widespread usage, and availability of code that can be easily adapted for both architecture and hyperparameter tuning.
* We consider in our benchmark both extensive and limited-budget optimizations for each fold and dataset through hundreds of trials.

It is worth noting that this paper is not intended to be a survey. Rather, it benchmarks selected models for tabular data generation through *meticulous fine-grained tuning*. Previous surveys on tabular data generation are either focused on privacy [7, 8, 9, 10, 11, 12] or do not cover recent diffusion models [2, 13]. Most survey only report the author's evaluations with default hyperparameters. A few benchmarks like [14, 11] perform hyperparameter search, with simple train/validation/test split, reduced search space, and small number of trials (usually from 20 to 50). Instead, we provide large-scale extensive benchmarks. We select representative models of the literature and we evaluate them on distinct datasets in cross validation, in which for each fold and dataset, we optimized the model's hyperparameters, feature encoding, and architecture through hundreds of trials. Unlike previous work, our benchmarks compare not only the *realism*, the *utility* and the *anonymity*, but also the *costs* and the *carbon footprint*.

Finally, we analyze the results of these benchmarks and derive some interesting insights about the models. While diffusion-based models like TabSyn and TabDDPM [15] generally outperform other models when left unconstrained, they do not significantly surpass their simpler counterparts when tuning and training budgets are limited. This is because models that do not rely on Transformers have a smaller memory footprint, allowing for more thorough optimization within the same GPU budget.

The paper is structured as follows. First we present the models' typology in Section 2. The benchmark's challengers, metrics, datasets and other experimental settings are presented in section 3. The extensive and limited-budget benchmarks are detailed in Sections 4 and 5 respectively. Finally the conclusions is presented in Section 6 and the future work in Section 7.

## 2. Typology of Models for Tabular Data Generation

Tabular data generation is a booming research field which gives birth every month to a host of new data synthesis algorithms. In this section we survey the existing families of tabular data generation methods with a specific focus on the models that we selected for our study. This includes models already covered in [2] and [13] as well as more recent diffusion-based models and LLM-based ones [16]. We chose models known for their strong
## 2.1. Tabular Data Generation Model's Overview

Among the neural approaches for tabular data generation, we make a distinction between the “push-forward” models which directly map noise into data, and the iterative models which require a decoding phase. We also consider a few non-neural approaches as baselines.

### 2.1.1. Non-iterative Neural Approaches

The most popular non-iterative or “push-forward” neural networks for tabular data generation are Variational Auto-Encoders (VAE) [15] and Generative Adversarial Networks (GAN) [16]. A few papers also consider self-normalizing flows [17]. One of the simplest VAE architecture for tabular data is TVAE [18]. In its original implementation [19], it consists of a one-hot encoding for categorical variables coupled with a Gaussian Mixture Model normalization scheme (GMM) for continuous features. The encoder/decoder architecture is a simple stack of linear layers. Several variants have been proposed to improve from this baseline. In [20] the GMM normalization is replaced by a two-step training that first fits the marginals then fits the inter-dependencies. In [21, 11], several normalization schemes were tested as a replacement for the GMM normalization. Other variants of VAE use differentiable oblivious trees to ensure privacy [22]. In [23] the VAE is coupled with a Graph Neural Network (GNN). For our benchmark, we followed [11] and used a customized version of TVAE which allows for an optimized choice of the architecture and of the feature encoder.

The most popular method to generate tabular data is certainly adversarial training [24, 25, 26, 27, 28, 18, 29, 21, 30, 7, 31]. It would not be exaggerated to affirm that every exotic variant of GANs has been tested on tabular data generation, but the most successful architectures seem to be the ones based on Wasserstein GANs [32] such as CTGAN [18]. CTGAN is the base architecture that we selected for our benchmark. As mentioned in [21, 11], the feature encoding scheme is critical, especially for numerical features. For this reason we customized the CTGAN code as we did for TVAE, to allow for a choice of architecture and feature encoders.

## 2.2. Iterative models

It has been shown that iterative generative models (either auto-regressive or by diffusion) almost systematically outperform push forward models when it comes to raw text, sound, or image generation [33]. We confirm here that
performance, widespread usage, and availability of code that can be easily
adapted for both architecture and hyperparameter tuning.

The following neural and non-neural approaches have been proposed for tabular data generation. Among the neural approaches, we make a distinction between the “push-forward” models which directly map noise into data, and the iterative models which require a decoding phase.

## 2.1. Non-iterative Neural Models

The most popular non-iterative or “push-forward” neural networks for tabular data generation are Variational Auto-Encoders (VAE) [17] and Generative Adversarial Networks (GAN) [18]. A few papers also consider self-normalizing flows [19]. One of the simplest VAE architecture for tabular data is TVAE [20]. In its original implementation [21], it consists of a one-hot encoding for categorical variables coupled with a Gaussian Mixture Model normalization scheme (GMM) for continuous features. The encoder/decoder architecture is a simple stack of linear layers. Several variants have been proposed to improve from this baseline. In [22] the GMM normalization is replaced by a two-step training that first fits the marginals then fits the inter-dependencies. In [23, 24], several normalization schemes were tested as a replacement for the GMM normalization. Other variants of VAE use differentiable oblivious trees to ensure privacy [25]. In [26] the VAE is coupled with a Graph Neural Network (GNN).

The most popular method to generate tabular data is certainly adversarial training [27, 28, 29, 30, 31, 20, 32, 23, 33, 13, 34]. It would not be exaggerated to affirm that every exotic variant of GANs has been tested on tabular data generation, but the most successful architectures seem to be the ones based on Wasserstein GANs [35] such as CTGAN [20]. CTGAN is the base architecture that we selected for our benchmark. As mentioned in [23, 24], the feature encoding scheme is critical, especially for numerical features. For this reason we customized the CTGAN code as we did for TVAE, to allow for a choice of architecture and feature encoders.

## 2.2. Iterative Neural Models

It has been shown that iterative generative models (either auto-regressive or by diffusion) almost systematically outperform push forward models when it comes to raw text, sound, or image generation [36]. We confirm here that tabular data do not escape this rule. However, this performance comes with a cost: the decoding phase is often slow and highly energy consuming.
tabular data do not escape this rule. However, this performance comes with a cost: the decoding phase is often slow and highly energy consuming.

### 2.2.1. Auto-regressive Language Models

After the recent breakthrough of large language models (LLM) [34], the usage of token-based language models to generate tabular data seems inevitable. Their main advantage against prior *ad-hoc* models is that they come without specific feature encodings for numerical or categorical columns: these values are directly fed to the model as raw sequences of tokens. Even if there is no clear agreement yet on how tabular instances should be serialized for LLMs, this general encoding ability opens the possibility for a pretraining/finetuning paradigm on heterogeneous tabular datasets [3, 5].

In a recent preprint survey [14], the authors tried to map the exuberant flow of preprint papers on tabular data and LLMs. Several of these preprints present only prompt engineering tricks that generate small tables, nonetheless some of the proposed models seem promising at a larger scale. To name a few of them, GReaT [13] (for "Generation of Realistic Tabular data") proposes to fine-tune GPT-2 [35] for tabular data generation. RealTabFormer [36] extends GReaT to multiple tables with shared indexes, and TAPTAP [5] experiments a pretraining of GReaT on 450 open tabular datasets.

We made a few experiments with GReaT and its variants and confirmed the remarks of [9, 37], which state that they struggle to capture the joint probability distribution on datasets where the categorical values names do not carry semantic information. Given the prohibitive computational cost of LLMs, the large number of hyperparameters and serialization schemes to consider, as well as the problematic fact that some datasets are already covered (i.e. used in the training data) by the foundation models training sets, we decided to postpone their evaluation for a future work.

### 2.2.2. Diffusion models

Another recent impressing breakthrough in generative modeling, especially in image generation, was the apparition of diffusion models [38, 39, 40, 41, 42]. The transposition of these models to tabular data gave rise to powerful synthesizers: TableDiffusion [43], Stasy [44], CODi [45], and TabDDPM [8]. In TabSyn [9], a more recent proposal, the authors transposed the idea of [46, 47] to make use of a transformer-based VAE in order to embed the diffusion in a latent space. We selected both TabDDPM and TabSyn for our benchmark.
### 2.2.1. Auto-regressive Language Models

After the recent breakthrough of large language models (LLM) [37], the usage of token-based language models to generate tabular data seems inevitable. Their main advantage against prior *ad-hoc* models is that they come without specific feature encodings for numerical or categorical columns: these values are directly fed to the model as raw sequences of tokens. Even if there is no clear agreement yet on how tabular instances should be serialized for LLMs, this general encoding ability opens the possibility for a pretraining/finetuning paradigm on heterogeneous tabular datasets [3, 5].

In a recent preprint survey [16], the authors tried to map the exuberant flow of preprint papers on tabular data and LLMs. Several of these preprints present only prompt engineering tricks that generate small tables, nonetheless some of the proposed models seem promising at a larger scale. To name a few of them, GReaT [38] (for "Generation of Realistic Tabular data") proposes to fine-tune GPT-2 [39] for tabular data generation. REaLTabFormer [40] extends GReaT to multiple tables with shared indexes, and TAPTAP [5] experiments a pretraining of GReaT on 450 open tabular datasets.

We made a few experiments with GReaT and its variants and confirmed the remarks of [15, 11], which state that they struggle to capture the joint probability distribution on datasets where the categorical values names do not carry semantic information. Given the prohibitive computational cost of LLMs, the large number of hyperparameters and serialization schemes to consider, as well as the problematic fact that some datasets are already covered (i.e. used in the training data) by the foundation models training sets, we decided to postpone their evaluation for a future work.

### 2.2.2. Diffusion models

Another recent impressing breakthrough in generative modeling, especially in image generation, was the apparition of diffusion models [41, 42, 43, 44, 45]. These models learn to iteratively transform random Gaussian noise to a sample from an unknown data distribution. This transformation is defined through a forward and a backward diffusion processes. The first one iteratively adds Gaussian noise to samples, while the second one iteratively “denoises” samples from the Gaussian distribution to obtain samples from the data distribution. The transposition of these models to tabular data gave rise to powerful synthesizers: TableDiffusion [46], Stasy [47], CoDi [48], and TabDDPM [14]. In TabSyn [15], a more recent proposal, the authors transposed the idea of [49, 50] to make use of a transformer-based VAE in
### 2.2.3. Non-neural baselines

The most common statistical approaches are based on *copulas* and *Probabilistic Graphical Models* (PGMs). Copulas [48] are functions that join or “couple” multivariate distribution functions to their one-dimensional marginals. They have been widely adopted for tabular data generation because they allow modeling the marginals and the feature inter-dependencies separately [19, 49, 50, 51, 52]. Nevertheless, parametric copulas have been shown to perform poorly on high-dimension data synthesis problems [18, 11].

On the other hand PGMs can model variable dependencies in high-dimension spaces [53, 54, 7, 55, 56, 57]. This approach has been shown to be quite efficient in the data privacy community [58, 59, 60, 37]. However, it often requires a prior knowledge on the dependency graph because graph inference from data is inefficient in high dimension, especially if the sample size is small [61].

The fact that ensembles of trees remain state of the art for predictive tasks on tabular data [6] motivated some interesting attempts to mimic the neural generative approaches with decision trees. It gave rise to adversarial forests [31], Forest-Fow, and Forest-VP (a variance-preserving diffusion algorithm) [62]. We tested Forest-VP but we encountered a severe scalability issue: contrary to neural diffusion models where the noise level is combined with input as an auxiliary variable, the Forest-VP algorithm trains a different ensemble of trees for each level of noise.

Another simple, but quite efficient way to generate new tabular data is to interpolate between existing instances. This geometric “nearest neighbors” approach called *Synthetic Minority Over-sampling Technique* (SMOTE) [12, 63, 64] is frequently used to resample instances before training predictive models on unbalanced datasets. It proceeds by picking a random instance with a fixed target value and finding its *k* nearest neighbors. New data points are then generated by interpolation between these neighbors. Although very simple, this model is a solid baseline for tabular data generation as shown in [8]. As explained in section 3.2.1, we considered two variants: SMOTE and ucSMOTE (for unconditional SMOTE).

### 2.3. Selected challengers

We finally selected five families for our benchmark, four of them neural: TVAE, CTGAN, TabDDPM, and TabSyn and one non-neural: SMOTE (with its variant ucSMOTE). As developed in Section 3, for each of these algorithms we had to delve into the code in order to enable a large scale hyperparameters,
order to embed the diffusion in a latent space. We selected both TabDDPM and TabSyn for our benchmark.

## 2.3. Non-neural Models

The most common statistical approaches are based on *copulas* and *Probabilistic Graphical Models* (PGMs). Copulas [51] are functions that join or “couple” multivariate distribution functions to their one-dimensional marginals. They have been widely adopted for tabular data generation because they allow modeling the marginals and the feature inter-dependencies separately [21, 52, 53, 54, 55]. Nevertheless, parametric copulas have been shown to perform poorly on high-dimension data synthesis problems [20, 24].

On the other hand PGMs can model variable dependencies in high-dimension spaces [56, 57, 13, 58, 59, 60]. This approach has been shown to be quite efficient in the data privacy community [61, 9, 10, 11]. However, it often requires a prior knowledge on the dependency graph because graph inference from data is inefficient in high dimension, especially if the sample size is small [62].

The fact that ensembles of trees remain state of the art for predictive tasks on tabular data [6] motivated some interesting attempts to mimic the neural generative approaches with decision trees. It gave rise to adversarial forests [34], Forest-Flow, and Forest-VP (a variance-preserving diffusion algorithm) [63]. We tested Forest-VP but we encountered a severe scalability issue: contrary to neural diffusion models, introduced in Section 2.2.2, where the noise level is combined with the input as an auxiliary variable, the Forest-VP algorithm trains a different ensemble of trees for each level of noise.

Another simple, but quite efficient way to generate new tabular data is to interpolate between existing instances. This geometric “nearest neighbors” approach called *Synthetic Minority Over-sampling Technique* (SMOTE) [64, 65, 66] is frequently used to resample instances before training predictive models on unbalanced datasets. It proceeds by picking a random instance with a fixed target value and finding its *k* nearest neighbors. New data points are then generated by interpolation between these neighbors. Although very simple, this model is a solid baseline for tabular data generation as shown in [14].
features encoding, and architecture optimization. In order to calibrate the
metrics, we also reported the trivial baseline which consists in resampling
directly the train set. It appears as “*Train Copy*” in result tables.

**3. Experimental Settings**

In this section, we first present the various evaluation metrics that we
report from our experiments. We then present the optimization framework
that we developed and we discuss the choices made to wrap the different
challengers into this framework. We finally present the different datasets.

3.1. *Evaluation Metrics*

Our purpose is to assess the quality of tabular data generation though
multiple facets. These facets can be summarized with four questions: (i) is
synthetic data realistic? does it respect the original distribution’s traits?
(ii) is it useful? i.e. can it be used to train machine learning models? (iii) do
synthesis preserve training data anonymity? does it overfit? (iv) what are
the model’s costs and CO2 impacts? Each of these questions is related to
specific metrics.

*The first and most important question is the realism of the generated data.* We rely on *Classifier Two Sample Test*<fnref n="1" /> (C2ST) [10] as a primary metric to address it. This metric is also the one that we used for hyperparameter optimization. It evaluates the performance of a classifier at discerning real data from synthetic data². To compute C2ST we use the same protocol as in [11] where the computed value is the mean ROC-AUC of XGBoost [65] over three folds. A C2ST around 1/2 means that XGBoost is unable to discern the test set from the generated set. A high C2ST means on the contrary that XGBoost is able to detect easily the fake data. The C2ST calculation procedure is summarized in Figure 1.

We also consider two other statistical metrics for data realism: column-
wise similarity and pair-wise correlation. To compute these metrics we used
the SDMetrics library [19]. The column-wise similarity measures how ac-
curately the synthetic data captures the shape of each column distribution

<footnote n="1">¹Also mentioned as *Detection test* in SDMetrics https://docs.sdv.dev/sdmetrics

²This is similar to what is done in GANs but with a fresh dataset. An overfitted GAN
can indeed have a poor C2ST on test set.</footnote>
### 3. Benchmarks Settings

We first present the selected challengers and the various evaluation metrics. Then, we present the optimization framework that we developed and we discuss the choices made to wrap the different challengers into this framework. Finally, We present the datasets.

#### 3.1. Selected Challengers

We selected five models for our benchmarks, two iterative neural: TVAE, CTGAN; two diffusion models: TabDDPM, and TabSyn; and one non-neural: SMOTE (with a variant unconditional SMOTE, namely ucSMOTE). We followed [24] and used a customized version of TVAE which allows for an optimized choice of the architecture and of the feature encoder.

For each of these algorithms we had to carefully examine the code to optimize large scale hyperparameters, features encoding and architecture. We also reported the trivial baseline which consists in resampling directly the train set, namely “Train Copy”.

#### 3.2. Evaluation Metrics

Our purpose is to assess the quality of tabular data generation though multiple facets. These facets can be summarized with four questions: (i) is synthetic data realistic? does it respect the original distribution's traits? (ii) is it useful? i.e. can it be used to train machine learning models? (iii) do synthesis preserve training data anonymity? does it overfit? (iv) what are the model's costs and CO₂ impacts? Each of these questions is related to specific metrics.

*The first and most important question is the realism of the generated data.* We rely on *Classifier Two Sample Test*<fnref n="1" /> (C2ST) [67] as a primary metric to address it. This metric is also the one that we used for hyperparameter optimization. It evaluates the performance of a classifier at discerning real data from synthetic data². To compute C2ST we use the same protocol as in [24] where the computed value is the mean ROC-AUC of XGBoost [68] over three folds. A C2ST around 1/2 means that XGBoost is unable to discern

<footnote n="1">¹Also mentioned as *Detection test* in SDMetrics https://docs.sdv.dev/sdmetrics

²This is similar to what is done in GANs but with a fresh dataset. An overfitted GAN can indeed have a poor C2ST on test set.</footnote>
<figure><img src="image_4.png" /><figcaption>**Figure 1:** C2ST Metric calculation.</figcaption></figure>

individually. It is reported as "Shape" in the result tables. Pair-wise correlation on the other hand captures how each column varies with each other. Pair-wise correlation is reported as "Pair" in the result tables. A naive generator that would assume independence of the columns might have a high "Shape" score but it would have a low "Pair" score.

*The second important question is the utility of generated data.* It is commonly measured by ML-Efficacy which evaluates the performance of a predictive model trained on synthetic data. To compute ML-Efficacy we use the same protocol as in [8] where CatBoost [66] is used as a predictor. We report the F1 score for classification tasks and the normalized R2 score for regression tasks. The procedure is summarized in Figure 2. It is important to evaluate the degradation of these scores against the ones obtained when training directly on real data (via *Train Copy*): a large degradation means a low utility.

<figure><img src="image_5.png" /><figcaption>**Figure 2:** ML-Efficacy Metric calculation.</figcaption></figure>

*The third question is anonymity.* We want to make sure that our generative model will not leak sensitive information by recopying or over-fitting the training instances. To do so we measure respectively the minimum distances
the test set from the generated set. A high C2ST means on the contrary that XGBoost is able to detect easily the fake data. The C2ST calculation procedure is summarized in Figure 1.

<figure><img src="image_2.png" /><figcaption>**Figure 1:** C2ST Metric calculation.</figcaption></figure>

We also consider two other statistical metrics for data realism: column-wise similarity and pair-wise correlation. To compute these metrics we used the SDMetrics library [21]. The column-wise similarity measures how accurately the synthetic data captures the shape of each column distribution individually. It is reported as "Shape" in the result tables. Pair-wise correlation on the other hand captures how each column varies with each other. Pair-wise correlation is reported as "Pair" in the result tables. A naive generator that would assume independence of the columns might have a high "Shape" score but it would have a low "Pair" score.

*The second important question is the utility of generated data.* It is commonly measured by ML-Efficacy which evaluates the performance of a predictive model trained on synthetic data. To compute ML-Efficacy we use the same protocol as in [14] where CatBoost [69] is used as a predictor. We report the F1 score for classification tasks and the normalized R2 score for regression tasks. The procedure is summarized in Figure 2. It is important to evaluate the degradation of these scores against the ones obtained when training directly on real data (via *Train Copy*): a large degradation means a low utility.

*The third question is anonymity.* We want to make sure that our generative model will not leak sensitive information by recopying or over-fitting the training instances. To do so we measure respectively the minimum distances of each generated instance to the train and test sets. The *Distance to Closest*
of each generated instance to the train and test sets. The *Distance to Closest Record Rate (DCR-Rate)* counts the proportion of generated instances that are closer to train set than test set [67]. The procedure is summarized in Figure 3.

A synthetic dataset is considered safe if it has a DCR-Rate that is close to 1/2. On the other hand, a plain copy of the train set, as *Train Copy* baseline does, would have a DCR-Rate of 1. This metric does not guarantee against all privacy breaches, but it provides a reasonable safeguard and ranking criterion for the models. It is also informative about overfitting, as an overfitted model would have samples that are systematically closer to the train set than a model with more generalization capabilities.

<figure><img src="image_6.png" /><figcaption>**Figure 3: DCR-Rate Metric calculation.**</figcaption></figure>

*The fourth question is the models' costs and their carbon impacts.* It requires an accurate estimate of three cost values: *time*, *energy consumption*, and *CO₂ impact*. Ideally, these values should be estimated for the three phases of (i) training (gradient descent) (ii) sampling, and (iii) hyperparameters search. For each dataset, each fold and each optimized model architecture, we ran the training and sampling phases on the exact same hardware and software architecture, a single Tesla V100 32 GB, and we measured accurately the cost values with the *CodeCarbon* library<fnref n="3" />. However, due to the massive nature of the experiments, we could not perform the whole hyperparameter search and training phases on such a uniform hardware and software architecture. We hence estimated the global search costs from the tuning logs by rescaling the training cost measures according to the effective number of steps performed

<footnote n="3">³https://codecarbon.io/</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 2:** ML-Efficacy Metric calculation.</figcaption></figure>

Record Rate (DCR-Rate) counts the proportion of generated instances that are closer to train set than test set [70]. The procedure is summarized in Figure 3.

A synthetic dataset is considered safe if it has a DCR-Rate that is close to 1/2. On the other hand, a plain copy of the train set, as *Train Copy* baseline does, would have a DCR-Rate of 1. This metric does not guarantee against all privacy breaches, but it provides a reasonable safeguard and ranking criterion for the models. It is also informative about overfitting, as an overfitted model would have samples that are systematically closer to the train set than a model with more generalization capabilities.

<figure><img src="image_4.png" /><figcaption>**Figure 3:** DCR-Rate Metric calculation.</figcaption></figure>

*The fourth question is the models' costs and their carbon impacts.* It requires an accurate estimate of three cost values: *time*, *energy consumption*, and *CO₂ impact*. Ideally, these values should be estimated for the three phases of (i) training (gradient descent) (ii) sampling, and (iii) hyperparameters search. For each dataset, each fold and each optimized model architecture, we ran the training and sampling phases on the exact same hardware and software
<figure><img src="image_7.png" /><figcaption>Figure 4: Hyperparameters trial optimization loop.</figcaption></figure>

and the number of GPU used (c.f. equation (1)).

$$ \text{total-gpu-cost} \approx \sum_{t \in \text{trials}} \frac{\text{init-cost}_t + \text{avg-cost-per-step} \times \text{num-steps}_t}{\text{trials-per-gpu}} \quad (1) $$

This is slightly overestimated since: (i) CodeCarbon assumes a 100% GPUs load while ours was roughly around 95%; and (ii) we measured the step costs on already optimized models which are usually slower because they often count more layers and parameters. Note that the number of trials that can be parallelized on a single GPU depends on the memory footprint of the model<fnref n="4" />.

## 3.2. Large Scale Optimization Framework and Implementation details

Providing a fair and reliable comparison of the different tabular generative models is a tough technical challenge. For this reason most existing benchmarks like [13, 9] only report the performance of models with their default hyperparameters. A few benchmarks like [8, 37] perform hyperparameters search for all models, but with a simple train/validation/test split, a reduced search space, and a small number of trials (usually from 20 to 50).

We wanted our experiment to be more extensive and more robust, so we decided to deploy it at a large scale on a super-computer equipped with

<footnote n="4">⁴For instance we could safely run 10 TVAE trials on the same V100 GPU while only 4 TabSyn's trials could fit because of the VAE transformer's footprint.</footnote>
architecture, a single Tesla V100 32 GB, and we measured accurately the cost values with the CodeCarbon library<fnref n="3" />. However, due to the massive nature of the experiments, we could not perform the whole hyperparameter search and training phases on such a uniform hardware and software architecture. We hence estimated the global search costs from the tuning logs by rescaling the training cost measures according to the effective number of steps performed and the number of GPU used (c.f. equation (1)).

$$ \text{total-gpu-cost} \approx \sum_{t \in \text{trials}} \frac{\text{init-cost}_t + \text{avg-cost-per-step} \times \text{num-steps}_t}{\text{trials-per-gpu}} \quad (1) $$

This is slightly overestimated since: (i) CodeCarbon assumes a 100% GPUs load while ours was roughly around 95%; and (ii) we measured the step costs on already optimized models which are usually slower because they often count more layers and parameters. Note that the number of trials that can be parallelized on a single GPU depends on the memory footprint of the model<fnref n="4" />.

### 3.3. Large Scale Optimization Framework and Implementation Details

Providing a fair and reliable comparison of the different tabular generative models is a tough technical challenge. For this reason most existing benchmarks like [38, 15] only report the performance of models with their default hyperparameters. A few benchmarks like [14, 11] perform hyperparameters search for all models, but with a simple train/validation/test split, a reduced search space, and a small number of trials (usually from 20 to 50).

We wanted our experiment to be more extensive and more robust, so we decided to deploy it at a large scale on a super-computer<fnref n="5" /> equipped with several nodes with 4 GPUs V100 32GB each<fnref n="6" />.

For this purpose, we used the *Ray tune* distributed library [71] coupled with *Hyperopt* [72] which is based on Tree Parzen Estimators (TPE) [73] and Asynchronous Successive Halving Algorithm (ASHA) as a scheduler [74] to optimize hyperparameters and architecture efficiently. Depending on the

<footnote n="3">³https://codecarbon.io/</footnote>

<footnote n="4">⁴For instance we could safely run 10 TVAE trials on the same V100 GPU while only 4 TabSyn's trials could fit because of the VAE transformer's footprint.</footnote>

<footnote n="5">⁵Jean-Zay super-computer www.idris.fr</footnote>

<footnote n="6">⁶Complementary experiments were also performed on a 2xRTX4090 workstation.</footnote>
several nodes with 4 GPUs V100 32GB each<fnref n="5" />.

For this purpose, we used the *Ray tune* library [68] coupled with *Hyperopt* [69] and a median elimination scheduler [70] to optimize hyperparameters and architecture efficiently. Depending on the model’s memory footprints, each GPU could host from four to twelve concurrent tuning trials. For each dataset and each model we performed a strict 3-fold cross validation, which means that for each tuple (dataset, fold, model) we performed an extensive hyperparameters search with 300 trials (except for TabSyn where we reduced this number to 100 for technical reasons explained in Section 3.2.2). We hence obtained a different optimized architecture for each (dataset, fold, model) tuple.

In [8] the parameters were optimized for ML-efficacy, in [37] the parame-
ters were optimized for an equal combination of realism, utility and privacy.
We chose to optimize for realism as in [11] through XGBoost-based C2ST
metric (c.f. Section 3.1). To obtain reliable evaluations with variance esti-
mates, for each dataset we evaluated the models by averaging all the metrics
on the three folds test sets with five synthetic samples for each fold.

As described in Figure 4, in order to work with our framework each algo-
rithm has to be wrapped into a generic *Synthesizer* class that provides three
methods: *prepare-fit* which prepares the dataset and the model according
to the hyperparameters, *train-step* which performs a training step roughly
equivalent to one or a few epochs, and *sample* which generates synthetic data.
After each train step, the model trial was evaluated and it was canceled out
by early stopping or by the Ray-Tune scheduler if it performed too poorly or
if the time budget was depleted.

3.2.1. *The importance of feature encoding*

Table 1 presents the encoding schemes that we used for the different
benchmark challengers. As pointed out in [6, 21, 11], categorical variables
are not the main weakness of neural networks but numerical feature encoding
is critical. Most recent neural models use a Quantile-based numerical feature
encoding and seem to work well with it. However, the original versions of
TVAE and CTGAN rely on a specific Cluster-Based normalization [18]. We
hence explored several encoding policies for these two models through hyper-

<footnote n="5">⁵Complementary experiments were also performed on a workstation with two RTX 4090.</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 4: Hyperparameters trial optimization loop.</figcaption></figure>

model's memory footprints, each GPU could host from four to twelve con-
current tuning trials. For each dataset and each model we performed a strict
3-fold cross validation, which means that for each tuple (dataset, fold, model)
we performed an extensive hyperparameters search with 300 trials (except for
TabSyn where we reduced this number to 100 for technical reasons explained
in Section 3.3.2). We hence obtained a different optimized architecture for
each (dataset, fold, model) tuple.

In [14] the parameters were optimized for ML-efficacy, in [11] the param-
eters were optimized for an equal combination of realism, utility and privacy.
We chose to optimize for realism as in [24] through XGBoost-based C2ST
metric (c.f. Section 3.2). To obtain reliable evaluations with variance esti-
mates, for each dataset we evaluated the models by averaging all the metrics
on the three folds test sets with five synthetic samples for each fold.

As described in Figure 4, in order to work with our framework each algo-
rithm has to be wrapped into a generic *Synthesizer* class that provides three
methods: *prepare_fit* which prepares the dataset and the model according
to the hyperparameters, *train_step* which performs a training step roughly
equivalent to one or a few epochs, and *sample* which generates synthetic data.
parameters optimization (see Tables A.6 and A.7 in the appendix section).

<table><caption>**Table 1:** Encoding schemes applied to the benchmark challengers during data preprocessing. We experimented with various encoding schemes on TVAE and CTGAN during hyperparameter search.</caption><thead><tr><th>Model</th><th>Num. Encoder</th><th>Cat. Encoder</th><th>Num. Target</th></tr></thead><tbody><tr><td>TVAE_base</td><td>Cluster-Based<sup>6</sup></td><td>One hot</td><td>-</td></tr><tr><td>CTGAN_base</td><td>Cluster-Based<sup>6</sup></td><td>One hot</td><td>-</td></tr><tr><td>TVAE</td><td><i>Optimized</i></td><td>One hot</td><td>-</td></tr><tr><td>CTGAN</td><td><i>Optimized</i></td><td>One hot</td><td>-</td></tr><tr><td>TabDDPM</td><td>Quantile</td><td>One Hot</td><td>Standardize</td></tr><tr><td>TabSyn</td><td>Quantile</td><td>Embedding</td><td>-</td></tr><tr><td>SMOTE</td><td>-</td><td>One hot</td><td>Median cut</td></tr><tr><td>ucSMOTE</td><td>-</td><td>One hot</td><td>Dummy</td></tr></tbody></table>

We kept the native Cluster-Based<fnref n="6" /> encoder as described in [18], along with the ones proposed in [11], namely: prototype encoding (PTP) [11], piece-wise linear encoder (PLE) [71], continuously distributed residuals (CDF) [72, 11], hybrid (PLE_CDF) [11]. We also added some standard *scikit-learn* transformers: MinMaxScaler and QuantileTransformer.

Contrary to QuantileTransformer which maps values deterministically, the CDF encoding uses randomization to produce continuously distributed residuals even when the original distribution is discrete or partially discrete [72]. The PLE encoder [71] performs a feature binning and normalizes each numerical value depending on the bin it belongs to. The PLE_CDF applies a CDF to the output of a PLE encoding. The PTP encoding, inspired by prototypical networks [73], encodes the input as a weighted average of fixed prototypes.

### 3.2.2. Model-specific implementation details

Wrapping heterogeneous models within a *Synthesizer* class required some implementation choices from our part, and despite all our efforts to keep a fair comparison, these choices had some impact on the compute time and performance of the models.

<footnote n="6">⁶https://docs.sdv.dev/rdt/transformers-glossary/numerical/clusterbasednormalizer</footnote>
After each train step, the model trial was evaluated and it was canceled out by early stopping or by the Ray-Tune scheduler if it performed too poorly or if the time budget was depleted.

### 3.3.1. The importance of feature encoding

Table 1 presents the encoding schemes that we used for the different benchmark challengers. As pointed out in [6, 23, 24], categorical variables are not the main weakness of neural networks but numerical feature encoding is critical. Most recent neural models use a Quantile-based numerical feature encoding and seem to work well with it. However, the original versions of TVAE and CTGAN rely on a specific Cluster-Based normalization [20]. We hence explored several encoding policies for these two models through hyper-parameters optimization (see Tables A.6 and A.7 in the appendix section).

<table><caption>**Table 1:** Encoding schemes applied to TVAE and CTGAN.</caption><thead><tr><th>Model</th><th>Num. Encoder</th><th>Cat. Encoder</th><th>Num. Target</th></tr></thead><tbody><tr><td>TVAE_base</td><td>Cluster-Based<sup>7</sup></td><td>One hot</td><td>-</td></tr><tr><td>CTGAN_base</td><td>Cluster-Based<sup>7</sup></td><td>One hot</td><td>-</td></tr><tr><td>TVAE</td><td><i>Optimized</i></td><td>One hot</td><td>-</td></tr><tr><td>CTGAN</td><td><i>Optimized</i></td><td>One hot</td><td>-</td></tr><tr><td>TabDDPM</td><td>Quantile</td><td>One Hot</td><td>Standardize</td></tr><tr><td>TabSyn</td><td>Quantile</td><td>Embedding</td><td>-</td></tr><tr><td>SMOTE</td><td>-</td><td>One hot</td><td>Median cut</td></tr><tr><td>ucSMOTE</td><td>-</td><td>One hot</td><td>Dummy</td></tr></tbody></table>

We kept the native Cluster-Based<fnref n="7" /> encoder as described in [20], along with the ones proposed in [24], namely: prototype encoding (PTP) [24], piece-wise linear encoder (PLE) [75], continuously distributed residuals (CDF) [76, 24], hybrid (PLE_CDF) [24]. We also added some standard *scikit-learn* transformers: MinMaxScaler and QuantileTransformer.

Contrary to QuantileTransformer which maps values deterministically, the CDF encoding uses randomization to produce continuously distributed residuals even when the original distribution is discrete or partially discrete [76]. The PLE encoder [75] performs a feature binning and normalizes each

<footnote n="7">⁷https://docs.sdv.dev/rdt/transformers-glossary/numerical/clusterbasednormalizer</footnote>
The first issue is the discrepancy of the training step's costs. The usual training step unit for most models is the *epoch* which corresponds to a single pass on all instances of the training set. However, depending on the hyperparameters some models like CTGAN perform both one pass through the generator and several passes through the discriminator at each training step. Other models such as TabDDPM are randomized and require several quick passes (almost one for each level of noise) for each instance. We hence had to caliper our wrapper's train step functions to perform a compute effort that is roughly equivalent to an epoch.

Another issue is the fact that TabSyn [9] combines two models and was not designed for hyperparameters tuning. According to the authors the model does not need hyperparameters tuning<sup>7</sup>. We decided to train a new transformer-based VAE for each trial because it hosts most of the parameters, compute-time and hyperparameters of TabSyn. Training a diffusion model on an unstable latent space would not be meaningful. We thus considered three technical options: (i) optimize first the VAE on a proxy metric (for instance the ability of its decoder to generate realistic data from a standard Gaussian), then optimize the denoiser in the latent space; (ii) wrap the VAE training steps into the *train step* function and retrain a new denoiser from scratch at each step; (iii) wrap the VAE training phase into the *prepare fit* function and loose the ability to prune its training steps. The first option is the cheapest and it is probably recommended for most practical applications, but it may be sub-optimal due to the proxy metric. The second option is extremely costly. We hence opted for the third option although it had a non-negligible cost. Indeed, we followed the recommendation of [9] to train the VAE through 4000 epochs which turns out to be huge knowing that most other models only utilized 400 epochs in our benchmark.

We also reduced the number of parallel trials per GPU because of the large memory footprint of the VAE's transformers. As a consequence, for TabSyn we had both to reduce the number of trials to 100, and to work on a sample of the largest dataset (*Covertype*) to get the results in a reasonable amount of time. It is worth noting that despite these handicaps, the optimized TabSyn remained better than its non-optimized version. To avoid this experimental

<footnote n="7"><sup>7</sup>Our experiment show however (c.f. Section 4), that even if the non-optimized TabSyn is quite good on most datasets, the hyperparameter tuning clearly improves the quality of the data it generates.</footnote>
bias for the second experiment in Section 5.2, we constrained TabSyn's VAE to use only 10 minutes for training. On *Adult* dataset, for instance, it resulted in roughly 560 epochs.

Finally, the last issue was the different ways the models deal with the target columns. A model conditioned on the target column may improve its ML-efficacy. TabDDPM and SMOTE implementations are natively conditioned on classification targets, while TVAE, CTGAN and TabSyn are not.

We did not modify TabDDPM, but we considered two variants in our experiment for SMOTE: the first, that we call SMOTE, follows the design of [8], it uses the train target distribution for classification datasets and a rough median split for regression targets (see Table 1). The second, that we call ucSMOTE (for unconditional SMOTE), adds a dummy target filled with zeros to the data before calling the SMOTE oversampling library. By doing so, all columns, including the original target, are considered equally.

### 3.3. Datasets

To evaluate the models, we picked datasets with various characteristics to assess their performances under different scenarios. The experiments were done with a 3-fold cross-validation procedure. Datasets were chosen in order to cover various sizes and dimensions, different types of tasks (regression, binary, and multi-class classification), various types of features (numerical, categorical, or mixed). We also added Moons a well known scikitlearn synthetic dataset. The complete list of datasets and their characteristics is presented in Table 2.

<footnote n="8">⁸https://www.openml.org</footnote>

<footnote n="9">⁹https://www.kaggle.com/datasets</footnote>
non-negligible cost. Indeed, we followed the recommendation of [15] to train
the VAE through 4000 epochs which turns out to be huge knowing that most
other models only utilized 400 epochs in our benchmark.

We also reduced the number of parallel trials per GPU because of the large
memory footprint of the VAE's transformers. As a consequence, for TabSyn
we had both to reduce the number of trials to 100, and to work on a sample
of the largest dataset (*Covertype*) to get the results in a reasonable amount of
time. It is worth noting that despite these handicaps, the optimized TabSyn
remained better than its non-optimized version. To avoid this experimental
bias for the second experiment in Section 5.2, we constrained TabSyn's VAE
to use only 10 minutes for training. On *Adult* dataset, for instance, it resulted
in roughly 560 epochs.

Finally, the last issue was the different ways the models deal with the
target columns. A model conditioned on the target column may improve its
ML-efficacy. TabDDPM and SMOTE implementations are natively condi-
tioned on classification targets, while TVAE, CTGAN and TabSyn are not.

We did not modify TabDDPM, but we considered two variants in our
experiment for SMOTE: the first, that we call SMOTE, follows the design
of [14], it uses the train target distribution for classification datasets and a
rough median split for regression targets (see Table 1). The second, that we
call ucSMOTE (for unconditional SMOTE), adds a dummy target filled with
zeros to the data before calling the SMOTE oversampling library. By doing
so, all columns, including the original target, are considered equally.

3.4. *Datasets*

To evaluate the models, we picked datasets with various characteristics
to assess their performances under different scenarios. Datasets were chosen
to cover various sizes, dimensions, different types of tasks (regression, binary,
and multi-class classification) and various types of features (numerical, cate-
gorical, or mixed). We also added *Moons* a well known *scikitlearn* synthetic
dataset. The complete list of datasets and their characteristics is presented
in Table 2. The *Covertype* dataset size was reduced for TabSyn tuning as
follows: 27500 in the training set, 18333 in the validation set, and 9167 in
the test set.

<footnote n="9">⁹https://www.openml.org</footnote>

<footnote n="10">¹⁰https://www.kaggle.com/datasets</footnote>
<table><caption>**Table 2:** List of datasets. Direct links to exact versions of datasets used can be found in Appendix B. The *Covertype* dataset size was reduced for TabSyn tuning as follows: 27500 in the training set, 18333 in the validation set, and 9167 in the test set.</caption><thead><tr><th>Name</th><th>Train</th><th>Validation</th><th>Test</th><th>Num</th><th>Categ.</th><th>Task</th></tr></thead><tbody><tr><td>Abalone<sup>8</sup></td><td>2088</td><td>696</td><td>1393</td><td>7</td><td>2</td><td>Binclass</td></tr><tr><td>Adult<sup>8</sup></td><td>24420</td><td>8141</td><td>16281</td><td>6</td><td>9</td><td>Binclass</td></tr><tr><td>Bank Marketing<sup>8</sup></td><td>22605</td><td>7535</td><td>15071</td><td>7</td><td>10</td><td>Regression</td></tr><tr><td>Black Friday<sup>8</sup></td><td>83410</td><td>27804</td><td>55607</td><td>6</td><td>4</td><td>Regression</td></tr><tr><td>Bike Sharing<sup>8</sup></td><td>8689</td><td>2897</td><td>5793</td><td>9</td><td>4</td><td>Regression</td></tr><tr><td>Covertype<sup>8</sup></td><td>290505</td><td>96836</td><td>193671</td><td>10</td><td>45</td><td>Multiclass</td></tr><tr><td>Cardio<sup>9</sup></td><td>34999</td><td>11667</td><td>23334</td><td>11</td><td>1</td><td>Binclass</td></tr><tr><td>Churn Modelling<sup>9</sup></td><td>4999</td><td>1667</td><td>3334</td><td>8</td><td>4</td><td>Binclass</td></tr><tr><td>Diamonds<sup>8</sup></td><td>26970</td><td>8990</td><td>17980</td><td>7</td><td>3</td><td>Regression</td></tr><tr><td>HELOC<sup>9</sup></td><td>5229</td><td>1743</td><td>3487</td><td>23</td><td>1</td><td>Binclass</td></tr><tr><td>Higgs<sup>8</sup></td><td>49024</td><td>16342</td><td>32684</td><td>28</td><td>1</td><td>Binclass</td></tr><tr><td>House 16H<sup>8</sup></td><td>11391</td><td>3798</td><td>7595</td><td>17</td><td>0</td><td>Regression</td></tr><tr><td>Insurance<sup>9</sup></td><td>669</td><td>223</td><td>446</td><td>4</td><td>3</td><td>Regression</td></tr><tr><td>King<sup>9</sup></td><td>10806</td><td>3602</td><td>7205</td><td>19</td><td>1</td><td>Regression</td></tr><tr><td>MiniBooNE<sup>8</sup></td><td>65031</td><td>21678</td><td>43355</td><td>50</td><td>1</td><td>Binclass</td></tr><tr><td>Two Moons</td><td>19999</td><td>6667</td><td>13334</td><td>2</td><td>1</td><td>Binclass</td></tr></tbody></table>

# 4. Extensive Experiment Results

As mentioned in Section 3.1, the evaluation and comparison of tabular generative models is based mainly on four criteria: realism, usefulness, anonymity, and cost. After a global multi-criteria overview of the results, we study and compare the model's behaviour according to each criterion individually.

## 4.1. *Multi-criteria Overview*

Figure 5 shows the average ranking over all datasets and folds of seven model variants over eight metrics (c.f. Section 3.1). To complete these rankings, Table 3 and Table 4 summarize respectively the quality metric and cost distributions among all datasets and folds. Recall that the best scores for the C2ST are around 0.50 (as it means poor AUC for the classifier at telling synthetic data apart from holdout data).

Overall, no model provides the best performance over all considered criteria. We observe, as expected, a strong correlation between energy and
<table><caption>**Table 2:** List of datasets. Direct links to exact versions of datasets used can be found in Appendix B.</caption><thead><tr><th>Name</th><th>Train</th><th>Validation</th><th>Test</th><th>Num</th><th>Categ.</th><th>Task</th></tr></thead><tbody><tr><td>Abalone<sup>9</sup></td><td>2088</td><td>696</td><td>1393</td><td>7</td><td>2</td><td>Binclass</td></tr><tr><td>Adult<sup>9</sup></td><td>24420</td><td>8141</td><td>16281</td><td>6</td><td>9</td><td>Binclass</td></tr><tr><td>Bank Marketing<sup>9</sup></td><td>22605</td><td>7535</td><td>15071</td><td>7</td><td>10</td><td>Regression</td></tr><tr><td>Black Friday<sup>9</sup></td><td>83410</td><td>27804</td><td>55607</td><td>6</td><td>4</td><td>Regression</td></tr><tr><td>Bike Sharing<sup>9</sup></td><td>8689</td><td>2897</td><td>5793</td><td>9</td><td>4</td><td>Regression</td></tr><tr><td>Covertype<sup>9</sup></td><td>290505</td><td>96836</td><td>193671</td><td>10</td><td>45</td><td>Multiclass</td></tr><tr><td>Cardio<sup>10</sup></td><td>34999</td><td>11667</td><td>23334</td><td>11</td><td>1</td><td>Binclass</td></tr><tr><td>Churn Modelling<sup>10</sup></td><td>4999</td><td>1667</td><td>3334</td><td>8</td><td>4</td><td>Binclass</td></tr><tr><td>Diamonds<sup>9</sup></td><td>26970</td><td>8990</td><td>17980</td><td>7</td><td>3</td><td>Regression</td></tr><tr><td>HELOC<sup>10</sup></td><td>5229</td><td>1743</td><td>3487</td><td>23</td><td>1</td><td>Binclass</td></tr><tr><td>Higgs<sup>9</sup></td><td>49024</td><td>16342</td><td>32684</td><td>28</td><td>1</td><td>Binclass</td></tr><tr><td>House 16H<sup>9</sup></td><td>11391</td><td>3798</td><td>7595</td><td>17</td><td>0</td><td>Regression</td></tr><tr><td>Insurance<sup>10</sup></td><td>669</td><td>223</td><td>446</td><td>4</td><td>3</td><td>Regression</td></tr><tr><td>King<sup>10</sup></td><td>10806</td><td>3602</td><td>7205</td><td>19</td><td>1</td><td>Regression</td></tr><tr><td>MiniBooNE<sup>9</sup></td><td>65031</td><td>21678</td><td>43355</td><td>50</td><td>1</td><td>Binclass</td></tr><tr><td>Two Moons</td><td>19999</td><td>6667</td><td>13334</td><td>2</td><td>1</td><td>Binclass</td></tr></tbody></table>

## 4. Extensive Benchmark

As mentioned in Section 3.2, the evaluation and comparison of tabular generative models is based mainly on four criteria: realism, usefulness, anonymity, and cost. After a global multi-criteria overview of the results, we study and compare the model's behaviour according to each criterion individually.

### 4.1. Multi-criteria Overview

Figure 5 shows the average ranking over all datasets and folds of seven model variants over eight metrics (c.f. Section 3.2). To complete these rankings, Table 3 and Table 4 summarize respectively the quality metric and cost distributions among all datasets and folds. Recall that the best scores for the C2ST are around 0.50 (as it means poor AUC for the classifier at telling synthetic data apart from holdout data).

Overall, no model provides the best performance over all considered criteria. We observe, as expected, a strong correlation between energy and GPU time as well as a strong correlation among “quality metrics” (i.e. C2ST,
<figure><img src="image_8.png" /><figcaption>**Figure 5:** Radar chart of the **extensive experiment** showing optimized model's average ranking on all datasets according to C2ST, DCR-Rate, *ML-Efficacy*, column-wise similarity ('Shape'), pair-wise correlation ('Pair'), training and sampling time and energy costs. For all models except "TabSyn_base", which refers to the model using its default hyperparameters, the training costs include both tuning and gradient descent.</figcaption></figure>

GPU time as well as a strong correlation among “quality metrics” (*i.e.* C2ST, *Shape*, and *Pair*). On the one hand, the diffusion models achieve the best performance in terms of quality metrics, especially the tuned version of *TabSyn*. As shown in Table 3, this model has a median C2ST value of 0.64. However, *TabSyn* is also one of the most expensive models in terms of training costs (*Train-Energy* as well as *Train and Sample Times*), as it requires to train both a transformer-based *VAE* and a denoiser model for each dataset.

On the other hand, the *SMOTE* baselines obtain the poorest quality and privacy ranking with a high DCR-Rate. The median DCR-Rate score for *SMOTE* and *ucSMOTE* is at 0.97 which means that most of the samples from these models are very similar to the training set. However, they achieve strong utility in terms of *ML-Efficacy* with quartiles very close to *Train Copy*.
<figure><img src="image_6.png" /><figcaption>**Figure 5:** Radar chart of the **extensive experiment** showing optimized model's average ranking on all datasets across various metrics. The training costs of optimized models include both tuning and gradient descent.</figcaption></figure>

*Shape*, and *Pair*). On the one hand, the diffusion models achieve the best performance in terms of quality metrics, especially the tuned version of Tab-Syn. As shown in Table 3, this model has a median C2ST value of 0.64. However, TabSyn is also one of the most expensive models in terms of training costs (*Train-Energy* as well as *Train and Sample Times*), as it requires to train both a transformer-based VAE and a denoiser model for each dataset.

On the other hand, the SMOTE baselines obtain the poorest quality and privacy ranking with a high DCR-Rate. The median DCR-Rate score for SMOTE and ucSMOTE is at 0.97 which means that most of the samples from these models are very similar to the training set. However, they achieve strong utility in terms of *ML-Efficacy* with quartiles very close to *Train Copy* and the same median value of 0.73. As expected for neighborhood-based algorithms, their training cost is negligible, but their deployment requires
and the same median value of 0.73. As expected for neighborhood-based
algorithms, their training cost is negligible, but their deployment requires a
neighborhood search which can be costly on large datasets. Finally the tuned
neural push-forward models (CTGAN and TVAE) achieve mitigated results in
term of both quality and utility but their deployment is clearly the cheapest.
We also note that all neural models achieve reasonable results in term of
privacy preservation. Although TabDDPM is clearly the slowest algorithm for
deployment, it is fast at training and it obtains homogeneous results over all
other metrics. We note that the tuned version of TabDDPM is performing
better than the base version of TabSyn.

For this extensive experiment we only limited mildly the time budget
and the number of epochs. However, some models like TabSyn and CTGAN
consumed much more GPU time than others, especially TabDDPM which was
very quick at performing an equivalent number of epochs (c.f. Section 3.2.2).
It is also important to compare these algorithms with a fair allocation of GPU
resource as we do in Section 5.

## 4.2. Detailed Analysis

In this section we study and compare the model's behaviour according
to each criterion taken individually. For each dataset we computed the per-
formance metrics over 3 folds and 5 synthetic samples per fold to provide a
stable central tendency and dispersion estimate. The full dataset-level results
are provided in Appendix Table C.12. We summarize these results among all
datasets and folds in Table 3 and Table 4. For quality metrics we compare
the models through critical difference diagrams.

### 4.2.1. Are synthetic data realistic ?

We show in Figure 6 the *critical difference diagrams* [74] of all tuned
models respectively for C2ST, pair-wise correlation and column-wise similar-
ity. These diagrams were obtained by aggregating the ranks of the seven
models over all datasets and folds. A thick horizontal line groups the set
of models for which the pairwise “*no significant difference*” test hypothesis
could not be rejected.

If we except the trivial *Train Copy* policy which is by construction the
most realistic generator, we note that TabSyn is significantly better in terms
of C2ST than all other models except TabDDPM. On the other side, the
SMOTE baselines are significantly worse than TabDDPM and TabSyn. The
absolute C2ST values in Table 3 corroborate these ranking results with three
a neighborhood search which can be costly on large datasets. Finally the tuned neural push-forward models (CTGAN and TVAE) achieve mitigated results in terms of both quality and utility but their deployment is clearly the cheapest. We also note that all neural models achieve reasonable results in terms of privacy preservation. Although TabDDPM is clearly the slowest algorithm for deployment, it is fast at training and it obtains homogeneous results over all other metrics. We note that the tuned version of TabDDPM is performing better than the base version of TabSyn.

For this extensive experiment we only limited mildly the time budget and the number of epochs. However, some models like TabSyn and CTGAN consumed much more GPU time than others, especially TabDDPM which was very quick at performing an equivalent number of epochs (c.f. Section 3.3.2). It is also important to compare these algorithms with a fair allocation of GPU resource as we do in Section 5.

## 4.2. Detailed Analysis

In this section we study and compare the model's behaviour according to each criterion taken individually. For each dataset we computed the performance metrics over 3 folds and 5 synthetic samples per fold to provide a stable central tendency and dispersion estimate. The full dataset-level results are provided in Appendix Table C.12. We summarize these results among all datasets and folds in Table 3 and Table 4. For quality metrics we compare the models through critical difference diagrams.

### 4.2.1. Are synthetic data realistic ?

We show in Figure 6 the *critical difference diagrams* [78] of all tuned models respectively for C2ST, pair-wise correlation and column-wise similarity. These diagrams were obtained by aggregating the ranks of the seven models over all datasets and folds. A thick horizontal line groups the set of models for which the pairwise “*no significant difference*” test hypothesis could not be rejected.

If we except the trivial *Train Copy* policy which is by construction the most realistic generator, we note that TabSyn is significantly better in terms of C2ST than all other models except TabDDPM. On the other side, the SMOTE baselines are significantly worse than TabDDPM and TabSyn. The absolute C2ST values in Table 3 corroborate these ranking results with three *sets* of models: (i) diffusion-based models TabSyn and TabDDPM with a C2ST lower than 0.80 on most datasets and median around 0.65;
<table><caption>**Table 3:** Summary results from the *extensive* experiment and the *base* models (using default hyperparameters). We provide performance dispersion at the 25th, 50th, and 75th percentiles over all datasets and folds. To save space, we respectively shortened DCR-rate to DCR-R and ML-efficacy to ML-EF.</caption><thead><tr><th>Model</th><th>Percentiles</th><th>C2ST ↓</th><th>DCR-R ↓</th><th>ML-EF ↑</th><th>Shape ↑</th><th>Pair ↑</th></tr></thead><tbody><tr><td rowspan="3">Train Copy</td><td>75%</td><td>0.50</td><td>1.00</td><td>0.90</td><td>0.99</td><td>0.99</td></tr><tr><td>50%</td><td>0.50</td><td>1.00</td><td>0.73</td><td>0.99</td><td>0.98</td></tr><tr><td>25%</td><td>0.50</td><td>1.00</td><td>0.63</td><td>0.99</td><td>0.91</td></tr><tr><td rowspan="3">TVAE</td><td>75%</td><td>0.88</td><td>0.65</td><td>0.79</td><td>0.96</td><td>0.95</td></tr><tr><td>50%</td><td>0.81</td><td>0.63</td><td>0.71</td><td>0.95</td><td>0.92</td></tr><tr><td>25%</td><td>0.74</td><td>0.61</td><td>0.50</td><td>0.94</td><td>0.85</td></tr><tr><td rowspan="3">TVAE_base</td><td>75%</td><td>1.00</td><td>0.62</td><td>0.70</td><td>0.92</td><td>0.90</td></tr><tr><td>50%</td><td>0.98</td><td>0.61</td><td>0.65</td><td>0.91</td><td>0.80</td></tr><tr><td>25%</td><td>0.96</td><td>0.60</td><td>0.44</td><td>0.87</td><td>0.74</td></tr><tr><td rowspan="3">CTGAN</td><td>75%</td><td>0.90</td><td>0.64</td><td>0.71</td><td>0.98</td><td>0.96</td></tr><tr><td>50%</td><td>0.83</td><td>0.63</td><td>0.69</td><td>0.97</td><td>0.91</td></tr><tr><td>25%</td><td>0.71</td><td>0.61</td><td>0.47</td><td>0.96</td><td>0.85</td></tr><tr><td rowspan="3">CTGAN_base</td><td>75%</td><td>1.00</td><td>0.62</td><td>0.63</td><td>0.92</td><td>0.93</td></tr><tr><td>50%</td><td>0.99</td><td>0.60</td><td>0.47</td><td>0.88</td><td>0.83</td></tr><tr><td>25%</td><td>0.93</td><td>0.60</td><td>0.30</td><td>0.87</td><td>0.76</td></tr><tr><td rowspan="3">TabDDPM</td><td>75%</td><td>0.78</td><td>0.64</td><td>0.80</td><td>0.98</td><td>0.98</td></tr><tr><td>50%</td><td>0.67</td><td>0.62</td><td>0.69</td><td>0.98</td><td>0.93</td></tr><tr><td>25%</td><td>0.63</td><td>0.61</td><td>0.58</td><td>0.95</td><td>0.84</td></tr><tr><td rowspan="3">TabDDPM_base</td><td>75%</td><td>0.86</td><td>0.64</td><td>0.75</td><td>0.99</td><td>0.96</td></tr><tr><td>50%</td><td>0.77</td><td>0.62</td><td>0.68</td><td>0.98</td><td>0.92</td></tr><tr><td>25%</td><td>0.65</td><td>0.61</td><td>0.44</td><td>0.94</td><td>0.74</td></tr><tr><td rowspan="3">TabSyn</td><td>75%</td><td>0.80</td><td>0.63</td><td>0.76</td><td>0.99</td><td>0.97</td></tr><tr><td>50%</td><td>0.64</td><td>0.62</td><td>0.68</td><td>0.97</td><td>0.93</td></tr><tr><td>25%</td><td>0.59</td><td>0.61</td><td>0.46</td><td>0.96</td><td>0.74</td></tr><tr><td rowspan="3">TabSyn_base</td><td>75%</td><td>0.86</td><td>0.64</td><td>0.75</td><td>0.98</td><td>0.97</td></tr><tr><td>50%</td><td>0.71</td><td>0.62</td><td>0.57</td><td>0.97</td><td>0.95</td></tr><tr><td>25%</td><td>0.63</td><td>0.61</td><td>0.29</td><td>0.95</td><td>0.88</td></tr><tr><td rowspan="3">SMOTE</td><td>75%</td><td>0.97</td><td>0.98</td><td>0.86</td><td>0.97</td><td>0.99</td></tr><tr><td>50%</td><td>0.90</td><td>0.97</td><td>0.73</td><td>0.95</td><td>0.95</td></tr><tr><td>25%</td><td>0.80</td><td>0.86</td><td>0.60</td><td>0.93</td><td>0.85</td></tr><tr><td rowspan="3">UC-SMOTE</td><td>75%</td><td>0.95</td><td>0.98</td><td>0.87</td><td>0.97</td><td>0.98</td></tr><tr><td>50%</td><td>0.88</td><td>0.97</td><td>0.73</td><td>0.95</td><td>0.95</td></tr><tr><td>25%</td><td>0.81</td><td>0.91</td><td>0.59</td><td>0.93</td><td>0.87</td></tr></tbody></table>
<table><caption>**Table 3:** Summary results from the *extensive* benchmark and the *base* models (i.e. with default hyperparameters), DCR-R stands for DCR-rate and ML-EF for ML-Efficacy.</caption><thead><tr><th>Model</th><th>Percentiles</th><th>C2ST ↓</th><th>DCR-R ↓</th><th>ML-EF ↑</th><th>Shape ↑</th><th>Pair ↑</th></tr></thead><tbody><tr><td rowspan="3">Train Copy</td><td>75%</td><td>0.50</td><td>1.00</td><td>0.90</td><td>0.99</td><td>0.99</td></tr><tr><td>50%</td><td>0.50</td><td>1.00</td><td>0.73</td><td>0.99</td><td>0.98</td></tr><tr><td>25%</td><td>0.50</td><td>1.00</td><td>0.63</td><td>0.99</td><td>0.91</td></tr><tr><td rowspan="3">TVAE</td><td>75%</td><td>0.88</td><td>0.65</td><td>0.79</td><td>0.96</td><td>0.95</td></tr><tr><td>50%</td><td>0.81</td><td>0.63</td><td>0.71</td><td>0.95</td><td>0.92</td></tr><tr><td>25%</td><td>0.74</td><td>0.61</td><td>0.50</td><td>0.94</td><td>0.85</td></tr><tr><td rowspan="3">TVAE_base</td><td>75%</td><td>1.00</td><td>0.62</td><td>0.70</td><td>0.92</td><td>0.90</td></tr><tr><td>50%</td><td>0.98</td><td>0.61</td><td>0.65</td><td>0.91</td><td>0.80</td></tr><tr><td>25%</td><td>0.96</td><td>0.60</td><td>0.44</td><td>0.87</td><td>0.74</td></tr><tr><td rowspan="3">CTGAN</td><td>75%</td><td>0.90</td><td>0.64</td><td>0.71</td><td>0.98</td><td>0.96</td></tr><tr><td>50%</td><td>0.83</td><td>0.63</td><td>0.69</td><td>0.97</td><td>0.91</td></tr><tr><td>25%</td><td>0.71</td><td>0.61</td><td>0.47</td><td>0.96</td><td>0.85</td></tr><tr><td rowspan="3">CTGAN_base</td><td>75%</td><td>1.00</td><td>0.62</td><td>0.63</td><td>0.92</td><td>0.93</td></tr><tr><td>50%</td><td>0.99</td><td>0.60</td><td>0.47</td><td>0.88</td><td>0.83</td></tr><tr><td>25%</td><td>0.93</td><td>0.60</td><td>0.30</td><td>0.87</td><td>0.76</td></tr><tr><td rowspan="3">TabDDPM</td><td>75%</td><td>0.78</td><td>0.64</td><td>0.80</td><td>0.98</td><td>0.98</td></tr><tr><td>50%</td><td>0.67</td><td>0.62</td><td>0.69</td><td>0.98</td><td>0.93</td></tr><tr><td>25%</td><td>0.63</td><td>0.61</td><td>0.58</td><td>0.95</td><td>0.84</td></tr><tr><td rowspan="3">TabDDPM_base</td><td>75%</td><td>0.86</td><td>0.64</td><td>0.75</td><td>0.99</td><td>0.96</td></tr><tr><td>50%</td><td>0.77</td><td>0.62</td><td>0.68</td><td>0.98</td><td>0.92</td></tr><tr><td>25%</td><td>0.65</td><td>0.61</td><td>0.44</td><td>0.94</td><td>0.74</td></tr><tr><td rowspan="3">TabSyn</td><td>75%</td><td>0.80</td><td>0.63</td><td>0.76</td><td>0.99</td><td>0.97</td></tr><tr><td>50%</td><td>0.64</td><td>0.62</td><td>0.68</td><td>0.97</td><td>0.93</td></tr><tr><td>25%</td><td>0.59</td><td>0.61</td><td>0.46</td><td>0.96</td><td>0.74</td></tr><tr><td rowspan="3">TabSyn_base</td><td>75%</td><td>0.86</td><td>0.64</td><td>0.75</td><td>0.98</td><td>0.97</td></tr><tr><td>50%</td><td>0.71</td><td>0.62</td><td>0.57</td><td>0.97</td><td>0.95</td></tr><tr><td>25%</td><td>0.63</td><td>0.61</td><td>0.29</td><td>0.95</td><td>0.88</td></tr><tr><td rowspan="3">SMOTE</td><td>75%</td><td>0.97</td><td>0.98</td><td>0.86</td><td>0.97</td><td>0.99</td></tr><tr><td>50%</td><td>0.90</td><td>0.97</td><td>0.73</td><td>0.95</td><td>0.95</td></tr><tr><td>25%</td><td>0.80</td><td>0.86</td><td>0.60</td><td>0.93</td><td>0.85</td></tr><tr><td rowspan="3">UC-SMOTE</td><td>75%</td><td>0.95</td><td>0.98</td><td>0.87</td><td>0.97</td><td>0.98</td></tr><tr><td>50%</td><td>0.88</td><td>0.97</td><td>0.73</td><td>0.95</td><td>0.95</td></tr><tr><td>25%</td><td>0.81</td><td>0.91</td><td>0.59</td><td>0.93</td><td>0.87</td></tr></tbody></table>
<figure><img src="image_9.png" /><figcaption>**Figure 6:** Models ranking with critical difference diagrams respectively for C2ST, pair-wise correlations, and column-wise shape similarity metrics over all datasets.</figcaption></figure>

sets of models: (i) diffusion-based models *Tabsyn* and *TabDDPM* with a C2ST lower than 0.80 on most datasets and median around 0.65; (ii) push-forward models *CTGAN* and *TVAE* with a median C2ST around 0.82; (iii) *SMOTE* algorithms with a C2ST higher than 0.80 on most datasets.

The rankings obtained according to the *column-wise similarity* are broadly the same as the one obtained for C2ST. In addition to the rank shown in Figure 6(b), we can see that all the models obtain good scores (usually around 0.96) as compared to the *Train Copy* baseline (0.99). This result suggests that all the models succeed at capturing univariate distributions. For *pair-wise correlation*, we note surprisingly high values for *SMOTE* and *ucSMOTE*.
<figure><img src="image_7.png" /><figcaption>**Figure 6:** Models ranking with critical difference diagrams for C2ST, pair-wise correlations, and column-wise metrics over all datasets.</figcaption></figure>

(ii) push-forward models CTGAN and TVAE with a median C2ST around 0.82; (iii) SMOTE algorithms with a C2ST higher than 0.80 on most datasets.

The rankings obtained according to the *column-wise similarity* are broadly the same as the one obtained for C2ST. In addition to the rank shown in Figure 6(b), we can see that all the models obtain good scores (usually around 0.96) as compared to the *Train Copy* baseline (0.99). This result suggests that all the models succeed at capturing univariate distributions. For *pair-wise correlation*, we note surprisingly high values for SMOTE and ucSMOTE baselines while the neural network models obtain broadly the same ranking as for C2ST but the gaps between models are less marked.

A side takeaway from this result is that XGBoost-based C2ST provides a stronger discriminative power than *column-wise similarity* and *pair-wise correlation* metrics.
baselines while the neural network models obtain broadly the same ranking
as for C2ST but the gaps between models are less marked.

A side takeaway from this result is that XGBoost-based C2ST provides
a stronger discriminative power than column-wise similarity and pair-wise
correlation metrics.

### 4.2.2. Can the synthetic data be used to train a machine learning model ?

<figure><img src="image_10.png" /><figcaption>**Figure 7:** Models ranking with critical difference diagram for Catboost ML-Efficacy over all datasets.</figcaption></figure>

According to the machine learning efficacy metric (ML-Efficacy), the most useful generators are the ones that are conditioned on their targets (namely SMOTE and TabDDPM) with median values respectively at 0.73 and 0.69 (against 0.73 for *Train Copy*).

TabDDPM outperforms both base and tuned versions of TabSyn for this metric. It is also safer than SMOTE and it performs its training iterations faster than the other evaluated models. If ML-Efficacy is of importance, it is advisable to use this model. As expected, all models are far from the performance obtained on real data (*Train Copy*).

### 4.2.3. Does synthetic data preserve anonymity?

<figure><img src="image_11.png" /><figcaption>**Figure 8:** Models ranking with a critical difference diagram for the DCR-Rate metric over all datasets.</figcaption></figure>
### 4.2.2. Can the synthetic data be used to train a machine learning model ?

<figure><img src="image_8.png" /><figcaption>**Figure 7:** Models ranking with critical difference diagram for Catboost ML-Efficacy over all datasets.</figcaption></figure>

According to the *machine learning efficacy* metric (ML-Efficacy), the most useful generators are the ones that are conditioned on their targets (namely SMOTE and TabDDPM) with median values respectively at 0.73 and 0.69 (against 0.73 for *Train Copy*).

TabDDPM outperforms both base and tuned versions of TabSyn for this metric. It is also safer than SMOTE and it performs its training iterations faster than the other evaluated models. If ML-Efficacy is of importance, it is advisable to use this model. As expected, all models are far from the performance obtained on real data (*Train Copy*).

### 4.2.3. Does synthetic data preserve anonymity?

<figure><img src="image_9.png" /><figcaption>**Figure 8:** Models ranking with a critical difference diagram for the DCR-Rate metric over all datasets.</figcaption></figure>

A data synthesizer that would only copy its training set would be of little value. If it generates new instances that are too close to its training set, it would obtain a good C2ST score, but it would be prone to over-fitting and it would leak private information from the training set. We assess the ability of a model to generate new data through the DCR-Rate metric (c.f. Section 3.2).

On Figure 8 we observe two significantly distinct groups of models. On the left-hand side a "leaky" group that contains both SMOTE and ucSMOTE,
A data synthesizer that would only copy its training set would be of little value. If it generates new instances that are too close to its training set, it would obtain a good C2ST score, but it would be prone to over-fitting and it would leak private information from the training set. We assess the ability of a model to generate new data through the DCR-Rate metric (c.f. Section 3.1).

On Figure 8 we observe two significantly distinct groups of models. On the left-hand side a "leaky" group that contains both SMOTE and ucSMOTE, and on the right-hand side, a "safe" group that contains all neural algorithms. The poor performance of SMOTE is mainly due to the way it generates new data points by interpolating between existing ones. Therefore, these models cannot be considered safe concerning data protection. By taking a look at Table 3, the DCR-Rate of the two SMOTE variants is almost always above 0.86. On the other hand, if we exclude the tiny "Insurance" dataset where CTGAN and TVAE overfitted, the DCR-Rate values of the "safe" group are quite uniform around 0.62 and almost always below 0.65: these models can be considered safe.

### 4.2.4. What are the models’ costs?

Model training, optimization, and data sampling have a cost and an environmental impact that varies greatly from one model to another. We hence measured and estimated time, energy consumption, and CO₂ impact for each model during three phases: (i) training (measured), (ii) hyperparameters search (estimated), and (iii) sampling (measured). These results are reported fully in Appendix D. We summarize the tuning and training process cost distributions over all datasets and fold in Table 4. As mentioned in Section 3.1, these values are estimated from the tuning logs by taking into account the effective number of training steps, the number of trials, and the average GPU resource usage per training step as reported in Table D.13.

With a median tuning time around 18 minutes as shown in Table 4, TabDDPM is the fastest neural model. As a result, it also consumes less energy and it has less emissions at the training stage. As shown in Figure 5, considering its other performance metrics, it is a suitable choice to achieve good results at a relatively low training cost.

As expected, Figure 5 shows that the push models CTGAN and TVAE are the fastest at sampling stage. We notice that although CTGAN achieves slightly better quality results than TVAE, it is also one of the costliest models at the training stage as shown in Table 4. In order to achieve reasonable performance while reducing training costs, TVAE is hence an option to consider
and on the right-hand side, a "safe" group that contains all neural algorithms. The poor performance of SMOTE is mainly due to the way it generates new data points by interpolating between existing ones. Therefore, these models cannot be considered safe concerning data protection. By taking a look at Table 3, the DCR-Rate of the two SMOTE variants is almost always above 0.86. On the other hand, if we exclude the tiny "Insurance" dataset where CTGAN and TVAE overfitted, the DCR-Rate values of the "safe" group are quite uniform around 0.62 and almost always below 0.65: these models can be considered safe.

### 4.2.4. What are the models’ costs?

<table><caption>**Table 4:** Summary of the costs of the extensive benchmark. Costs dispersion of the 25th, 50th, and 75th percentiles are provided over all datasets and folds.</caption><thead><tr><th>Model</th><th>Percentiles</th><th>Duration (HH:MM) ↓</th><th>Emission (Kg) ↓</th><th>Energy (kWh) ↓</th></tr></thead><tbody><tr><td rowspan="3">TVAE</td><td>75%</td><td>01:00</td><td>1.05</td><td>15.57</td></tr><tr><td>50%</td><td>00:24</td><td>0.42</td><td>6.22</td></tr><tr><td>25%</td><td>00:12</td><td>0.21</td><td>3.08</td></tr><tr><td rowspan="3">CTGAN</td><td>75%</td><td>02:20</td><td>2.45</td><td>36.36</td></tr><tr><td>50%</td><td>01:37</td><td>1.67</td><td>24.84</td></tr><tr><td>25%</td><td>01:06</td><td>1.16</td><td>17.14</td></tr><tr><td rowspan="3">TabDDPM</td><td>75%</td><td>00:26</td><td>0.36</td><td>5.30</td></tr><tr><td>50%</td><td>00:18</td><td>0.29</td><td>4.37</td></tr><tr><td>25%</td><td>00:12</td><td>0.19</td><td>2.82</td></tr><tr><td rowspan="3">TabSyn</td><td>75%</td><td>03:25</td><td>1.92</td><td>22.65</td></tr><tr><td>50%</td><td>01:59</td><td>1.03</td><td>14.20</td></tr><tr><td>25%</td><td>01:29</td><td>0.72</td><td>9.09</td></tr><tr><td rowspan="3">SMOTE</td><td>75%</td><td>00:02</td><td>0.00</td><td>0.01</td></tr><tr><td>50%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr><tr><td>25%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr><tr><td rowspan="3">UC-SMOTE</td><td>75%</td><td>00:03</td><td>0.00</td><td>0.01</td></tr><tr><td>50%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr><tr><td>25%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr></tbody></table>

Model training, optimization, and data sampling have a cost and an environmental impact that varies greatly from one model to another. We hence measured and estimated time, energy consumption, and CO₂ impact for each model during three phases: (i) training (measured), (ii) hyperpa-
<table><caption>**Table 4:** Summary of the *costs* of the *extensive* experiment. We provide *costs* dispersion at the 25th, 50th, and 75th percentiles over all datasets and folds. The values are aggregated from the tuning costs estimated per dataset in Table D.14. More information about the estimation process can be found in Section 3.1.</caption><thead><tr><th>Model</th><th>Percentiles</th><th>Duration (HH:MM) ↓</th><th>Emission (Kg) ↓</th><th>Energy (kWh) ↓</th></tr></thead><tbody><tr><td rowspan="3">TVAE</td><td>75%</td><td>01:00</td><td>1.05</td><td>15.57</td></tr><tr><td>50%</td><td>00:24</td><td>0.42</td><td>6.22</td></tr><tr><td>25%</td><td>00:12</td><td>0.21</td><td>3.08</td></tr><tr><td rowspan="3">CTGAN</td><td>75%</td><td>02:20</td><td>2.45</td><td>36.36</td></tr><tr><td>50%</td><td>01:37</td><td>1.67</td><td>24.84</td></tr><tr><td>25%</td><td>01:06</td><td>1.16</td><td>17.14</td></tr><tr><td rowspan="3">TabDDPM</td><td>75%</td><td>00:26</td><td>0.36</td><td>5.30</td></tr><tr><td>50%</td><td>00:18</td><td>0.29</td><td>4.37</td></tr><tr><td>25%</td><td>00:12</td><td>0.19</td><td>2.82</td></tr><tr><td rowspan="3">TabSyn</td><td>75%</td><td>03:25</td><td>1.92</td><td>22.65</td></tr><tr><td>50%</td><td>01:59</td><td>1.03</td><td>14.20</td></tr><tr><td>25%</td><td>01:29</td><td>0.72</td><td>9.09</td></tr><tr><td rowspan="3">SMOTE</td><td>75%</td><td>00:02</td><td>0.00</td><td>0.01</td></tr><tr><td>50%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr><tr><td>25%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr><tr><td rowspan="3">UC-SMOTE</td><td>75%</td><td>00:03</td><td>0.00</td><td>0.01</td></tr><tr><td>50%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr><tr><td>25%</td><td>00:00</td><td>0.00</td><td>0.00</td></tr></tbody></table>
rameters search (estimated), and (iii) sampling (measured). These results are reported fully in Appendix D. We summarize the tuning and training process cost distributions over all datasets and fold in Table 4. As mentioned in Section 3.2, these values are estimated from the tuning logs by taking into account the effective number of training steps, the number of trials, and the average GPU resource usage per training step as reported in Table D.13.

With a median tuning time around 18 minutes as shown in Table 4, TabDDPM is the fastest neural model. As a result, it also consumes less energy and it has less emissions at the training stage. As shown in Figure 5, considering its other performance metrics, it is a suitable choice to achieve good results at a relatively low training cost.

As expected, Figure 5 shows that the push models CTGAN and TVAE are the fastest at sampling stage. We notice that although CTGAN achieves slightly better quality results than TVAE, it is also one of the costliest models at the training stage as shown in Table 4. In order to achieve reasonable performance while reducing training costs, TVAE is hence an option to consider prior to CTGAN.

The training and tuning of TabSyn is one of the most demanding (with a median time of 2 hours for tuning). In the end, however, it delivers the best performance in terms of quality metrics. This model also has the advantage of providing a set of default hyperparameters that can have a *reasonable* performance although, if we follow the author's recommendation of 4000 VAE epochs, its training remains costly by comparison to other models. Indeed, Figure 5 show that, even if we consider the whole tuning+training pipeline TabDDPM and TVAE remain cheaper to train than TabSyn_base.

In terms of sampling cost, TabDDPM is the worst-performing model. It takes longer than the other models (c.f. Figure 5) and hence consumes more energy with more CO₂ emissions at this step. TabSyn reduces the number of denoising steps by using VAE embedding and linear noises to reduce its sampling time [15]. It hence achieves better performance at inference than TabDDPM.

The two baselines SMOTE and ucSMOTE being based on neighbourhood interpolation their training and tuning cost is negligible. However, as shown in Appendix Table C.12, their sampling process requiring a nearest neighbor search, it is slower on large datasets than push-forward models like TVAE or CTGAN.
prior to CTGAN.

The training and tuning of TabSyn is one of the most demanding (with a median time of 2 hours for tuning). In the end, however, it delivers the best performance in terms of quality metrics. This model also has the advantage of providing a set of default hyperparameters that can have a *reasonable* performance although, if we follow the author's recommendation of 4000 VAE epochs, its training remains costly by comparison to other models. Indeed, Figure 5 shows that, even if we consider the whole tuning+training pipeline TabDDPM and TVAE remain cheaper to train than TabSyn_base.

In terms of sampling cost, TabDDPM is the worst-performing model. It takes longer than the other models (c.f. Figure 5) and hence consumes more energy with more CO₂ emissions at this step. TabSyn reduces the number of denoising steps by using VAE embedding and linear noises to reduce its sampling time [9]. It hence achieves better performance at inference than TabDDPM.

The two baselines SMOTE and uCSMOTE being based on neighbourhood interpolation their training and tuning cost is negligible. However, as shown in Appendix Table C.12, their sampling process requiring a nearest neighbor search, it is slower on large datasets than push-forward models like TVAE or CTGAN.

### 4.3. Is it worth optimizing the hyperparameters for all models ?

As mentioned in the previous section, even with the help of sophisticated search algorithms, hyperparameter tuning is costly and the performance-versus-tuning-budget curve is following a *diminishing returns law*. We were hence interested in comparing heavily tuned models against non-tuned models.

To assess this, we trained the neural models using the hyperparameters provided by the authors in the original papers. These results are detailed in Appendix E.

In Figure 9(a) we can see a, sometimes huge, C2ST performance improvement of all models when tuned. This improvement is statistically significant between optimized TVAE and CTGAN and their base versions. Looking at the absolute C2ST values in Table 3, it confirms that these models should not be used with their default hyperparameters.

For TabDDPM and TabSyn, we also notice a 10 points improvement on the median C2ST but this gap is not large enough to provide statistical guarantees.
<figure><img src="image_10.png" /><figcaption>**Figure 9:** Models' ranking with critical difference diagrams on C2ST, DCR-Rate, and ML-Efficacy metrics over all datasets: base models (i.e. with default hyperparameters) versus extensively tuned models.</figcaption></figure>

### 4.3. Is it worth optimizing the hyperparameters for all models ?

As mentioned in the previous section, even with the help of sophisticated search algorithms, hyperparameter tuning is costly and the performance-versus-tuning-budget curve is following a *diminishing returns law*. We were hence interested in comparing heavily tuned models against non-tuned models.

To assess this, we trained the neural models using the hyperparameters provided by the authors in the original papers. These results are detailed in Appendix E.

In Figure 9(a) we can see a, sometimes huge, C2ST performance improvement of all models when tuned. This improvement is statistically significant between optimized TVAE and CTGAN and their base versions. Looking at the absolute C2ST values in Table 3, it confirms that these models should not be used with their default hyperparameters.
<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>**Figure 9:** Models' ranking with critical difference diagrams on C2ST, DCR-Rate, and ML-Efficacy metrics over all datasets: base models (using default hyperparameters) versus models obtained after an extensive hyperparameter tuning.</figcaption></figure>

Although it was not the main target for tuning, we also observe an ML-Efficacy gain for all models. This gain is marked at the 25th percentile (i.e. for the hardest datasets).

We can therefore conclude that for all neural tabular generative models that we considered (including TabSyn), it is worth optimizing the hyperparameters specifically for each dataset if we want to improve performance. But the trade-off between the optimization cost and the performance gain is highly correlated to the size and design of the hyperparameter's space (c.f. Appendix A). In the next section we propose a reduced search space and study the impact of a "light" hyperparameters optimization with a limited budget.
For TabDDPM and TabSyn, we also notice a 10 points improvement on the median C2ST but this gap is not large enough to provide statistical guarantees.

Although it was not the main target for tuning, we also observe an ML-Efficacy gain for all models. This gain is marked at the 25th percentile (i.e. for the hardest datasets).

We can therefore conclude that for all neural tabular generative models that we considered (including TabSyn), it is worth optimizing the hyperparameters specifically for each dataset if we want to improve performance. But the trade-off between the optimization cost and the performance gain is highly correlated to the size and design of the hyperparameter's space (c.f. Appendix A). In the next section we propose a reduced search space and study the impact of a "light" hyperparameters optimization with a limited budget.

# 5. Limited-Budget Benchmark

As underlined in Section 4.3, optimizing the hyperparameters for each dataset can significantly improve the quality of the generators. But a large-scale optimization like the one we performed is technically difficult, costly, and it has a non-negligible carbon impact (c.f. Table 4). Researchers and practitioners may hence be interested in reducing this cost without deteriorating too much the model's performance.

In this section, we leverage the results of our extensive tuning experiment to: (i) suggest reduced search spaces achieving reasonable performance at a much lower cost; (ii) assess and compare the performance of the models when tuned and trained with the same limited budget. By comparing the model's performance after this limited-budget tuning/training with our previous results (respectively with heavy tuning or without tuning at all), we gain new insights into the models.

## 5.1. *Hyperparameters Search Space Reduction*

We carried out this experiment for the most hyperparameter-heavy models, namely: TVAE, CTGAN, TabDDPM, and TabSyn. To reduce their search space, we independently considered each hyperparameter/architecture configuration variable and kept only the values that were the most frequently selected during the large-scale tuning phase. For discrete variables we kept
# 5. Limited-Budget Experiment Results

As underlined in Section 4.3, optimizing the hyperparameters for each dataset can significantly improve the quality of the generators. But a large-scale optimization like the one we performed is technically difficult, costly, and it has a non-negligible carbon impact (c.f. Table 4). Researchers and practitioners may hence be interested in reducing this cost without deteriorating too much the model's performance.

In this section, we leverage the results of our extensive tuning experiment to: (i) suggest reduced search spaces achieving reasonable performance at a much lower cost; (ii) assess and compare the performance of the models when tuned and trained with the same limited budget. By comparing the model's performance after this "light" tuning/training with our previous results (respectively with heavy tuning or without tuning at all), we gain new insights into the models.

## 5.1. Hyperparameters Search Space Reduction

We carried out this experiment for the most hyperparameter-heavy models, namely: TVAE, CTGAN, TabDDPM, and TabSyn. To reduce their search space, we independently considered each hyperparameter/architecture configuration variable and kept only the values that were the most frequently selected during the large-scale tuning phase. For discrete variables we kept the 80% most frequent values, and for continuous variables we kept the value ranging between the 10th and 90th percentiles.

For instance, on CTGAN the large-scale tuning included six encoder options: CDF, PLE_CDF, PTP, Quantile, MinMax, and CBN. But only CDF and PLE_CDF were selected on most datasets. We could also drastically reduce TabSyn VAE's learning rate range from $(10^{-5}, 10^{-2})$ to $(10^{-3}, 7 \cdot 10^{-3})$. We present all these reduced search spaces in Appendix A.

## 5.2. Putting the Reduced Search Spaces to the Test

To evaluate the reduced search spaces, we ran a new tuning on all datasets with only 50 trials and a strict limit of 20 minutes per trial. Since TabSyn's training is done in two steps, we allocated 10 minutes for the VAE training and 10 minutes for the denoiser. Each hyperparameter search was performed with the same 3-fold splits and the same methodology as in the extensive experiment.
the 80% most frequent values, and for continuous variables we kept the value ranging between the 10th and 90th percentiles.

For instance, on CTGAN the large-scale tuning included six encoder options: CDF, PLE_CDF, PTP, Quantile, MinMax, and CBN. But only CDF and PLE_CDF were selected on most datasets. We could also drastically reduce TabSyn VAE's learning rate range from (10⁻⁵, 10⁻²) to (10⁻³, 7·10⁻³). We present all these reduced search spaces in Appendix A.

## 5.2. Putting the Reduced Search Spaces to the Test

To evaluate the reduced search spaces, we ran a new tuning on all datasets with only 50 trials and a strict limit of 20 minutes per trial. Since TabSyn's training is done in two steps, we allocated 10 minutes for the VAE training and 10 minutes for the denoiser. Each hyperparameter search was performed with the same 3-fold splits and the same methodology as in the extensive experiment.

### 5.2.1. A Cost-Performance Trade-off

In Figure 10 we summarize the performance of the models after limited-budget hyperparameter search against the ones obtained with the base models and the ones obtained after the extensive tuning of Section 3.3. On the x-axis we display the total GPU-time (search+optimization) needed to obtain the corresponding model and on the y-axis we show the C2ST performance. The diameters of the dots represents the number of possible configurations of the hyperparameter search space as described in Appendix A.

If we compare against the non-tuned base models, the C2ST performance of all models except TabSyn is clearly improved by the limited-budget hyperparameter tuning. For CTGAN, and TVAE the gain is huge: after only a few minutes of tuning with the reduced search space we reach the same performance as the one obtained after hours of heavy tuning. For TabD-DPM, we also improved the base performance by running the limited-budget hyperparameter tuning. However, we did not reach the same performance as we did with an extensive search.

In the limited-budget experiment, the TabSyn's performance degradation against the base model reveals the importance of a well-trained VAE which leads to better samples from the diffusion process. Indeed, on Adult dataset for instance, within its 10 minutes budget, it was trained with less than 600 epochs against 4000 for the base model. This suggests to increase the number of epochs or the training time for TabSyn's VAE in further experiments.
<figure><img src="image_15.png" /><figcaption>**Figure 10:** Average C2ST performance of the models under various setups. We show the performance of the base models (appended with "_B"), the models obtained from the light experiment described in this section (appended with "_L"), and the ones from the extensive experiment (appended with "_E"). The C2ST axis is reversed to show the best models at the top. The dot diameter indicates the complexity of the search space. The duration axis and dot diameters are log-scaled for better visualization.</figcaption></figure>

### 5.2.1. A Cost-Performance Trade-off

In Figure 10 we summarize the performance of the models after this light hyperparameter search against the ones obtained with the base models and the ones obtained after the extensive tuning of Section 3.2. On the x-axis we display the total GPU-time (search+optimization) needed to obtain the corresponding model and on the y-axis we show the C2ST performance. The diameters of the dots represents the number of possible configurations of the hyperparameter search space as described in Appendix A.

If we compare against the non-tuned base models, the C2ST performance
<figure><img src="image_11.png" /><figcaption>**Figure 10:** Average C2ST performance of the models under various setups (base models are appended with "_B", limited-budget with "_L" and extensive with "_E"). The C2ST axis shows the best models at the top, the dot diameter indicates the complexity of the search space. The duration axis and dot diameters are log-scaled for better visualization.</figcaption></figure>

Finally, as shown in Figure 10, the costs induced by the heavily optimized TabSyn model are high compared to the base one. The performance gain and the gap in cost should be considered depending on the task and constraints as the base model already delivers *decent* performance.

It is worth noting that, due to its quick implementation, TabDDPM performed sometimes more training steps within its 20 minutes time budget on the limited-budget experiment than it did on the extensive search where the number of steps was bounded. But on the other hand, its performance was also affected by the reduced number of trials (50 instead of 300).
of all models except TabSyn is clearly improved by the light hyperparameter tuning. For CTGAN, and TVAE the gain is huge: after only a few minutes of tuning with the reduced search space we reach the same performance as the one obtained after hours of heavy tuning. For TabDDPM, we also improved the base performance by running the light hyperparameter tuning. However, we did not reach the same performance as we did with an extensive search.

In the light experiment, the TabSyn's performance degradation against the base model reveals the importance of a well-trained VAE which leads to better samples from the diffusion process. Indeed, on Adult dataset for instance, within its 10 minutes budget, it was trained with less than 600 epochs against 4000 for the base model. This suggests to increase the number of epochs or the training time for TabSyn's VAE in further experiments.

Finally, as shown in Figure 10, the costs induced by the heavily optimized TabSyn model are high compared to the base one. The performance gain and the gap in cost should be considered depending on the task and constraints as the base model already delivers decent performance.

It is worth noting that, due to its quick implementation, TabDDPM performed sometimes more training steps within its 20 minutes time budget on the light experiment than it did on the extensive search where the number of steps was bounded. But on the other hand, its performance was also affected by the reduced number of trials (50 instead of 300).

### 5.2.2. Multi-criteria Analysis of Light Tuning

In Figure 11 we summarize the model's relative performances according to the five metrics for the light tuning experiment. To complete this point of view, Table 5 summarizes the quality metric distributions among all datasets and folds. The full results are detailed in Appendix F.

A first remark about Figure 11 is that it is more balanced than Figure 5: with a fair and limited GPU budget allocation, the models tend to perform similarly among all criteria.

There is no more clear leading model although TabDDPM obtains the best C2ST performance and TabSyn slightly dominates in terms of privacy (DCR-Rate). We however have a safe median DCR-Rate score of about 0.63 for this experiment on all neural models (against 0.62 for the extensive experiment). For ML-Efficacy, TabDDPM remains slightly the best but there is no clear leader either.

The reduced search space that we provide is hence a good starting point to perform a quick dataset-specific hyperparameters optimization that will
<table><caption>**Table 5:** Performance dispersion across all datasets in the *limited-budget* benchmark. DCR-R and ML-EF stand for DCR-Rate and ML-Efficacy respectively.</caption><thead><tr><th>Model</th><th>Percentiles</th><th>C2ST ↓</th><th>DCR-R ↓</th><th>ML-EF ↑</th><th>Shape ↑</th><th>Pair ↑</th></tr></thead><tbody><tr><td rowspan="3">Train Copy</td><td>75%</td><td>0.50</td><td>1.00</td><td>0.90</td><td>0.99</td><td>0.99</td></tr><tr><td>50%</td><td>0.50</td><td>1.00</td><td>0.73</td><td>0.99</td><td>0.98</td></tr><tr><td>25%</td><td>0.50</td><td>1.00</td><td>0.63</td><td>0.99</td><td>0.91</td></tr><tr><td rowspan="3">TVAE</td><td>75%</td><td>0.92</td><td>0.65</td><td>0.80</td><td>0.96</td><td>0.95</td></tr><tr><td>50%</td><td>0.80</td><td>0.63</td><td>0.70</td><td>0.95</td><td>0.93</td></tr><tr><td>25%</td><td>0.70</td><td>0.61</td><td>0.51</td><td>0.94</td><td>0.87</td></tr><tr><td rowspan="3">CTGAN</td><td>75%</td><td>0.89</td><td>0.64</td><td>0.71</td><td>0.98</td><td>0.96</td></tr><tr><td>50%</td><td>0.85</td><td>0.63</td><td>0.68</td><td>0.97</td><td>0.92</td></tr><tr><td>25%</td><td>0.70</td><td>0.62</td><td>0.45</td><td>0.95</td><td>0.85</td></tr><tr><td rowspan="3">TabDDPM</td><td>75%</td><td>0.83</td><td>0.67</td><td>0.83</td><td>0.97</td><td>0.95</td></tr><tr><td>50%</td><td>0.72</td><td>0.63</td><td>0.69</td><td>0.96</td><td>0.91</td></tr><tr><td>25%</td><td>0.63</td><td>0.62</td><td>0.50</td><td>0.95</td><td>0.81</td></tr><tr><td rowspan="3">TabSyn</td><td>75%</td><td>0.88</td><td>0.63</td><td>0.71</td><td>0.97</td><td>0.96</td></tr><tr><td>50%</td><td>0.81</td><td>0.62</td><td>0.65</td><td>0.95</td><td>0.93</td></tr><tr><td>25%</td><td>0.72</td><td>0.61</td><td>0.34</td><td>0.94</td><td>0.82</td></tr></tbody></table>

### 5.2.2. Multi-criteria Analysis of the Limited-Budget Tuning

In Figure 11 we summarize the model's relative performances according to the five metrics for the limited-budget tuning experiment. To complete this point of view, Table 5 summarizes the quality metric distributions among all datasets and folds. The full results are detailed in Appendix F.

A first remark about Figure 11 is that it is more balanced than Figure 5: with a fair and limited GPU budget allocation, the models tend to perform similarly among all criteria.

There is no more clear leading model although TabDDPM obtains the best C2ST performance and TabSyn slightly dominates in terms of privacy (DCR-Rate). We however have a safe median DCR-Rate score of about 0.63 for this experiment on all neural models (against 0.62 for the extensive experiment). For ML-Efficacy, TabDDPM remains slightly the best but there is no clear leader either.

The reduced search space that we provide is hence a good starting point to perform a quick dataset-specific hyperparameters optimization that will fit on a medium-size workstation.

The relatively balanced results of this new experiment confirms that the superiority of a tabular data generator is not only due to its model but also to the whole tuning and training compute effort.
<table><caption>**Table 5:** Results from the *light* experiment: performance dispersion at the 25th, 50th, and 75th percentiles across all models. These results are obtained by aggregating the scores across all datasets. To save space, we respectively shortened DCR-rate to DCR-R and ML-efficacy to ML-EF.</caption><thead><tr><th>Model</th><th>Percentiles</th><th>C2ST ↓</th><th>DCR-R ↓</th><th>ML-EF ↑</th><th>Shape ↑</th><th>Pair ↑</th></tr></thead><tbody><tr><td rowspan="3">Train Copy</td><td>75%</td><td>0.50</td><td>1.00</td><td>0.90</td><td>0.99</td><td>0.99</td></tr><tr><td>50%</td><td>0.50</td><td>1.00</td><td>0.73</td><td>0.99</td><td>0.98</td></tr><tr><td>25%</td><td>0.50</td><td>1.00</td><td>0.63</td><td>0.99</td><td>0.91</td></tr><tr><td rowspan="3">TVAE</td><td>75%</td><td>0.92</td><td>0.65</td><td>0.80</td><td>0.96</td><td>0.95</td></tr><tr><td>50%</td><td>0.80</td><td>0.63</td><td>0.70</td><td>0.95</td><td>0.93</td></tr><tr><td>25%</td><td>0.70</td><td>0.61</td><td>0.51</td><td>0.94</td><td>0.87</td></tr><tr><td rowspan="3">CTGAN</td><td>75%</td><td>0.89</td><td>0.64</td><td>0.71</td><td>0.98</td><td>0.96</td></tr><tr><td>50%</td><td>0.85</td><td>0.63</td><td>0.68</td><td>0.97</td><td>0.92</td></tr><tr><td>25%</td><td>0.70</td><td>0.62</td><td>0.45</td><td>0.95</td><td>0.85</td></tr><tr><td rowspan="3">TabDDPM</td><td>75%</td><td>0.83</td><td>0.67</td><td>0.83</td><td>0.97</td><td>0.95</td></tr><tr><td>50%</td><td>0.72</td><td>0.63</td><td>0.69</td><td>0.96</td><td>0.91</td></tr><tr><td>25%</td><td>0.63</td><td>0.62</td><td>0.50</td><td>0.95</td><td>0.81</td></tr><tr><td rowspan="3">TabSyn</td><td>75%</td><td>0.88</td><td>0.63</td><td>0.71</td><td>0.97</td><td>0.96</td></tr><tr><td>50%</td><td>0.81</td><td>0.62</td><td>0.65</td><td>0.95</td><td>0.93</td></tr><tr><td>25%</td><td>0.72</td><td>0.61</td><td>0.34</td><td>0.94</td><td>0.82</td></tr></tbody></table>

fit on a medium-size workstation.

The relatively balanced results of this new experiment confirms that the superiority of a tabular data generator is not only due to its model but also to the whole tuning and training compute effort.

Overall, TabDDPM provides a good balance between realism, privacy, utility, and cost. TVAE is a viable alternative if the utility constraint can be relaxed. It can be tuned using the light experiment search space to quickly achieve good performance. If resources are available and cost is not a priority, it is advisable to tune TabSyn, which achieves good realism results but at a higher cost. Finally, for quick results when privacy is not a concern and when the main focus is utility, SMOTE is the recommended approach.
<figure><img src="image_12.png" /><figcaption>**Figure 11:** Radar chart of the optimized models in the limited-budget benchmark across various metrics. "Shape" stands for *column-wise similarity*, and "Pair" stands for *pair-wise correlation*.</figcaption></figure>

Overall, TabDDPM provides a good balance between realism, privacy, utility, and cost. TVAE is a viable alternative if the utility constraint can be relaxed. It can be tuned using the limited-budget experiment search space to quickly achieve good performance. If resources are available and cost is not a priority, it is advisable to tune TabSyn, which achieves good realism results but at a higher cost. Finally, for quick results when privacy is not a concern and when the main focus is utility, SMOTE is the recommended approach.

# 6. Conclusion

We proposed a typology of state-of-art models for tabular data generation and we benchmarked five of them on 16 datasets with a strict 3-fold
<figure><img src="image_16.png" /><figcaption>**Figure 11:** Radar chart of the optimized models under the **light experiment** according to C2ST, DCR-Rate, ML-Efficacy, column-wise similarity (named "Shape"), and pair-wise correlation (named "Pair"). We take the ranks of the models to obtain normalized metrics. The models were trained with a maximum time budget of 20 minutes and tuned with 50 trials per fold.</figcaption></figure>

## 6. Conclusion

We benchmarked extensively recent tabular data generation models on 16 datasets with a strict 3-fold cross-validation procedure. We first performed both a large-scale tuning experiment on a super computer from which we derived a reduced search-space. We then performed a quick tuning experiment that fits on a medium-size workstation. Leveraging these experiments we were able to provide several insights on the models while answering to three technical questions: (i) is it worth optimizing the hyper-parameters/preprocessing specifically for each dataset? (ii) can we propose a reduced search space that fits well for all datasets? (iii) is there a clear trade-off between training/sampling costs, and synthetic data quality?

For the two first questions, Figure 10 is certainly the best summary: most models, including TabSyn, benefit greatly from a dataset-specific tuning. But the whole tuning process is costly (time, money, energy, CO₂ emissions) and
cross-validation procedure. The number of folds was limited to 3 because
increasing it would have dramatically increased the costs and carbon foot-
print. We performed extensive large-scale tuning on a super computer from
which we derived a reduced search-space. Moreover, we performed a limited-
budget benchmark that fits on a medium-size workstation. Leveraging these
benchmarks we were able to provide several insights on the models while
answering to three technical questions: (i) is it worth optimizing the hyper-
parameters/preprocessing specifically for each dataset? (ii) can we propose
a reduced search space that fits well for all datasets? (iii) is there a clear
trade-off between training/sampling costs, and synthetic data quality?

For the two first questions, Figure 10 is certainly the best summary: most
models, including TabSyn, benefit greatly from a dataset-specific tuning.
However, the whole tuning process is costly in terms of time, money, energy,
and CO2 emissions, and there is clearly a "diminishing return" effect. We
conclude that, even for TabSyn, a quick dataset-specific model tuning based
on the reduced search spaces we provide in Appendix A is enough to get
most of the performance at the scale of a medium-size workstation. We also
recommend using a subset of the dataset for hyperparameter tuning. The
obtained hyperparameters can then be used to train the model at full-scale.

Regarding the trade-off question on the multiple considered criteria: re-
alism, privacy, utility and costs, Figure 5 and Figure 11 confirm that if we do
not limit the compute power, the two diffusion-based models TabDDPM and
TabSyn are the most recommended solutions for tabular data generation.
Nevertheless, the performance gaps between models become quite narrow
with a limited compute power.

7. Future Work

Our study focuses on models trained, tuned, and evaluated on a single
table and can be extended to evaluate cross-table models that fits with mul-
tiple table structures. This is an important step towards foundation models
for tabular data generation. Models such as GReaT [38] offers an interesting
approach for a cross-table approach by converting the data generation pro-
cess into a text generation one. However, when evaluated on a single tables,
it struggle against models like CTGAN [20] for joint probability distribu-
tion [15, 11]. Working on a cross-table foundation model also introduces
additional considerations towards data poisoning or adversarial attacks as
foundation models are prone to being poisoned [79].
there is clearly a "diminishing return" effect. We conclude that, even for
Tabsyn, a quick dataset-specific model tuning on a carefully designed search
space as we did in Section 5 is enough to get most of the performance at the
scale of a medium-size workstation.

Regarding the trade-off question on the multiple considered criteria: re-
alism, privacy, utility and costs, Figure 5 and Figure 11 confirm that if we do
not limit the compute power, the two diffusion-based models TabDDPM and
Tabsyn are the most recommended solutions for tabular data generation. But
with a fairly limited compute power, the performance gaps between models
become quite narrow.

7. Ackowlegment

This work was granted access to the HPC resources of IDRIS under the al-
locations 2023-AD011014381, 2023-AD011011407R3, and 2023-AD011012220R2
made by GENCI.

References

[1] D. Libes, D. Lechevalier, S. Jain, Issues in synthetic data generation for advanced manufacturing, in: 2017 IEEE International Conference on Big Data (Big Data), IEEE, 2017, pp. 1746–1754.

[2] V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, G. Kasneci, Deep neural networks and tabular data: A survey, IEEE Transactions on Neural Networks and Learning Systems (2022).

[3] L. Zhang, S. Zhang, K. Balog, Table2vec: Neural word and entity embeddings for table population and retrieval, in: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, 2019, pp. 1029–1032.

[4] S. Chitlangia, A. Muralidhar, R. Agarwal, Self supervised pre-training for large scale tabular data, in: NeurIPS 2022 Workshop on Table Representation Learning, 2022.

[5] T. Zhang, S. Wang, S. Yan, L. Jian, Q. Liu, Generative table pre-training empowers models for tabular prediction, in: H. Bouamor,
Another important research direction is the models’ interpretability. Understanding the generation procedure is also important to build user’s trust for decision making. Such interpretability study can be performed starting from a tree-based model such as Forest-Flow [63].

## 8. Ackowlement

This work was granted access to the HPC resources of IDRIS under the al-
locations 2023-AD011014381, 2023-AD011011407R3, and 2023-AD011012220R2
made by GENCI.

References

[1] D. Libes, D. Lechevalier, S. Jain, Issues in synthetic data generation for advanced manufacturing, in: 2017 IEEE International Conference on Big Data (Big Data), IEEE, 2017, pp. 1746–1754.

[2] V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, G. Kasneci, Deep neural networks and tabular data: A survey, IEEE Transactions on Neural Networks and Learning Systems (2022).

[3] L. Zhang, S. Zhang, K. Balog, Table2vec: Neural word and entity embeddings for table population and retrieval, in: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, 2019, pp. 1029–1032.

[4] S. Chitlangia, A. Muralidhar, R. Agarwal, Self supervised pre-training for large scale tabular data, in: NeurIPS 2022 Workshop on Table Representation Learning, 2022.

[5] T. Zhang, S. Wang, S. Yan, L. Jian, Q. Liu, Generative table pre-training empowers models for tabular prediction, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Singapore, 2023, pp. 14836–14854. URL: https://aclanthology.org/2023.emnlp-main.917. doi:10.18653/v1/2023.emnlp-main.917.
J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Singapore, 2023, pp. 14836-14854. URL: https://aclanthology.org/2023.emnlp-main.917. doi:10.18653/v1/2023.emnlp-main.917.

[6] L. Grinsztajn, E. Oyallon, G. Varoquaux, Why do tree-based models still outperform deep learning on typical tabular data?, in: Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL: https://openreview.net/forum?id=Fp7__phQszn.

[7] J. Fonseca, F. Bacao, Tabular and latent space synthetic data generation: a literature review, Journal of Big Data 10 (2023) 115.

[8] A. Kotelnikov, D. Baranchuk, I. Rubachev, A. Babenko, Tabddpm: Modelling tabular data with diffusion models, in: International Conference on Machine Learning, PMLR, 2023, pp. 17564-17579.

[9] H. Zhang, J. Zhang, Z. Shen, B. Srinivasan, X. Qin, C. Faloutsos, H. Rangwala, G. Karypis, Mixed-type tabular data synthesis with score-based diffusion in latent space, in: The Twelfth International Conference on Learning Representations, 2024.

[10] D. Lopez-Paz, M. Oquab, Revisiting classifier two-sample tests, in: International Conference on Learning Representations, 2016.

[11] E. H. Zein, T. Urvoy, Tabular data generation: Can we fool XGBoost?, in: NeurIPS 2022 First Table Representation Workshop, 2022.

[12] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: synthetic minority over-sampling technique, Journal of artificial intelligence research 16 (2002) 321-357.

[13] V. Borisov, K. Sessler, T. Leemann, M. Pawelczyk, G. Kasneci, Language models are realistic tabular data generators, in: The Eleventh International Conference on Learning Representations (ICLR), 2023.

[14] X. Fang, W. Xu, F. A. Tan, J. Zhang, Z. Hu, Y. Qi, S. Nickleach, D. Socolinsky, S. Sengamedu, C. Faloutsos, Large language models on tabular data-a survey, arXiv preprint arXiv:2402.17944 (2024).
[6] L. Grinsztajn, E. Oyallon, G. Varoquaux, Why do tree-based models still outperform deep learning on typical tabular data?, in: Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL: https://openreview.net/forum?id=Fp7__phQszn.

[7] C. M. Bowen, J. Snoke, Comparative study of differentially private synthetic data algorithms from the nist pscr differential privacy synthetic data challenge, Journal of Privacy and Confidentiality 11 (2021). URL: https://journalprivacyconfidentiality.org/index.php/jpc/article/view/748. doi:10.29012/jpc.748.

[8] C. Arnold, M. Neunhoeffer, Really useful synthetic data–a framework to evaluate the quality of differentially private synthetic data, arXiv e-prints (2020) arXiv:2004.0748.

[9] Y. Tao, R. McKenna, M. Hay, A. Machanavajjhala, G. Miklau, Benchmarking differentially private synthetic data generation algorithms (2022).

[10] Y. Hu, F. Wu, Q. Li, Y. Long, G. Garrido, C. Ge, B. Ding, D. Forsyth, B. Li, D. Song, Sok: Privacy-preserving data synthesis, in: 2024 IEEE Symposium on Security and Privacy (SP), IEEE Computer Society, 2023, pp. 2–2.

[11] Y. Du, N. Li, Towards principled assessment of tabular data synthesis algorithms, arXiv preprint arXiv:2402.06806 (2024).

[12] G. Zabërgja, A. Kadra, J. Grabocka, Tabular data: Is attention all you need?, 2024. arXiv:2402.03970.

[13] J. Fonseca, F. Bacao, Tabular and latent space synthetic data generation: a literature review, Journal of Big Data 10 (2023) 115.

[14] A. Kotelnikov, D. Baranchuk, I. Rubachev, A. Babenko, Tabddpm: Modelling tabular data with diffusion models, in: International Conference on Machine Learning, PMLR, 2023, pp. 17564–17579.

[15] H. Zhang, J. Zhang, Z. Shen, B. Srinivasan, X. Qin, C. Faloutsos, H. Rangwala, G. Karypis, Mixed-type tabular data synthesis with score-based diffusion in latent space, in: The Twelfth International Conference on Learning Representations, 2024.
[15] D. P. Kingma, M. Welling, Auto-encoding variational bayes, stat 1050 (2014) 1.

[16] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Communications of the ACM 63 (2020) 139–144.

[17] E. G. Tabak, C. V. Turner, A family of nonparametric density estimation algorithms, Communications on Pure and Applied Mathematics 66 (2013) 145–164.

[18] L. Xu, M. Skoularidou, A. Cuesta-Infante, K. Veeramachaneni, Modeling tabular data using conditional gan, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 32, Curran Associates, Inc., 2019.

[19] N. Patki, R. Wedge, K. Veeramachaneni, The synthetic data vault, in: IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2016, pp. 399–410. doi:10.1109/DSAA.2016.49.

[20] C. Ma, S. Tschiatschek, R. Turner, J. M. Hernández-Lobato, C. Zhang, Vaem: a deep generative model for heterogeneous mixed type data, Advances in Neural Information Processing Systems 33 (2020) 11237–11247.

[21] Z. Zhao, A. Kunar, R. Birke, L. Y. Chen, Ctab-gan: Effective table data synthesizing, in: Asian Conference on Machine Learning, PMLR, 2021, pp. 97–112.

[22] L. V. H. Vardhan, S. Kok, Generating privacy-preserving synthetic tabular data using oblivious variational autoencoders, in: Proceedings of the Workshop on Economics of Privacy and Data Labor at the 37 th International Conference on Machine Learning, 2020.

[23] T. Liu, Z. Qian, J. Berrevoets, M. van der Schaar, GOGGLE: Generative modelling for tabular data by learning relational structure, in: The Eleventh International Conference on Learning Representations, 2023. URL: https://openreview.net/forum?id=fPVRcJqspu.
[16] X. Fang, W. Xu, F. A. Tan, J. Zhang, Z. Hu, Y. Qi, S. Nickleach, D. Socolinsky, S. Sengamedu, C. Faloutsos, Large language models on tabular data-a survey, arXiv preprint arXiv:2402.17944 (2024).

[17] D. P. Kingma, M. Welling, Auto-encoding variational bayes, stat 1050 (2014) 1.

[18] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Communications of the ACM 63 (2020) 139-144.

[19] E. G. Tabak, C. V. Turner, A family of nonparametric density estimation algorithms, Communications on Pure and Applied Mathematics 66 (2013) 145-164.

[20] L. Xu, M. Skoularidou, A. Cuesta-Infante, K. Veeramachaneni, Modeling tabular data using conditional gan, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems (NeurIPS), volume 32, Curran Associates, Inc., 2019.

[21] N. Patki, R. Wedge, K. Veeramachaneni, The synthetic data vault, in: IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2016, pp. 399-410. doi:10.1109/DSAA.2016.49.

[22] C. Ma, S. Tschiatschek, R. Turner, J. M. Hernández-Lobato, C. Zhang, Vaem: a deep generative model for heterogeneous mixed type data, Advances in Neural Information Processing Systems 33 (2020) 11237-11247.

[23] Z. Zhao, A. Kunar, R. Birke, L. Y. Chen, Ctab-gan: Effective table data synthesizing, in: Asian Conference on Machine Learning, PMLR, 2021, pp. 97-112.

[24] E. H. Zein, T. Urvoy, Tabular data generation: Can we fool XGBoost?, in: NeurIPS 2022 First Table Representation Workshop, 2022.

[25] L. V. H. Vardhan, S. Kok, Generating privacy-preserving synthetic tabular data using oblivious variational autoencoders, in: Proceedings of the Workshop on Economics of Privacy and Data Labor at the 37 th International Conference on Machine Learning, 2020.
[24] J. Jordon, J. Yoon, M. van der Schaar, Pate-gan: Generating synthetic data with differential privacy guarantees, in: International Conference on Learning Representations, 2018. URL: https://api.semanticscho lar.org/CorpusID:53342261.

[25] N. Park, M. Mohammadi, K. Gorde, S. Jajodia, H. Park, Y. Kim, Data synthesis based on generative adversarial networks, Proceedings of the VLDB Endowment 11 (2018).

[26] R. D. Camino, C. Hammerschmidt, et al., Generating multi-categorical samples with generative adversarial networks, in: ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models, 2018.

[27] M. K. Baowaly, C.-C. Lin, C.-L. Liu, K.-T. Chen, Synthesizing electronic health records using improved generative adversarial networks, Journal of the American Medical Informatics Association: JAMIA 26 (2019) 228-241.

[28] H. Chen, S. Jajodia, J. Liu, N. Park, V. Sokolov, V. Subrahmanian, Faketables: Using gans to generate functional dependency preserving tables with bounded real data, in: 28th International Joint Conference on Artificial Intelligence, IJCAI 2019, International Joint Conferences on Artificial Intelligence, 2019, pp. 2074-2080.

[29] A. Koivu, M. Sairanen, A. Airola, T. Pahikkala, Synthetic minority oversampling of vital statistics data with generative adversarial networks, Journal of the American Medical Informatics Association 27 (2020) 1667-1674.

[30] R. Sauber-Cole, T. M. Khoshgoftaar, The use of generative adversarial networks to alleviate class imbalance in tabular data: a survey, Journal of Big Data 9 (2022) 98.

[31] D. S. Watson, K. Blesch, J. Kapar, M. N. Wright, Adversarial random forests for density estimation and generative modeling, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2023, pp. 5357-5375.
[26] T. Liu, Z. Qian, J. Berrevoets, M. van der Schaar, GOGGLE: Generative modelling for tabular data by learning relational structure, in: The Eleventh International Conference on Learning Representations, 2023. URL: https://openreview.net/forum?id=fPVRcJqspu

[27] J. Jordon, J. Yoon, M. van der Schaar, Pate-gan: Generating synthetic data with differential privacy guarantees, in: International Conference on Learning Representations, 2018. URL: https://api.semanticscho lar.org/CorpusID:53342261.

[28] N. Park, M. Mohammadi, K. Gorde, S. Jajodia, H. Park, Y. Kim, Data synthesis based on generative adversarial networks, Proceedings of the VLDB Endowment 11 (2018).

[29] R. D. Camino, C. Hammerschmidt, et al., Generating multi-categorical samples with generative adversarial networks, in: ICML 2018 work- shop on Theoretical Foundations and Applications of Deep Generative Models, 2018.

[30] M. K. Baowaly, C.-C. Lin, C.-L. Liu, K.-T. Chen, Synthesizing electronic health records using improved generative adversarial networks, Journal of the American Medical Informatics Association: JAMIA 26 (2019) 228-241.

[31] H. Chen, S. Jajodia, J. Liu, N. Park, V. Sokolov, V. Subrahmanian, Faketables: Using gans to generate functional dependency preserving tables with bounded real data, in: 28th International Joint Conference on Artificial Intelligence, IJCAI 2019, International Joint Conferences on Artificial Intelligence, 2019, pp. 2074-2080.

[32] A. Koivu, M. Sairanen, A. Airola, T. Pahikkala, Synthetic minority oversampling of vital statistics data with generative adversarial networks, Journal of the American Medical Informatics Association 27 (2020) 1667-1674.

[33] R. Sauber-Cole, T. M. Khoshgoftaar, The use of generative adversarial networks to alleviate class imbalance in tabular data: a survey, Journal of Big Data 9 (2022) 98.

[34] D. S. Watson, K. Blesch, J. Kapar, M. N. Wright, Adversarial random forests for density estimation and generative modeling, in: International
[32] M. Arjovsky, S. Chintala, L. Bottou, Wasserstein generative adversarial networks, in: International conference on machine learning, PMLR, 2017, pp. 214–223.

[33] S. Bond-Taylor, A. Leach, Y. Long, C. G. Willcocks, Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2022) 7327–7347. doi:10.1109/TPAMI.2021.3116668.

[34] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, R. Lowe, Training language models to follow instructions with human feedback, in: A. H. Oh, A. Agarwal, D. Belgrave, K. Cho (Eds.), Advances in Neural Information Processing Systems, 2022. URL: https://openreview.net/forum?id=TG8KACxEON.

[35] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, J. Dean, S. Ghemawat, Language models are unsupervised multitask learners, in: OSDI’04: Sixth Symposium on Operating System Design and Implementation, ????, pp. 137–150.

[36] A. V. Solatorio, O. Dupriez, Realtabformer: Generating realistic relational and tabular data using transformers, arXiv preprint arXiv:2302.02041 (2023).

[37] Y. Du, N. Li, Towards principled assessment of tabular data synthesis algorithms, arXiv preprint arXiv:2402.06806 (2024).

[38] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, in: International Conference on Machine Learning, PMLR, 2015, pp. 2256–2265.

[39] J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840–6851.

[40] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, Score-based generative modeling through stochastic differential equations, in: International Conference on Learning Representations, 2021. URL: https://openreview.net/forum?id=PxTIG12RRHS.
Conference on Artificial Intelligence and Statistics, PMLR, 2023, pp. 5357-5375.

[35] M. Arjovsky, S. Chintala, L. Bottou, Wasserstein generative adversarial networks, in: International conference on machine learning, PMLR, 2017, pp. 214-223.

[36] S. Bond-Taylor, A. Leach, Y. Long, C. G. Willcocks, Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2022) 7327-7347. doi:10.1109/TPAMI.2021.3116668.

[37] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, R. Lowe, Training language models to follow instructions with human feedback, in: A. H. Oh, A. Agarwal, D. Belgrave, K. Cho (Eds.), Advances in Neural Information Processing Systems, 2022. URL: https://openreview.net/forum?id=TG8KACxEON.

[38] V. Borisov, K. Sessler, T. Leemann, M. Pawelczyk, G. Kasneci, Language models are realistic tabular data generators, in: The Eleventh International Conference on Learning Representations (ICLR), 2023.

[39] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, Language models are unsupervised multitask learners, OpenAI (2019). URL: https://cdn.openai.com/better-language-models/language_mod els_are_unsupervised_multitask_learners.pdf, accessed: 2024- 11-15.

[40] A. V. Solatorio, O. Dupriez, Realtabformer: Generating realistic relational and tabular data using transformers, arXiv preprint arXiv:2302.02041 (2023).

[41] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, in: International Conference on Machine Learning, PMLR, 2015, pp. 2256-2265.

[42] J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840-6851.
[41] P. Dhariwal, A. Nichol, Diffusion models beat gans on image synthesis, Advances in neural information processing systems 34 (2021) 8780-8794.

[42] T. Karras, M. Aittala, T. Aila, S. Laine, Elucidating the design space of diffusion-based generative models, Advances in Neural Information Processing Systems 35 (2022) 26565-26577.

[43] G. Truda, Generating tabular datasets under differential privacy, arXiv preprint arXiv:2308.14784 (2023). TableDiffusion.

[44] J. Kim, C. Lee, N. Park, STaSy: Score-based tabular data synthesis (2022).

[45] C. Lee, J. Kim, N. Park, Codi: Co-evolving contrastive diffusion models for mixed-type tabular synthesis, in: Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023.

[46] A. Vahdat, K. Kreis, J. Kautz, Score-based generative modeling in latent space, Advances in neural information processing systems 34 (2021) 11287-11302.

[47] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10684-10695.

[48] R. B. Nelsen, An Introduction to Copulas, second ed., Springer, New York, NY, USA, 2006.

[49] Z. Li, Y. Zhao, J. Fu, Sync: A copula based framework for generating synthetic data from aggregated sources, in: 2020 International Conference on Data Mining Workshops (ICDMW), IEEE, 2020, pp. 571-578.

[50] S. Kamthe, S. Assefa, M. Deisenroth, Copula flows for synthetic data generation, arXiv preprint arXiv:2101.00598 (2021).

[51] D. Meyer, T. Nagler, Synthia: multidimensional synthetic data generation in python, Journal of Open Source Software 6 (2021) 2863. URL: https://doi.org/10.21105/joss.02863. doi:10.21105/joss.02863.
[43] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, Score-based generative modeling through stochastic differential equations, in: International Conference on Learning Representations, 2021. URL: https://openreview.net/forum?id=PdTIG12RRHS.

[44] P. Dhariwal, A. Nichol, Diffusion models beat gans on image synthesis, Advances in neural information processing systems 34 (2021) 8780-8794.

[45] T. Karras, M. Aittala, T. Aila, S. Laine, Elucidating the design space of diffusion-based generative models, Advances in Neural Information Processing Systems 35 (2022) 26565-26577.

[46] G. Truda, Generating tabular datasets under differential privacy, arXiv preprint arXiv:2308.14784 (2023). TableDiffusion.

[47] J. Kim, C. Lee, N. Park, STaSy: Score-based tabular data synthesis (2022).

[48] C. Lee, J. Kim, N. Park, Codi: Co-evolving contrastive diffusion models for mixed-type tabular synthesis, in: Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023.

[49] A. Vahdat, K. Kreis, J. Kautz, Score-based generative modeling in latent space, Advances in neural information processing systems 34 (2021) 11287-11302.

[50] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10684-10695.

[51] R. B. Nelsen, An Introduction to Copulas, second ed., Springer, New York, NY, USA, 2006.

[52] Z. Li, Y. Zhao, J. Fu, Sync: A copula based framework for generating synthetic data from aggregated sources, in: 2020 International Conference on Data Mining Workshops (ICDMW), IEEE, 2020, pp. 571-578.

[53] S. Kamthe, S. Assefa, M. Deisenroth, Copula flows for synthetic data generation, arXiv preprint arXiv:2101.00598 (2021).
[52] D. Meyer, T. Nagler, R. J. Hogan, Copula-based synthetic data augmentation for machine-learning emulators, Geoscientific Model Development 14 (2021) 5205-5215. URL: https://doi.org/10.5194/gmd-14-5205-2021. doi:10.5194/gmd-14-5205-2021.

[53] R. McKenna, D. Sheldon, G. Miklau, Graphical-model based estimation and inference for differential privacy, in: K. Chaudhuri, R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of *Proceedings of Machine Learning Research*, PMLR, 2019, pp. 4435-4444. URL: https://proceedings.mlr.pres.s/v97/mckenna19a.html.

[54] Z. Zhang, T. Wang, N. Li, J. Honorio, M. Backes, S. He, J. Chen, Y. Zhang, {PrivSyn}: Differentially private data synthesis, in: 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 929-946.

[55] J. Dahmen, D. J. Cook, Synsys: A synthetic data generation system for healthcare applications, Sensors (Basel, Switzerland) 19 (2019). URL: https://api.semanticscholar.org/CorpusID:75136844.

[56] B. Tang, H. He, Kerneladasyn: Kernel based adaptive synthetic data generation for imbalanced learning, in: 2015 IEEE Congress on Evolutionary Computation (CEC), 2015, pp. 664-671. doi:10.1109/CEC.2015.7256954.

[57] M. Baak, S. Brugman, I. Fridman Rojas, L. Dalmeida, R. E.Q. Urlus, J.-B. Oger, Synthsonic: Fast, probabilistic modeling and synthesis of tabular data, in: G. Camps-Valls, F. J. R. Ruiz, I. Valera (Eds.), Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of *Proceedings of Machine Learning Research*, PMLR, 2022, pp. 4747-4763. URL: https://proceedings.mlr.pres.s/v151/baak22a.html.

[58] R. McKenna, G. Miklau, D. Sheldon, Winning the nist contest: A scalable and general approach to differentially private synthetic data, Journal of Privacy and Confidentiality 11 (2021). URL: https://journa.lprivacyconfidentiality.org/index.php/jpc/article/view/778. doi:10.29012/jpc.778.
[54] D. Meyer, T. Nagler, Synthia: multidimensional synthetic data generation in python, Journal of Open Source Software 6 (2021) 2863. URL: https://doi.org/10.21105/joss.02863. doi:10.21105/joss.02863.

[55] D. Meyer, T. Nagler, R. J. Hogan, Copula-based synthetic data augmentation for machine-learning emulators, Geoscientific Model Development 14 (2021) 5205-5215. URL: https://doi.org/10.5194/gmd-14-5205-2021. doi:10.5194/gmd-14-5205-2021.

[56] R. Mckenna, D. Sheldon, G. Miklau, Graphical-model based estimation and inference for differential privacy, in: K. Chaudhuri, R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of *Proceedings of Machine Learning Research*, PMLR, 2019, pp. 4435-4444. URL: https://proceedings.mlr.pres/s/v97/mckenna19a.html.

[57] Z. Zhang, T. Wang, N. Li, J. Honorio, M. Backes, S. He, J. Chen. Y. Zhang, {PrivSyn}: Differentially private data synthesis, in: 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 929-946.

[58] J. Dahmen, D. J. Cook, Synsys: A synthetic data generation system for healthcare applications, Sensors (Basel, Switzerland) 19 (2019). URL: https://api.semanticscholar.org/CorpusID:75136844.

[59] B. Tang, H. He, Kerneladasyn: Kernel based adaptive synthetic data generation for imbalanced learning, in: 2015 IEEE Congress on Evolutionary Computation (CEC), 2015, pp. 664-671. doi:10.1109/CEC.2015.7256954.

[60] M. Baak, S. Brugman, I. Fridman Rojas, L. Dalmeida, R. E.Q. Urlus, J.-B. Oger, Synthersonic: Fast, probabilistic modeling and synthesis of tabular data, in: G. Camps-Valls, F. J. R. Ruiz, I. Valera (Eds.), Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of *Proceedings of Machine Learning Research*, PMLR, 2022, pp. 4747-4763. URL: https://proceedings.mlr.pres/s/v151/baak22a.html.

[61] R. McKenna, G. Miklau, D. Sheldon, Winning the nist contest: A scalable and general approach to differentially private synthetic data,
[59] Y. Tao, R. McKenna, M. Hay, A. Machanavajjhala, G. Miklau, Benchmarking differentially private synthetic data generation algorithms (2022).

[60] Y. Hu, F. Wu, Q. Li, Y. Long, G. Garrido, C. Ge, B. Ding, D. Forsyth, B. Li, D. Song, Sok: Privacy-preserving data synthesis, in: 2024 IEEE Symposium on Security and Privacy (SP), IEEE Computer Society, 2023, pp. 2–2.

[61] J. Jewson, L. Li, L. Battaglia, S. Hansen, D. Rossell, P. Zwiernik, Graphical model inference with external network data, cemmap working paper CWP20/22, London, 2022. URL: https://hdl.handle.net/10419/272832. doi:10.47004/wp.cem.2022.2022.

[62] A. Jolicoeur-Martineau, K. Fatras, T. Kachman, Generating and imputing tabular data via diffusion and flow-based gradient-boosted trees, in: S. Dasgupta, S. Mandt, Y. Li (Eds.), Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of *Proceedings of Machine Learning Research*, PMLR, 2024, pp. 1288–1296. URL: https://proceedings.mlr.press/v238/jolicoeur-martineau24a.html.

[63] L. Torgo, R. Ribeiro, B. Pfahringer, P. Branco, Smote for regression, volume 8154, 2013, pp. 378–389. doi:10.1007/978-3-642-40669-0_33.

[64] L. Camacho, G. Douzas, F. Bacao, Geometric smote for regression, Expert Systems with Applications 193 (2022) 1–8. doi:10.1016/j.eswa.2021.116387, camacho, L., Douzas, G., & Bacao, F. (2022). Geometric SMOTE for regression. Expert Systems with Applications, 193(May), 1–8. [116387]. https://doi.org/10.1016/j.eswa.2021.116387.

[65] T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, in: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794.

[66] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, A. Gulin, Catboost: unbiased boosting with categorical features, Advances in neural information processing systems 31 (2018).

[67] M. Platzer, T. Reutterer, Holdout-based empirical assessment of mixed-type synthetic data, Frontiers in big Data 4 (2021) 679939.
Journal of Privacy and Confidentiality 11 (2021). URL: https://journa
lprivacyconfidentiality.org/index.php/jpc/article/view/778.
doi:10.29012/jpc.778.

[62] J. Jewson, L. Li, L. Battaglia, S. Hansen, D. Rossell, P. Zwiernik, Graph-
ical model inference with external network data, cemmap working paper
CWP20/22, London, 2022. URL: https://hdl.handle.net/10419/2
72832. doi:10.47004/wp.cem.2022.2022.

[63] A. Jolicoeur-Martineau, K. Fatras, T. Kachman, Generating and im-
puting tabular data via diffusion and flow-based gradient-boosted trees,
in: S. Dasgupta, S. Mandt, Y. Li (Eds.), Proceedings of The 27th Inter-
national Conference on Artificial Intelligence and Statistics, volume 238
of Proceedings of Machine Learning Research, PMLR, 2024, pp. 1288–
1296. URL: https://proceedings.mlr.press/v238/jolicoeur-mar
tineau24a.html.

[64] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: synthetic minority over-sampling technique, Journal of artificial intelligence research 16 (2002) 321–357.

[65] L. Torgo, R. Ribeiro, B. Pfahringer, P. Branco, Smote for regression, volume 8154, 2013, pp. 378–389. doi:10.1007/978-3-642-40669-0_33.

[66] L. Camacho, G. Douzas, F. Bacao, Geometric smote for regression, Expert Systems with Applications 193 (2022) 1-8. doi:10.1016/j.eswa.2021.116387, camacho, L., Douzas, G., & Bacao, F. (2022). Geometric SMOTE for regression. Expert Systems with Applications, 193(May), 1-8. [116387]. https://doi.org/10.1016/j.eswa.2021.116387.

[67] D. Lopez-Paz, M. Oquab, Revisiting classifier two-sample tests, in: International Conference on Learning Representations, 2016.

[68] T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, in: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794.

[69] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, A. Gulin, Catboost: unbiased boosting with categorical features, Advances in neural information processing systems 31 (2018).
[68] R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, I. Stoica, Tune: A research platform for distributed model selection and training, arXiv preprint arXiv:1807.05118 (2018).

[69] J. Bergstra, D. Yamins, D. Cox, Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures, in: International conference on machine learning, PMLR, 2013, pp. 115-123.

[70] L. Li, K. Jamieson, A. Rostamizadeh, E. Gonina, J. Ben-tzur, M. Hardt, B. Recht, A. Talwalkar, A system for massively parallel hyperparameter tuning, in: I. Dhillon, D. Papailiopoulos, V. Sze (Eds.), Proceedings of Machine Learning and Systems, volume 2, 2020, pp. 230-246. URL: https://proceedings.mlsys.org/paper_files/paper/2020/file/a06f20b349c6cf09a6b171c71b88bbfc-Paper.pdf.

[71] Y. Gorishniy, I. Rubachev, A. Babenko, On embeddings for numerical features in tabular deep learning, Advances in Neural Information Processing Systems 35 (2022) 24991-25004.

[72] P. K. Dunn, G. K. Smyth, Randomized quantile residuals, Journal of Computational and graphical statistics 5 (1996) 236-244.

[73] P. Mettes, E. van der Pol, C. G. M. Snoek, Hyperspherical Prototype Networks, Curran Associates Inc., Red Hook, NY, USA, 2019.

[74] J. Demšar, Statistical comparisons of classifiers over multiple data sets, The Journal of Machine learning research 7 (2006) 1-30.

## Appendix A. Benchmark challengers Hyperparameters

Hyperparameters search space of TVAE is in Table A.6, for CTGAN in A.7, for TabSyn's VAE and MLP in A.8, for TabDDPM in A.9, for SMOTE and ucSMOTE in A.10. These tables also present the reduced search spaces suggested and applied for the experiment of Section 5.

<footnote n="10">¹⁰ClusterBasedNormalizer</footnote>
[70] M. Platzer, T. Reutterer, Holdout-based empirical assessment of mixed-type synthetic data, Frontiers in big Data 4 (2021) 679939.

[71] R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, I. Stoica, Tune: A research platform for distributed model selection and training, arXiv preprint arXiv:1807.05118 (2018).

[72] J. Bergstra, D. Yamins, D. Cox, Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures, in: International conference on machine learning, PMLR, 2013, pp. 115–123.

[73] J. Bergstra, R. Bardenet, Y. Bengio, B. Kégl, Algorithms for hyper-parameter optimization, in: J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, K. Weinberger (Eds.), Advances in Neural Information Processing Systems, volume 24, Curran Associates, Inc., 2011. URL: https://proceedings.neurips.cc/paper_files/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf.

[74] L. Li, K. Jamieson, A. Rostamizadeh, E. Gonina, J. Ben-tzur, M. Hardt, B. Recht, A. Talwalkar, A system for massively parallel hyperparameter tuning, in: I. Dhillon, D. Papailiopoulos, V. Sze (Eds.), Proceedings of Machine Learning and Systems, volume 2, 2020, pp. 230–246. URL: https://proceedings.mlsys.org/paper_files/paper/2020/file/a06f20b349c6cf09a6b171c71b88bbfc-Paper.pdf.

[75] Y. Gorishniy, I. Rubachev, A. Babenko, On embeddings for numerical features in tabular deep learning, Advances in Neural Information Processing Systems 35 (2022) 24991–25004.

[76] P. K. Dunn, G. K. Smyth, Randomized quantile residuals, Journal of Computational and graphical statistics 5 (1996) 236–244.

[77] P. Mettes, E. van der Pol, C. G. M. Snoek, Hyperspherical Prototype Networks, Curran Associates Inc., Red Hook, NY, USA, 2019.

[78] J. Demšar, Statistical comparisons of classifiers over multiple data sets, The Journal of Machine learning research 7 (2006) 1–30.
<table><caption>**Table A.6:** Hyperparameter search spaces of TVAE: extensive and reduced. "Extensive" refers to the search space used during the extensive tuning done in this paper and "Reduced" refers to the reduced experiment performed in Section 5. for the extensive and reduced tuning experiments. For the reduced experiment the number of epochs was bounded by the time budget. A description of numerical encoders can be found in Section 3.2.1 and [11].</caption><thead><tr><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td>Learning rate</td><td>qLogUniform(1e-4, 1e-2, 1e-4)</td><td>qLogUniform(1e-4, 7.3e-03, 1e-4)</td></tr><tr><td>Batch size</td><td>[100, 500, 2000]</td><td>[100]</td></tr><tr><td>Embedding dim.</td><td>[16, 32, 64, 128, 256, 512]</td><td>[16, 32, 64]</td></tr><tr><td>Encoder dim.</td><td>[64, 128, 256, 512]</td><td>[256, 512]</td></tr><tr><td>Encoder depth</td><td>[2, 3, 4]</td><td>[2]</td></tr><tr><td>Decoder dim.</td><td>[64, 128, 256, 512]</td><td>[256, 512]</td></tr><tr><td>Decoder depth</td><td>[2, 3, 4]</td><td>[2, 4]</td></tr><tr><td>Loss factor</td><td>[3, 2, 1, 0.5]</td><td>[3, 2]</td></tr><tr><td>L2 scale</td><td>qLogUniform(1e-5, 1e-4, 1e-5)</td><td>qLogUniform(1e-5, 6.3e-5, 1e-6)</td></tr><tr><td>Numerical encoder</td><td>[CDF, PLE_CDF, PTP, QuantileTransformer, MinMaxScaler, CBN<sup>10</sup>]</td><td>[CDF]</td></tr><tr><td>Categorical encoder</td><td>[one-hot-encoder]</td><td>[one-hot-encoder]</td></tr><tr><td>Epochs</td><td>[400]</td><td>∞</td></tr><tr><td>Number of trials per fold</td><td>300</td><td>50</td></tr></tbody></table>

<table><caption>**Table A.7:** Hyperparameter search spaces of CTGAN for the extensive and reduced tuning experiments. For the reduced experiment the number of epochs was bounded by the time budget.</caption><thead><tr><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td>Discriminator learning rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(4e-4, 2.1e-03, 5e-5)</td></tr><tr><td>Generator learning rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(5e-5, 1.3e-3, 5e-5)</td></tr><tr><td>Batch size</td><td>[50, 100, 250, 500, 1000]</td><td>[100, 500, 1000]</td></tr><tr><td>Embedding dim.</td><td>[32, 64, 128, 256]</td><td>[32, 128]</td></tr><tr><td>Generator dim.</td><td>[128, 256]</td><td>[128]</td></tr><tr><td>Generator depth</td><td>[2, 3, 4]</td><td>[3, 4]</td></tr><tr><td>Discriminator dim.</td><td>[128, 256]</td><td>[256]</td></tr><tr><td>Discriminator depth</td><td>[2, 3]</td><td>[2, 3]</td></tr><tr><td>Generator decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 6.4e-6, 1e-7)</td></tr><tr><td>Discriminator decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 8e-6, 1e-6)</td></tr><tr><td>Log frequency</td><td>[False, True]</td><td>[False, True]</td></tr><tr><td>Numerical encoder</td><td>[CDF, PLE_CDF, PTP, QuantileTransformer, MinMaxScaler, CBN<sup>10</sup>]</td><td>[CDF, PLE_CDF]</td></tr><tr><td>Categorical encoder</td><td>[one-hot-encoder]</td><td>[one-hot-encoder]</td></tr><tr><td>Epochs</td><td>[400]</td><td>∞</td></tr><tr><td>Number of trials per fold</td><td>300</td><td>50</td></tr></tbody></table>
[79] A. Wan, E. Wallace, S. Shen, D. Klein, Poisoning language models during instruction tuning, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023.

## Appendix A. Benchmark challengers Hyperparameters

Hyperparameters search space of TVAE is in Table A.6, for CTGAN in A.7, for TabSyn’s VAE and MLP in A.8, for TabDDPM in A.9, for smote and ucsmote in A.10. These tables also present the reduced search spaces suggested and applied for the experiment of Section 5.

<table><caption>**Table A.6:** Hyperparameter search spaces of TVAE: extensive and limited-budget benchmarks.</caption><thead><tr><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td>Learning rate</td><td>qLogUniform(1e-4, 1e-2, 1e-4)</td><td>qLogUniform(1e-4, 7.3e-03, 1e-4)</td></tr><tr><td>Batch size</td><td>[100, 500, 2000]</td><td>[100]</td></tr><tr><td>Embedding dim.</td><td>[16, 32, 64, 128, 256, 512]</td><td>[16, 32, 64]</td></tr><tr><td>Encoder dim.</td><td>[64, 128, 256, 512]</td><td>[256, 512]</td></tr><tr><td>Encoder depth</td><td>[2, 3, 4]</td><td>[2]</td></tr><tr><td>Decoder dim.</td><td>[64, 128, 256, 512]</td><td>[256, 512]</td></tr><tr><td>Decoder depth</td><td>[2, 3, 4]</td><td>[2, 4]</td></tr><tr><td>Loss factor</td><td>[3, 2, 1, 0.5]</td><td>[3, 2]</td></tr><tr><td>L2 scale</td><td>qLogUniform(1e-5, 1e-4, 1e-5)</td><td>qLogUniform(1e-5, 6.3e-5, 1e-6)</td></tr><tr><td>Numerical encoder</td><td>[CDF, PLE_CDF, PTP, QuantileTransformer, MinMaxScaler, CBN<sup>11</sup>]</td><td>[CDF]</td></tr><tr><td>Categorical encoder</td><td>[one-hot-encoder]</td><td>[one-hot-encoder]</td></tr><tr><td>Epochs</td><td>[400]</td><td>∞</td></tr><tr><td>Number of trials per fold</td><td>300</td><td>50</td></tr></tbody></table>

<footnote n="11">¹¹ClusterBasedNormalizer</footnote>
<table><caption>**Table A.8:** Hyperparameter search spaces of TabSyn's VAE and MLP for the extensive and reduced tuning experiments. For the reduced experiment the number of epochs was bounded by a time budget (10 minutes for the VAE, 10 minutes for the denoiser).</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td rowspan="10">VAE</td><td>Learning rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(1.1e-3, 7.2e-3, 5e-5)</td></tr><tr><td>Batch size</td><td>[1024, 2048, 4096]</td><td>[1024, 2048]</td></tr><tr><td>Weight decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 5e-6, 1e-6)</td></tr><tr><td>Token dim.</td><td>[2, 4]</td><td>[4]</td></tr><tr><td>Number of head</td><td>[1, 2]</td><td>[1, 2]</td></tr><tr><td>Factor</td><td>[8, 16, 32, 64]</td><td>[8, 16, 64]</td></tr><tr><td>Number of layers</td><td>[1, 2, 3, 4]</td><td>[1, 2, 4]</td></tr><tr><td>Max. beta</td><td>[1e-2]</td><td>[1e-2]</td></tr><tr><td>Min. beta</td><td>[1e-5]</td><td>[1e-5]</td></tr><tr><td>Lambda</td><td>[0.7, 0.8, 0.85, 0.9, 0.95]</td><td>[0.8, 0.85, 0.9, 0.95]</td></tr><tr><td>Epochs</td><td>[4000]</td><td>∞</td></tr><tr><td rowspan="5">MLP</td><td>Learning Rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(7.7e-4, 2.5e-3, 1e-5)</td></tr><tr><td>Weight Decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 3.3e-6, 1e-7)</td></tr><tr><td>Batch size</td><td>[1024, 2048, 4096]</td><td>[1024, 4096]</td></tr><tr><td>MLP's hidden dimension</td><td>[512, 1024]</td><td>[1024]</td></tr><tr><td>Epochs</td><td>[2000]</td><td>∞</td></tr><tr><td colspan="2">Number of trials per fold</td><td>100</td><td>50</td></tr></tbody></table>

<table><caption>**Table A.9:** Hyperparameter search spaces of TabDDPM for the extensive and reduced tuning experiments.</caption><thead><tr><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td>Batch size</td><td>[256, 4096]</td><td>[4096]</td></tr><tr><td>Dropout</td><td>[0.0]</td><td>[0.0]</td></tr><tr><td>Number of timesteps</td><td>[1000]</td><td>[1000]</td></tr><tr><td>Learning rate</td><td>qLogUniform(1e-5, 1e-3, 1e-5)</td><td>qLogUniform(3.5e-4, 9.2e-4, 1e-5)</td></tr><tr><td>Number of layers</td><td>[2, 4, 6, 8]</td><td>[2, 4, 6]</td></tr><tr><td>First layer's dim.</td><td>[128, 256, 512, 1024]</td><td>[256, 512, 1024]</td></tr><tr><td>Middle layer's dim.</td><td>[128, 256, 512, 1024]</td><td>[512, 1024]</td></tr><tr><td>Last layer's dim.</td><td>[128, 256, 512, 1024]</td><td>[256, 512, 1024]</td></tr><tr><td>Training iterations</td><td>[20000]</td><td>∞</td></tr><tr><td>Number of trials per fold</td><td>300</td><td>50</td></tr></tbody></table>

<table><caption>**Table A.10:** Hyperparameter search space of SMOTE and uCSMOTE.</caption><thead><tr><th>Parameter</th><th>Possible values</th></tr></thead><tbody><tr><td>K-Neighbors</td><td>Grid search in range [2, 20]</td></tr><tr><td>Number of trials per fold</td><td>38</td></tr></tbody></table>
<table><caption>**Table A.7:** Hyperparameter search spaces of CTGAN for the extensive and limited-budget benchmarks.</caption><thead><tr><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td>Discriminator learning rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(4e-4, 2.1e-03, 5e-5)</td></tr><tr><td>Generator learning rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(5e-5, 1.3e-3, 5e-5)</td></tr><tr><td>Batch size</td><td>[50, 100, 250, 500, 1000]</td><td>[100, 500, 1000]</td></tr><tr><td>Embedding dim.</td><td>[32, 64, 128, 256]</td><td>[32, 128]</td></tr><tr><td>Generator dim.</td><td>[128, 256]</td><td>[128]</td></tr><tr><td>Generator depth</td><td>[2, 3, 4]</td><td>[3, 4]</td></tr><tr><td>Discriminator dim.</td><td>[128, 256]</td><td>[256]</td></tr><tr><td>Discriminator depth</td><td>[2, 3]</td><td>[2, 3]</td></tr><tr><td>Generator decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 6.4e-6, 1e-7)</td></tr><tr><td>Discriminator decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 8e-6, 1e-6)</td></tr><tr><td>Log frequency</td><td>[False, True]</td><td>[False, True]</td></tr><tr><td>Numerical encoder</td><td>[CDF, PLE_CDF, PTP, QuantileTransformer, MinMaxScaler, CBN<sup>11</sup>]</td><td>[CDF, PLE_CDF]</td></tr><tr><td>Categorical encoder</td><td>[one-hot-encoder]</td><td>[one-hot-encoder]</td></tr><tr><td>Epochs</td><td>[400]</td><td>∞</td></tr><tr><td>Number of trials per fold</td><td>300</td><td>50</td></tr></tbody></table>

<table><caption>**Table A.8:** Hyperparameter search spaces of TabSyn's VAE and MLP for the extensive and limited-budget benchmarks. In the second benchmark, the number of epochs was bounded by a time budget (10 minutes for the VAE, 10 minutes for the denoiser).</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td rowspan="10">VAE</td><td>Learning rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(1.1e-3, 7.2e-3, 5e-5)</td></tr><tr><td>Batch size</td><td>[1024, 2048, 4096]</td><td>[1024, 2048]</td></tr><tr><td>Weight decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 5e-6, 1e-6)</td></tr><tr><td>Token dim.</td><td>[2, 4]</td><td>[4]</td></tr><tr><td>Number of head</td><td>[1, 2]</td><td>[1, 2]</td></tr><tr><td>Factor</td><td>[8, 16, 32, 64]</td><td>[8, 16, 64]</td></tr><tr><td>Number of layers</td><td>[1, 2, 3, 4]</td><td>[1, 2, 4]</td></tr><tr><td>Max. beta</td><td>[1e-2]</td><td>[1e-2]</td></tr><tr><td>Min. beta</td><td>[1e-5]</td><td>[1e-5]</td></tr><tr><td>Lambda</td><td>[0.7, 0.8, 0.85, 0.9, 0.95]</td><td>[0.8, 0.85, 0.9, 0.95]</td></tr><tr><td>Epochs</td><td>[4000]</td><td>∞</td></tr><tr><td rowspan="5">MLP</td><td>Learning Rate</td><td>qLogUniform(5e-5, 1e-2, 5e-5)</td><td>qLogUniform(7.7e-4, 2.5e-3, 1e-5)</td></tr><tr><td>Weight Decay</td><td>qLogUniform(1e-6, 1e-5, 1e-6)</td><td>qLogUniform(1e-6, 3.3e-6, 1e-7)</td></tr><tr><td>Batch size</td><td>[1024, 2048, 4096]</td><td>[1024, 4096]</td></tr><tr><td>MLP's hidden dimension</td><td>[512, 1024]</td><td>[1024]</td></tr><tr><td>Epochs</td><td>[2000]</td><td>∞</td></tr><tr><td colspan="2">Number of trials per fold</td><td>100</td><td>50</td></tr></tbody></table>
## Appendix B. Datasets Links

We provide the list of links toward the datasets that were used in this paper in Table B.11 below.

<table><caption>**Table B.11:** Links to the datasets</caption><thead><tr><th>Dataset</th><th>URL</th></tr></thead><tbody><tr><td>Abalone</td><td>www.openml.org/d/183</td></tr><tr><td>Adult</td><td>www.openml.org/d/1590</td></tr><tr><td>Bank</td><td>www.openml.org/d/1461</td></tr><tr><td>Black Friday</td><td>www.openml.org/d/41540</td></tr><tr><td>Bike Sharing</td><td>www.openml.org/d/42712</td></tr><tr><td>Covertype</td><td>www.openml.org/d/150</td></tr><tr><td>Cardio</td><td>www.kaggle.com/sulianova/datasets</td></tr><tr><td>Churn</td><td>www.kaggle.com/datasets/shrutimechlearn</td></tr><tr><td>Diamonds</td><td>www.openml.org/d/42225</td></tr><tr><td>HELOC</td><td>www.kaggle.com/averkiyoliabev/datasets</td></tr><tr><td>Higgs</td><td>www.openml.org/d/4532</td></tr><tr><td>House 16H</td><td>www.openml.org/d/574</td></tr><tr><td>Insurance</td><td>www.kaggle.com/datasets/mirichoi0218/insurance</td></tr><tr><td>King</td><td>www.kaggle.com/datasets/harlfoxem/housesalesprediction</td></tr><tr><td>MiniBooNE</td><td>www.openml.org/d/41150</td></tr><tr><td>Moons</td><td>scikit-learn.org/stable/modules/classes.html</td></tr></tbody></table>

## Appendix C. Dataset-Level Results for Large-Scale Experiment

Table C.12 present the per-dataset performance according to the metrics described in Section 3.1 averaged on 3-folds with 5 samples per-fold.

<table><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Model</th><th colspan="7">Metrics</th></tr><tr><th>C2ST ↓</th><th>DCR-Rate ↓</th><th>ML-Efficacy ↑</th><th>Shape ↑</th><th>Pair ↑</th><th>Train time ↓</th><th>Sample time ↓</th></tr></thead><tbody><tr><td rowspan="7">Abalone</td><td>Train Copy</td><td>0.51 ± 0.00</td><td>1.00 ± 0.00</td><td>0.23 ± 0.01</td><td>0.96 ± 0.01</td><td>0.88 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.73 ± 0.03</td><td>0.63 ± 0.02</td><td>0.17 ± 0.01</td><td>0.93 ± 0.01</td><td>0.86 ± 0.03</td><td>685 ± 398.05</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.75 ± 0.07</td><td>0.64 ± 0.05</td><td>0.23 ± 0.02</td><td>0.91 ± 0.02</td><td>0.86 ± 0.04</td><td>113 ± 3.57</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.78 ± 0.01</td><td>0.67 ± 0.01</td><td>0.23 ± 0.01</td><td>0.94 ± 0.00</td><td>0.85 ± 0.03</td><td>169 ± 28.82</td><td>03 ± 0.62</td></tr><tr><td>TabSyn</td><td>0.78 ± 0.01</td><td>0.63 ± 0.01</td><td>0.22 ± 0.01</td><td>0.94 ± 0.02</td><td>0.87 ± 0.00</td><td>1056 ± 183.20</td><td>00 ± 0.08</td></tr><tr><td>SMOTE</td><td>1.00 ± 0.00</td><td>0.85 ± 0.02</td><td>0.50 ± 0.02</td><td>0.85 ± 0.01</td><td>0.72 ± 0.01</td><td>-</td><td>00 ± 0.01</td></tr><tr><td>UC-SMOTE</td><td>0.89 ± 0.01</td><td>0.92 ± 0.02</td><td>0.51 ± 0.03</td><td>0.95 ± 0.01</td><td>0.88 ± 0.03</td><td>-</td><td>00 ± 0.00</td></tr></tbody></table>
<table><caption>**Table A.9:** Hyperparameter search spaces of TabDDPM for the extensive and limited-budget benchmarks.</caption><thead><tr><th rowspan="2">Parameter</th><th colspan="2">Possible values</th></tr><tr><th>Extensive</th><th>Reduced</th></tr></thead><tbody><tr><td>Batch size</td><td>[256, 4096]</td><td>[4096]</td></tr><tr><td>Dropout</td><td>[0.0]</td><td>[0.0]</td></tr><tr><td>Number of timesteps</td><td>[1000]</td><td>[1000]</td></tr><tr><td>Learning rate</td><td>qLogUniform(1e-5, 1e-3, 1e-5)</td><td>qLogUniform(3.5e-4, 9.2e-4, 1e-5)</td></tr><tr><td>Number of layers</td><td>[2, 4, 6, 8]</td><td>[2, 4, 6]</td></tr><tr><td>First layer's dim.</td><td>[128, 256, 512, 1024]</td><td>[256, 512, 1024]</td></tr><tr><td>Middle layer's dim.</td><td>[128, 256, 512, 1024]</td><td>[512, 1024]</td></tr><tr><td>Last layer's dim.</td><td>[128, 256, 512, 1024]</td><td>[256, 512, 1024]</td></tr><tr><td>Training iterations</td><td>[20000]</td><td>∞</td></tr><tr><td>Number of trials per fold</td><td>300</td><td>50</td></tr></tbody></table>

<table><caption>**Table A.10:** Hyperparameter search space of SMOTE and ucSMOTE.</caption><thead><tr><th>Parameter</th><th>Possible values</th></tr></thead><tbody><tr><td>K-Neighbors</td><td>Grid search in range [2, 20]</td></tr><tr><td>Number of trials per fold</td><td>38</td></tr></tbody></table>
Results per datasets and models under diverse metrics for the extensive search (continued).

<table><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Model</th><th colspan="7">Metrics</th></tr><tr><th>C2ST ↓</th><th>DCR-Rate ↓</th><th>ML-Efficacy ↑</th><th>Shape ↑</th><th>Pair ↑</th><th>Train time ↓</th><th>Sample time ↓</th></tr></thead><tbody><tr><td rowspan="6">Adult</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.71 ± 0.01</td><td>0.99 ± 0.00</td><td>0.98 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.74 ± 0.02</td><td>0.72 ± 0.00</td><td>0.67 ± 0.01</td><td>0.97 ± 0.00</td><td>0.88 ± 0.01</td><td>2755 ± 920.97</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.77 ± 0.01</td><td>0.72 ± 0.00</td><td>0.63 ± 0.03</td><td>0.96 ± 0.00</td><td>0.91 ± 0.02</td><td>1404 ± 2.70</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.65 ± 0.00</td><td>0.62 ± 0.01</td><td>0.67 ± 0.01</td><td>0.98 ± 0.00</td><td>0.95 ± 0.01</td><td>443 ± 44.97</td><td>10 ± 2.22</td></tr><tr><td>TabSyn</td><td>0.64 ± 0.01</td><td>0.62 ± 0.00</td><td>0.66 ± 0.01</td><td>0.98 ± 0.00</td><td>0.96 ± 0.01</td><td>5042 ± 2119.47</td><td>01 ± 1.06</td></tr><tr><td>SMOTE</td><td>0.93 ± 0.00</td><td>0.85 ± 0.00</td><td>0.69 ± 0.01</td><td>0.95 ± 0.00</td><td>0.90 ± 0.01</td><td>-</td><td>06 ± 0.11</td></tr><tr><td rowspan="6">Bank marketing</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.54 ± 0.01</td><td>0.99 ± 0.00</td><td>0.98 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.72 ± 0.01</td><td>0.63 ± 0.00</td><td>0.47 ± 0.02</td><td>0.98 ± 0.00</td><td>0.95 ± 0.01</td><td>3159 ± 65.25</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.81 ± 0.04</td><td>0.63 ± 0.00</td><td>0.45 ± 0.08</td><td>0.96 ± 0.01</td><td>0.92 ± 0.02</td><td>1089 ± 628.11</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.65 ± 0.01</td><td>0.61 ± 0.00</td><td>0.52 ± 0.01</td><td>0.99 ± 0.01</td><td>0.96 ± 0.01</td><td>461 ± 42.63</td><td>10 ± 1.98</td></tr><tr><td>TabSyn</td><td>0.61 ± 0.02</td><td>0.62 ± 0.01</td><td>0.49 ± 0.02</td><td>0.99 ± 0.01</td><td>0.97 ± 0.01</td><td>2783 ± 471.56</td><td>02 ± 0.03</td></tr><tr><td>SMOTE</td><td>0.79 ± 0.01</td><td>0.98 ± 0.00</td><td>0.53 ± 0.02</td><td>0.97 ± 0.00</td><td>0.95 ± 0.00</td><td>-</td><td>09 ± 0.16</td></tr><tr><td rowspan="6">Bike sharing</td><td>Train Copy</td><td>0.50 ± 0.01</td><td>1.00 ± 0.00</td><td>0.94 ± 0.00</td><td>0.99 ± 0.00</td><td>0.53 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.90 ± 0.01</td><td>0.61 ± 0.00</td><td>0.68 ± 0.02</td><td>0.97 ± 0.01</td><td>0.53 ± 0.00</td><td>3772 ± 2301.33</td><td>00 ± 0.05</td></tr><tr><td>TVAE</td><td>0.81 ± 0.01</td><td>0.61 ± 0.00</td><td>0.78 ± 0.03</td><td>0.96 ± 0.01</td><td>0.53 ± 0.00</td><td>540 ± 15.46</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.81 ± 0.01</td><td>0.62 ± 0.00</td><td>0.79 ± 0.01</td><td>0.97 ± 0.01</td><td>0.53 ± 0.00</td><td>332 ± 13.26</td><td>07 ± 0.90</td></tr><tr><td>TabSyn</td><td>0.86 ± 0.02</td><td>0.62 ± 0.01</td><td>0.51 ± 0.07</td><td>0.96 ± 0.00</td><td>0.53 ± 0.00</td><td>1843 ± 556.82</td><td>00 ± 0.01</td></tr><tr><td>SMOTE</td><td>0.98 ± 0.00</td><td>0.99 ± 0.01</td><td>0.85 ± 0.00</td><td>0.95 ± 0.00</td><td>0.67 ± 0.08</td><td>-</td><td>01 ± 0.06</td></tr><tr><td rowspan="6">Black friday</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.53 ± 0.01</td><td>1.00 ± 0.00</td><td>0.99 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.82 ± 0.02</td><td>0.66 ± 0.01</td><td>0.46 ± 0.02</td><td>0.97 ± 0.01</td><td>0.87 ± 0.00</td><td>10111 ± 8081.87</td><td>00 ± 0.12</td></tr><tr><td>TVAE</td><td>0.87 ± 0.01</td><td>0.68 ± 0.01</td><td>0.42 ± 0.02</td><td>0.95 ± 0.01</td><td>0.91 ± 0.02</td><td>4978 ± 1.32</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.87 ± 0.01</td><td>0.64 ± 0.00</td><td>0.47 ± 0.02</td><td>0.99 ± 0.01</td><td>0.98 ± 0.01</td><td>366 ± 13.15</td><td>39 ± 14.32</td></tr><tr><td>TabSyn</td><td>0.87 ± 0.01</td><td>0.68 ± 0.00</td><td>0.17 ± 0.01</td><td>0.99 ± 0.00</td><td>0.46 ± 0.00</td><td>7533 ± 563.20</td><td>06 ± 3.54</td></tr><tr><td>SMOTE</td><td>0.80 ± 0.00</td><td>0.97 ± 0.00</td><td>0.50 ± 0.01</td><td>0.95 ± 0.00</td><td>0.94 ± 0.00</td><td>-</td><td>31 ± 0.16</td></tr><tr><td rowspan="6">Cardio</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.72 ± 0.01</td><td>1.00 ± 0.00</td><td>0.98 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.62 ± 0.01</td><td>0.64 ± 0.00</td><td>0.70 ± 0.02</td><td>0.99 ± 0.01</td><td>0.96 ± 0.01</td><td>4678 ± 59.79</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.72 ± 0.02</td><td>0.64 ± 0.01</td><td>0.72 ± 0.01</td><td>0.97 ± 0.01</td><td>0.95 ± 0.02</td><td>1708 ± 984.53</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.55 ± 0.01</td><td>0.62 ± 0.00</td><td>0.72 ± 0.01</td><td>0.99 ± 0.00</td><td>0.98 ± 0.01</td><td>185 ± 73.66</td><td>13 ± 8.85</td></tr><tr><td>TabSyn</td><td>0.56 ± 0.00</td><td>0.64 ± 0.00</td><td>0.72 ± 0.00</td><td>0.99 ± 0.00</td><td>0.98 ± 0.01</td><td>2960 ± 578.82</td><td>03 ± 0.01</td></tr><tr><td>SMOTE</td><td>0.95 ± 0.00</td><td>0.98 ± 0.00</td><td>0.73 ± 0.00</td><td>0.93 ± 0.00</td><td>0.97 ± 0.01</td><td>-</td><td>00 ± 0.02</td></tr><tr><td rowspan="6">Churn</td><td>Train Copy</td><td>0.50 ± 0.01</td><td>1.00 ± 0.00</td><td>0.59 ± 0.02</td><td>0.95 ± 0.00</td><td>0.87 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.63 ± 0.02</td><td>0.64 ± 0.01</td><td>0.36 ± 0.03</td><td>0.93 ± 0.01</td><td>0.84 ± 0.02</td><td>2505 ± 953.49</td><td>00 ± 0.03</td></tr><tr><td>TVAE</td><td>0.64 ± 0.01</td><td>0.63 ± 0.00</td><td>0.51 ± 0.00</td><td>0.92 ± 0.01</td><td>0.84 ± 0.01</td><td>294 ± 6.20</td><td>00 ± 0.01</td></tr><tr><td>TabDDPM</td><td>0.66 ± 0.07</td><td>0.64 ± 0.05</td><td>0.50 ± 0.05</td><td>0.89 ± 0.06</td><td>0.82 ± 0.06</td><td>618 ± 35.47</td><td>25 ± 2.36</td></tr><tr><td>TabSyn</td><td>0.58 ± 0.02</td><td>0.61 ± 0.01</td><td>0.56 ± 0.03</td><td>0.93 ± 0.01</td><td>0.48 ± 0.00</td><td>1627 ± 340.95</td><td>00 ± 0.22</td></tr><tr><td>SMOTE</td><td>0.76 ± 0.01</td><td>0.86 ± 0.03</td><td>0.50 ± 0.02</td><td>0.87 ± 0.01</td><td>0.81 ± 0.02</td><td>-</td><td>01 ± 0.06</td></tr><tr><td></td><td>UC-SMOTE</td><td>0.78 ± 0.02</td><td>0.93 ± 0.02</td><td>0.12 ± 0.09</td><td>0.87 ± 0.01</td><td>0.80 ± 0.02</td><td>-</td><td>00 ± 0.05</td></tr></tbody></table>
## Appendix B. Datasets Links

We provide the list of links toward the datasets that were used in this paper in Table B.11 below.

<table><caption>**Table B.11:** Domains and links to the datasets</caption><thead><tr><th>Domains</th><th>Dataset</th><th>URL</th></tr></thead><tbody><tr><td>Environment</td><td>Abalone</td><td>www.openml.org/d/183</td></tr><tr><td>Social</td><td>Adult</td><td>www.openml.org/d/1590</td></tr><tr><td>Finance / Marketing</td><td>Bank</td><td>www.openml.org/d/1461</td></tr><tr><td>Marketing</td><td>Black Friday</td><td>www.openml.org/d/41540</td></tr><tr><td>Social / Environment</td><td>Bike Sharing</td><td>www.openml.org/d/42712</td></tr><tr><td>Environment</td><td>Covertype</td><td>www.openml.org/d/150</td></tr><tr><td>Health</td><td>Cardio</td><td>www.kaggle.com/sulianova/datasets</td></tr><tr><td>Finance / Marketing</td><td>Churn</td><td>www.kaggle.com/datasets/shrutimechlearn</td></tr><tr><td>Finance</td><td>Diamonds</td><td>www.openml.org/d/42225</td></tr><tr><td>Finance / Social</td><td>HELOC</td><td>www.kaggle.com/averkiyoliabev/datasets</td></tr><tr><td>Physics</td><td>Higgs</td><td>www.openml.org/d/4532</td></tr><tr><td>Finance / Social</td><td>House 16H</td><td>www.openml.org/d/574</td></tr><tr><td>Health / Insurance</td><td>Insurance</td><td>www.kaggle.com/datasets/mirichoi0218/insurance</td></tr><tr><td>Finance / Social</td><td>King</td><td>www.kaggle.com/datasets/harlfoxem/housesalesprediction</td></tr><tr><td>Physics</td><td>MiniBooNE</td><td>www.openml.org/d/41150</td></tr><tr><td>Synthetic</td><td>Moons</td><td>scikit-learn.org/stable/modules/classes.html</td></tr></tbody></table>

## Appendix C. Dataset-Level Results for Large-Scale Experiment

Table C.12 present the per-dataset performance according to the metrics described in Section 3.2 averaged on 3-folds with 5 samples per-fold.

<table><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Model</th><th colspan="8">Metrics</th></tr><tr><th>C2ST ↓</th><th>DCR-Rate ↓</th><th>ML-Efficacy ↑</th><th>Shape ↑</th><th>Pair ↑</th><th>Train time ↓</th><th>Sample time ↓</th></tr></thead><tbody><tr><td rowspan="7">Abalone</td><td>Train Copy</td><td>0.51 ± 0.00</td><td>1.00 ± 0.00</td><td>0.23 ± 0.01</td><td>0.96 ± 0.01</td><td>0.88 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.73 ± 0.03</td><td>0.63 ± 0.02</td><td>0.17 ± 0.01</td><td>0.93 ± 0.01</td><td>0.86 ± 0.03</td><td>685 ± 398.05</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.75 ± 0.07</td><td>0.64 ± 0.05</td><td>0.23 ± 0.02</td><td>0.91 ± 0.02</td><td>0.86 ± 0.04</td><td>113 ± 3.57</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.78 ± 0.01</td><td>0.67 ± 0.01</td><td>0.23 ± 0.01</td><td>0.94 ± 0.00</td><td>0.85 ± 0.03</td><td>169 ± 28.82</td><td>03 ± 0.62</td></tr><tr><td>TabSyn</td><td>0.78 ± 0.01</td><td>0.63 ± 0.01</td><td>0.22 ± 0.01</td><td>0.94 ± 0.02</td><td>0.87 ± 0.00</td><td>1056 ± 183.20</td><td>00 ± 0.08</td></tr><tr><td>SMOTE</td><td>1.00 ± 0.00</td><td>0.85 ± 0.02</td><td>0.50 ± 0.02</td><td>0.85 ± 0.01</td><td>0.72 ± 0.01</td><td>-</td><td>00 ± 0.01</td></tr><tr><td>UC-SMOTE</td><td>0.89 ± 0.01</td><td>0.92 ± 0.02</td><td>0.51 ± 0.03</td><td>0.95 ± 0.01</td><td>0.88 ± 0.03</td><td>-</td><td>00 ± 0.00</td></tr></tbody></table>
Results per datasets and models under diverse metrics for the extensive search (continued).

<table><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Model</th><th colspan="7">Metrics</th></tr><tr><th>C2ST ↓</th><th>DCR-Rate ↓</th><th>ML-Efficacy ↑</th><th>Shape ↑</th><th>Pair ↑</th><th>Train time ↓</th><th>Sample time ↓</th></tr></thead><tbody><tr><td rowspan="6">Covertype</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.90 ± 0.00</td><td>1.00 ± 0.00</td><td>1.00 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.97 ± 0.01</td><td>0.60 ± 0.01</td><td>0.70 ± 0.01</td><td>0.98 ± 0.00</td><td>0.96 ± 0.01</td><td>51644 ± 17257.41</td><td>04 ± 0.60</td></tr><tr><td>TVAE</td><td>0.90 ± 0.01</td><td>0.60 ± 0.00</td><td>0.77 ± 0.01</td><td>0.98 ± 0.00</td><td>0.96 ± 0.01</td><td>23219 ± 18238.02</td><td>01 ± 0.44</td></tr><tr><td>TabDDPM</td><td>0.94 ± 0.01</td><td>0.59 ± 0.04</td><td>0.66 ± 0.04</td><td>0.95 ± 0.01</td><td>0.91 ± 0.01</td><td>868 ± 30.61</td><td>249 ± 25.67</td></tr><tr><td>TabSyn</td><td>0.63 ± 0.02</td><td>0.62 ± 0.01</td><td>0.75 ± 0.02</td><td>0.99 ± 0.00</td><td>0.69 ± 0.00</td><td>4937 ± 1556.02</td><td>03 ± 0.05</td></tr><tr><td>SMOTE</td><td>0.97 ± 0.00</td><td>0.97 ± 0.00</td><td>0.90 ± 0.00</td><td>1.00 ± 0.00</td><td>1.00 ± 0.01</td><td>-</td><td>125 ± 0.69</td></tr><tr><td rowspan="6">Diamonds</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.98 ± 0.00</td><td>0.99 ± 0.00</td><td>0.77 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.86 ± 0.01</td><td>0.65 ± 0.01</td><td>0.94 ± 0.01</td><td>0.97 ± 0.01</td><td>0.80 ± 0.04</td><td>3389 ± 2.57</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.79 ± 0.03</td><td>0.64 ± 0.00</td><td>0.96 ± 0.01</td><td>0.94 ± 0.01</td><td>0.74 ± 0.04</td><td>1104 ± 637.85</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.71 ± 0.01</td><td>0.61 ± 0.00</td><td>0.97 ± 0.00</td><td>0.98 ± 0.01</td><td>0.70 ± 0.02</td><td>374 ± 13.27</td><td>16 ± 6.28</td></tr><tr><td>TabSyn</td><td>0.87 ± 0.01</td><td>0.65 ± 0.00</td><td>0.79 ± 0.10</td><td>0.98 ± 0.01</td><td>0.76 ± 0.04</td><td>3255 ± 360.89</td><td>02 ± 1.12</td></tr><tr><td>SMOTE</td><td>0.97 ± 0.00</td><td>0.93 ± 0.01</td><td>0.92 ± 0.01</td><td>0.97 ± 0.00</td><td>0.77 ± 0.02</td><td>-</td><td>03 ± 0.02</td></tr><tr><td rowspan="6">Heloc</td><td>Train Copy</td><td>0.50 ± 0.01</td><td>1.00 ± 0.00</td><td>0.70 ± 0.01</td><td>0.99 ± 0.00</td><td>0.97 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.92 ± 0.01</td><td>0.64 ± 0.01</td><td>0.66 ± 0.05</td><td>0.97 ± 0.01</td><td>0.94 ± 0.02</td><td>4330 ± 3393.17</td><td>00 ± 0.10</td></tr><tr><td>TVAE</td><td>0.87 ± 0.00</td><td>0.66 ± 0.01</td><td>0.70 ± 0.01</td><td>0.95 ± 0.00</td><td>0.94 ± 0.01</td><td>421 ± 249.85</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.72 ± 0.01</td><td>0.64 ± 0.01</td><td>0.70 ± 0.01</td><td>0.97 ± 0.00</td><td>0.95 ± 0.01</td><td>80 ± 9.69</td><td>01 ± 0.53</td></tr><tr><td>TabSyn</td><td>0.70 ± 0.02</td><td>0.66 ± 0.01</td><td>0.70 ± 0.02</td><td>0.97 ± 0.00</td><td>0.95 ± 0.01</td><td>1870 ± 302.36</td><td>00 ± 0.00</td></tr><tr><td>SMOTE</td><td>0.92 ± 0.01</td><td>1.00 ± 0.00</td><td>0.69 ± 0.01</td><td>0.94 ± 0.00</td><td>0.95 ± 0.01</td><td>-</td><td>00 ± 0.04</td></tr><tr><td rowspan="6">Higgs</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.74 ± 0.00</td><td>0.99 ± 0.00</td><td>0.99 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.85 ± 0.02</td><td>0.60 ± 0.00</td><td>0.70 ± 0.01</td><td>0.99 ± 0.01</td><td>0.98 ± 0.00</td><td>12696 ± 4197.49</td><td>00 ± 0.13</td></tr><tr><td>TVAE</td><td>0.92 ± 0.01</td><td>0.60 ± 0.00</td><td>0.70 ± 0.02</td><td>0.94 ± 0.01</td><td>0.98 ± 0.01</td><td>5752 ± 116.50</td><td>00 ± 0.08</td></tr><tr><td>TabDDPM</td><td>0.57 ± 0.00</td><td>0.63 ± 0.00</td><td>0.73 ± 0.00</td><td>0.99 ± 0.00</td><td>0.99 ± 0.00</td><td>223 ± 10.78</td><td>22 ± 2.25</td></tr><tr><td>TabSyn</td><td>0.57 ± 0.01</td><td>0.61 ± 0.00</td><td>0.73 ± 0.00</td><td>0.94 ± 0.00</td><td>0.99 ± 0.00</td><td>7503 ± 2581.02</td><td>04 ± 2.17</td></tr><tr><td>SMOTE</td><td>0.84 ± 0.00</td><td>1.00 ± 0.00</td><td>0.73 ± 0.00</td><td>0.96 ± 0.00</td><td>0.98 ± 0.01</td><td>-</td><td>00 ± 0.05</td></tr><tr><td rowspan="6">House 16h</td><td>Train Copy</td><td>0.50 ± 0.01</td><td>1.00 ± 0.00</td><td>0.64 ± 0.01</td><td>0.99 ± 0.01</td><td>0.99 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.84 ± 0.01</td><td>0.62 ± 0.00</td><td>0.47 ± 0.01</td><td>0.98 ± 0.01</td><td>0.97 ± 0.01</td><td>983 ± 559.75</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.82 ± 0.03</td><td>0.61 ± 0.01</td><td>0.48 ± 0.06</td><td>0.95 ± 0.01</td><td>0.98 ± 0.01</td><td>922 ± 6.68</td><td>00 ± 0.01</td></tr><tr><td>TabDDPM</td><td>0.60 ± 0.01</td><td>0.61 ± 0.00</td><td>0.61 ± 0.02</td><td>0.98 ± 0.00</td><td>0.99 ± 0.01</td><td>95 ± 21.99</td><td>02 ± 0.88</td></tr><tr><td>TabSyn</td><td>0.74 ± 0.01</td><td>0.62 ± 0.01</td><td>0.40 ± 0.02</td><td>0.97 ± 0.00</td><td>0.99 ± 0.00</td><td>2370 ± 810.03</td><td>01 ± 0.01</td></tr><tr><td>SMOTE</td><td>0.87 ± 0.00</td><td>0.86 ± 0.05</td><td>0.62 ± 0.01</td><td>0.92 ± 0.01</td><td>0.99 ± 0.00</td><td>-</td><td>00 ± 0.03</td></tr><tr><td rowspan="6">Insurance</td><td>Train Copy</td><td>0.48 ± 0.01</td><td>1.00 ± 0.00</td><td>0.85 ± 0.03</td><td>0.96 ± 0.01</td><td>0.91 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.67 ± 0.03</td><td>0.92 ± 0.01</td><td>0.71 ± 0.01</td><td>0.94 ± 0.00</td><td>0.87 ± 0.02</td><td>265 ± 23.11</td><td>00 ± 0.00</td></tr><tr><td>TVAE</td><td>0.67 ± 0.02</td><td>0.92 ± 0.01</td><td>0.77 ± 0.01</td><td>0.91 ± 0.01</td><td>0.86 ± 0.02</td><td>32 ± 4.50</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.68 ± 0.02</td><td>0.62 ± 0.01</td><td>0.83 ± 0.03</td><td>0.93 ± 0.02</td><td>0.89 ± 0.02</td><td>181 ± 18.27</td><td>03 ± 1.01</td></tr><tr><td>TabSyn</td><td>0.60 ± 0.02</td><td>0.61 ± 0.02</td><td>0.82 ± 0.00</td><td>0.94 ± 0.01</td><td>0.88 ± 0.00</td><td>1156 ± 74.54</td><td>00 ± 0.03</td></tr><tr><td>SMOTE</td><td>0.68 ± 0.01</td><td>0.80 ± 0.03</td><td>0.81 ± 0.01</td><td>0.94 ± 0.01</td><td>0.87 ± 0.01</td><td>-</td><td>00 ± 0.01</td></tr><tr><td></td><td>UC-SMOTE</td><td>0.67 ± 0.01</td><td>0.85 ± 0.02</td><td>0.80 ± 0.02</td><td>0.93 ± 0.02</td><td>0.87 ± 0.02</td><td>-</td><td>00 ± 0.00</td></tr></tbody></table>
Results per datasets and models under diverse metrics for the extensive search (continued).

<table><thead><tr><th rowspan="2">Dataset</th><th rowspan="2">Model</th><th colspan="7">Metrics</th></tr><tr><th>C2ST ↓</th><th>DCR-Rate ↓</th><th>ML-Efficacy ↑</th><th>Shape ↑</th><th>Pair ↑</th><th>Train time ↓</th><th>Sample time ↓</th></tr></thead><tbody><tr><td rowspan="6">Adult</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.71 ± 0.01</td><td>0.99 ± 0.00</td><td>0.98 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.74 ± 0.02</td><td>0.72 ± 0.00</td><td>0.67 ± 0.01</td><td>0.97 ± 0.00</td><td>0.88 ± 0.01</td><td>2755 ± 920.97</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.77 ± 0.01</td><td>0.72 ± 0.00</td><td>0.63 ± 0.03</td><td>0.96 ± 0.00</td><td>0.91 ± 0.02</td><td>1404 ± 2.70</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.65 ± 0.00</td><td>0.62 ± 0.01</td><td>0.67 ± 0.01</td><td>0.98 ± 0.00</td><td>0.95 ± 0.01</td><td>443 ± 44.97</td><td>10 ± 2.22</td></tr><tr><td>TabSyn</td><td>0.64 ± 0.01</td><td>0.62 ± 0.00</td><td>0.66 ± 0.01</td><td>0.98 ± 0.00</td><td>0.96 ± 0.01</td><td>5042 ± 2119.47</td><td>01 ± 1.06</td></tr><tr><td>SMOTE</td><td>0.93 ± 0.00</td><td>0.85 ± 0.00</td><td>0.69 ± 0.01</td><td>0.95 ± 0.00</td><td>0.90 ± 0.01</td><td>-</td><td>06 ± 0.11</td></tr><tr><td rowspan="6">Bank marketing</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.54 ± 0.01</td><td>0.99 ± 0.00</td><td>0.98 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.72 ± 0.01</td><td>0.63 ± 0.00</td><td>0.47 ± 0.02</td><td>0.98 ± 0.00</td><td>0.95 ± 0.01</td><td>3159 ± 65.25</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.81 ± 0.04</td><td>0.63 ± 0.00</td><td>0.45 ± 0.08</td><td>0.96 ± 0.01</td><td>0.92 ± 0.02</td><td>1089 ± 628.11</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.65 ± 0.01</td><td>0.61 ± 0.00</td><td>0.52 ± 0.01</td><td>0.99 ± 0.01</td><td>0.96 ± 0.01</td><td>461 ± 42.63</td><td>10 ± 1.98</td></tr><tr><td>TabSyn</td><td>0.61 ± 0.02</td><td>0.62 ± 0.01</td><td>0.49 ± 0.02</td><td>0.99 ± 0.01</td><td>0.97 ± 0.01</td><td>2783 ± 471.56</td><td>02 ± 0.03</td></tr><tr><td>SMOTE</td><td>0.79 ± 0.01</td><td>0.98 ± 0.00</td><td>0.53 ± 0.02</td><td>0.97 ± 0.00</td><td>0.95 ± 0.00</td><td>-</td><td>09 ± 0.16</td></tr><tr><td rowspan="6">Bike sharing</td><td>Train Copy</td><td>0.50 ± 0.01</td><td>1.00 ± 0.00</td><td>0.94 ± 0.00</td><td>0.99 ± 0.00</td><td>0.53 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.90 ± 0.01</td><td>0.61 ± 0.00</td><td>0.68 ± 0.02</td><td>0.97 ± 0.01</td><td>0.53 ± 0.00</td><td>3772 ± 2301.33</td><td>00 ± 0.05</td></tr><tr><td>TVAE</td><td>0.81 ± 0.01</td><td>0.61 ± 0.00</td><td>0.78 ± 0.03</td><td>0.96 ± 0.01</td><td>0.53 ± 0.00</td><td>540 ± 15.46</td><td>00 ± 0.00</td></tr><tr><td>TabDDPM</td><td>0.81 ± 0.01</td><td>0.62 ± 0.00</td><td>0.79 ± 0.01</td><td>0.97 ± 0.01</td><td>0.53 ± 0.00</td><td>332 ± 13.26</td><td>07 ± 0.90</td></tr><tr><td>TabSyn</td><td>0.86 ± 0.02</td><td>0.62 ± 0.01</td><td>0.51 ± 0.07</td><td>0.96 ± 0.00</td><td>0.53 ± 0.00</td><td>1843 ± 556.82</td><td>00 ± 0.01</td></tr><tr><td>SMOTE</td><td>0.98 ± 0.00</td><td>0.99 ± 0.01</td><td>0.85 ± 0.00</td><td>0.95 ± 0.00</td><td>0.67 ± 0.08</td><td>-</td><td>01 ± 0.06</td></tr><tr><td rowspan="6">Black friday</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.53 ± 0.01</td><td>1.00 ± 0.00</td><td>0.99 ± 0.00</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.82 ± 0.02</td><td>0.66 ± 0.01</td><td>0.46 ± 0.02</td><td>0.97 ± 0.01</td><td>0.87 ± 0.00</td><td>10111 ± 8081.87</td><td>00 ± 0.12</td></tr><tr><td>TVAE</td><td>0.87 ± 0.01</td><td>0.68 ± 0.01</td><td>0.42 ± 0.02</td><td>0.95 ± 0.01</td><td>0.91 ± 0.02</td><td>4978 ± 1.32</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.87 ± 0.01</td><td>0.64 ± 0.00</td><td>0.47 ± 0.02</td><td>0.99 ± 0.01</td><td>0.98 ± 0.01</td><td>366 ± 13.15</td><td>39 ± 14.32</td></tr><tr><td>TabSyn</td><td>0.87 ± 0.01</td><td>0.68 ± 0.00</td><td>0.17 ± 0.01</td><td>0.99 ± 0.00</td><td>0.46 ± 0.00</td><td>7533 ± 563.20</td><td>06 ± 3.54</td></tr><tr><td>SMOTE</td><td>0.80 ± 0.00</td><td>0.97 ± 0.00</td><td>0.50 ± 0.01</td><td>0.95 ± 0.00</td><td>0.94 ± 0.00</td><td>-</td><td>31 ± 0.16</td></tr><tr><td rowspan="6">Cardio</td><td>Train Copy</td><td>0.50 ± 0.00</td><td>1.00 ± 0.00</td><td>0.72 ± 0.01</td><td>1.00 ± 0.00</td><td>0.98 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.62 ± 0.01</td><td>0.64 ± 0.00</td><td>0.70 ± 0.02</td><td>0.99 ± 0.01</td><td>0.96 ± 0.01</td><td>4678 ± 59.79</td><td>00 ± 0.01</td></tr><tr><td>TVAE</td><td>0.72 ± 0.02</td><td>0.64 ± 0.01</td><td>0.72 ± 0.01</td><td>0.97 ± 0.01</td><td>0.95 ± 0.02</td><td>1708 ± 984.53</td><td>00 ± 0.02</td></tr><tr><td>TabDDPM</td><td>0.55 ± 0.01</td><td>0.62 ± 0.00</td><td>0.72 ± 0.01</td><td>0.99 ± 0.00</td><td>0.98 ± 0.01</td><td>185 ± 73.66</td><td>13 ± 8.85</td></tr><tr><td>TabSyn</td><td>0.56 ± 0.00</td><td>0.64 ± 0.00</td><td>0.72 ± 0.00</td><td>0.99 ± 0.00</td><td>0.98 ± 0.01</td><td>2960 ± 578.82</td><td>03 ± 0.01</td></tr><tr><td>SMOTE</td><td>0.95 ± 0.00</td><td>0.98 ± 0.00</td><td>0.73 ± 0.00</td><td>0.93 ± 0.00</td><td>0.97 ± 0.01</td><td>-</td><td>00 ± 0.02</td></tr><tr><td rowspan="6">Churn</td><td>Train Copy</td><td>0.50 ± 0.01</td><td>1.00 ± 0.00</td><td>0.59 ± 0.02</td><td>0.95 ± 0.00</td><td>0.87 ± 0.01</td><td>-</td><td>-</td></tr><tr><td>CTGAN</td><td>0.63 ± 0.02</td><td>0.64 ± 0.01</td><td>0.36 ± 0.03</td><td>0.93 ± 0.01</td><td>0.84 ± 0.02</td><td>2505 ± 953.49</td><td>00 ± 0.03</td></tr><tr><td>TVAE</td><td>0.64 ± 0.01</td><td>0.63 ± 0.00</td><td>0.51 ± 0.00</td><td>0.92 ± 0.01</td><td>0.84 ± 0.01</td><td>294 ± 6.20</td><td>00 ± 0.01</td></tr><tr><td>TabDDPM</td><td>0.66 ± 0.07</td><td>0.64 ± 0.05</td><td>0.50 ± 0.05</td><td>0.89 ± 0.06</td><td>0.82 ± 0.06</td><td>618 ± 35.47</td><td>25 ± 2.36</td></tr><tr><td>TabSyn</td><td>0.58 ± 0.02</td><td>0.61 ± 0.01</td><td>0.56 ± 0.03</td><td>0.93 ± 0.01</td><td>0.48 ± 0.00</td><td>1627 ± 340.95</td><td>00 ± 0.22</td></tr><tr><td>SMOTE</td><td>0.76 ± 0.01</td><td>0.86 ± 0.03</td><td>0.50 ± 0.02</td><td>0.87 ± 0.01</td><td>0.81 ± 0.02</td><td>-</td><td>01 ± 0.06</td></tr><tr><td></td><td>UC-SMOTE</td><td>0.78 ± 0.02</td><td>0.93 ± 0.02</td><td>0.12 ± 0.09</td><td>0.87 ± 0.01</td><td>0.80 ± 0.02</td><td>-</td><td>00 ± 0.05</td></tr></tbody></table>