# Challenges of Using Synthetic Data Generation Methods for Tabular Microdata
## 3 Results  
Table 2 outlines the RF performance of various models, including the original, CTGAN, TVAE, CopulaGAN, and Copula, across different datasets. Results for the low-dimensional three datasets AP, AQ, and BC will be analyzed first. The original model serves as a baseline, demonstrating high accuracy across all datasets: 0.782 \(\pm\) 0.003 for AP, 0.896 \(\pm\) 0.005 for AQ and 0.880 \(\pm\) 0.006 for BC. The CTGAN model exhibits varied performance across the three datasets and different training configurations. Notably, CTGAN's performance improves with an increase in epochs and a decrease in batch size. For the AQ dataset, the accuracy starts low (worse than random guess) at \(-\)0.037 \(\pm\) 0.147 (300 epochs & 500 batch size) but significantly improves to 0.656 \(\pm\) 0.029 (10,000 epochs & 60 batch size), though this is still below the baseline results. The AP dataset shows a similar trend, with improved performance at higher \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Abbreviation**} & \multirow{2}{*}{**Name**} & **\# Train-** & **\# Test-** & **\# Numerical** & **\# Categorical** & **Task Type** \\  & & **Set (80\%)** & **Set (20\%)** & **(cont.)** & **\# Categorical** & **Task Type** \\ \hline AP & Apartment & 2345 & 587 & 6 & 12 & Regression \\ AQ & Air Quality & 661 & 166 & 13 & 0 & Regression \\ BC & Breast Cancer & 455 & 114 & 30 & 2 & Regression \\ MB & Metabric & 873 & 219 & 493 & 200 & Regression \\ MU & Musk & 380 & 166 & 168 & 1 & Classification \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Details on several UCI Machine Learning datasets used in the study. epochs and lower batch sizes, although the improvement is less pronounced compared to AQ. Interestingly, the BC dataset, despite being the smallest, achieves the best results with CTGAN, even surpassing the baseline accuracy. The accuracy reaches 0.896 \(\pm\) 0.015 (10,000 epochs & 500 batch size), indicating that CTGAN effectively learns the general data distribution. This superior performance on the BC dataset can be attributed to the smaller number of instances (455), allowing the model to learn more efficiently and accurately with extensive training. As this pattern can be observed with other models as well (TVAE and CopulaGAN), we assume that the dataset is not challenging. The TVAE model shows a more consistent performance improvement compared to CTGAN. For the AQ dataset, the accuracy starts at 0.363 \(\pm\) 0.039 (300 epochs & 60 batch size) and improves to 0.671 \(\pm\) 0.029 (10,000 epochs & 60 batch size). For the AP and BC datasets, the TVAE model consistently performs well, with accuracies reaching up to 0.707 \(\pm\) 0.014 (1000 epochs & 60 batch size) for AP and 0.899 \(\pm\) 0.019 (300 epochs & 60 batch size) for BC. This consistency highlights the robustness of the TVAE model in handling different datasets and suggests that it can effectively learn from the data with less sensitivity to parameter changes compared to CTGAN. The CopulaGAN model exhibits similar mixed results as CTGAN. For the AQ dataset, the accuracy improves from \(-\)0.069 \(\pm\) 0.111 (300 epochs & 60 batch size) to 0.682 \(\pm\) 0.029 (10,000 epochs & 60 batch size). On the AP and BC datasets, the performance fluctuates slightly less than CTGAN, with accuracies ranging from 0.524 \(\pm\) 0.031 (300 epochs & 60 batch size) to 0.672 \(\pm\) 0.017 (10,000 epochs & 500 batch size) for AP and from 0.385 \(\pm\) 0.002 (300 epochs & 500 batch size) to 0.863 \(\pm\) 0.040 (1000 epochs & 60 batch size) for BC. While CTGAN shows potential for high performance, its sensitivity to training parameters makes it less predictable. In contrast, CopulaGAN's more stable performance across different datasets and training configurations can be advantageous in real-world applications where model reliability is critical. Blending the Copula and GAN approach tends to outperform the plain Copula model, demonstrating the added value of adversarial training in capturing complex data patterns.