# Fake It Till You Make It: Guidelines for Effective Synthetic Data Generation
## 4 Results and Discussion  
In this section we elaborate on the experiments followed to answer each question, and we present the results of these experiments. The section is organized into 4 subsections, one for each question. Question 1: Does Pre-Processing Real Data Prior to the Generation of Synthetic Data Improve the Utility of the Generated Synthetic Data? To answer this question, we compare the propensity score of data synthesized from raw real data (Path 1) using the four generators, versus the propensity score of data synthesized from preprocessed real data (Path 2). The results are shown in Figures 3 and 4. As evident from Figure 3, Propensity scores display a preference to data synthesized from raw real datasets across all synthesizers and particularly for the SP synthesizers. When we take the average propensity score across all datasets (Figure 4), this conclusion becomes more evident. Thus, overall, propensity scores display better results when data is synthesized from raw real datasets. This result supports what has been followed in the literature, i.e., synthesizing from original raw data, and it also supports the desires of data scientists to have a dataset that looks like the original, and offers them the power to pre-process as they wish. Figure 4: Propensity scores per synthesizer (across datasets). Figure 3: Propensity scores per dataset across all classification algorithms. The x-axis represents the different datasets ordered according to their attributes number \(d\). Question 2: When Calculating Prediction Accuracy, Does Importing the Real Data Tuning Settings to the Generated Synthetic Data Lead to Improved Accuracy Results? The aim of this question is to guide data holders on whether to include additional information when sharing synthetic data. Such information can be shared at a later stage, once scientists have a decision regarding the methods and algorithms to run on the data. To answer this question, we study the relative accuracy of the synthetic datasets in Case 1 versus Case 2 using the four aforementioned machine learning algorithms. The results are shown in Figures 5-7. Relative accuracy represents the synthetic data accuracy relative to the real data accuracy (100\(\times\) synthetic accuracy/real accuracy). Figure 5: Relative accuracy for each synthesizer across all datasets for cases 1 and 2. The \(x\)-axis represents the ML model. Figure 6: Relative accuracy across all synthesizers and all datasets for each ML model. Figure 5 shows that case 1 offers better accuracy across all ML algorithms and for all synthesizers. Figure 6 shows the results from a different perspective. It displays the accuracy average across the four synthesizers, it shows an overall increase in accuracy when real data tuning is imported. When taken per synthesizer (Figure 7), and as expected, Case 1 shows an overall better accuracy across all synthesizers. Question 3: Should We Apply Tuning on the Synthetic Datasets Generated from Raw Unprocessed Data? Or Would Non-Tuning Lead to a Better Accuracy? In general, parameter tuning and feature selection is essential to achieve high quality models. It is often the case that applying \"out-of-the-box\" machine learning algorithms to the data leads to inferior results [46]. However, with synthetic data, parameters are optimized using the synthetic data itself and final utility is tested on the real data. Therefore, if the synthetic data is generated from an inaccurate or biased model, the optimized parameters could further this bias, leading to low utility (on real data). As tuning is a highly time-consuming activity, it is essential to guide scientists working with synthetic data on the value of tuning their datasets. To best answer this question, we study the relative accuracy of the machine learning models with and without tuning (in other words we compare Case 2 and Case 3). For the non-tuned dataset, training is performed using the default parameters. The results of the experiments are shown in Figures 8-10.