# Fake It Till You Make It: Guidelines for Effective Synthetic Data Generation
## 5 Conclusions  
Synthetic data is still in the experimental stage and is often used to carry out preliminary research such as exploratory analyses or for the public release of data (such as census data, data used for education purposes, and data used in competitions). It is often the case that the final research analysis is obtained from the original dataset [48]. To understand whether synthetic data can be used for more than the exploratory data analysis tasks, an exploration into the utility of synthetic data needs to be done. That includes investigations into the efficiency of data synthesis methods, comparisons between synthetic data generation methods and other privacy enhancing technologies, as well as investigations into the best data generation settings/paths. While the first and second points are the subject of recent investigations (with the second being less explored but starting to get more attention), the third point has not yet been explored in prior research. As such, in this article, we investigate different synthetic data generation settings with a goal to guide data holders into the best synthetic data generation and usage path. Our results show that there is no benefit from pre-processing real data prior to synthesizing it (Q1). Switching to Machine learning models, our results show that tuning the ML models when using synthetic datasets (versus default parameters and features) does not enhances the performance of the generated models (Q3). However, importing the real data tuning-settings onto the synthetic data models is beneficial and presented an improvement in accuracy across all generators. This signifies a benefit in sharing the tuning setting of the real data along with the synthetic data, once/if the user is aware of the type of analysis to be performed on the data (Q2). Lastly, our results suggest that propensity scores are good forecasters for prediction accuracy when the synthetic data is generated using SP-p and DS synthesizers, which could be helpful for data holders in benchmarking their synthetic datasets generated via SP-p and DS (Q4). Figure 11: Correlation coefficients per synthesizer. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Case** & **Synthesizer** & **Rel Accuracy** & **AD** & **Prop** \\ \cline{2-5}  & SP-np & 96.653917 & 3.1445598 & 0.104347 (1) \\ \cline{2-5}   Synthesized from original real data & DS & 96.025506 & 3.5491086 & 0.139671 (2) \\ \cline{2-5}   Accuracy for Case 3 & SDV & 92.117844 & 7.1893318 & 0.158985 (4) \\ \cline{2-5}  & SP-p & 89.770557 & 9.60027587 & 0.141708 (3) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The Propensity, relative accuracy and accuracy difference scores for each synthesizer. The table is ordered by accuracy and the number in bracket next to propensity scores indicate their order (note that lower propensity indicates better utility). To summarize the above, these preliminary experiments suggest that there is no need to tune synthetic data when using supervised machine learning models, that it may be beneficial to share some \"privacy-safe\" pre-classification results along with the synthetic data, and that propensity score is a sufficient predictor of accuracy when using the SP-p and DS generators. It is important to note that our results are limited by our testing strategies. Some of these questions are more complex in real life than what is being tested. For example, there are more approaches for preprocessing available to analysts than what was assumed in the experiments. Our goal from this paper was to raise these pertinent questions as much as to try to tackle them from certain dimensions. More experiments should be done to assert and strengthen the obtained results with a purpose to inform on the best synthetic data pipeline. Conceptualization, F.K.D. ; methodology, F.K.D. ; software, F.K.D. and M.I. ; validation, F.K.D. and M.I. ; formal analysis, F.K.D. ; investigation, F.K.D. ; resources, F.K.D. and M.I. ; data curation, F.K.D. and M.I. ; writing--original draft preparation, F.K.D.