# Fake It Till You Make It: Guidelines for Effective Synthetic Data Generation
## 3 Materials and Methods
### Paths Tested
#### 3.1.2 Data Training Paths  
To answer questions related to hyper-parameter tuning and feature selection (Questions 2 and 3), we tested three data training cases:  
**Case 1**: Perform feature selection and hyper-parameter tuning on the real data and apply the obtained results on the generated synthetic data. **Case 2**: Perform tuning independently on real and on synthetic datasets, and
**Case 3**: Apply no tuning, instead use the default parameter values and include all features in the model (note that, this path is the one followed across all literature). In all three cases, the synthetic data is generated from raw, unprocessed real data (i.e., using the data generated in Path 2). For Case 2, given a synthetic dataset \(SD\) generated from the real training data \(DT\), we perform model selection and parameter tuning on \(SD\) using the five-fold cross validation method (Figures 1 and 2) \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Dataset** & **Short** & **Number of** & **Number of** & **Autributes** & **Categorical** & **Number of** & **Total Synthetic** & **Origin** \\
**Name** & **Name** & **Observations** & **(Predictors)** & **Predictors** & **Labels** & **Generated** & **Origin** \\ \hline BankNote & \(D_{1}\) & 1372 & 4 & 0 & 2 & 160 & UCI \\ \hline Titanic & \(D_{2}\) & 891 & 7 & 7 & 2 & 160 & Kaggle \\ \hline Ecoli & \(D_{3}\) & 336 & 7 & 0 & 8 & 160 & UCI \\ \hline Diabetes & \(D_{4}\) & 768 & 9 & 2 & 2 & 160 & UCI \\ \hline Cleveland & \(D_{5}\) & 297 & 13 & 8 & 2 & 160 & UCI \\ heart & \(D_{6}\) & 48,843 & 14 & 8 & 2 & 144 1  & UCI \\ \hline Breast cancer & \(D_{7}\) & 570 & 30 & 0 & 2 & 160 & UCI \\ \hline Dermatology & \(D_{8}\) & 366 & 34 & 33 & 6 & 160 & UCI \\ \hline SPECTF & \(D_{9}\) & 267 & 44 & 0 & 2 & 160 & UCI \\ \hline Z-Alizadeh & \(D_{10}\) & 303 & 55 & 34 & 2 & 160 & UCI \\ Sani & \(D_{11}\) & 287 & 68 & 6 & 2 & 160 & UCI \\ \hline Colposcopies & \(D_{12}\) & 841 & 71 & 3 & 2 & 160 & OpenML \\ \hline Mice Protein & \(D_{13}\) & 1080 & 80 & 3 & 8 & 160 & UCI \\ \hline Diabetic & \(D_{14}\) & 281 & 97 & 92 & 2 & 135 2  & OpenML \\ \hline Tecator & \(D_{15}\) & 240 & 124 & 0 & 2 & 160  & OpenML \\ \hline \hline \end{tabular}  
* 24 synthetic datasets were generated for DS due to time inefficiency. 2 15 synthetic datasets were generated for SDV due to time inefficiency; 3 For DS, the number of parents’ nodes allowed was changed from default to 2 due to time inefficiency.  
\end{table} 
Table 1: Datasets description. Figure 1: A representation of 5 folds cross validation method. The figure shows the first step of the method, which consists of forming sets of different hyperparameter values. The Figure is taken from [43]. Figure 2: Details of 5 folds cross validation method for a given set of hyperparameter values. Where the steps consist of: Dividing the synthetic dataset \(SD\) randomly into 5 equal subsets or folds (**A**). Then, for each hyperparameter values-set, the model is trained 5 times using a different testing (or validation) fold every time (4 folds are used as training and one is used for validation) (**A,B**). The average of the performance metrics is then reported (**A**). The hyperparameter set that gives the best average performance is chosen to produce the final model (**C**). The final model is then tested on the real testing data. The Figure is taken from [43]. Similarly, for Case 1, given the synthetic dataset \(SD\) and its corresponding real training dataset \(DT\), we perform tuning on \(DT\) using the five-fold cross validation method described above. The chosen tuning parameters are then applied to the synthetic dataset \(SD\). The obtained model is then tested on the corresponding real testing data. To answer Question 2, we compared the performance of Case 1 and Case 2. The comparison is informative on whether importing the tuning parameters of the real data is beneficial. Finally, to answer Question 3, we compared the performance of Case 2 and Case 3. The comparison is informative on whether there is any benefit in tuning synthetic data.