# GAN-Based Approaches for Generating Structured Data in the Medical Domain
## 2 Methods
### Data, Experimental Setup, and Evaluation Framework
#### 2.2.2 Experiments and Evaluation Framework  
The given dataset (here, BCW or BCC) is first divided into 70% Train and 30% Test datasets. The Train data are used to teach the generative models, i.e., to train the generator and discriminator networks in each GAN model to generate synthetic data. Once the data are generated by GAN variants, their quality needs to be evaluated to determine whether their statistics could reflect real data. To this end, we have developed an evaluation framework that measures the accuracy of a binary classification (malignant/benign) trained on a combination of synthetic and real data. A schematic overview of the evaluation method is illustrated in Figure 2. The Test dataset is kept separate and used only to verify the quality of the generated data via measuring the accuracy of the classifiers. The Train data, together with synthetic data, however, are used to train the classifiers. To investigate the impact of the size of the training data on the accuracy of the classifiers, the Train dataset is divided into several subsets for each given dataset. The BCW is divided into 10 subsets containing 5, 6, 7, 8, 9, 10, 20, 30, 40, or 50% of the Train data, respectively, while the BCC is split into nine subsets, each consisting of 10, 20, 30, 40, 50, 60, 70, 80, and 90% of the Train data. Moreover, two different sampling strategies are used for each dataset, random selection (RS) and random extension (RE). In the RS, data for each subset are randomly and independently selected from the Train dataset, while in the RE, the smallest subset is first randomly selected from the Train dataset and then extended by adding more random data to create a larger dataset with a desired size. In this way, the larger subsets contain all the subsets generated in previous steps of the RE. The goal of the RS strategy is to test a complete random selection where random effects on the results are expected. The RE strategy, on the other hand, is concerned with how much additional data should be added to an existing data subset to obtain reliable and robust results. The latter strategy is particularly important in the medical field to evaluate improved model  
Figure 2: Schematic overview of the evaluation of data generated by GAN variants: Data are divided into Train and Test, the GAN models generate synthetic data based on the Train data; Train and generated data are combined into an extended dataset. The classifier is trained once with only the original Train data (Silver Standard), and then using the extended dataset (including generated data) corresponding to each GAN variants. The classifiers are in the end evaluated with Test data. performance (in this case, classification accuracy) by incorporating newly available data when limited data are present, such as for rare diseases. The GAN models are trained using each data subset with RS or RE sampling to generate synthetic data. The same amount of generated data are then added to the original data subset, and the classifier is eventually trained using this larger extended dataset. For each setup, the experiments are performed on 10 random samples, and finally, the results of different samples are averaged, and the statistical mean accuracy and standard error are measured. Data generated by all GAN variants were benchmarked using two different classification methods, namely Support Vector Machines (SVM) and classical neural networks, Multi-Layer Perceptron (MLP). Studies on the performance of different classification models for the BCW dataset have shown that the SVM and MLP methods outperform other classifiers in terms of accuracy [62; 63; 64]. While the MLP has the highest accuracy in most cases and is superior for large and complex datasets, SVM performs best in a small sample setting when there are only few data points with many features, which is the case in this study [62; 65]. We note that to measure the quality of generated synthetic data, one should consider other statistical metrics to better compare the real and synthetic distributions.