# Synthetic Medical Imaging Generation with Generative Adversarial Networks for Plain Radiographs
## 2 Materials and Methods
### Experimentation  
#### 2.5.1 Initial Experimentation  
For each dataset, the GIST pipeline was utilized to train models, generate synthetic images, and evaluate the performance of the GAN. While the platform is agnostic to the GPU environment used, the pipeline ran on a high-performance compute environment using SLURM, an open-source job scheduler for Linux and Unix-like kernels. Training times varied depending on the complexity of the data being used but could take two days for models trained on simpler data (e.g., KneeXrayOA-simple) to reach convergence, while it could take up to a week for more complex data (e.g., MURA). This high-performance compute environment leveraged GPUs in a SLURM environment.  
An example training process for the KneeXrayOA dataset can be seen in Figure 2. The generator begins by presenting a random collection of pixels to the discriminator. As the  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Name of Dataset** & **Location** & **Type of Data** & **Image Count** & & **Shortcomings/Challenges** \\ \hline KneeXrayOA-simple & Kaggle & Osteoarthritic Knee X-rays (JPG) & \(10\)k & \(\bullet\) & Similarity of images \\  & & & X-rays (JPG) & \(\bullet\) & Lack of labels (e.g., medical implants) \\  & & & & \(\bullet\) & Light saturation \\ \hline MURA & Stanford & Musculoskeletal Radiographs (PNGs) & \(5\)k (elbow) & \(\bullet\) & Inconsistent positioning of X-ray \\  & & & & \(\bullet\) & Lack of labels \\ \hline UMD Elbow & University of Maryland & Elbow X-rays (DICOM) & \(<\)1k & \(\bullet\) & Dataset Size \\  & & & & \(\bullet\) & Inconsistent positioning of X-ray \\ \hline \hline \end{tabular}
\end{table}
Table 1: Datasets used for training.  
error is backpropagated, the model improves, and one begins to see the X-ray of the knee form. Training continues to progress until convergence is reached and a fairly realistic knee X-ray remains.  
As training progressed, the fid50_full score was leveraged, which calculated the FID score for the current model against the training set of images. Eventually, convergence was reached. In addition to observing the fid50_full score, synthetic images from the models at the point of convergence were generated and manually inspected for artifacts. Depending on model performance, the hyperparameters were tuned, data were added, and data augmentation was performed (e.g., increasing image count by flipping the images and including the flipped version) in attempts to improve the model performance.  
While the two public datasets used in this study had sufficient data for training, medical image datasets are often small in size due to the issues previously mentioned. Therefore, future users of the pipeline may not have access to as much data for training. As such, an investigation was performed to determine if a threshold (minimal) amount of data was required for convergence of the fid50_full score and a lack of artifacts. In other words, what minimal training dataset size is required for successful image generation. This could inform future users if they had enough data to pursue use of the pipeline. To establish this threshold, the same experiment was run for various training dataset sizes and the performance was evaluated. Due to the use of the PyTorch package, it was not guaranteed that two training runs, trained on the same dataset with the same hyperparameter values, would converge to the exact same result. This implies that a direct comparison between runs may include slightly different results, an implication which was validated with the later analysis. However, the overall comparison is still useful as the variability between the same runs was less than the difference between runs with varying amounts of data or different hyperparameters.