# Synthesize and Segment: Towards Improved Catheter Segmentation via Adversarial Augmentation
## 4 Experiments
### Quantitative Results  
This section provides quantitative analysis of the segmentation models trained with synthetic images. In Table 1, the performance of several state-of-the-art-methods (SOTA) evaluated on 50 real X-ray test images are presented in terms of Dice scores. Among the evaluated methods, Linknet trained with 140 labelled images showed the most improved performance (0.82960). Additionally, Linknet achieved a higher dice score when trained with labeled images and an additional generated images. As for the images generated by our catheter synthesis method, U-Net reports the highest dice score and largely outperforms the rest i.e., PSPNet, PAN, and Linknet. Based on Table 2, we noted that U-Net, PAN, and LinkNet report significant performance improvements when trained in mixed settings i.e., both camera and synthetic catheter samples with 140 annotated images, as compared to models trained with synthetic and annotated images only. Notably, camera samples enable the models to learn more complex and thin structures, and consequently result in an overall efficiency improvement when combined. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Training Images** & **Dice Score** \\ \hline \multirow{3}{*}{UNet [26]} & 140 Labeled Images (Baseline) & 0.8156 \\  & 30,000 Synthetic catheter (Generated Images) & 0.8023 \\  & 140 Labeled Images + 30,000 Synthetic catheter (Generated Images) & 0.8595 \\ \hline \multirow{3}{*}{PSPNet [43]} & 140 Labeled Images & 0.7589 \\  & 30,000 Synthetic catheter (Generated Images) & 0.7455 \\  & 140 Labeled Images + Synthetic catheter (Generated Images) & 0.8133 \\ \hline \multirow{3}{*}{PAN [44]} & 140 Labeled Images & 0.8072 \\  & 30,000 Synthetic catheter (Generated Images) & 0.7954 \\  & 140 Labeled Images + 30,000 Synthetic catheter (Generated Images) & 0.8671 \\ \hline \multirow{3}{*}{Linknet [42]} & 140 Labeled Images & 0.8296 \\  & 30,000 Synthetic catheter (Generated Images) & 0.8044 \\ \cline{1-1}  & 140 Labeled Images + 30,000 Synthetic catheter (Generated Images) & 0.8806 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of segmentation models with labeled and generated images. All of the segmentation models are evaluated on real catheter X-Ray test set. Figure 5(a) presents additional experiments with data augmentation included to confirm whether the synthetic images are useful. We selected subsets of the generated images sequentially to train the U-Net model to verify the effect of increasing the number of generated images on performance. The augmented data samples were created using transformations [47], such as scaling (from 0.5 to 1.5 ratio), horizontal flipping, blurring with gaussian filters, elastic deformations with different scaling factors and elasticity coefficients [48], as well as random rotations on each input image. We augmented 140 labeled images to 1000. The U-Net [26] model trained with augmented images achieved a higher dice score compared to the model using 1000 generated images. However, when we increased the number of the generated images to 5000, both models trained using traditional augmentation and synthetic images showed similar performance. Furthermore, even better results are obtained by training the segmentation model with 30,000 generated images. However, the performance saturated when trained with 40,000 images. We assume that saturation in performance occurs because of the error accumulation of inconsistently produced images. Additional experiments were performed to further analyze whether camera catheter images indeed enhance performance (see Figure 5(b)) by considering a setting with equal splits of samples i.e., a 50-50 split for camera and synthetic images generated from the labeled set. Herein, [PERSON] was trained on the aforementioned split.