# RT-GAN: GAN Based Architecture for Precise Segmentation of Railway Tracks
## 4 Experiment
### Ablation Study  
The essence of RT-GAN is to improve the accuracy of image segmentation networks through adversarial training. Compared to the usual GAN model, RT-GAN uses a loss function containing gradient information and a discriminator input. To verify the effectiveness of these improvements, this subsection compares the effects of different generators, loss functions, and discriminator inputs on RT-GAN.  
**Different generators.** DeepLabV3+ and Segformer are selected as generators for comparison tests with the proposed RT-seg. The experimental results are shown in Table 3, and the numbers in parentheses are the IoUs without adversarial training. It can be seen that (1) using RT-seg as generator is superior to using other generators; and (2) adversarial training can substantially improve the segmentation accuracy of the image segmentation network. For the Mit-b5 network, the IoU for unaugmented images (86.42%) is improved to a level similar to the baseline (88.07%), indicating that adversarial training can effectively reduce the demand of the transformer model for training sample size and improve the training efficiency.  
**Different image segmentation losses.** The effects of different loss functions on the segmentation accuracy of RT-seg and adversarial training of RT-GAN are shown in Table 4. Compared with using only \(L_{cls}\), adding \(L_{side}\) improves the IoU of the RT-seg for augmentation images by 4.64% (from 64.98% to 69.62%), while the improvement for adversarial training is smaller, with only 0.08% (augmentation, from 78.97% to 79.05%) and 0.1% (no augmentation, from 87.13% to 87.23%). The accuracy is further improved if \(L_{cls}\), \(L_{side}\) and \(L_{grad}\) are used simultaneously. Before adversarial training, the IoUs increase by 6.79% (augmentation, from 64.98% to 71.77%) and 4.68% (no augmentation, from 79.95% to 84. 63%), and after adversarial training, the IoUs increase by 2.37% (augmentation, from 78.97% to 81.34%) and 0.94% (no augmentation, from 87.13% to 88.07%).  
**Different discriminator inputs.** Table 5 shows the effect of different discriminator inputs on the segmentation accuracy of RT-GAN. If only semantic images are used as discriminator inputs, the segmentation IoUs are 79.9% (augmentation) and 83.04% (no augmentation). After adding the input image as auxiliary information to the track discriminator, the IoUs improve by 1.87% (augmentation, from 79.9% to 81.17%) and 4.7% (no augmentation, from 83.04% to 87.74%). The segmentation IoU improves to 81.34% (augmentation) and 88.07% (no augmentation) after further addition of gradient information to the discriminator input.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \(L_{cls}\) & \(L_{side}\) & \(L_{grad}\) & GAN & IoU (Aug.) & IoU (No Aug.) \\ \hline \(\surd\) & & & & 0.6498 & 0.7995 \\ \(\surd\) & \(\surd\) & & & 0.6962 & 0.7974 \\ \(\surd\) & \(\surd\) & \(\surd\) & & 0.7177 & 0.8463 \\ \hline \(\surd\) & & & & 0.7897 & 0.8713 \\ \(\surd\) & \(\surd\) & & \(\surd\) & 0.7905 & 0.8723 \\ \(\surd\) & \(\surd\) & \(\surd\) & & **0.8134** & **0.8807** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance comparison of different loss functions.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Methods** & **IoU (Aug.)** & **IoU (No Aug.)** \\ \hline baseline & **0.8134** & **0.8807** \\ DeepLab + GAN & 0.7680 (0.6169) & 0.8331 (0.7884) \\ Mit-b2 + GAN & 0.7567 (0.5086) & 0.8160 (0.7017) \\ Mit-b5 + GAN & 0.7418 (0.5079) & 0.8642 (0.7113) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performance comparison of different generators.