## 6 Experiments
### Architecture and Training Tricks in Combination with SoftGAN  
The previous experiments have confirmed the benefits of the SoftGAN in terms of its ability to explore modes within both simple and complex datasets. Apart from the mode coverage, the quality of generated samples is also crucial in evaluating generative models. Consequently, this section aims to quantitatively compare the variation in generation quality between the SoftGAN and other GANs using three datasets: CIFAR-10 [30], STL-10 [45] and CelebA [46]. To assess the quality, we employ the widely recognized Frechet Inception Distance [47] (FID) as the metric. The FID measures the dissimilarity between the distribution of generated images and the distribution of real images used for training. Unlike pixel-based comparisons, the FID evaluates the mean and standard deviation of the output from the intermediate layer of Inception v3 [48], which approximates the human perception of image similarity. Over the years, the FID has become the standard metric for evaluating GAN quality. As previously mentioned, our dynamic borderline softening mechanism is completely independent of other generator architectures and training techniques used in GANs. To demonstrate the versatility of our approach, we conducted experiments by integrating it into other models. Specifically, we compared GAN models using three different architecture combinations: linear generator + linear discriminator, ResNet generator + skip discriminator and style generator + linear discriminator. The network architectures of the various components are depicted in Figure 10, and the detailed FID comparison results are presented in Table 6.  
Now, we proceed to examine the experimental outcomes utilizing identical network architectures and adversarial loss. Following 100 epochs of training, we present the FID scores achieved by our SoftGAN model across the CIFAR-10 [30], STL-10 [45] and CelebA [46] datasets. We exclusively incorporate models trained in a fully unsupervised manner to ensure a fair comparison. In summary, our proposed model surpasses the baseline models and achieves a state-of-the-art performance within the same architectural configurations. \begin{table}
\begin{tabular}{l r r r} \hline \hline
**Model** & \multicolumn{3}{c}{**FID**} \\  & **CIFAR-10 [30]** & **STL-10 [45]** & **CelebA [46]** \\ \hline Dropout-GAN [24] & 88.6 & & 36.36 \\ DCGAN [2] & 37.7 & & \\ DCGAN+TUR [47] & 36.9 & & \\ QSNGAN [49] & 31.966 & 59.611 & 29.417 \\ WGAN-GP [14] & 29.3 & 55.1 & \\ SNGAN [linear G + linear D) & 29.3 & 53.1 & \\ MGAN [37] & 26.7 & & \\ SoftGAN (linear G + linear D) & **21.34** & **51.67** & **10.33** \\ \hline WGAN-GP+TUR [47] & 24.8 & & \\ SNGAN [ResNet G + skip D) [23] & 21.7 & 40.1 & \\ PeerGAN [36] & 21.55 & 51.37 & 13.95 \\ BigGAN [42] & 14.73 & & \\ AutoGAN [50] & **12.42** & 31.01 & \\ SoftGAN (ResNet G + skip D) & 12.50 & **30.87** & **6.65** \\ \hline StyleGAN2 [21] & 11.07 & & 5.06 \\ LT-GAN [51] & 9.80 & 31.35 & 16.84 \\ SoftGAN (style G + linear D) & **3.07** & **23.92** & **2.79** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: FIDs (where lower values indicate better performance) are reported for three datasets across various network architectures. The best performance achieved within each network architecture is highlighted in bold. Linear generator + linear discriminator: Our architecture closely adheres to the principles of DCGAN [2] for an optimal performance. To ensure effective adversarial learning, we employ hinge loss, which aligns with the SNGAN framework [23]. Impressively, our approach achieves a remarkable FID score of 21.34 on the CIFAR-10 [30] dataset, surpassing other GAN models utilizing the same linear architecture. This includes single-discriminator models, like DCGAN [2], WGAN-GP [14], QSNGAN [49] and SNGAN [23], as well as multi-discriminator models, like Dropout-GAN [24] and MGAN [37]. However, the advantage of the proposed approach is that the proposed approach is not only applicable to the training data, but also to the training data.