# Enhanced Generative Adversarial Networks with Restart Learning Rate in Discriminator
## 4 Evaluation
### Result Analysis  
Firstly, we ran our method on different data sets (CIFAR-10, single class images in CIFAR-10 and Align CelebA) to check its efficiency in generation results. Table 1 shows that we have obtained different results (measured with FID) when we performed the experiments with stable \(lr_{d}\) and restart \(lr_{d}\) (\(lr_{d}\) means the learning rate in discriminator). In the experiments, we used the same DCGAN structure but different inputs. In the first row, the input is the whole CIFAR-10, and the input in the last row is Align Celeba. We also checked the availability of our method in single class images in CIFAR-10 and present them in other rows. From the results in Table 1, we could see that the performance of the DCGAN trained with restart learning rate has shown different degrees of improvement on multi-category data (shown in the first row and the last row) and single-category data in CIFAR-10 (shown in the other rows). We also presented the corresponding visual results in Figure 3: when we trained DCGAN with stable \(lr_{d}\) on CIFAR-10, there are blurred parts in (a), while (b) has exhibited higher image resolution; on Align Celeba, almost all the faces in (c) are distorted and the use of Restart \(lr_{d}\) could decrease the faces distortion (shown in (d) of Figure 3). In conclusion, the plunge of Restart \(lr_{d}\) is beneficial to GAN in terms of generation ability. In the next experiments, we will see whether these improvements come from the improvement of networks stability. Table 2 shows the generated results (measured by FID score) by various GANs on CIFAR-10. We performed these experiments in different GANs for two purposes: (1) SNGAN and WGAN-GP are typical GANs to stable the training process, so we want to compare our method with SNGAN and WGAN-GP; (2) as our method only affects learning rate, we also want to check if it can improve the performance in SNGAN and WGAN-GP. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **With Stable \(lr_{d}\)** & **With Restart \(lr_{d}\)** \\ \hline CIFAR-10 & 69.988 & 56.735 \\ Aeroplane images in & 91.059 & 82.226 \\ CIFAR-10 & 108.720 & 101.736 \\ Barl images in CIFAR-10 & 99.202 & 88.669 \\ Cat images in CIFAR-10 & 114.093 & 82.071 \\ Deer images in CIFAR-10 & 80.956 & 63.793 \\ Dog images in CIFAR-10 & 107.805 & 94.317 \\ Frog images in CIFAR-10 & 65.511 & 58.339 \\ Horse images in CIFAR-10 & 100.841 & 85.706 \\ Ship images in CIFAR-10 & 91.693 & 79.851 \\ Truck images in CIFAR-10 & 93.601 & 73.968 \\ Align Celeba & 123.521 & 99.587 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance of DCGAN trained with stable learning rate and restart learning rate in 10,000 iterations for different datasets: (1) CIFAR-10; (2) single class in CIFAR-10 (3); Align Celeba. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & **1** & **2** & **3** & **4** \\ \hline \multirow{4}{*}{**Model**} & \multirow{2}{*}{**Updates**} & \(lr_{d}=1\times 10^{-3}\) & \multirow{4}{*}{**Restart \(lr_{d}\)**} & \(lr_{d}=1\times 10^{-4}\) & \multirow{4}{*}{**Restart \(lr_{d}\)**} \\  & & & **Stable \(lr_{d}\)** & & **Stable \(lr_{d}\)** & **Restart \(lr_{d}\)** \\  & & & & **(Mean/Std)** & **(Mean/Std)** \\ \hline
1 & DCGAN & 20 k & 164.254 & 81.641 & - & - \\
2 & 40 k & 234.214 & 75.214 & 133.241/2.730 & **58.735**/3.657 \\ \hline
3 & SNGAN & 20 k & 124.257 & 84.526 & - & - \\
4 & & 40k & 125.775 & 64.256 & 85.625/2.041 & **63.785**/3.124 \\ \hline
5 & WGAN-GP & 20 k & 62.853 & 92.772 & - & - \\
6 & & 40 k & 59.919 & 92.090 & **59.245**/2.841 & 72.836/14.040 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Generated FID results on CIFAR-10 with stable/restart learning rate in different GANs: DCGAN [2]; SNGAN [9]; WGAN-GP [23]. Bold denotes the smallest distance. Note that when we performed the experiments in Table 1, we tried to train models sufficiently (total updates = 100 k) with opposite learning rate value.