# DC-MMD-GAN: A New Maximum Mean Discrepancy Generative Adversarial Network Using Divide and Conquer
## 4 Experiment  
In this section, we conducted experiments on unsupervised image generation to evaluate the performance of proposed DC-MMD-GANs. The experiments are conducted on two popular datasets, CIFAR-10 [36] (10 categories, \(32\times 32\)), and CelebA [37] (face images, resized and cropped to \(160\times 160\)). Models were trained on Nvidia GTX 1080 Ti GPUs. CelebA was trained on a generator with 10-layer ResNet as in [29] and a discriminator with 5-layer DCGAN architecture [2]. For CIFAR-10, we used a standard CNN structure of [38], which contains a 4-layer generator and a 7-layer discriminator. Figure 3: The distribution of deviation between the square of empirical estimate and \(\text{MMD}{}^{2}\). Besides the original model, we set the number of sub-models of DC-MMD-GANs to 2, 4, and 8. The distribution of deviation of original MMD-GANs (green curve) have high variance, which means that the loss function of MMD-GANs has loose deviation bound. The loss function of DC-MMD-GANs has relatively tight deviation bound and the bound becomes tighter as the number of sub-models increases. To compare generation quality of GANs, we adopted Frechet Inception Distance (FID) [39] and Kernel Inception Distance (KID) [14] as evaluation metrics. FID measures similarity between generated images and real images. FID fits intermediate representations of generated images and real images into two multidimensional Gaussian distributions and computes Wasserstein-2 distance between the two distributions. KID adopts a squared polynomial-kernel MMD between generated images and real images as evaluation distance. The lower scores of FID and KID, the better generation quality will be. The number of real images used in computing FID and KID was set to 10,000. According to [39], the FID \(d_{\text{FID}}(,)\) is shown as  
\[d_{\text{FID}}((m_{r},C_{r}),(m_{s},C_{s}))=\Big{(}\|m_{r}-m_{s}\|_{2}^{2}+Tr(C_ {r}+C_{s}-2(C_{r}C_{s})^{\frac{1}{2}})\Big{)}^{\frac{1}{2}} \tag{14}\]  
where \((m_{r},C_{r})\) is Gaussian distribution obtained from \(\mathbb{P}_{r}\) and \((m_{s},C_{s})\) is obtained from \(\mathbb{Q}_{s}\). According to [29], the KID \(d_{\text{KID}}(,)\) is shown as  
\[d_{\text{KID}}(R,S)=\text{MMD}_{\text{em}}{}^{2}(\psi(R),\psi(S)) \tag{15}\]  
where \(\psi(\cdot)\) is representation of generated images and real images in the Inception model [40]. For CelebA, auto-encoder was pre-trained for 20,000 iterations on 20,000 training images. We got embedding of all images by auto-encoder. According to the clusters of k-means on the learned embedding by auto-encoder, we divided 10,000 images into 2, 4 and 8 subsets. For CIFAR-10, auto-encoder was pre-trained for 10,000 iterations on 60,000 images and divided 30,000 images into 2, 4 and 8 subsets. As a comparison to DC-MMD-GANs, we also trained the standard model which is equivalent to the proposed model when \(c=1\). DC-MMD-GANs contain DC-SMMD-GAN and DC-MMD-GAN. We used 2, 4, and 8 sub-models in DC-MMD-GAN and DC-SMMD-GAN. We compared two related GANs models, MMD-GAN [14; 15] and Scaled MMD-GAN (SMMD-GAN) [29]. We limited the number of training iterations to 15,000 for CelebA to compare learning efficiency and evaluation scores between different models. Because the generating quality of DC-MMD-GANs is high when the training step is 15,000. For CIFAR-10, we limited number of training iterations to 5000 in DC-SMMD-GAN and DC-MMD-GAN. To show differences in the performance of each model clearly, we computed FID and KID every 500 generator iterations. The learning rate was set to 0.0001. The batch size of each sub-model was set to 64. According to [14], the discriminator updated 5 times during 1 generator iteration. According to [29], the dimension of discriminator outputs was set to 1 in DC-SMMD-GAN, while it was 16 in DC-MMD-GAN. DC-SMMD-GAN used a radial-basis-function kernel with bandwidth equaling to 0.5. DC-MMD-GAN used a mixture of rational quadratic kernels as in [14]. In DC-MMD-GAN, the gradient penalty and \(L^{2}\) penalty on discriminator layers were both set to 1.