# CDL-GAN: Contrastive Distance Learning Generative Adversarial Network for Image Generation
## 1 Introduction  
Generative Adversarial Networks (GANs) [1] have shown incredible success as effective data-driven models for image synthesis, but have become exposed to inevitable training obstacles. To find the theoretic Nash equilibrium with non-convex objective functions, GAN needs to exploit image information from a continuous and high-dimensional parameter space. Because GAN training is substantially more complicated than a standard neural network, it is challenging to keep the training stable. As a result, the outputs of a generative model frequently become uncontrollable and are of poor quality. To handle these challenges, many solutions have been introduced to improve GANs' performance. In recent times, numerous proposals for better designs and optimization of basic GANs have been reported. [PERSON] and [PERSON] [2], [PERSON] et al. [3], [PERSON] et al. [4] proposed a re-engineered network architecture based on conditional generation. Conditional GANs (cGANs) learn a conditional probability distribution from auxiliary information about real data. [PERSON] et al. [5], [PERSON], [PERSON] et al. [6], and [PERSON] et al. [7] modeled generative-discriminative network pairs to increase the generation capacity of the generator. With multiple generators or discriminators, GANs can get more constructive gradient signals to learn intermediate representation. [PERSON] et al. [8], [PERSON] et al. [9], [PERSON] et al. [10], [PERSON], [PERSON] [11], and [PERSON], [PERSON] et al. [12] use the most common encoder-decoder architecture to learn image features from latent space. These hybrid models are useful for addressing mode collapse. In the face of the oscillation in model parameters, previous work by [PERSON] et al. [13], [PERSON] et al. [14], [PERSON] et al. [15], and [PERSON] et al. [16] utilized appropriate loss functions to tacklestability issues. Some added interventions such as normalization [13; 14] and regularization [17; 18] to the discriminator [19], the generator [20] or both together [21]. Others found new probability distance metric [15; 16; 22] to replace JS divergence. Concerning the optimization algorithm, several researchers proposed the use of another gradient descent optimization technique [23; 24], or modified the training technique [25; 26]. Within the solutions mentioned above, diverse characteristic GANs variants have made a difference to some extent in the GANs literature, but some issues still remain unsolved. In practice, these methods more or less result in poor image quality due to training instability or mode collapse. In this work, we undertake a comprehensive and effective approach using Contrastive Distance Learning (CDL) to make GANs perform better. Motivated by Improved Consistency Regularized GAN (ICR-GAN) [21] and Mode Seeking GAN (MS-GAN) [20], we present Consistent Contrastive Distance (CoCD) to modulate the sensitivity of the generator with prior changes in the noise. In light of the work of [PERSON] [22] and [PERSON] [27], we establish additional Characteristic Contrastive Distance (ChCD) to capture more informative image features for the discriminator. The CoCD is aimed at mitigating mode collapse to improve the training stability, while the ChCD forces the discriminator to remember more useful high-level semantics to further improve image synthesis quality. In particular, to alleviate computational cost with two additional auxiliary losses, we design our framework with the Siamese modules. In our experiments, we conduct comparisons on CDL-GAN across the existing optimized GAN models for three public datasets. CDL-GAN yields state-of-the-art image synthesis results among the existing models. In extensive qualitative and quantitative studies, we show that our work offers multi-faceted improvements. It achieves lower Frechet Inception Distance (FID) [28] scores under the same training and evaluation conditions with different datasets. Meanwhile, it works well across a large range of GAN models with different hyperparameter sets of the Adam optimizer.