# CDL-GAN: Contrastive Distance Learning Generative Adversarial Network for Image Generation
## 3 Methodology: Contrastive Distance Learning
### Characteristic Contrastive Distance  
To stabilize training, we utilize Characteristic Function Distance (CFD) introduced in CFD-GAN [22] as a regularization for the discriminator. Meanwhile, the transformation vector introduced in TraVeLGAN [40], which uses a function to represent high-level semantics in some latent space, is aimed at mapping images to some space with the same relationship between the original and generated versions. TraVeLGAN gives us a feasible scheme to add the CFD regularization to the discriminator. We extract semantic information by a characteristic function and utilize a transformation vector to achieve Characteristic Contrastive Distance (ChCD). To elaborate, we firstly extract the output of the second-to-last layer as high-level semantic information when a real image or generated  
Figure 1: Illustration of the Contrastive Distance Learning-Generative Adversarial Network (CDL-GAN) framework. Part A: the generator, \(G\), consists of a Siamese module, \(S_{1}\). The discriminator, \(D\), includes a Siamese module, \(S_{2}\), and a fully connected (FC) layer. \(L_{ccd}^{G}\) means Consistent Contrastive Distance (CoCD), while \(L_{ccd}^{D}\) means Characteristic Contrastive Distance (ChCD). \(S_{1}\) shares the weights \(W_{1}\) when noise vector \(z_{1}\) and \(z_{2}\) are fed to \(G\). \(S_{2}\) shares the weights \(W_{2}\) when real and fake images pass through \(D\).  
image is encoded by the discriminator. Then, we take a characteristic function as the transformation mode and utilize the transformation vector to learn characteristic semantic information. Next, we turn characteristic semantic information into characteristic vectors with a finite-dimensional approximation in Euler space. Finally, we compute the distance between real characteristic vectors and generated characteristic vectors.  
Unlike the density function, the Euler space always exists with uniform continuity, differentiability, and boundedness. From the perspective of manifold learning, a characteristic vector is regarded as an essential element of images, and we could get the discrepancy between images by contrasting the distance between characteristic vectors. For GAN, ChCD can keep the parameters of the discriminator continuous and differentiable almost everywhere and provide a more informative signal to the discriminator for feature representations. The proofs of CFD properties are stated in CFD-GAN [21], which the reader is highly encouraged to read.  
Letting t be the input argument of the characteristic vector \(V:\{v_{1},\ldots,v_{n}\}\), the characteristic function \(\hat{\psi}\) is a weighted sum of characteristic vectors transformed in Euler space:  
\[\hat{\psi}(t)=\frac{1}{n}\sum_{j=1}^{n}e^{i(t\bigodot v_{j})} \tag{7}\]  
where \(i=\sqrt{(-1)}\), \(|e^{ix}|\leq 1\), \(t\in R^{d}\), \(\bigodot\) means vector dot product, and \(t\) is a random variable of a degenerated sampling distribution \(\delta_{V}\). Given \(X:=\{x_{1},\ldots,x_{n}\}\) and \(Y:=\{y_{1},\ldots,y_{n}\}\) with \(x_{i},y_{i}\in R^{d}\) are samples from the distributions \(\mathbb{P}\) and \(\mathbb{Q}\), respectively, and let \(t_{1},\ldots,t_{k}\) be samples from \(\delta_{V}\). We define the characteristic contrastive distance between \(V_{\mathbb{P}}\) and \(V_{\mathbb{Q}}\) as  
\[L^{D}_{ccd}(V_{\mathbb{P}},V_{\mathbb{Q}})=\frac{1}{k}\sum_{i=1}^{k}\lVert \hat{\psi}_{\mathbb{P}}(t_{i})-\hat{\psi}_{\mathbb{Q}}(t_{i})\rVert^{2} \tag{8}\]  
where \(\hat{\psi}_{\mathbb{P}}\) and \(\hat{\psi}_{\mathbb{Q}}\) are the characteristic functions of characteristic vectors, computed by \(X\) and \(Y\), respectively. With the characteristic contrastive distance, the new objective function of the discriminator can be written as follows:  
\[L^{\prime}_{D}=L_{D}+\lambda_{dis}L^{D}_{ccd} \tag{9}\]  
where \(\lambda_{dis}\) denotes the control of the importance of the \(L^{D}_{ccd}\) and \(L_{D}\) denotes the original loss.