# Express Construction for GANs from Latent Representation to Data Distribution
## 3 Problem Statement  
GANs always want to search for a mapping from random noise to the latent space. Unfortunately, the searching will not converge or will converge to one continuous branch of the target mapping, leading to the mode collapse problem. Instead, the mapping from the latent representation to the data distribution is easier to search, and is analogous to the mapping from random noise to the latent representation. That means it is effective to tackle the mode collapse problem and potentially speed the training up. Thus, we have proposed a training methodology for GANs, named Express Construction, and provide mathematical statements as follows. We will first recall the regularity analysis for mode collapse [6]. The generator map \(g_{\theta}:(\mathcal{Z},\zeta)\longrightarrow(\sum\mu_{\theta})\) can be further decomposed into two steps,\[g_{\theta}:(\mathcal{Z},\zeta)\stackrel{{ T}}{{\longrightarrow}}( \mathcal{Z},\mu)\stackrel{{ S}}{{\longrightarrow}}(\Sigma,\mu_{\theta}) \tag{1}\]  
where \(T\) is a transportation map, mapping the noise \(\zeta\) to \(\mu\) in the latent space \(\mathcal{Z}\), \(g\) is the manifold parameterization, mapping local coordinates in the latent space to the manifold \(\Sigma\). That is to say, \(g\) provides a local chart of the data manifold, and \(T\) realizes the probability measure transformation. By manifold structure assumption, the local chart representation \(g:\mathcal{Z}\longrightarrow\Sigma\) is continuous. However, according to the regularity theory of the optimal transportation map, except in very rare situations, the transportation map \(T\) is always discontinuous. This intrinsic conflict leads to the mode collapse problem. Instead, Express Construction is able to search for more transportation maps by decomposing the procedure into two phases, making the manifold transformation in Equation (1) smoother rather than a drastic transformation from a low dimension to a high one. We now compare them in terms of training dynamics and total transportation cost, as follows. We first recall the theory of the perfect discriminator, following [16]. Definition 1: Let \(\mathcal{M}\) and \(\mathcal{P}\) be two boundary free regular submanifolds of \(\mathbb{R}^{d}\). Let \(x\in\mathcal{M}\cap\mathcal{P}\) be an intersection point of the two manifolds. \(\mathcal{M}\) and \(\mathcal{P}\) intersect transversally in \(x\) if \(T_{x}\mathcal{M}+T_{x}\mathcal{P}=T_{x}\mathbb{R}^{d}\), where \(T_{x}\mathcal{M}\) means the tangent space of \(\mathcal{M}\) around \(x\). Accordingly, \(\mathcal{M}\) and \(\mathcal{P}\) perfectly align if there is an \(x\in\mathcal{M}\cap\mathcal{P}\) such that \(\mathcal{M}\) and \(\mathcal{P}\) do not intersect transversally in \(x\). Lemma 1: Let \(\mathcal{M}\) and \(\mathcal{P}\) be two regular submanifolds of \(\mathbb{R}^{d}\) that do not perfectly align and do not have full dimension. Let \(\mathcal{L}=\mathcal{M}\cap\mathcal{P}\). If \(\mathcal{M}\) and \(\mathcal{P}\) do not have a boundary, then \(\mathcal{L}\) is also a manifold, and has a strictly lower dimension than both the one of \(\mathcal{M}\) and the one of \(\mathcal{P}\). If they have a boundary, \(\mathcal{L}\) is a union of, at most, four strictly lower dimensional manifolds. In both cases, \(\mathcal{L}\) measures 0 in both \(\mathcal{M}\) and \(\mathcal{P}\). Lemma 2: Let \(\mathbb{P}_{r}\) (data distribution) and \(\mathbb{P}_{g}\) (generated distribution) be two distributions that have support contained in two closed manifolds \(\mathcal{M}\) and \(\mathcal{P}\) that do not perfectly align and do not have full dimension. We further assume that \(\mathbb{P}_{r}\) and \(\mathbb{P}_{g}\) are continuous in their respective manifolds, meaning that if there is a set \(A\) with measure 0 in \(\mathcal{M}\), then \(\mathbb{P}_{r}(A)=0\) (and analogously for \(\mathbb{P}_{g}\)).