# Express Construction for GANs from Latent Representation to Data Distribution
## 1 Introduction  
Generative methods are one of the most promising approaches toward automatically learning features from a given high-dimension data distribution and then producing new samples approximating the truth [1]. Currently, the most prominent approaches are auto-regressive models [2, 3], variational auto-encoders (VAEs) [4], generative adversarial networks (GANs), and a unifying framework combining the best of VAEs and GANs like Wasserstein auto-encoders (WAE) [5]. They all have significant strengths and weakness, and among them, GANs have the most significant impact. GANs always consist of two networks: a generator network, which maps a random noise \(\zeta\) to a data distribution \(P_{r}\) ofreal ones without calculating the sample likelihood, and a discriminator network trained to assess whether the input sample from the data distribution. However, this formulation has multiple potential problems, such as training instability and mode collapse, especially the latter, which is the most urgent and difficult problem in GANs. Generators and Discriminators in GANs are always composed of deep neural networks, which can only represent continuous mappings. However, as pointed out by these works, the transport maps may be discontinuous if there are too many modes in the data distribution. This intrinsic conflict will lead to mode collapse or mode mixture [6, 7, 8]. Although a number of alternatives that can increase diversity have been proposed, the problem remains unsolved. In deep learning, the manifold distribution hypothesis is well accepted, which assumes the distribution of a specific class of natural data is concentrated on a low dimensional manifold embedded in the high dimensional data space [9]. Auto-encoders (AEs) have demonstrated the capability of learning a subspace for dimensionality reduction, which consistently attempts to find the encoding maps between the data manifold embedded in the in the image space, and the decoding maps between the data manifold embedded in the latent space [10]. Generally, the data distribution will be embedded into the latent space \(\Omega\) in AEs. Compared to random noises, the latent representation sampled from the latent space is a low-dimensional representation mapped from the real data distribution. Both the transportation map from that to the data distribution, and the one from random noise to the latent representation, are easier to search and construct. Note that this implies that it could ease up the intrinsic conflict. Thus, we tackle the mode collapse problem for GANs by decomposing the generating procedure into two phases: the generation from random noise to the latent representation and the generation from the latent representation to the data distribution. In this paper, we have proposed a training methodology for GANs named Express Construction. An auto-encoder is trained with mean absolute error (MAE) loss first to embed the image manifold \(\mathcal{X}\) into the latent manifold \(\Omega\). The distribution sampled from \(\Omega\) as the latent representation serves as the target of the first stage. To this end, a standard GAN model with a small scale is trained to map random noises into the latent representation by creating new samples that are intended to come from the same distribution as the training data. Then, the output of this model will be used as the input for the next stage. Another large-scale GAN model is trained to generate new samples close to the data distribution from the latent representation rather than random noises. We can reduce the task complexity and search for more transportation maps to benefit from indirect construction and smoother transformation. Finally, we provide theoretical steps toward understanding the training dynamics of these procedures and prove our assumptions. Extensive experiments are conducted to verify the performance and contrast with other works. Results show that Express Construction effectively alleviates the mode collapse problem and speeds the training up.