## 3 Generative Adversarial Network StyleGAN  
Generative Adversarial Networks were created in 2014 by [PERSON] et al. [3]. The GANs consist of two networks named Generator (\(G\)) and Discriminator (\(D\)). Both mentioned networks make GANs extremely complex and sensitive with respect to the (hyper)parameters. The reason is that \(G\) and \(D\) networks are based on a game theory and must be aligned perfectly [3; 6]. The goal of GANs is to train a generator network \(G(z;\theta^{(G)})\) that creates instances (in this case images) from data distribution, \(p_{data}(x)\), transforming the noise vectors z into samples \(x=G(z;\theta^{(G)})\). The letter \(z\) denotes the latent features of the images being generated, \(G\) is the generator, and \(\theta^{(G)}\) is the neural network model. The training signal for \(G\) is provided by the Discriminator network \(D(x)\). This network is trained to distinguish samples (images) from the distribution of the Generator \(p_{data}(x)\) from the actual data. State-of-the-art GANs generate artificial or fake images of extremely high quality. It is practically impossible to distinguish such fake images from real images by observing just the visual image characteristics [1; 2; 6; 14; 15; 16; 17]. There are many variations of GANs that seek to improve training and model convergence on test data [6]. The StyleGAN neural network was used in an experimental part of this research. This state-of-the-art network is one of the latest GANs that achieves superior results with respect to the  
Figure 2: Stanford car dataset image noise examples [7]. FID metric [1]. The StyleGAN is the evolution of the progressive GAN [2]. It was implemented using the Tensorflow framework [18]. Similarly to the progressive GAN, the StyleGAN applies progression of the image size (resolution to some extent) during training. This means that the training starts by using smaller images, by which only layers in the generator that output this specific size of images are trained. At the same time, only layers with this specific image input size are trained in the discriminator. After 8.4 M images, the training continues by using the images of full 1024 \(\times\) 1024 pixel (resolution) size from the dataset. This technique improves the performance of the training in terms of speed and stability of the GAN drastically [1,2]. The generator architecture of the StyleGAN is depicted in Figure 3. The traditional generator feeds the latent code only through the input layer, while the style generator maps the input to an intermediate latent space W, which controls the generator through adaptive instance normalization at each convolutional layer. Gaussian noise is added after each convolution. \"A\" stands for learned affine transform, and \"B\" applies learned per-channel scaling factors to the noise input. Most GANs use latent code that is provided to the generator through the input layer. However, the StyleGAN omits the input layer completely, and starts from a trained constant tensor instead (i.e., Const 4 \(\times\) 4 \(\times\) 512 in Figure 3) [1]. This network starts training by using a sample of 4 \(\times\) 4 pixels and upsamples the image progressively to the maximum size of 1024 \(\times\) 1024 pixels. The image sizes (resolutions) are denoted in the bottom right corner of each layer of the synthesis network (see Figure 3). The adaptive instance normalization (AdaIN) is used to apply the style transfer to the StyleGAN if style is needed [19]. The AdaIN is defined in Equation (1). It can be observed that each feature map \(x_{i}\) is normalized separately. The normalized feature map is then scaled and biased by using the appropriate style scalar components \(y_{s}\) and \(y_{b}\). Denotation \(\sigma(x)\) stands for normalized content input, while \(\mu(x)\) denotes a shift. The AdaIN that receives the content \(x\) and a style \(y\) as inputs simply aligns the channelwise mean and variance of \(x\) to match those of \(y\).