# Evaluation of StyleGAN-CLIP Models in Text-to-Image Generation of Faces
## 2 Review of Generative Models
### Review of StyleGAN Models  
In this section, we briefly review the evolution of the StyleGAN models.  
StyleGAN: The style-based generator architecture was introduced by [PERSON] et al. in 2019 [18]. The idea was to improve the traditional GAN generator in its ability to generate and interpolate face images with high fidelity and high-quality resolution. The novel GAN model can control both the coarse-grained (pose, face form) and fine-grained (eyes, hair) details. This is completed by projecting the noise vector into an intermediate latent space, which is then fed to the synthesis network. In spite of its powerful ability to generate realistic images, StyleGAN outputs suffer mainly from three issues [19]: (1) water droplet artifacts, (2) smoother with less texture detail images caused by the nature of the cost function, and (3) phase artifacts produced due to the adopted progressive growing strategy. StyleGAN was improved in later versions.  
StyleGAN2: To address the above-mentioned StyleGAN issues, the second variant StyleGAN2 [19] was proposed in 2020. A set of revisions in the generator normalisation form was made to avoid the appearance of water droplet artifacts, including weighted demodulation integration. Smoother outputs were obtained after adding a new regularisation term to the loss function. To resolve the phase artifacts problem in StyleGAN, the progressive growing strategy model was revised and replaced with skip connections in the generator and residual layers in the discriminator. Furthermore, the training process was revised by training larger models, which helps to improve the image quality generation.  
StyleGAN2 Adaptive (StyleGAN2-ADA): In 2020, the NVIDIA research team noted that training the model with a small amount of data causes the discriminator to overfit [25]. To overcome this, a novel StyleGAN2 variant, StyleGAN2 Adaptive (StyleGAN2-ADA), was developed. StyleGAN2-ADA integrates an adaptive discriminator augmentation strategy that considerably stabilises training in small data regimes. The new method adopts a wide range of augmentations to prevent the discriminator from overfitting. It has proved its capability to generate good results using only a few thousand training images.  
StyleGAN2 Distillation (StyleGAN2-Dist): This StyleGAN variant [26] addresses the inference time of StyleGAN2 while image quality is preserved. This is completed by uniting unconditional image generation with paired image-to-image translation to accelerate the StyleGAN2 image generation process. The novel framework, termed StyleGAN2 Distillation (StyleGAN2-Dist), aims to distill a specific image manipulation in StyleGAN2 latent code into one image-to-image translation. Despite its impressive quality, StyleGAN2 Distillation suffers from a number of limitations. Firstly, the framework's latent space is not disentangled enough, and the transformations produced by the model are not very natural-looking. Another limitation is the need to distil each transformation to an independent model, where a universal model could be trained.  
StyleGAN3: Despite the different improvements made by StyleGAN2 and its variants, some image details seem to be generated at fixed pixel coordinates. This makes the images look less natural. This lack of translation equivariance in StyleGAN2 outputs decreases its competence in video and animation generation. In 2021, the third version of the StyleGAN models was developed [20]. The goal of StyleGAN3 is to counter the texture sticking problem (aliasing) in StyleGAN2. The new equivariant generator to translation and rotation helps the model generate images with a more natural transition animation and achieves video-quality generative performance. However, although the model maintains image quality, it somewhat trades off spatial consistency and editability since it no longer fixes textures/features to specific image coordinates.