## 4 Results and Discussion
### RQ3: What Evaluation Methods Are Used to Assess the Found Architecture?
#### 4.3.1 Evaluation Metrics
Some, like EAGAN and AutoGAN, explicitly use parameter count as an optimization criterion or reportable metric. Others, such as AdversarialNAS and NSGA-II DCGAN, dynamically adjust size during training or use multi-objective optimization. Methods like E-GAN and Lipizzaner focus on pruning and efficiency. However, not all approaches prioritize or report size metrics, with some like AGAN and E\({}^{2}\)GAN emphasizing performance over explicit size considerations. A more holistic approach to architecture optimization is necessary, considering factors beyond just size, such as interpretability, robustness, and adaptability. This comprehensive view could yield more balanced and effective GAN architectures, avoiding the pitfalls of overly simplistic size-based optimizations while still maintaining efficiency. * Mode Collapse Resistance: Mode collapse is a common failure mode in GANs where the generator produces a limited variety of outputs, failing to capture the full diversity of the target distribution. In the context of NAS, architectures should be evaluated on their resistance to mode collapse [48]. This can be assessed through diversity metrics applied to generated samples, or by analyzing the distribution of generated outputs in feature space. A good NAS solution for GANs should prioritize architectures that maintain output diversity while still producing high-quality samples. * Convergence Stability: GAN training instability, characterized by mode collapse and oscillating losses, remains a significant challenge. Convergence stability in NAS for GANs is crucial, measured by consistent performance across multiple initializations [49]. In the reviewed literature, multiple solutions have been developed to address this issue, such as adaptive training techniques used by AGAN. Some methods focus on balancing Generator and Discriminator performance used in NSGA-II DCGAN, while Mustangs employs multi-agent systems or co-evolutionary algorithms COEGAN, Lipizzaner that continuously measure stability. Advanced techniques like alpha-divergence minimization used by alphaGAN and EWSGAN's Wasserstein distance optimization have also been explored. Despite these diverse approaches, the field lacks a comprehensive comparison of their effectiveness, and the trade-offs between stability, performance, and computational cost remain unclear. Moreover, the reliance on existing evaluation metrics may not fully capture the nuances of convergence stability, suggesting a need for more robust assessment methods. * Sample Quality: While automated metrics like IS and FID are valuable, they don't always align perfectly with human perception. Therefore, subjective evaluation of generated images by human raters remains an important aspect of GAN assessment [50]. This typically involves showing raters a mix of real and generated images and asking them to judge qualities such as realism, coherence, and aesthetic appeal. For NAS in GANs, this human evaluation can be used as a final validation step for top-performing architectures. However, it's important to note that human evaluation is time-consuming and can be subject to biases, so it's often used in conjunction with automated metrics rather than as the sole evaluation criterion. All presented work conducted sample quality check in their experiments.