## 4 Results and Discussion
### RQ3: What Evaluation Methods Are Used to Assess the Found Architecture?
#### 4.3.1 Evaluation Metrics  
* Inception Score (IS): IS [43] attempts to measure image realism and diversity using a pre-trained InceptionV3 network [44], calculating scores based on predicted class probability distributions. However, IS has critical flaws: it lacks direct comparison to training data, can miss mode collapse, and introduces ImageNet biases. Its non-intuitive nature hinders meaningful interpretation of score differences. Despite these limitations, IS remains widely used, highlighting the need for more robust evaluation metrics. Our review of representitive works (e.g., [12; 14; 22]) shows IS scores vary slightly across methods, with NAS approaches demonstrating more consistent high-quality images. Notably, the state-of-the-art IS on this benchmark comes from EWSGAN, an evolutionary method. While this highlights the potential of evolutionary approaches, it also underscores the success of NAS methods in consistently achieving high IS scores. Given IS's vulnerability to adversarial examples and other shortcomings, it should be used cautiously and always supplemented with additional evaluation metrics for a comprehensive assessment of generative models. * Frechet Inception Distance (FID): FID [45] improves on IS by comparing generated and real image statistics using InceptionV3 features, calculating the Frechet distance between feature distributions. While considered more robust than IS and better correlated with human judgment, FID's limitations include assuming Gaussian distributions, relying on a potentially biased pre-trained network, and possibly overlooking certain aspects of image quality or diversity. Despite these drawbacks, FID remains widely used for evaluating generative models. Analysis of reported scores from major NAS-GAN approaches [12; 14; 29] reveals significant variation across methods, with NAS approaches consistently outperforming manual methods. AdversarialNAS, EAGAN, and EWSGAN demonstrate impressive scores across datasets, suggesting NAS's effectiveness in generating high-quality images. This data indicates that automated searches often find more optimal architectures than human designers. Recent work using EA algorithms has achieved state-of-the-art results, though the longevity of this trend remains uncertain. However, these findings warrant further investigation into the factors contributing to NAS's success and potential limitations of current evaluation metrics. * Computational Efficiency: NAS-GAN needs to improve efficiency and performance while managing computational resources. Search time, measured in GPU Days, is critical in NAS as it explores architecture space to identify promising GAN structures [46]. To address computational challenges, researchers have developed several strategies. Weight sharing reduces parameters and computational load, used in [14; 30; 31]. Adaptive mechanisms and progressive growing dynamically adjust network complexity, as seen in AGAN [34]. Evolutionary algorithms iteratively improve GAN architectures, adopted by (e.g., [30; 36; 40]). Multi-objective optimization balances performance and computational cost, implemented by [28; 32]. Ensemble methods use multiple discriminators for improved diversity and efficiency, utilized by E-GAN. Coevolutionary algorithms evolve Generators and Discriminators simultaneously, employed by Lipizzaner and COEGAN. Lastly, multi-stage training gradually increases model complexity, an approach adopted by Mustangs. * Model Size: Model size is a critical yet complex factor in NAS for GANs, requiring a delicate balance between performance and efficiency. While NAS algorithms often incorporate size constraints, this approach has significant limitations. The focus on smaller models, though advantageous for memory usage and speed, can lead to oversimplification of trade-offs, bias towards suboptimal architectures, and difficulty in accurately assessing the impact on performance. Moreover, it risks overlooking larger innovative architectures with unique benefits [47]. Reviewed methods address model size differently.