## 4 Results and Discussion
### RQ1: What NAS Approaches Are Applied to GANs in the Literature?
#### 4.1.1 Evolutionary Algorithms Approaches
(2022) [29] proposed EAS-GAN, an evolutionary architectural search method that searches for generator using a multi-objective function. Generator architectures evolve using three objective functions as mutation operators: minimax, least-squares, and hinge loss, and then a traditional adversarial training for the discriminator weights. The search process takes 1 GPU Day. [PERSON] et al. (2019) [13] developed COEGAN, a neuroevolutionary and coevolutionary approach that evolves both generator and discriminator architectures. Utilizing different objectives for the generator and discriminator populations, the discriminator objective is the adversarial loss while the generator objective is the FID score. [PERSON] et al. (2022) [30] introduced EAGAN, a two-stage evolutionary NAS that decouples the search for generator and discriminator networks. It uses Multi-objective Pareto optimization, considering model size, IS, and FID as objectives. It efficiently searches for the optimal GAN in 1.2 GPU Days. Finally, [PERSON] et.al. (2024) [31] introduced EWSGAN, the method employs a two-step process: first training a super net generator using weight sharing and single-path sampling, then utilizing NSGA-II to search for optimal subnets. EWSGAN focuses solely on searching for generator architectures while using a fixed discriminator. The search process is highly efficient, completing in just 1 GPU Day. It has a multi-objective function, simultaneously optimizing IS and FID. The approach offers several advantages, including an efficient search process due to weight sharing and low-fidelity evaluation, improved stability through fair single-path sampling and a commonality-based discarding strategy. However, a potential drawback is the discarding strategy that may require further investigation, and possible challenges in scaling to higher-resolution datasets. Recent work has explored conducting hyperparameter optimization (HPO) and NAS jointly. [PERSON] and [PERSON] (2020) [32] proposed searching the architecture and hyperparameters of GANs using multi-objective evolutionary algorithms. They used Cartesian Genetic Programming (CGP) with NSGA-II (NSGA-II with CPG) to evolve generator and discriminator architectures simultaneously, optimizing various hyperparameters. The multi-objective fitness function maximized IS score and minimized FID. It is considered one of the first works that implemented HPO and NAS for GANs. However, a limitation of the work is the restriction of the network size, limiting the scalability of the method. Also, although training time was not reported, the authors stated that the search efficiency needs to be improved. Common benefits across these approaches include improved generative performance, increased architectural efficiency, and enhanced adaptability to different datasets. The use of multi-objective optimization in several approaches allowed for balancing multiple performance criteria simultaneously, leading to more robust solutions. Despite that, EAapproaches' performance tends to rely heavily on the objective function. Another limitation to EA approaches is the search space formulation: EA approaches were mainly used with discrete search space.