## 4 Results and Discussion
### RQ1: What NAS Approaches Are Applied to GANs in the Literature?
#### 4.1.3 Gradient-Based Approaches  
Gradient-based algorithms are a fundamental class of optimization methods [38]. These techniques leverage the gradient of an objective function to iteratively update model parameters, efficiently navigating high-dimensional parameter spaces to minimize loss or maximize performance. Gradient-based algorithms are not typically used directly for searching neural network architectures. However, there are related approaches that use gradients to help with architecture search. Gradient-based NAS methods have been successfully applied to optimize GAN architectures. AdversarialNAS, proposed by [PERSON] et al. (2020) [39], uses a differentiable approach to simultaneously search for both generator and discriminator architectures. It employs an adversarial search mechanism and can discover high-performing architectures in just 1 GPU Day. The method boasts a large search space and demonstrates good transferability and scalability. [PERSON] and [PERSON] (2021) [40] introduced DEGAS, a gradient-based method focusing solely on generator architecture search. DEGAS reformulates the problem as a differentiable optimization task and utilizes the Global Latent Optimization technique to avoid adversarial training instabilities during search. It can find competitive generator architectures in 1.16 GPU Days, significantly faster than most previous reinforcement learning-based methods. A limitation of DEGAS is that it only searches for Generator architectures. Searching for Discriminator architectures may have an impact on results. GAN Compression, introduced by [PERSON] et al. (2020) [41], presents a gradient-based framework for compressing the generator component of conditional GANs. Rather than relying on reinforcement learning or evolutionary methods, this approach decouples network training from architecture search by first training a \"once-for-all\" generator via standard gradient descent with weight sharing. In this phase, the network is designed to support a vast array of sub-networks through flexible channel configurations. Once trained, the method efficiently evaluates these sub-networks using differentiable losses--including intermediate feature distillation--to select the most efficient architecture that satisfies a specified computational budget. This gradient-based sub-network selection allows GAN Compression to dramatically reduce both the inference time and model size; empirical results show reductions in MACs by up to 21\(\times\) on models such as CycleGAN, pix2 pix, and GauGAN while preserving high visual fidelity. A key strength of GAN Compression is its model-agnostic nature and its ability to stabilize GAN training via knowledge transfer, thereby enabling interactive conditional GAN applications on resource-constrained devices. [PERSON] et al. (2021) [14] proposed alphaGAN, a fully differentiable architecture search framework that formulates the problem as a bi-level minimax optimization. A key innovation is the use of the duality gap as a differentiable evaluation metric. alphaGAN can efficiently discover high-performing architectures for both conventional GANs and StyleGAN2 in just 3 GPU Hours, demonstrating strong transferability and scalability. [PERSON] et al. (2024) [42] presented Differentiable architecture search with attention mechanisms for GANs (DAMGAN) an innovative evolution in gradient-based NAS approaches. Unlike traditional methods that rely on fixed architectural parameters, DAMGAN leverages a dual-attention strategy to guide the search process, thereby enhancing training stability and efficiency. In this approach, a generator supernet is constructed with two distinct attention mechanisms: up-attention (UA) and down-attention (DA). UA is used to select the most salient feature maps before candidate operations are applied, effectively reducing computational overhead and ensuring that only the most informative features contribute to the network's evolution.