## 4 Results and Discussion
### RQ3: What Evaluation Methods Are Used to Assess the Found Architecture?
#### 4.3.2 Datasets  
In NAS-GAN, datasets play a crucial role in both training and evaluating generated architectures. During the search process, the dataset is used to train candidate GAN architectures, allowing the NAS algorithm to assess their performance. The search typically involves iteratively sampling architectures, training them on a subset of the data, and evaluating their performance using metrics. Once a promising architecture is found, it is then trained on the full dataset to produce the final GAN model. The test set is used to evaluate the generalization capability of the trained model, ensuring it can generate high-quality images beyond what it has seen during training. The MNIST [51] dataset comprises 70,000 grayscale images of handwritten digits (0-9), each 28 \(\times\) 28 pixels in size. It's divided into 60,000 training images and 10,000 test images, with each digit class represented by roughly 6000 training and 1000 test images, ensuring balanced distribution. Despite its widespread use in machine learning for image recognition, MNIST has significant shortcomings. Its simplistic format fails to capture the complexity of real-world image processing challenges. MNIST's lack of complexity in terms of color, texture, and context limits its utility in representing modern image processing tasks. Success on this dataset doesn't necessarily indicate an algorithm's effectiveness on more sophisticated problems. While MNIST's simplicity limits its utility for modern tasks, it remains a common benchmark for NAS-GAN approaches. Previous works have used diverse evaluation metrics: NSGA-II DCGAN reported TPR = 0.9802 and FPR = 0.0042, while COEGAN achieved an IS of 1.7 \(\pm\) 0.6. As shown in Table 4, Mustangs achieved superior FID scores (42.24) compared to E-GAN's (466.1) and Lipizzaner variants (Lip-BCE: 48.96, Lip-MSE: 371.6, Lip-HEU: 52.53), demonstrating enhanced stability in low-resolution image generation. The CelebA [52] dataset is a comprehensive and widely used benchmark in computer vision and deep learning, particularly for tasks related to face recognition and attribute analysis. It comprises 202,599 high-quality color images of celebrity faces, featuring 10,177 unique identities. The dataset is notable for its large scale and rich annotations, making it invaluable for a wide range of facial analysis tasks. Each image in the CelebA dataset is annotated with 40 binary attributes, covering various facial features and characteristics such as hair color, facial expression, and the presence of accessories like glasses. Additionally, the dataset provides 5 landmark locations for each face, enhancing its utility for tasks involving facial geometry and alignment. The dataset's strength lies in its diversity and complexity. It encompasses a wide range of pose variations, background clutter, and demographic diversity, closely mimicking real-world scenarios. For generative modeling tasks, Table 4 reveals Mustangs outperformed Lipizzaner variants on FID scores (36.15 vs. Lip-BCE: 36.25, Lip-HEU: 37.87), despite CelebA's challenging high-resolution nature. This suggests our method better captures facial attribute diversity while maintaining generation quality. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Method** & **MNIST (FID \(\downarrow\) *)** & **CelebA (FID \(\downarrow\) *)** \\ \hline COEGAN [13] & 43.0 \(\pm\) 4.0 & - \\ E-GAN [22] & 466.1 & - \\ Lip-BCE [23] & 48.96 & 36.25 \\ Lip-MSE [23] & 371.6 & 158.7 \\ Lip-HEU [23] & 52.53 & 37.87 \\
**Mustangs\({}^{\dagger}\)[24]** & **42.24** & **36.15** \\ \hline \hline \end{tabular}  
* The down arrow (\(\downarrow\)) indicates that lower values are better.
* Bold values indicate the best performance across all methods.  
\end{table} 
Table 4: Comparative FID Scores on MNIST and CelebA Datasets. The LSUN [53] is a large-scale computer vision dataset for scene classification, featuring 10 diverse indoor and outdoor categories.