## 4 Experimental Results
Even though performance on PGD and FGSM is slightly lower than that of [PERSON]'s model or the Unrestricted Adversarial Model, our model largely outperforms them on other attacks, such as SPSA, MIM, and BIM. The accuracy of [PERSON]'s model specifically drops by up to half on these attacks. This drop in accuracy is because models that use previously proposed adversarial training methods defend a model from only a small number of inherently similar attacks. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Attacks** & **Clean Model** & **Madry’s Model** & **Unrest. Adv Model** & **Our Model (w/o Mixup)** & **Our Model** \\ \hline Clean & 96.4\% & 95.5\% & 91.2\% & 94.2\% & 96.8\% \\ SPSA & 26.4\% & 70.5\% & 88.5\% & 90.5\% & 91.7\% \\ MIM & 10.3\% & 50.2\% & 92.9\% & 94.1\% & 94.5\% \\ BIM & 7.4\% & 70.3\% & 88.6\% & 90.0\% & 93.1\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: CelebA accuracy comparison between a clean model, a model trained on PGD images, and models trained on our proposed method. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Attacks** & **Clean Model** & **Madry’s Model** & **Unrest. Adv Model** & **Our Model (w/o Mixup)** & **Our Model** \\ \hline Clean & 97.1\% & 98.7\% & 98.5\% & 98.4\% & 98.4\% \\ PGD & 13.5\% & 99.9\% & 98.7\% & 98.6\% & 98.3\% \\ FGSM & 13.4\% & 98.8\% & 98.4\% & 98.4\% & 98.8\% \\ Unrestricted & 13.9\% & 91.1\% & 99.9\% & 99.1\% & 99.1\% \\ SPSA & 37.7\% & 76.9\% & 90.3\% & 94.4\% & 99.8\% \\ MIM & 4.9\% & 50.0\% & 92.9\% & 90.2\% & 94.0\% \\ BIM & 9.3\% & 75.5\% & 89.5\% & 92.9\% & 98.5\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: MNIST accuracy comparison between a clean model, a model trained on PGD images, and models trained on our proposed method. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Average** & **Variance** \\ \hline Madry’s Model & 83.43\% & \(3\times 10^{-2}\) \\ Unrestricted Adv. Model & 94.70\% & \(2\times 10^{-3}\) \\ Proposed Model & 97.48\% & \(7.2\times 10^{-4}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: MNIST average and variance accuracy comparison between models (showing the variance and average performance of the model accuracy provided in Table 1). On the other hand, it can be shown that the performance improvement of the highest-performing model is due to the removal of the soft-constraint loss \(L_{1}\) and addition of the \(D2D-CE\) loss function proposed in [28]. This modification allows the AC-GAN to search for and find adversarial samples that are impossible to generate using standard methods such as PGD and FGSM. One thing to note here is that the unrestricted adversarial model, which was trained on unrestricted examples based on the original architecture in [20], outperforms our proposed model with regard to unrestricted attacks because the augmented adversarial training samples are in the same set as the attack examples. However, when comparing [PERSON]'s model with our model, the overall attack performance across all attacks is better, as shown in Table 3. In the case of the MNIST dataset, our proposed model has performance degradation on clean images; however, when considering the case of the CelebA dataset, its performance is better. There was a performance drop from 98.9% to 98.4% on clean samples when increasing the number of augmented samples, from which it can be speculated that there is a limit on how much augmentation improves the overall model performance without affecting performance on the clean dataset, and that this limit depends on the dataset. Mixup augmentation, explained in detail in the previous section, is another factor that significantly improved the adversarial robustness of the model. Our findings indicate that mixup augmentation resulted in an average accuracy improvement of 2.13% on the MNIST dataset and 1.82% on the CelebA dataset, which is noticeably superior to not using this method. The experimental results in Tables 2-4 show gaps in the defenses created by adversarial training that attackers might exploit. Research by [PERSON] et al.