## 3 Methodology
### Modifying AC-GAN Architecture for Adversarial Training  
Taking a closer look at the AC-GAN architecture proposed in [17], which is used in constructing Unrestricted Adversarial Examples from scratch, it can be noted that three different loss functions are proposed. According to the authors, the first loss \(L_{0}\) improves the performance of the target classifier \(f\) in predicting \(y_{target}\). It is expected that the classifier classifies, or rather misclassifies, the input with a source label \(y_{source}\) to the target label \(y_{target}\), as shown in Equation (9). \[L_{0}\triangleq-\text{log}\,f\big{(}y_{target}|_{\theta}(z,y_{source})\big{)} \tag{9}\]  
On the other hand, the loss term \(L_{1}\), shown in Equation (10) below, places a soft constraint on the search region of the noise \(z\) such that more diverse samples can be produced. In this paper, we argue that the loss function restricts the search region instead, and that replacing it with another term would be more efficient. This improvement is discussed further in the later parts of this section. \[L_{1}\triangleq\frac{1}{m}\sum_{i=1}^{m}\max\Big{\{}\Big{|}z_{i}-z_{i}^{0} \Big{|}-\varepsilon,0\Big{\}} \tag{10}\]  
The last loss component, as shown in the equation below, encourages the auxiliary classifier \(c_{\varphi}\) to correctly classify the generated images, i.e., classify them as their source label. \[L_{2}\triangleq-\text{log}\,c_{\varphi}\big{(}y_{source}|_{\theta}(z,y_{source })\big{)} \tag{11}\]  
To summarize all the loss functions mentioned above, the total loss helps the model find the latent space \(z\) that produces quality unrestricted adversarial examples; hence, minimizing the loss \(L\) helps to optimize the latent space provided to the Generator. The first loss \(L_{0}\) helps the target classifier \(f\) to predict \(y_{target}\), which is the wrong target that an attacker might exploit. On the other hand, \(L_{1}\) constrains the search range of the latent space to a certain degree. Our research shows that this restriction reduces the efficacy and universality of the generated adversarial samples and weakens adversarial training on the said samples. Lastly, \(L_{2}\) allows the auxiliary classifier to make the correct prediction \(y_{source}\). As shown in Figure 2, the Generator is trying to minimize the summation of all three loss functions. The hypothesis is that the soft constraint \(L_{1}\) limits the search range of the Generator, thereby not allowing for an efficient defense. This is shown in the experiments section by comparing adversarial training on the original AC-GAN architecture proposed in [20] with our modified method. The experiments involve removing different loss terms, including the soft-constraint loss \(L_{1}\), adding a custom loss function from [28] named Data-to-Data Cross-Entropy Loss (D2D-CE), and training the proposed model on unrestricted examples to test against different types of attacks. [PERSON] et al. 2021 [28] improved the performance of traditional AC-GAN by creating data-to-class and data-to-data relationships, making the produced samples more visually similar to the original target data. Therefore, testing different combinations of loss functions from the above four has proven that using D2D-CE loss and removing the soft constraint generates unrestricted samples, in turn creating a more robust adversarial training output. The formulation of the D2D-CE loss as shown by [PERSON] et al. (2021) [PERSON] et al. (2021) is provided in the equation below. \[ \begin{split}\text{L}_{\text{D2D-CE}}\triangleq\frac{1}{\text{N}} \sum_{i=1}^{\text{N}}\log(\frac{\exp([f_{i}^{\top}v_{y_{i}}-m_{p}]_{-}/\tau)}{ \exp([f_{i}^{\top}v_{y_{i}}-m_{p}]_{-}/\tau)+\sum_{j\in N(i)}\exp([f_{i}^{\top} v_{y_{i}}-m_{p}]_{+}/\tau)})\end{split}  \tag{12}\]  
Figure 3 below compares the proposed model to the original adversarial model trained on unrestricted examples from [20]. The models were tested on a range of attacks in order to test their universality.