## 4 Analysis of the Representations
### Visualisation of Representations  
The visualisation of representations presents a challenge due to the high dimensionality of representation vectors. Nonetheless, such visualisation can be useful for analysis purposes. To address this issue, we employed principal component analysis (PCA) to reduce the dimensionality of representations from 512 to 2. We limited our experimentation to ResNet18 due to computational constraints.  
We present the visualisation of representations for different combinations of models and norms of PGD attacks in Figures 4-8. Figure 4 depicts the representations of samples from CIFAR-10 for a regularly trained ResNet18, which serves as a baseline case. In Figures 5 and 6, we visualise the data as adversarial samples with \(L_{2}\)- and \(L_{\infty}\)-norms of attack, respectively. Furthermore, we examine the representations of adversarial samples for adversarially trained models in Figures 7 and 8. To challenge the models, we use alter norms of attacks from training ones: the model trained on PGD with an \(L_{2}\)-norm is tested on an \(L_{\infty}\)-norm PGD (Figure 8), and vice versa (Figure 7). We group the representations of different classes by colours in all figures to comprehend how the representations of different classes intermingle under adversarial attacks.  
The representations of the regularly trained networks are clustered according to the classes in the data set, as shown in Figure 4. However, when subjected to adversarial attacks, all representations become heavily mixed, resulting in a more challenging classification task due to overlapping representations of different classes. The most mixed representations were observed on the regularly trained network through the \(L_{\infty}\)-norm PGD attack (Figure 6). On the contrary, adversarial training, as shown in Figures 7 and 8, resulted in some class representations (e.g., \"Automobile\" or \"Truck\") remaining clustered while becoming closer to each other than those without attacks. The same pattern was observed for the \(L_{2}\)-norm attack (Figure 6).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Attack** & **Parameters** & **Regular Model** & **Adv. Trained, \(L_{2}\)** & **Adv. Trained, \(L_{\infty}\)** \\ \hline \multirow{2}{*}{PGD} & \(L_{2}\), \(\epsilon=0.25\), \(steps=100\) & 0.686 & 0.981 & 0.989 \\  & \(L_{\infty}\), \(\epsilon=0.025\), \(steps=100\) & **0.587** & **0.751** & **0.822** \\ \hline \multirow{2}{*}{FGSM} & \(L_{2}\), \(\epsilon=0.25\) & 0.833 & 0.982 & 0.991 \\  & \(L_{\infty}\), \(\epsilon=0.025\) & 0.618 & 0.78 & 0.837 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Mean correlation coefficient for different models. The values of the lowest mean correlation coefficients for each model are bold.  
Figure 4: Representations of regularly trained ResNet18 on clean data set (CIFAR-10).  
Figure 5: Representations of regularly trained ResNet18 on adversarial data set (CIFAR-10), PGD with L\({}_{2}\)**-norm** and 20 steps.  
Figure 6: Representations of regularly trained ResNet18 on adversarial data set (CIFAR-10), PGD with L\({}_{\infty}\)**-norm** and 20 steps.  
Figure 7: Representations of adversarially trained ResNet18 (PGD, \(L_{\infty}\)-norm) on adversarial data set (CIFAR-10), PGD with L\({}_{2}\)**-norm** and 20 steps.