## 3 Generalisation of Robustly and Adversarially Trained Models
### Equivalence of \(L_{2}\) and \(L_{\infty}\) Perturbation Norms  
To explain the lack of generalisation in the previous experiment, we used the theorem of the equivalence of vector norms on finite-dimensional spaces. The theorem states that for a given two norms \(\|.\|_{a}\) and \(\|.\|_{b}\) on the finite-dimensional vector space \(V\) over \(C\), there exists a pair of real numbers \(0<C_{1}\leq C_{2}\) such that, for all \(x\in V\), the following inequality holds:  
\[C_{1}\|x\|_{b}\leq\|x\|_{a}\leq C_{2}\|x\|_{b} \tag{5}\]  
In other words, all norms are equal to each other with respect to some coefficients. Images and adversarial perturbations are finite-dimensional vectors over \(R\); thus, the same inequality holds for the \(L_{2}\)- and \(L_{\infty}\)-norms. The \(\epsilon\)-s that we used in the previous experiments, such as \(\epsilon=0.03\) for the \(L_{\infty}\)-norm and \(\epsilon=0.25\) for the \(L_{2}\)-norm on the CIFAR-10 data set, are widely used [7; 8]. In practice, such a choice seems reasonable because the corresponding perturbations are indistinguishable by the human eye. However, such \(\epsilon\)-s might be too loose, which can explain the superiority of \(L_{\infty}\)-norm attacks. 1. Take the CIFAR-10 test data set and two attacks: PGD with an \(L_{2}\)-norm of attack and \(\epsilon=0.25\); and PGD with an \(L_{\infty}\)-norm and \(\epsilon=0.03\). 2. Generate adversarial perturbations by these attacks. Save only the perturbation that misclassifies a network: the prediction of such an adversarial sample must not be equal to the true label and prediction of the original image. 3. Flatten the perturbation matrices into vectors and calculate the norms. 4. Iterating over the perturbations, take \(C_{1}\) and \(C_{2}\) as \(C_{1}=min(\frac{x_{opt_{2}}}{x_{opt_{\infty}}})\) and \(C_{2}=max(\frac{x_{opt_{2}}}{x_{opt_{\infty}}})\). Therefore, in Equation (5) we picked \(a=2\), \(b=\infty\), but this was an arbitrary choice. 5. Repeat for three models: regular ResNet-50 and adversarially trained ResNets-50 with \(L_{2}\)- and \(L_{\infty}\)-norms of attacks. In addition, for all cases, calculate the percentage of adversarial misclassified samples by the model. The results are presented in Table 4. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{4}{c}{**Attack Norm**} \\  & \multicolumn{2}{c}{\(L_{2}\)} & \multicolumn{2}{c}{\(L_{\infty}\)} \\ \hline  & \(C_{1}\) & \(C_{2}\) & **\% of Misclassified** & \(C_{1}\) & \(C_{2}\) & **\% of Misclassified** \\ \hline Regular ResNet-50 & 4.6 & 14.4 & 86 & 43.6 & 55.4 & 96 \\ \hline Adv. trained ResNet-50 (\(L_{2}\)) & 3.3 & 14.3 & 8 & 45.8 & 55.4 & 52 \\ \hline Adv. trained ResNet-50 (\(L_{\infty}\)) & 1.6 & 7.3 & 10 & 45.4 & 55.4 & 44 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Coefficients for Equation (5). Testing attacks are PGD with \(L_{2}\) and \(L_{\infty}\) perturbation norms; testing data set is CIFAR-10. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Regular Model** & **Adv. Trained, \(L_{\infty}\)-Norm** & **Adv. Trained, \(L_{2}\)-Norm** \\ \hline Accuracy on CINIC-10 (no attack) & 76\% & 75\% & 72\% \\ Accuracy on CIFAR-10 (no attack) & 95\% & 94\% & 91\% \\ \hline Accuracy on CINIC-10 (PGD attack) & 7\%—\(L_{\infty}\) attack, & 37\%—\(L_{\infty}\) attack, & 40 \%—\(L_{\infty}\) attack, \\  & 11\%—\(L_{2}\) attack & 36\%—\(L_{2}\) attack & 43\%—\(L_{2}\) attack \\ Accuracy on CIFAR-10 (PGD attack) & 7\%—\(L_{\infty}\) attack, & 57\%—\(L_{\infty}\) attack, & 61 \%—\(L_{\infty}\) attack, \\  & 27\%—\(L_{2}\) attack & 55\%—\(L_{2}\) attack & 64 \%—\(L_{2}\) attack \\ \hline Accuracy on CINIC-10 (FGSM attack) & 49 \%—\(L_{\infty}\) attack & 67 \%—\(L_{\infty}\) attack & 66 \%—\(L_{\infty}\) attack \\ Accuracy on CIFAR-10 (FGSM attack) & 72 \%—\(L_{\infty}\) attack & 89 \%—\(L_{\infty}\) attack & 87 \%—\(L_{\infty}\) attack \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Accuracy of ResNet18-s, trained on CIFAR-10 data set.