# Empirical Perturbation Analysis of Two Adversarial Attacks: Black Box versus White Box
## 3 Local Effect of the Adversarial Noise on the Target CNN
### Is the Global Random Aggregation of Local Adversarial Effect Sufficient to Fool the CNNs? First, replacing all patches simultaneously and at the correct location is, by definition, enough for a targeted misclassification, since its completion leads to the adversarial image. Second, most of the patches taken individually have a local adversarial impact, but none are sufficient to individually achieve a targeted attack. The issue addressed here is whether the global aggregation of the local adversarial effect is strong enough, independent of the location of the patches, to create the global adversarial effect that we are aiming at. We proceed as follows. Given an image \(I\) and an integer \(s\) such that patches of size \(s\times s\) create a partition of \(I\), \(sh(I,s)\) is a shuffled image deduced from \(I\) by randomly swapping all its patches. With these notations, \(sh(\mathcal{D}_{k}^{\textit{e}tk}(\mathcal{A}_{q}^{p}),s)\) (with \(\textit{e}tk=BIM\) or \(EA\)) is sent to \(\mathcal{C}_{k}\) CNN. One obtains the \(c_{a}\) and \(c_{t}\)-label values as well as the dominant category (which may differ from \(c_{a},c_{t}\)). The values of \(s\) used in our tests are \(16,32,56\), and \(112\), leading to partitions of the \(224\times 224\) images into \(196\), \(49\), \(16\), and \(4\) patches, respectively. Table 2 gives the outcome of these tests. For each value \(s\), each cell is composed of a triplet of numbers. The left one corresponds to the tests with the ancestor images, the middle one corresponds to the tests with images obtained by the EA, and the right one corresponds to the tests with images obtained by BIM. Each number is the percentage of images \(sh(\mathcal{A}_{q}^{p},s)\) or of images \(sh(\mathcal{D}_{k}^{\mathit{E}ak}(\mathcal{A}_{q}^{p}),s)\) taken for all ancestor images \(\mathcal{A}_{q}^{p}\), all (ancestor and target) category pairs, and all \(\mathcal{C}_{1},\cdots,\mathcal{C}_{10}\), which are classified in category \(c\), where \(c\) is the ancestor category \(c_{a}\), the target category \(c_{t}\), or any other class. To allow for comparisons, the randomly selected swapping order of the patches is performed only once per value of \(s\). For each \(s\), this uniquely defined sequence is applied in the same manner to create the \(sh(\mathcal{A}_{q}^{p},s)\), \(sh(\mathcal{D}_{k}^{EA}(\mathcal{A}_{q}^{p}),s)\), and \(sh(\mathcal{D}_{k}^{BIM}(\mathcal{A}_{q}^{p}),s)\) shuffled images. Contrary to what occurs with \(s=32,56\) and \(112\), the proportion of shuffled ancestors \(sh(\mathcal{A}_{q}^{p},s)\) classified as \(c_{a}\) is negligible for \(s=16\). Therefore, \(s=16\) seems to lead to patches that are too small for a \(224\times 224\) image to allow for a meaningful comparison between the ancestor and adversarials and is consequently disregarded in the remainder of this subsection. At all other values of \(s\), the classification of the shuffled adversarial image as a class different from \(c_{a}\) (\(c_{t}\) or other) is more common with BIM than with the EA. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \(\mathbf{s}\) & **Number of Patches** & \(\mathbf{c=c_{a}}\) & \(\mathbf{c\
otin\{c_{a},c_{t}\}}\) & \(\mathbf{c=c_{t}}\) \\ \hline
16 & 196 & 0.4, 0.1, 0.1 & 99, 69, 99, 99, 99 & 0.0, 0.0, 0.0 \\ \hline
32 & 49 & 18.0, 9.2, 5.3 & 82.0, 90.8, 94.4 & 0.0, 0.0, 0.3 \\ \hline
56 & 16 & 67.6, 39.3, 15.8 & 32.4, 60.3, 70.1 & 0.0, 0.4, 14.1 \\ \hline
112 & 4 & 88.4, 62.3, 22.3 & 11.6, 33.2, 35.9 & 0.0, 4.5, 41.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Percentages of shuffled images \(sh(\mathcal{A}_{q}^{p},s)\) (first percentage), \(sh(\mathcal{D}_{k}^{EA}(\mathcal{A}_{q}^{p}),s)\) (second percentage), and \(sh(\mathcal{D}_{k}^{BIM}(\mathcal{A}_{q}^{p}),s)\) (third percentage) for which the predicted class is \(c\). Figure 1: Single-patch replacement for \(\mathcal{A}_{q}^{4}\) and \(\mathcal{C}=\mathcal{C}_{6}\). The four pairs of graphs correspond to patches with sizes of \(16\times 16\), \(32\times 32\), \(56\times 56\), and \(112\times 112\).