# Empirical Perturbation Analysis of Two Adversarial Attacks: Black Box versus White Box
## 5 Transferability and Texture Bias
### How Does Texture Change Relate to Adversarial Impact on the CNNs? In this subsection, BagNet-17 is used to visualize, thanks to heatmaps, whether texture change correlates with the adversarial impact of the obtained images for the 10 CNNs \(\mathcal{C}_{1},\cdots,\mathcal{C}_{10}\). Although we have seen that both attacks affect BagNet-17's \(c_{a}\) probability on average, here, we attempt to find the image areas in which these changes are most prominent and to compare the locations in the \(\mathcal{C}_{k}\) adversarials that have the largest impact on BagNet-17 and on \(\mathcal{C}_{k}\). To achieve this, we proceed in a similar manner as in Section 3.1, with the difference that we allow overlaps. We replace all overlapping \(17\times 17\) patches of the ancestor \(\mathcal{A}_{q}^{p}\) with patches from the same location in \(\mathcal{D}_{k}^{\textit{attk}}(\mathcal{A}_{q}^{p})\), a single patch at a time, and we extract and store the \(c_{a}\) and \(c_{t}\) probabilities given by \(\mathcal{C}_{k}\) of the obtained hybrid image \(I\) at each step. Contrary to the situation in Section 3.1, note that there are as many patches as pixels in this case. Simultaneously, these patches are also fed to BagNet-17 (leading to 50,176 predictions for each adversarial image) to extract the \(c_{a}\) and \(c_{t}\)-label values of these patches. The stored \(c_{a}\) and \(c_{t}\) label values (and combinations of them) can be displayed in a square box of size \(224\times 224\) (hence of sizes equal to the size of the handled images), resulting in a heatmap. More precisely, given an ancestor image \(\mathcal{A}_{q}^{p}\), all hybrid adversarial images obtained as above via the EA lead to 5 heatmaps, and all those obtained by BIM lead to 5 heatmaps as well. For both attacks, the first four heatmaps are obtained using BagNet-17, and the fifth is obtained using \(\mathcal{C}_{k}\) for comparison purposes. Each heatmap assesses the 10% largest variations in the following sense. We have the first sequence \((c_{a}(P(\mathcal{D}(\mathcal{A}))))_{P}\) of \(c_{a}\)-label values obtained from the evaluation by BagNet-17 of the patches of the adversarial images and a second similar sequence \((c_{a}(P(\mathcal{A})))_{P}\) of \(c_{a}\)-label values coming from the patches of the ancestor images. Both sequences are naturally indexed by the same successive patch locations \(P\). We then consider that the sequence, also indexed by the patches, was made up of the differences \(c_{a}(P(\mathcal{D}(\mathcal{A})))-c_{a}(P(\mathcal{A}))\). The selection of locations of the smallest 10% out of this sequence of differences leads to the first heatmap. One proceeds similarly for the second heatmap by selecting the location of the largest 10% of the values of \(c_{t}(P(\mathcal{D}(\mathcal{A})))-c_{t}(P(\mathcal{A}))\) (with obvious notations). The process is similar for the third and fourth heatmaps, where one considers the location of the largest 10% of the values of \(c_{a}(P(\mathcal{D}(\mathcal{A})))-c_{t}(P(\mathcal{D}(\mathcal{A})))\) for the third heatmap and of the values of \(c_{t}(P(\mathcal{D}(\mathcal{A})))-c_{a}(P(\mathcal{D}(\mathcal{A})))\) for the fourth heatmap. Finally, the fifth heatmap is obtained by considering the largest 10% of the values \(c_{t}(I_{P(\mathcal{D}(\mathcal{A}))})-c_{t}(\mathcal{A})\), where the two members of the difference are the \(c_{t}\)-label values given by the CNN \(\mathcal{C}_{k}\) for a full image: the right one for the ancestor image and the left one for the hybrid image, obtained as explained above. Figure 6 shows the outcome of this process for \(\mathcal{C}_{6}\)= ResNet-50 and ancestor \(\mathcal{A}_{10}^{8}\) (see Figure A5 in Appendix B.1 for other examples). Figure 6a shows the adversarial images \(\mathcal{D}_{6}^{EA}(\mathcal{A}_{10}^{8})\) (top) and \(\mathcal{D}_{6}^{BIM}(\mathcal{A}_{10}^{8})\) (bottom).